New Zealand

Takashi Kuremoto, Takaomi Hirata, Masanao Obayashi, Shingo Mabu and Kunikazu Kobayashi (April 3rd 2019). Training Deep Neural Networks with Reinforcement Learning for Time Series Forecasting, Time Series Analysis - Data, Methods, and Applications, Chun-Kit Ngan, IntechOpen, DOI: 10.5772/intechopen.85457. 

Kunikazu Kobayashi, Koji Nakano, Takashi Kuremoto and Masanao Obayashi (February 1st 2010). Objective-based Reinforcement Learning System for Cooperative Behavior Acquisition, Application of Machine Learning, Yagang Zhang, IntechOpen, DOI: 10.5772/8615. 

Celia Escamilla-Rivera (May 1st 2020). Bayesian Deep Learning for Dark Energy [Online First], IntechOpen, DOI: 10.5772/intechopen.91466.

Large-scale screening for potential threats with limited resources and capacity for screening is a problem of interest at airports, seaports, and other ports of entry. Adversaries can observe screening procedures and arrive at a time when there will be gaps in screening due to limited resource capacities. To capture this game between ports and adversaries, this problem has been previously represented as a Stackelberg game, referred to as a Threat Screening Game (TSG). Given the significant complexity associated with solving TSGs and uncertainty in arrivals of customers, existing work has assumed that screenees arrive and are allocated security resources at the beginning of the time-window. In practice, screenees such as airport passengers arrive in bursts correlated with flight time and are not bound by fixed time-windows. To address this, we propose an online threat screening model in which the screening strategy is determined adaptively as a passenger arrives while satisfying a hard bound on acceptable risk of not screening a threat. To solve the online problem, we first reformulate it as a Markov Decision Process (MDP) in which the hard bound on risk translates to a constraint on the action space and then solve the resultant MDP using Deep Reinforcement Learning (DRL). To this end, we provide a novel way to efficiently enforce linear inequality constraints on the action output in DRL. We show that our solution allows us to significantly reduce screenee wait time without compromising on the risk.

AAMAS 2020

Solving Online Threat Screening Games using Constrained Action Space Reinforcement Learning

correlated learning

resource constrained reinforcement learning

sequential matching

ride pooling/sharing

threat screening games

optlearnmas

game theory

deep learning

security

reinforcement learning

AAMAS is the leading scientific conference for research in autonomous agents and multi-agent systems. The AAMAS conference series was initiated in 2002 as the merging of three respected scientific meetings: the International Conference on Multi-Agent Systems (ICMAS), the International Workshop on Agent Theories, Architectures, and Languages (ATAL), and the International Conference on Autonomous Agents (AA). The aim of the joint conference is to provide a single, high-profile, internationally-respected archival forum for scientific research in the theory and practice of autonomous agents and multi-agent systems.

Browse keynotes, discussions, panels and over 300 presentations.


AAMAS is the leading scientific conference for research in autonomous agents and multi-agent systems. The AAMAS conference series was initiated in 2002 as the merging of three respected scientific meetings: the International Conference on Multi-Agent Systems (ICMAS), the International Workshop on Agent Theories, Architectures, and Languages (ATAL), and the International Conference on Autonomous Agents (AA). 

technical paper

Mitingating the Negative Side Effects of Reasoning with Imperfect Models: A Mult - Objective Approach

A variety of constructive manipulation, control, and bribery for approval-based multi-winner voting have been extensively studied very recently. However, their destructive counterparts seem to be less studied in the literature so far. This paper aims to fill this gap by exploring the complexity of several destructive bribery problems under five prestigious approval-based multi-winner voting rules. Generally speaking, these problems are to determine if a number of given candidates can be excluded from any winning committees by performing a series of modification operations yet without exceeding a given budget. We consider five operations. We offer a complete landscape of the complexity of the problems studied in this paper, and for NP-hard problems we study their parameterized complexity with respect to meaningful parameters.

On the Complexity of Destructive Bribery in Approval-Based Multiwinner Voting

"Peer-to-peer ridesharing (P2P-RS) enables people to arrange one-time rides with their own private cars, without the involvement of professional drivers. It is a prominent collective intelligence application producing significant benefits both for individuals (reduced costs) and for the entire community (reduced pollution and traffic), as we showed in a recent publication where we proposed an online approximate solution algorithm for large-scale P2P-RS.
In this paper we tackle the fundamental question of assessing the benefit of predicting ridesharing requests in the context of P2P-RS optimisation. 
Results on a public real-world show that, by employing a perfect predictor, the total reward can be improved by 5.27% with a forecast horizon of $1$ minute.
On the other hand, a vanilla long short-term memory neural network cannot improve upon a baseline predictor that simply replicates the previous day's requests, whilst achieving an almost-double accuracy"

Predicting Requests in Large-Scale Online P2P Ridesharing

Motivated by the apparent societal need to design complex autonomous systems whose decisions and actions are humanly intelligible, the study of explainable artificial intelligence, and with it, research on explainable autonomous agents has gained increased attention from the research community. One important objective of research on explainable agents is the evaluation of explanation approaches in human-computer interaction studies. In this demonstration paper, we present a way to facilitate such studies by implementing explainable agents and multi-agent systems that i) can be deployed as static files, not requiring the execution of server-side code, which minimizes administration and operation overhead, and ii) can be embedded into web front ends and other JavaScript-enabled user interfaces, hence increasing the ability to reach a broad range of users. We then demonstrate the approach with the help of an application that was designed to assess the effect of different explainability approaches on the human intelligibility of an unmanned aerial vehicle simulation.

Explainable Agents as Static Web Pages: UAV Simulation Example

We propose a novel method for analyzing and visualizing the complexity of standard reinforcement learning (RL) benchmarks based on score distributions. A large number of policy networks are generated by randomly guessing their parameters, and then evaluated on the benchmark task; the study of their aggregated results provide insights into the benchmark complexity. Our method guarantees objectivity of evaluation by sidestepping learning altogether: the policy network parameters are generated using Random Weight Guessing (RWG), making our method agnostic to (i) the classic RL setup, (ii) any learning algorithm, and (iii) hyperparameter tuning. We show that this approach isolates the environment complexity, highlights specific types of challenges, and provides a proper foundation for the statistical analysis of the task's difficulty. We test our approach on a variety of classic control benchmarks from the OpenAI Gym, where we show that small untrained networks can provide a robust baseline for a variety of tasks. The networks generated often show good performance even without gradual learning, incidentally highlighting the triviality of a few popular benchmarks.

Analyzing Reinforcement Learning Benchmarks using Random Weight Guessing

Multiagent systems can use commitments as the core of a general coordination infrastructure, supporting both cooperative and non-cooperative interactions. For agents whose objectives are aligned, and where one agent can help another achieve greater reward by sacrificing some of its own reward, their cooperative commitment should be chosen to maximize their joint reward. We present a solution to the problem of how cooperative agents can efficiently find an (approximately) optimal commitment by querying about carefully selected commitment choices. We prove structural properties of the agents' values as functions of the parameters of the commitment specification, and develop a greedy method for composing a query with provable approximation bounds, that we empirically show can find nearly optimal commitments in a fraction of the time that methods that lack our insights require.

Efficient Querying for Cooperative Commitments

In scenarios that require teamwork, we usually have at hand a variety of specific tasks, for which we need to form a team in order to carry out each one. Here we target the problem of matching teams with tasks within the context of education, and specifically in the context of forming teams of students and allocating them to internship programs. We provide a formalization of the problem, and show the computational hardness of solving it optimally.
Thereafter, we propose a heuristic anytime algorithm that generates an initial team allocation which later on attempts to improve in an iterative process.
Moreover, we present the results of our evaluation, and compare the results with a state-of-the-art optimizer.

TAIP: an anytime algorithm for allocating student teams to internship programs

Theory of Mind is commonly defined as the ability to attribute mental states (e.g., beliefs, goals) to oneself, and to others. A large body of previous work---from the social sciences to artificial intelligence---has observed that Theory of Mind capabilities are central to providing an explanation to another agent or when explaining that agent’s behaviour. In this paper, we build and expand upon previous work by providing an account of explanation in terms of the beliefs of agents and the mechanism by which agents revise their beliefs given possible explanations. Our account satisfies a number of important desiderata which we posit must be met by accounts of explanation that purport to employ Theory of Mind, in contrast to previous work which has addressed some but not all of these desiderata.

Towards the Role of Theory of Mind in Explanation

Maximizing Plan Legibility in Stochastic Environments (Extended Abstract)

In this work, the Distributed Bayesian (D-Bay) algorithm is applied to an autonomous search use case. Within the use case multiple unmanned aerial vehicles equipped with cameras cooperatively search an area and minimize the required time. The use case is mod- eled within the continuous Distributed Constraint Optimization Problem (DCOP) framework. This framework extends the (discrete) DCOP framework by allowing variables with continuous domains. Compared to similar DCOP solvers, the characteristics of the D- Bay algorithm are well suited for the use case and allow for the implementation on autonomous vehicles with limited resources (computational power, memory, and communication bandwidth). Experimental results are given and these results are used to vali- date a simulation environment. Within the simulation environment various scenarios are implemented. The D-Bay algorithm was able to find solutions within 3.5 % of the optimal solution with a limited amount of samples per agent.

Downloads

Next from AAMAS 2020

Mitingating the Negative Side Effects of Reasoning with Imperfect Models: A Mult - Objective Approach

Similar lecture

Who and When to Screen: Multi-Round Active Screening for Recurrent Infectious Diseases Under Uncertainty

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from AAMAS 2020

Mitingating the Negative Side Effects of Reasoning with Imperfect Models: A Mult - Objective Approach

Similar lecture

Who and When to Screen: Multi-Round Active Screening for Recurrent Infectious Diseases Under Uncertainty

Downloads