Modeling and inference with relational dynamic bayesian networks

Many domains in the real world are richly structured, containing a diverse set of agents characterized by different set of features and related to each other in a variety of ways. Moreover, uncertainty both on the objects observations and on their relations can be present. This is the case of many problems as, for example, multi-target tracking, activity recognition, automatic surveillance and traffic monitoring. The common ground of these types of problems is the necessity of recognizing and understanding the scene, the activities that are going on, who are the actors, their role and estimate their positions. When the environment is particularly complex, including several distinct entities whose behaviors might be correlated, automated reasoning becomes particularly challenging. Even in cases where humans can easily recognize activities, current computer programs fail because they lack of commonsense reasoning, and because the current limitation of automated reasoning systems. As a result surveillance supervision is so far mostly delegated to humans. The explicit representation of the interconnected behaviors of agents can provide better models for capturing key elements of the activities in the scene. In this Thesis we propose the use of relations to model particular correlations between agents features, aimed at improving the inference task. We propose the use of relational Dynamic Bayesian Networks, an extension of Dynamic Bayesian Networks with First Order Logic, to represent the dependencies between an agent’s attributes, the scene’s elements and the evolution of state variables over time. In this way, we can combine the advantages of First Order Logic (that can compactly represent structured environments), with those of probabilistic models (that provide a mathematically sound framework for inference in face of uncertainty). In particular, we investigate the use of Relational Dynamic Bayesian Networks to represent the dependencies between the agents’ behaviors in the context of multi-agents tracking and activity recognition. We propose a new formulation of the transition model that accommodates for relations and present a filtering algorithm that extends the Particle Filter algorithm in order to directly track relations between the agents. The explicit recognition of the relationships between interacting objects can improve the understanding of their dynamic domain. The inference algorithm we develop in this Thesis is able to take into account relations between interacting objects and we demonstrate with experiments that the performance of our relational approach outperforms those of standard non-relational methods. While the goal of emulating human-level inference on scene understanding is out of reach for the current state of the art, we believe that this work represents an important step towards better algorithms and models to provide inference in complex multi-agent systems. Another advantage of our probabilistic model is its ability to make inference online, so that the appropriate cause of action can be taken when necessary (e.g., raise an alarm). This is an important requirement for the adoption of automatic surveillance systems in the real world, and avoid the common problems associated with human surveillance.