Graphical models for continuous time inference and decision making

Reasoning about evolution of system in time is both an important and challenging task. We are interested in probability distributions over time of events where often observations are irregularly spaced over time. Probabilistic models have been widely used to accomplish this task but they have some limits. Indeed, Hidden Markov Models and Dynamic Bayesian Networks in general require the specification of a time granularity between consecutive observations. This requirement leads to computationally inefficient learning and inference procedures when the adopted time granularity is finer than the time spent between consecutive observations, and to possible losses of information in the opposite case. The framework of Continuous Time Bayesian Networks (CTBN) overcomes this limit, allowing the representation of temporal dynamics over a structured state space. In this dissertation an overview of the semantic and inference aspects of the framework of the CTBNs is proposed. The limits of exact inference are overcome using approximate inference, in particular the cluster-graph message passing algorithm and the Gibbs Sampling has been investigated. The CTBN has been applied to a real case study of diagnosis of cardiogenic heart failure, developed in collaboration with domain experts. Moving from the task of simply reasoning under uncertainty, to the task of deciding how to act in the world, a part of the dissertation is devoted to graphical models that allow the inclusion of decisions. We describe Influence Diagrams, which extend Bayesian Networks by introducing decisions and utilities. We then discuss an approach for approximate representation of optimal strategies in influence diagrams. The contributions of the dissertation are the following: design and development of a CTBN software package implementing two of the most important inference algorithms (Expectation Propagation and Gibbs Sampling), development of a realistic diagnosis scenario of cardiogenic heart failure (to the best of our knowledge it is the first clinical application of this type), the approach of information enhancement to reduce the domain of the policy in large influence diagrams together with an important contribution concerning the identification of informational links to add in the graph.