In the present work, we employ backward stochastic differential equations (BSDEs) to study the optimal control problem of semi-Markov processes on a finite horizon, with general state and action spaces. More precisely, we prove that the value function and the optimal control law can be represented by means of the solution of a class of BSDEs driven by a semi-Markov process or, equivalently, by the associated random measure. We also introduce a suitable Hamilton–Jacobi–Bellman (HJB) equation. With respect to the pure jump Markov framework, the HJB equation in the semi-Markov case is characterized by an additional differential term ∂a. Taking into account the particular structure of semi-Markov processes, we rewrite the HJB equation in a suitable integral form which involves a directional derivative operator D related to ∂a. Then, using a formula of Ito^ type tailor-made for semi-Markov processes and the operator D, we are able to prove that a BSDE of the above-mentioned type provides the unique classical solution to the HJB equation, which identifies the value function of our control problem

Bandini, E., Confortola, F. (2017). Optimal control of semi-Markov processes with a backward stochastic differential equations approach. MATHEMATICS OF CONTROL SIGNALS AND SYSTEMS, 29(1) [10.1007/s00498-016-0181-6].

Optimal control of semi-Markov processes with a backward stochastic differential equations approach

Bandini, E;Confortola, F.
2017

Abstract

In the present work, we employ backward stochastic differential equations (BSDEs) to study the optimal control problem of semi-Markov processes on a finite horizon, with general state and action spaces. More precisely, we prove that the value function and the optimal control law can be represented by means of the solution of a class of BSDEs driven by a semi-Markov process or, equivalently, by the associated random measure. We also introduce a suitable Hamilton–Jacobi–Bellman (HJB) equation. With respect to the pure jump Markov framework, the HJB equation in the semi-Markov case is characterized by an additional differential term ∂a. Taking into account the particular structure of semi-Markov processes, we rewrite the HJB equation in a suitable integral form which involves a directional derivative operator D related to ∂a. Then, using a formula of Ito^ type tailor-made for semi-Markov processes and the operator D, we are able to prove that a BSDE of the above-mentioned type provides the unique classical solution to the HJB equation, which identifies the value function of our control problem
Articolo in rivista - Articolo scientifico
Backward stochastic differential equations; Marked point processes; Optimal control problems; Semi-Markov processes;
English
2017
29
1
1
reserved
Bandini, E., Confortola, F. (2017). Optimal control of semi-Markov processes with a backward stochastic differential equations approach. MATHEMATICS OF CONTROL SIGNALS AND SYSTEMS, 29(1) [10.1007/s00498-016-0181-6].
File in questo prodotto:
File Dimensione Formato  
MCSS.pdf

Solo gestori archivio

Dimensione 677.73 kB
Formato Adobe PDF
677.73 kB Adobe PDF   Visualizza/Apri   Richiedi una copia

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/10281/179702
Citazioni
  • Scopus 10
  • ???jsp.display-item.citation.isi??? 8
Social impact