A regularized maximum likelihood estimation for hidden Markov models with covariates

Hidden Markov models are widely used for analyzing longitudinal data, capturing unobserved heterogeneity through a latent process that generally follows a first-order Markov chain. When covariates affect the conditional distribution of the response variables, high latent state separation may arise when computing maximum likelihood estimation of parameters, particularly with binary responses. This issue can lead to instability in parameter estimates. To overcome this problem, we propose to penalize the log-likelihood function with a term that depends on the relative distance between support points, modifying the usual expectation-maximization algorithm. We present simulation results comparing estimates obtained with and without penalization under different scenarios and apply the method to data on hypotension occurrences.