Advances in Applied Probability
- Adv. in Appl. Probab.
- Volume 48, Number 1 (2016), 255-273.
Perturbation analysis of inhomogeneous finite Markov chains
In this paper we provide a perturbation analysis of finite time-inhomogeneous Markov processes. We derive closed-form representations for the derivative of the transition probability at time t, with t > 0. Elaborating on this result, we derive simple gradient estimators for transient performance characteristics either taken at some fixed point in time t, or for the integrated performance over a time interval [0 , t]. Bounds for transient performance sensitivities are presented as well. Eventually, we identify a structural property of the derivative of the generator matrix of a Markov chain that leads to a significant simplification of the estimators.
Adv. in Appl. Probab., Volume 48, Number 1 (2016), 255-273.
First available in Project Euclid: 8 March 2016
Permanent link to this document
Mathematical Reviews number (MathSciNet)
Zentralblatt MATH identifier
Primary: 60J27: Continuous-time Markov processes on discrete state spaces
Secondary: 65C05: Monte Carlo methods
Heidergott, Bernd; Leahu, Haralambie; Löpker, Andreas; Pflug, Georg. Perturbation analysis of inhomogeneous finite Markov chains. Adv. in Appl. Probab. 48 (2016), no. 1, 255--273. https://projecteuclid.org/euclid.aap/1457466165