The Annals of Applied Probability
- Ann. Appl. Probab.
- Volume 8, Number 4 (1998), 1270-1290.
Multi-armed bandits in discrete and continuous time
We analyze Gittins' Markovian model, as generalized by Varaiya, Walrand and Buyukkoc, in discrete and continuous time. The approach resembles Weber's modification of Whittle's, within the framework of both multi-parameter processes and excursion theory. It is shown that index-priority strategies are optimal, in concert with all the special cases that have been treated previously.
Ann. Appl. Probab., Volume 8, Number 4 (1998), 1270-1290.
First available in Project Euclid: 9 August 2002
Permanent link to this document
Digital Object Identifier
Mathematical Reviews number (MathSciNet)
Zentralblatt MATH identifier
Primary: 60G40: Stopping times; optimal stopping problems; gambling theory [See also 62L15, 91A60]
Secondary: 60J55: Local time and additive functionals 60G44: Martingales with continuous parameter
Kaspi, Haya; Mandelbaum, Avishai. Multi-armed bandits in discrete and continuous time. Ann. Appl. Probab. 8 (1998), no. 4, 1270--1290. doi:10.1214/aoap/1028903380. https://projecteuclid.org/euclid.aoap/1028903380