Optimal Stopping Model with Unknown Transition Probabilities

• Autorzy: Masayuki Horiguchi , Alexey B. Piunovskiy
• Języki publikacji: EN

Abstrakty

EN

This article concerns the optimal stopping problem for a discrete-time Markov chain with observable states, but with unknown transition probabilities. A stopping policy is graded via the expected total-cost criterion resulting from the non-negative running and terminal costs. The Dynamic Programming method, combined with the Bayesian approach, is developed. A series of explicitly solved meaningful examples illustrates all the theoretical issues. (original abstract)

Słowa kluczowe

EN

Markov chain; Dynamic programming; Bayesian inference

PL

Łańcuch Markowa; Programowanie dynamiczne; Wnioskowanie bayesowskie

• Czasopismo: Control and Cybernetics
• Rocznik: 2013
• Tom: 42
• Numer: nr 3
• Strony: 593--612

Twórcy

autor

Masayuki Horiguchi

• Kanagawa University, Japan

autor

Alexey B. Piunovskiy

• University of Liverpool, United Kingdom

Bibliografia

• Bäuerle, N. and Rieder, U. (2011) Markov Decision Processes with Applications to Finance. Springer-Verlag, Berlin.
• Bertsekas, D.P. and Shreve, S.E. (1978) Stochastic Optimal Control. Academic Press, New York.
• Dufour, F. and Piunovskiy, A. (2010) Multiobjective stopping problem for discrete-time Markov processes: the convex analytic approach. Journal of Applied Probability 47: 947-966.
• Dynkin, E.B. and Yushkevich A.A. (1979) Controlled Markov Processes and their Applications. Springer-Verlag, New York - Berlin.
• Easley, D. and Kiefer, N.M. (1988) Controlling a stochastic process with unknown parameters. Econometrica 56: 1045-1064.
• Ekström, E. and Lu, B. (2011) Optimal selling of an asset under incomplete information. Int. J. Stoch. Anal. Art. ID 543590, 17 ,2090-3340.
• Ferguson, T.S. (1967) Mathematical Statistics. Academic Press, New York- London.
• González-Trejo, J.I., Hernández-Lerma, O. and Hoyos-Reyes, L.F. (2003) Minimax control of discrete-time stochastic systems. SIAM J.Control Optim 41: 1626-1659.
• DeGroot, M.H. (1970) Optimal Statistical Decisions. McGraw-Hill Book Co., New York.
• van Hee, K.M. (1978) Bayesian Control of Markov Chains. Mathematical Centre Tracts, No. 95. Mathematisch Centrum, Amsterdam.
• Hernández-Lerma, O. and Marcus, S.I. (1985) Adaptive control of discounted Markov decision chains. Journal of Optimization Theory and Applications 46: 227-235.
• Hernández-Lerma, O. (1989) Adaptive Markov Control Processes, volume 79 of Applied Mathematical Sciences. Springer-Verlag, New York.
• Hernández-Lerma, O. and Lasserre, J.B. (1996) Discrete-time Markov Control Processes. Springer, New York.
• Hordijk, A. (1974) Dynamic Programming and Markov Potential Theory. Mathematical Centre Tracts, No. 51. Mathematisch Centrum, Amsterdam.
• Horiguchi, M. (2001a) Markov decision processes with a stopping time constraint. Mathematical Methods of Operations Research 53: 279-295.
• Horiguchi, M. (2001b) Stopped Markov decision processes with multiple constraints. Mathematical Methods of Operations Research 54: 455-469.
• Kurano, M. (1972) Discrete-time Markovian decision processes with an unknown parameter. Average return criterion. Journal of the Operations Research Society of Japan 15: 67-76.
• Kurano, M. (1983) Adaptive policies in Markov decision processes with uncertain transition matrices. Journal of Information & Optimization Sciences 4: 21-40
• Mandl, P. (1974) Estimation and control in Markov chains. Advances in Applied Probability 6: 40-60.
• Martin, J.J. (1967) Bayesian Decision Problems and Markov Chains. Publications in Operations Research, No. 13. John Wiley & Sons Inc., New York.
• Piunovskiy, A. B. (2006) Dynamic programming in constrained Markov decision processes. Control and Cybernetics 35: 645-660.
• Puterman, M. (1994) Markov Decision Processes: Discrete Stochastic Dynamic Programming. Wiley, New York.
• Raiffa, H and Schlaifer, R. (1961) Applied Statistical Decision Theory. Studies in Managerial Economics. Division of Research, Graduate School of Business Administration, Harvard University, Boston, Mass.
• Rieder, U. (1975) Bayesian Dynamic Programming. Advances in Applied Probability 7: 330-348.
• Ross, S.M. (1970) Applied Probability Models with Optimization Applications. Holden-Day, San Francisco.
• Ross, S.M. (1983) Introduction to Stochastic Dynamic Programming. Academic Press, San Diego, CA.
• Stadje, W. (1997) An optimal stopping problem with two levels of incomlete information. Mathem. Methods of Oper. Research 45: 119-131.
• Wald, A. (1950) Statistical Decision Functions. John Wiley & Sons Inc., New York.
• Wang, X. and Yi, Y. (2009) An optimal investment and consumption model with stochastic returns. Applied Stoch. Models in Business and Industry 25: 45-55.
• White, D.J. (1969) Dynamic Programming. Mathematical Economic Texts, 1. Oliver&Boyd, Edinburgh-London.