Markov Decision Processes with Constraints
Edwin Chong, Dept. of Electrical & Computer Engineering and Dept. of Mathematics, Colorado State University
Monday, August 30, 2010
4:00 p.m., room 223, Weber Bldg
We consider Markov decision processes subject to expected long-term average constraints, with a particular focus on the multichain case. Motivated by observations of Haviv, we derive a form of Bellman's optimality principle for such multichain processes. We then respond to Haviv's comments by illustrating some key ideas using examples, including the following. Suppose you set a goal of saving for retirement with some specific constraint on how much you will need for a comfortable post-career life. If you win the lottery, how should your subsequent spending and saving decisions change?