【Time】10:30am-11:30am, March 18, 2014 (Tuesday)

【Venue】Room N510, Shunde Building 

【Title】 Simulation-based optimization of Markov decision processes (MDPs) and risk-aware MDPs 

【Speaker】Dr. William B. Haskell, University of Southern California 

【Host】Dr.Max Shen

【Abstract】This talk presents results from two arms of my research: Simulation-based optimization of Markov decision processes (MDPs) and risk-aware MDPs.  First, I present a new algorithm for simulation-based value and policy iteration in MDPs.  This algorithm, and its novel convergence proof technique, offers finite sample probabilistic convergence guarantees.   Second, I show that a wide class of risk-aware MDPs can be modeled and solved with the convex analytic approach: writing the risk-aware MDP as a static optimization problem in a space of occupation measures.  I emphasize some risk-aware MDPs based on expected utility theory that yield linear programming problems in this fashion.  The talk concludes by discussing how these two arms can be combined for online data-driven risk management.

【Bio】Will Haskell earned his B.S. Mathematics and M.S. Econometrics from the University of Massachusetts Amherst, and his M.A. Mathematics and Ph.D. Operations Research degrees from the University of California Berkeley.  His Ph.D. advisors are Z. Max Shen and J. George Shanthikumar.  He is currently a postdoctoral researcher at the University of Southern California, working with Rahul Jain and Milind Tambe.  Will’s research emphasizes risk-awareness and robustness in single-agent, multi-agent, and dynamic decision making.