The course aims at introducing a set of selected fundamental sequential decision problems under uncertainty and at providing tools towards exact or approximate solutions. The course is primarily devoted to stochastic models and more precisely to Markov Decision Processes, but also presents some recent advances in optimal decision making in adversarial scenarios and games. Numerous examples of applications are provided.

After completing this course, you will be able to rigorously formulate and classify sequential decision problems, to estimate their tractability, and to propose and efficiently implement methods towards their solutions.

Keywords. Dynamic programming, Markov Decision Process, Multi-armed bandit, Kalman filter, Online optimization.

Credits: 8 points (two home exams)