Mathematics of Reinforcement Learning

What it is about

You will learn the mathematical foundations of reinforcement learning: Markov decision processes (MDPs), tabular RL methods such as Monte Carlo, Temporal Difference, SARSA and Q-Learning, as well as basics of stochastic approximation to analyze the convergence of these algorithms. In the end you can formulate dynamic decision problems under uncertainty as MDPs and solve them with tabular RL algorithms.

What you will be able to do

• Formulate MDPs
• Apply tabular reinforcement learning algorithms (Monte Carlo, TD, SARSA, Q-Learning)
• Understand basics of stochastic approximation
• Perform convergence analyses of the algorithms

What the module consists of

• LectureConveying the theoretical foundations and concepts
• TutorialWorking on theoretical and programming-oriented exercises for deepening; partly in team work

Teaching method

• LectureIntroduction and explanation of the mathematical foundations
• Tutorial sessionsDeveloping problem-solving under guidance for deepening and practice
• Programming assignmentsApplying the algorithms and understanding through implementation