SF2852/FSF3852 Optimal Control,   2023, 7.5hp.

Course registration: If you have problems registering for the course or for exams, please contact the Student affairs office , e.g., via email: studentoffice@math.kth.se.

Some material from the lectures can be found on this page

Examiner and lecturer:
Johan Karlsson, email: johan.karlsson@math.kth.se,
room 3550, Lindstedtsv 25, phone: 790 8440

Tutorial exercises:
Michele Mascherpa, micmas@kth.se,
room 3415, Lindstedtsv 25.

Office hours:
Mondays 13.00-14.00 (4/9-23/10).
Room 3415, Lindstedtsvägen 25, or zoom: https://kth-se.zoom.us/j/67993457534. (Ask in the zoom to be let in.)

Optimal control is the problem of determining the control function for a dynamical system to minimize a performance index. The subject has its roots in the calculus of variations but it evolved to an independent branch of applied mathematics and engineering in the 1950s. The rapid development of the subject during this period was due to two factors. The first are two key innovations, namely the maximum principle by L. S. Pontryagin and the dynamic programming principle by R. Bellman. The second was the space race and the introduction of the digital computer, which led to the development of numerical algorithms for the solution of optimal control problems. The field of optimal control is still very active and it continues to find new applications in diverse areas such as robotics, finance, economics, and biology.

Course goals
The goal of the course is to provide an understanding of the main results in optimal control and how they are used in various applications in engineering, economics, logistics, and biology. After the course you should be able to

For the highest grade you should be able to integrate the tools you have learnt during the course and apply them to more complex problems. In particular you should be able to

Course topics
  • Dynamic Programming Discrete dynamic programming, principle of optimality, Hamilton-Jacobi-Bellman equation, verification theorem.
  • Pontryagin minimum principle Several versions of Pontryagin Minimum Principle (PMP) will be discussed.
  • Infinite Horizon Optimal Control Optimal control over an infinite time horizon, stability, LQ optimal control.
  • Model Predictive Control Explicit and implicit model predictive control.
  • Applications Examples from economics, logistics, aeronautics, and robotics will be discussed.
  • Computational Algorithms The most common methods for numerical solution of optimal control problems are presented.

    Course material
    The required course material consists of the following lecture and exercise notes on sale at Kårbokhandeln. [Lecture notes], [Exercise notes].

    The student is required to have passed the course optimization SF1841/SF1811 or a course with similar content. The student should hence be familiar with concepts and theory for optimization: linear, quadratic, and nonlinear optimization; optimality conditions, lagrangian relaxation and duality theory. Familiarity with systems theory and state space is not required but recommended.

    Course requirements
    The course requirements consista of three mandatory homework sets and a final written examination. The homework sets may also give you bonus credits in the examination.

    PhD course SF3852
    It is possible to read this course as a PhD level course. For this, an extra project and at least a B on the exam is required. Email the examiner to get details regarding the project.

    Homework sets
    Homework set 0: This homework set provides some review of systems theory and optimization as well as a Matlab exercise that use the toolbox CVX. I recommend that everyone does problem 2. Homework set 0 is optional and does not give bonus points to the exam, however, you can get feedback on your solutions if you hand it in before the deadline.

    Each of the homework sets 1-3 are mandatory and consists of three-five problems. The first two-three problems are methodology problems where you practice on the topics of the course and apply them to examples. Among the last two problems, one will focus on more theoretical nature and helps you to understand the mathematics behind the course. It can, for example, be to derive an extension of a result in the course or to provide an alternative proof of a result in the course. The other will focus on implementation and the student is required to make a Matlab program that solve a problem numerically.

    You are required to get at least 10 points on each of the homeworks 1 and 3, and do the project in homework 2. Each successfully completed homework set handed in on time also gives you maximally 2 bonus points for the exam. The bonus is only valid during the year it is acquired. The exact requirements will be posted on each separate homework set. The homework sets will be posted on the homepage roughly two weeks before the deadline. The solutions to the homeworks should be uploaded on Canvas. Please prepare the solutions as a pdf in LaTeX or comparable software.

    Matlab code

    Here are some Matlab routines that are used in the excerise notes. You may use this for the solution of your homeworks.

    Written exam
    You may use Beta Mathematics Handbook and the following formula sheet (pdf) . The exam will consist of five problems that give maximally 50 points. These problems will be similar to those in the homework assignments and the tutorial exercises. The preliminary grade levels are distributed according to the following rule, where the total score is the sum of your exam score and maximally 6 bonus points from the homework assignments (max credit is 56 points). These grade limits can only be modified to your advantage.

    Total credit (points) Grade
    45-56 A
    39-44 B
    33-38 C
    28-32 D
    25-27 E
    24 FX
    The grade FX means that you are allowed to make an extra assignment, see below.

    The grade FX
    If your total score (exam score + maximum 6 bonus points from the homework assignments and the computational exercises) is 24 points then you are allowed to do an extra assignment. The solutions should be handed in to the examiner in written form and you must be able to defend your solutions in an oral examination. Contact the examiner no later than three weeks after the final exam if you want to do this.

    Course evaluation
    At the end of the course you will be asked to complete a course evaluation form online.

    Preliminary schedule for 2023
    Type Date Time Room Topic Content (Preliminary)
    L1 2023-08-28 10:15 D41 Introduction
    Discrete dynamic programming
    Pages 17-23
    L2 2023-08-29 10:15 E51 Discrete dynamic programming
    Discrete PMP
    Pages 22-24
    L3 2023-08-30 13:15 D32 Discrete dynamic programming
    Infinite time horizon
    Pages 24-26
    E1 2023-08-31 13:15 E52 Discrete dynamic programming
    Linear systems

    L4 2023-09-04 10:15 M33 Model predictive control
    E2 2023-09-05 15:15 E51 Model predictive control

    L5 2023-09-06 10:15 Q22 Dynamic programming
    Pages 35-39
    E3 2023-09-07 08:15 E35 Dynamic programming

    L6 2023-09-11 10:15 E35 Dynamic programming
    and review
    Pages 5-7, 39-44
    L7 2023-09-12 15:15 D34 Mathematical preliminaries
    (ODE theory etc)
    Pages 47-54
    L8 2023-09-13 13:15 M33 Pontryagins minimum principle
    (PMP) (using small variations)
    Pages 59-62 and a basic example
    E4 2023-09-15 13:15 D34 PMP I

    L9 2023-09-18 10:15 E51 PMP (control constraints)
    Examples on pages 62-63 and 74-75
    L10 2023-09-19 13:15 E51 PMP (optimal control
    to a manifold)
    Pages 71-81
    L11 2023-09-22 10:15 W37 PMP (generalizations)
    Pages 81-88
    E5 2023-09-25 10:15 D41 PMP II:
    Time optimal control

    L12 2023-09-27 10:15 Q17 Numerical Methods
    Pages 121-130
    E6 2023-09-28 13:15 Q26 PMP III

    L13 2023-10-02 10:15 E52 PMP applications
    Pages 90-96
    E7 2023-10-03 15:15 E35 PMP IV

    L14 2023-10-05 13:15 E35 Computational methods Seminar
    (student presentation)

    L15 2023-10-11 08:15 E51 Topics: Infinite time horizon
    optimal control
    Pages 97-109
    L16 2023-10-12 13:15 E51 Review

    E8 2023-10-13 13:15 D34 Infinite time horizon optimal control and
    Review: old exams

    Exam 2023-10-25 08:00 Exam

    Some of last years exams can be found here:
    exam and solutions 20221027.

    exam and solutions 20211028.

    exam and solutions 20201022.
    exam and solutions 20201216.
    exam and solutions
    exam and solutions
    exam and solutions
    exam and solutions
    exam and solutions
    exam and solutions
    exam and solutions