printlogo
http://www.ethz.ch/index_EN
Institute for Dynamic Systems and Control
  
News | About Us | People | Partnerships
Contact | Sitemap | Help
Research D'Andrea | Research Guzzella | Lectures | Theses/Projects
Positions | Choose Advisor/Tutor | Calendar | Intranet | Downloads
print
  
ETH Zurich - DMAVT - IDSC - Lectures Overview - Archives - Optimal Control Archive - Optimal Control 
Lectures Overview
Control Lab (Messlabor)
Control Systems I
Control Systems II
Digital Control Systems
Embedded Control Systems
Engine Systems
Engin. Tools II: Matlab
Optimal Control
Recursive Estimation
Signals and Systems
Stochastic Systems
System Modeling
Vehicle Propulsion Systems
Optimal Control
Optimal Control
Fall 08
Optimal Control

Dynamic Programming and Optimal Control (Fall 2009)

Dynamic Programming and Optimal Control

151-0563-01

4 credit points

Start: September 2009
End: December 2009

Frequency:  Annually, fall semester

 

Lecturer

Raffaello D'Andrea

 

Assistants

Angela Schoellig, Sebastian Trimpe

 

Day/Time/Location

Wednesdays

13:00-16:00, ML H 44

 

Book

Dynamic Programming and Optimal Control by Dimitri P. Bertsekas, Vol. I, 3rd edition, 2005, 558 pages, hardcover.

Description:

Dynamic Programming Algorithm; Deterministic Systems and Shortest Path Problems; Infinite Horizon Problems; Value/Policy Iteration; Deterministic Continuous-Time Optimal Control; Problems with Perfect State Information.

Book:  Dynamic Programming and Optimal Control by Dimitri P. Bertsekas, Vol. I, 3rd edition, 2005, 558 pages, hardcover.

Requirements:  Knowledge of advanced calculus, introductory probability theory, and matrix-vector algebra.

Announcements

Feb 18
 		Final Grades:
  • The results of the final exam and your overall grade are posted here.
  • We have reported your final grades to the student administration, so you should find them soon on your mystudies account.
  • You may look at your exam on Feb 23 from 12:30 to 14:30 in ML K33.
  • An exemplary solution of the final exam can be downloaded here.
Jan 26
 Summary of information regarding the final examination:
  • Date: Jan 28, 2010; start 9:00 (plan on being there early)
  • Place: ETA F 5
  • Permitted aids: one A4 sheet of paper (double sided)
Jan 15
 For the final examination, all material that was covered in the lecture or the exercise class is relevant.
Jan 13
 During the semester break, we offer office hours on Jan 19 (Tue), Jan 25 (Mon) and Jan 26 (Tue) at the usual time and location (see section 'Office Hours' below).
Dec 17
 		Final examination
The final exam takes place on Jan 28, 2009 from 09:00-11:30. You can bring one A4 sheet of paper (double sided). No calculators or other aids are allowed.

Last exercise class (Dec 16)
You can find the Matlab code for the inverted pendulum example and other relevant information of the last exercise class here.

Programming Exercise 2
The results of programming exercise 2 have been updated.  Resubmissions of the code were taken into account.
Dec 14 This year's last office hour is on Tuesday, Dec 15. We will offer office hours again in the week before the exam. We will publish more details on time and location soon.
Dec 14 The results and solutions of Quiz 2 are available for download.  See section Quizzes and Programming Exercises.  You may take a look at the correction of your quiz during office hours.
Dec 11
 The Problem Set Problems with Perfect State Information is online.
Dec 09
 The results of Programming exercise 2 are online (see Section Quizzes and Programming Exercises).  

The students whose initially submitted code was not correct have the option of resubmitting their solution until Dec 16.  Please submit by e-mail to aschoellig(at)ethz.ch.

Dec 03
 Information regarding the second quiz taking place on Dec 09:
  • Start: 13:15 sharp, duration: 45 min
  • Afterwards two hours of lecture (no exercise class)
  • Location: the usual class room (ML H 44)
  • Material covered: Deterministic Continuous-Time Optimal Control
  • No aids (books, notes etc.) permitted
  • Paper will be provided
Nov 26
 The results of programming exercise 1 have been updated.  Resubmissions of the code were taken into account.
Nov 18
 The Problem Set Deterministic Continuous-time Optimal Control is online.
Nov 18
 The results of Programming exercise 1 are online (see Section Quizzes and Programming Exercises).  

The students whose initially submitted code was not correct have the option of resubmitting their solution until Nov 25.  Please submit by e-mail to strimpe(at)ethz.ch.

Nov 18
 Guideline for Programming Exercise 2: The value iteration algorithm can be terminated if |Jk(i)-J*(i)|<0.1.
Nov 16
 Note: In Programming Exercise 2, script.m, it should read: [valueIterationJ, valueIterationF] = ssp_ValueIt(P, policyIterationJ). A corrected version is online.
Nov 16 Slides on the Stochastic Shortest Path Problem are found here.
Nov 13
 		The Problem Set Infinite Horizon Problems is online.
Nov 03
 The Problem Set Infinite Horizon Problems will include the problems 7.1, 7.3, 7.12, and 1.23 from the class textbook.  The solutions will be available next week.
Oct 30
 The results and solutions of Quiz 1 are available for download.  See section Quizzes and Programming Exercises.  You may take a look at the correction of your quiz during office hours.
Oct 21
 The programming exercise 1 is online.  It is due on Nov 04.
Oct 15
 An updated version of the Problem Set 1 has been uploaded. 

Please keep reporting errors that you may find in the problem sets to the TAs.  We will keep a list of all corrections of previous versions of the problem sets, which you can download at the end of the problem sets section or here.
Oct 13
 There is a typo in the fact sheet that we distributed in the first lecture.  Instead of "Imperfect State Information" in lecture 12 and 13 and the corresponding problem set, it should say "Perfect State Information."  The updated sheet can be downloaded here.
Oct 08
 Information regarding the quiz taking place on Oct 14:
  • Start: 13:15 sharp, duration: 45 min
  • Afterwards two hours of lecture (no exercise class)
  • Location: the usual class room (ML H 44)
  • Material covered: The Dynamic Programming Algorithm
  • No aids (books, notes etc.) permitted
  • Paper will be provided
Oct 07 The slide shown in the exercise class which summarizes the Dynamic Programming Algorithm is found here.
Sep 24
 We got a new class room:  from now on, the lectures and exercises will take place in ML H44.
Sep 23
 The class textbook is being sold by the Polybuchhandlung during the break of the next lecture (Sep 30, 14:00).  It will cost 159 CHF (139 CHF for Master students) and has to be paid in cash.
You can also order the book through the publisher (Athena Scientific), or any other store you like.
Sep 22
 The first lecture takes place on Sep 23.  Exercises and office hours start the week after.

The fact sheet that will be handed out in the first lecture can be downloaded here.
Sep 22
 The website has been updated (office hours, quizzes/programming exercises, problem set 1 uploaded).
Aug 19 The class website has been updated with information on grading, quizzes/programming exercises, and problem sets.
Jul 22
 There will be more information about the class Dynamic Programming and Optimal Control (Fall 09) on this page soon.

Last year's class website (Fall 08) can be found here.

Class Facts

Instructor Prof. Raffaello D'Andrea
Teaching Assistants Angela Schoellig, Sebastian Trimpe
Lecture Wednesday, 13:15 to 15:00, ML H44
Exercise Wednesday, 15:15 to 16:00, ML H44
Office hours
 Monday, 18:30 to 19:30, ML K37.1 (IDSC library), Angela

Tuesday, 16:00 to 17:00, ML K37.1 (IDSC library), Sebastian

During the semester: every Monday and Tuesday;

After the semester: Jan 19, Jan 25, Jan 26.
Exam
 Final written exam during the examination session, covers all material
Grading
 40% quizzes/programming exercises, 60% final exam if the grade for quizzes and programming exercises is better than the grade in the final exam;
100% final exam otherwise.
  PhD students will get credits for the class if they pass the class (final grade of 4.0 or higher).
Repetition
 The final exam is only offered in the session after the course unit. Repetition is only possible after re-enrolling. Students who took the class in Fall 08 and have to retake the course have the option of keeping their old homework grades. They have to inform the teaching assistants before the beginning of the new class.

Lectures

# date topic reading
 1
 Sep 23
 The Dynamic Programming Algorithm
 1.1 - 1.4
 2
 Sep 30
 The Dynamic Programming Algorithm
 1.1 - 1.4
 3 Oct 07
 The Dynamic Programming Algorithm
 1.1 - 1.4
 4 Oct 14
 Deterministic Systems and the Shortest Path Problem
 2.1 - 2.3
 5 Oct 21
 Deterministic Systems and the Shortest Path Problem 2.1 - 2.3
 6 Oct 28
Infinite Horizon Problems, Value Iteration, Policy Iteration
 7.1 - 7.3
 7 Nov 04
 Infinite Horizon Problems, Value Iteration, Policy Iteration 7.1 - 7.3
 8
 Nov 11
 Infinite Horizon Problems, Value Iteration, Policy Iteration 7.1 - 7.3
 9 Nov 18
 Deterministic Continuous-Time Optimal Control
 3.1 - 3.4
 10 Nov 25
Deterministic Continuous-Time Optimal Control 3.1 - 3.4
 11
 Dec 02
Deterministic Continuous-Time Optimal Control 3.1 - 3.4
 12 Dec 09
Problems with Perfect State Information: Linear Systems and Quadratic Cost
 4.1
 13
 Dec 16
 Problems with Perfect State Information: Linear Systems and Quadratic Cost 4.1

Quizzes and Programming Exercises

During the semester, there will be graded quizzes and programming exercises, which can be used to improve the final grade for the course (see "grading"). Quizzes will take place at the beginning of the lecture and will test the student's understanding of the corresponding topic. The programming exercises will require the student to apply the lecture material.

Up to three students can work together on the programming exercises.  If they do, they have to hand in one solution per group and will all receive the same grade.

# type topic dates download
Q1 Quiz The Dynamic Programming Algorithm Oct 14
 Results
Solutions
P1 Programming Deterministic Systems and the Shortest Path Problem Oct 21 (issued)
Nov 04 (due)
 Exercise
MatlabTemplate

Results
P2 Programming Value/Policy Iteration Nov 11 (issued)
Nov 25 (due)
 Exercise
MatlabTemplate

Results
Q2 Quiz Deterministic Continuous-Time Optimal Control Dec 09
 Results
Solutions

Problem Sets

We will make sets of problems and solutions available online for the chapters covered in the lecture. It is the student's responsibility to solve the problems and understand their solutions. The TAs will answer questions in office hours and some of the problems might be covered during the exercises.

# topic
download
1
 The Dynamic Programming Algorithm ProblemSet1.pdf
2 Deterministic Systems and the Shortest Path Problem ProblemSet2.pdf
3 Infinite Horizon Problems, Value Iteration, Policy Iteration ProblemSet3.pdf
4 Deterministic Continuous-Time Optimal Control ProblemSet4.pdf
5 Problems with Perfect State Information: Linear Systems and Quadratic Cost ProblemSet5.pdf

List of corrected errors in previous versions of the problem sets.

 

Wichtiger Hinweis:
Diese Website wird in älteren Versionen von Netscape ohne graphische Elemente dargestellt. Die Funktionalität der Website ist aber trotzdem gewährleistet. Wenn Sie diese Website regelmässig benutzen, empfehlen wir Ihnen, auf Ihrem Computer einen aktuellen Browser zu installieren. Weitere Informationen finden Sie auf
folgender Seite.

Important Note:
The content in this site is accessible to any browser or Internet device, however, some graphics will display correctly only in the newer versions of Netscape. To get the most out of our site we suggest you upgrade to a newer browser.
More information

© 2013 ETH Zurich | Imprint | Disclaimer | 24 August 2013
top