
^{4 credit points}
_{Start: }_{26.09.12}
_{End: }_{12.12.12}
_{Frequency: Annually, fall semester}
Nico Hübel,
Mohanarajah Gajamohan
Wednesdays
13:0016:00, ML H 44
Dynamic Programming and Optimal Control by Dimitri P. Bertsekas, Vol. I, 3rd edition, 2005, 558 pages, hardcover.
Description:
Dynamic Programming Algorithm; Deterministic Systems and Shortest Path Problems; Infinite Horizon Problems; Value/Policy Iteration; Deterministic ContinuousTime Optimal Control.
Book: Dynamic Programming and Optimal Control by Dimitri P. Bertsekas, Vol. I, 3rd edition, 2005, 558 pages, hardcover.
Requirements: Knowledge of advanced calculus, introductory probability theory, and linear algebra.
Feb 22 
Final Grades:

Feb 22 
Dear students, I would like to thank you for your feedback on the course. Based on your comments, I see the following possibilities for improvement (and will suggest it to the next lecturer at ETH):
I wish you all the best for the future!  Angela Schoellig 
Jan 26  The results of Programming Exercise 2 are online. 
Jan 21  The results of Quiz 2 are online. 
Jan 19  You are allowed to bring one A4 sheet of paper (2 pages, handwritten or computer typed) as a written aid (cheat sheet) for the final examination. 
Dec 18  Quiz 2 solutions are online. Click here to download. 
Dec 18  The slides and the examples of the last recitation class (12,Dec) are online. (slides, examples) 
Dec 10  Quiz 2 will take place in HG E7 (13:1514:05, Wed, Dec 12). 
Dec 07 
Additional office hours in January, ML K 35: (1) Thu, January 10, 2013, 17:00 – 18:00 (2) Mon, January 14, 2013, 12:00 – 14:00 (3) Thu, January 17, 2013, 12:00 – 14:00 (4) Mon, January 21, 2013, 12:00 – 14:00 
Nov 27  The results of Programming Exercise 1 are online. If you want to discuss your result, please send an email to Nico. 
Nov 21 
Visit ETH EduApp and answer the two teaser questions on the HamiltonJacobiBellman (HJB) equation! 
Nov 14  Programming Exercise 2 is online. 
Nov 08 
The slides of Lecture 7 are found here. 
Nov 06  Quiz 1 results are out. If you are interested, you can take a look at your solutions/grading during the office hours; starting from November 15. 
Oct 27  Quiz 1 solutions are online. Click here to download. 
Oct 16  No books or cheat sheets will be allowed for the quiz. 
Oct 16  Notes of the second recitation class can be downloaded from here. 
Oct 06  The first quiz (Q1, Oct 24) will cover the Dynamic Programming algorithm including parts of the lecture and the recitation class on October 17. It will not cover Deterministic Systems and Shortest Path problems. 
Oct 4  The quizzes and the subsequent lecture (Oct 24, Dec 12) will take place in HG E7. 
Oct 2  The second recitation session on Oct 10, 15:1516:00, is moved to Oct 16 (Tue), 18:0018:45, due to the unavailability of both teaching assistants. The venue will remain the same (ML H 44). We apologize for the inconvenience. 
Aug 25 
We have setup the ETH EduApp for the lecture. You can download the app for free or use the web interface to give us feedback. We are happy to receive suggestions, comments, etc. regarding the class. 
Jul 20 
The recommended text book [Dynamic Programming and Optimal Control, Dimitri P. Bertsekas] will be for sale on Oct 03, 15:00 in front of the class room (Price: 126 Chf/ 119 Chf with student ID). The books can also be ordered from the publisher directly using the following link: http://www.athenasc.com/ordering.html. 
Jul 20 
The first lecture will be held on September 26th. Exercise and Office hours start a week after. The class fact sheet can be downloaded from here and copies will be handed out during the first lecture. 
Instructor  Angela Schoellig 
Teaching Assistants  Nico Hübel, Mohanarajah Gajamohan 
Lecture 
Wednesday, 13:15 to 15:00, ML H 44 
Exercise  Wednesday, 15:15 to 16:00, ML H 44 
Office hours 
Thursday, 17:00 to 18:00, ML K 35 
Feedback 
Please use the ETH EduApp for giving us feedback. 
Exam 
TBA (during examination session, covers all material) 
Grading 
40% quiz/programming exercises, 60% final exam if the grade for quiz and programming exercises is better than the grade in the final exam; 100% final exam otherwise. 
Only the three best grades from the quizzes and the programming exercises will count towards the 40% above.  
PhD students will get credits for the class if they pass the class (final grade of 4.0 or higher).  
Repetition 
The final exam is only offered in the session after the course unit. Repetition is only possible after reenrolling. 
#  date  topic  reading 
L1 
Sep 26 
The Dynamic Programming Algorithm 
1.1  1.4 
L2 
Oct 03 
The Dynamic Programming Algorithm 
1.1  1.4 
L3 
Oct 10 
The Dynamic Programming Algorithm 
1.1  1.4 
L4 
Oct 17 
Deterministic Systems and the Shortest Path Problem 
2.1  2.3 
Q1 + L5 
Oct 24 
Q1: The Dynamic Programming Algorithm L5: Deterministic Systems and the Shortest Path Problem 
1.1  1.4 (Q1) 2.1  2.3 (L5) 
L6 
Oct 31 
Infinite Horizon Problems, Value Iteration, Policy Iteration 
7.1  7.3 
L7 
Nov 07 
Infinite Horizon Problems, Value Iteration, Policy Iteration 
7.1  7.3 
L8 
Nov 14 
Infinite Horizon Problems, Value Iteration, Policy Iteration 
7.1  7.3 
L9 
Nov 21 
Deterministic ContinuousTime Optimal Control 
3.1  3.4 
L10  Nov 28  Deterministic ContinuousTime Optimal Control 
3.1  3.4 
L11 
Dec 05  Deterministic ContinuousTime Optimal Control 
3.1  3.4 
Q2 + L12  Dec 12 
Quiz 2: Deterministic ContinuousTime Optimal Control L12: Final recitation class 
3.1  3.4 (Q2) 
During the semester, there will be graded quizzes and programming exercises, which can be used to improve the final grade for the course (see "grading"). Quizzes will take place at the beginning of the lecture and will test the student's understanding of the corresponding topic.
The programming exercises will require the student to apply the lecture material.
Up to three students can work together on the programming exercises. If they do, they have to hand in one solution per group and will all receive the same grade.
#  type  topic  dates  download 
Q1  Quiz  The Dynamic Programming Algorithm  Oct 24 
Solutions results 
P1  Programming  Deterministic Systems and the Shortest Path Problem 
Oct 24 (issued) Nov 07 (due) 
Programming Exercise 1 Programming Exercise 1 template 
P2  Programming  Value/Policy Iteration 
Nov 14 (issued) Nov 28 (due) 
Programming Exercise 2 FAQ errata results 
Q2  Quiz  Deterministic ContinuousTime Optimal Control 
Dec 12 
Solutions Results 
Past Quizzes and Exams
type  download 
Quiz 2011 (The Dynamic Programming Algorithm)  Quiz & Solutions 
Quiz 2011 (Deterministic ContinuousTime Optimal Control)  Quiz & Solutions 
Final Exam 2011  Final 2011 & Solutions 
Quiz 2010 (The Dynamic Programming Algorithm)  Quiz & Solutions 
Quiz 2010 (Deterministic ContinuousTime Optimal Control)  Quiz & Solutions 
Final Exam 2010  Final 2010 & Solutions 
Quiz 2009 (The Dynamic Programming Algorithm)  Quiz & Solutions 
Quiz 2009 (Deterministic ContinuousTime Optimal Control)  Quiz & Solutions 
Final Exam 2009  Final 2009 & Solutions 
Midterm Exam 2008  Midterm 2008 & Solutions 
Final Exam 2008  Final 2008 & Solutions 
We will make sets of problems and solutions available online for the chapters covered in the lecture. It is the student's responsibility to solve the problems and understand their solutions. The TAs will answer questions in office hours and some of the problems might be covered during the exercises.
The problem sets are password protected. The password will be announced in class or can be obtained from the teaching assistants.
# 
topic 
download 
1 
The Dynamic Programming Algorithm  ProblemSet1 
2  Deterministic Systems and the Shortest Path Problem  ProblemSet2 
3  Infinite Horizon Problems, Value Iteration, Policy Iteration 
ProblemSet3 script_P73c.m 
4  Deterministic ContinuousTime Optimal Control  ProblemSet4 
Wichtiger Hinweis:
Diese Website wird in älteren Versionen von Netscape ohne
graphische Elemente dargestellt. Die Funktionalität der
Website ist aber trotzdem gewährleistet. Wenn Sie diese
Website regelmässig benutzen, empfehlen wir Ihnen, auf
Ihrem Computer einen aktuellen Browser zu installieren. Weitere
Informationen finden Sie auf
folgender
Seite.
Important Note:
The content in this site is accessible to any browser or
Internet device, however, some graphics will display correctly
only in the newer versions of Netscape. To get the most out of
our site we suggest you upgrade to a newer browser.
More
information