 |
|
||||||||||
Dynamic Programming and Optimal Control (Fall 2012)
Dynamic Programming and Optimal Control
4 credit points
Start: 26.09.12
End: 12.12.12
Frequency: Annually, fall semester
Lecturer
Assistants
Nico Hübel,
Mohanarajah Gajamohan
Day/Time/Location
Wednesdays
13:00-16:00, ML H 44
Book
Dynamic Programming and Optimal Control by Dimitri P. Bertsekas, Vol. I, 3rd edition, 2005, 558 pages, hardcover.
Description:
Dynamic Programming Algorithm; Deterministic Systems and Shortest Path Problems; Infinite Horizon Problems; Value/Policy Iteration; Deterministic Continuous-Time Optimal Control.
Book: Dynamic Programming and Optimal Control by Dimitri P. Bertsekas, Vol. I, 3rd edition, 2005, 558 pages, hardcover.
Requirements: Knowledge of advanced calculus, introductory probability theory, and linear algebra.
Announcements
| Feb 22 |
Final Grades:
|
|
Feb 22 |
Dear students, I would like to thank you for your feedback on the course. Based on your comments, I see the following possibilities for improvement (and will suggest it to the next lecturer at ETH):
I wish you all the best for the future! - Angela Schoellig |
| Jan 26 | The results of Programming Exercise 2 are online. |
| Jan 21 | The results of Quiz 2 are online. |
| Jan 19 | You are allowed to bring one A4 sheet of paper (2 pages, handwritten or computer typed) as a written aid (cheat sheet) for the final examination. |
| Dec 18 | Quiz 2 solutions are online. Click here to download. |
| Dec 18 | The slides and the examples of the last recitation class (12,Dec) are online. (slides, examples) |
| Dec 10 | Quiz 2 will take place in HG E7 (13:15-14:05, Wed, Dec 12). |
| Dec 07 |
Additional office hours in January, ML K 35: (1) Thu, January 10, 2013, 17:00 – 18:00 (2) Mon, January 14, 2013, 12:00 – 14:00 (3) Thu, January 17, 2013, 12:00 – 14:00 (4) Mon, January 21, 2013, 12:00 – 14:00 |
| Nov 27 | The results of Programming Exercise 1 are online. If you want to discuss your result, please send an email to Nico. |
|
Nov 21 |
Visit ETH EduApp and answer the two teaser questions on the Hamilton-Jacobi-Bellman (HJB) equation! |
| Nov 14 | Programming Exercise 2 is online. |
| Nov 08 |
The slides of Lecture 7 are found here. |
| Nov 06 | Quiz 1 results are out. If you are interested, you can take a look at your solutions/grading during the office hours; starting from November 15. |
| Oct 27 | Quiz 1 solutions are online. Click here to download. |
| Oct 16 | No books or cheat sheets will be allowed for the quiz. |
| Oct 16 | Notes of the second recitation class can be downloaded from here. |
| Oct 06 | The first quiz (Q1, Oct 24) will cover the Dynamic Programming algorithm including parts of the lecture and the recitation class on October 17. It will not cover Deterministic Systems and Shortest Path problems. |
| Oct 4 | The quizzes and the subsequent lecture (Oct 24, Dec 12) will take place in HG E7. |
| Oct 2 | The second recitation session on Oct 10, 15:15-16:00, is moved to Oct 16 (Tue), 18:00-18:45, due to the unavailability of both teaching assistants. The venue will remain the same (ML H 44). We apologize for the inconvenience. |
|
Aug 25 |
We have setup the ETH EduApp for the lecture. You can download the app for free or use the web interface to give us feedback. We are happy to receive suggestions, comments, etc. regarding the class. |
|
Jul 20 |
The recommended text book [Dynamic Programming and Optimal Control, Dimitri P. Bertsekas] will be for sale on Oct 03, 15:00 in front of the class room (Price: 126 Chf/ 119 Chf with student ID). The books can also be ordered from the publisher directly using the following link: http://www.athenasc.com/ordering.html. |
|
Jul 20 |
The first lecture will be held on September 26th. Exercise and Office hours start a week after. The class fact sheet can be downloaded from here and copies will be handed out during the first lecture. |
Class Facts
| Instructor | Angela Schoellig |
| Teaching Assistants | Nico Hübel, Mohanarajah Gajamohan |
| Lecture |
Wednesday, 13:15 to 15:00, ML H 44 |
| Exercise | Wednesday, 15:15 to 16:00, ML H 44 |
|
Office hours |
Thursday, 17:00 to 18:00, ML K 35 |
|
Feedback |
Please use the ETH EduApp for giving us feedback. |
|
Exam |
TBA (during examination session, covers all material) |
|
Grading |
40% quiz/programming exercises, 60% final exam if the grade for quiz and programming exercises is better than the grade in the final exam; 100% final exam otherwise. |
| Only the three best grades from the quizzes and the programming exercises will count towards the 40% above. | |
| PhD students will get credits for the class if they pass the class (final grade of 4.0 or higher). | |
|
Repetition |
The final exam is only offered in the session after the course unit. Repetition is only possible after re-enrolling. |
Class Schedule
| # | date | topic | reading |
|
L1 |
Sep 26 |
The Dynamic Programming Algorithm |
1.1 - 1.4 |
|
L2 |
Oct 03 |
The Dynamic Programming Algorithm |
1.1 - 1.4 |
| L3 |
Oct 10 |
The Dynamic Programming Algorithm |
1.1 - 1.4 |
| L4 |
Oct 17 |
Deterministic Systems and the Shortest Path Problem |
2.1 - 2.3 |
| Q1 + L5 |
Oct 24 |
Q1: The Dynamic Programming Algorithm L5: Deterministic Systems and the Shortest Path Problem |
1.1 - 1.4 (Q1) 2.1 - 2.3 (L5) |
| L6 |
Oct 31 |
Infinite Horizon Problems, Value Iteration, Policy Iteration |
7.1 - 7.3 |
| L7 |
Nov 07 |
Infinite Horizon Problems, Value Iteration, Policy Iteration |
7.1 - 7.3 |
|
L8 |
Nov 14 |
Infinite Horizon Problems, Value Iteration, Policy Iteration |
7.1 - 7.3 |
| L9 |
Nov 21 |
Deterministic Continuous-Time Optimal Control |
3.1 - 3.4 |
| L10 | Nov 28 | Deterministic Continuous-Time Optimal Control |
3.1 - 3.4 |
|
L11 |
Dec 05 | Deterministic Continuous-Time Optimal Control |
3.1 - 3.4 |
| Q2 + L12 | Dec 12 |
Quiz 2: Deterministic Continuous-Time Optimal Control L12: Final recitation class |
3.1 - 3.4 (Q2) |
Quizzes and Programming Exercises
During the semester, there will be graded quizzes and programming exercises, which can be used to improve the final grade for the course (see "grading"). Quizzes will take place at the beginning of the lecture and will test the student's understanding of the corresponding topic.
The programming exercises will require the student to apply the lecture material.
Up to three students can work together on the programming exercises. If they do, they have to hand in one solution per group and will all receive the same grade.
| # | type | topic | dates | download |
| Q1 | Quiz | The Dynamic Programming Algorithm | Oct 24 |
Solutions results |
| P1 | Programming | Deterministic Systems and the Shortest Path Problem |
Oct 24 (issued) Nov 07 (due) |
Programming Exercise 1 Programming Exercise 1 template |
| P2 | Programming | Value/Policy Iteration |
Nov 14 (issued) Nov 28 (due) |
Programming Exercise 2 FAQ errata results |
| Q2 | Quiz | Deterministic Continuous-Time Optimal Control |
Dec 12 |
Solutions Results |
Past Quizzes and Exams
| type | download |
| Quiz 2011 (The Dynamic Programming Algorithm) | Quiz & Solutions |
| Quiz 2011 (Deterministic Continuous-Time Optimal Control) | Quiz & Solutions |
| Final Exam 2011 | Final 2011 & Solutions |
| Quiz 2010 (The Dynamic Programming Algorithm) | Quiz & Solutions |
| Quiz 2010 (Deterministic Continuous-Time Optimal Control) | Quiz & Solutions |
| Final Exam 2010 | Final 2010 & Solutions |
| Quiz 2009 (The Dynamic Programming Algorithm) | Quiz & Solutions |
| Quiz 2009 (Deterministic Continuous-Time Optimal Control) | Quiz & Solutions |
| Final Exam 2009 | Final 2009 & Solutions |
| Midterm Exam 2008 | Midterm 2008 & Solutions |
| Final Exam 2008 | Final 2008 & Solutions |
Problem Sets
We will make sets of problems and solutions available online for the chapters covered in the lecture. It is the student's responsibility to solve the problems and understand their solutions. The TAs will answer questions in office hours and some of the problems might be covered during the exercises.
The problem sets are password protected. The password will be announced in class or can be obtained from the teaching assistants.
| # |
topic |
download |
|
1 |
The Dynamic Programming Algorithm | ProblemSet1 |
| 2 | Deterministic Systems and the Shortest Path Problem | ProblemSet2 |
| 3 | Infinite Horizon Problems, Value Iteration, Policy Iteration |
ProblemSet3 script_P73c.m |
| 4 | Deterministic Continuous-Time Optimal Control | ProblemSet4 |
Additional Reading
- Tutorial on viscosity solutions to the HJB equation.
- Additional material for those interested in the history of the Maximum Principle from a mathematical perspective.
- Optional Reading: http://spectrum.ieee.org/geek-life/profiles/2010-medal-of-honor-winner-andrew-j-viterbi (Some good insights into the development of the Viterbi algorithm).
- A good read on continuos time optimal control. Optimization-Based Control: Chapter 2 [A different view with Lagrange multipliers]
Wichtiger Hinweis:
Diese Website wird in älteren Versionen von Netscape ohne
graphische Elemente dargestellt. Die Funktionalität der
Website ist aber trotzdem gewährleistet. Wenn Sie diese
Website regelmässig benutzen, empfehlen wir Ihnen, auf
Ihrem Computer einen aktuellen Browser zu installieren. Weitere
Informationen finden Sie auf
folgender
Seite.
Important Note:
The content in this site is accessible to any browser or
Internet device, however, some graphics will display correctly
only in the newer versions of Netscape. To get the most out of
our site we suggest you upgrade to a newer browser.
More
information
