Course Material

For Undergraduate Level Control Course

• Preliminaries
        
  
  




• Practice Problems
  
         Problem Set 01: Modeling and linearization
  
  

For Graduate Level Optimal Control Course

• In-class Quizzes
  
         Quiz 0: Assessment for mathematical preparation
  
         Quiz 1: Background on optimization, convex/nonconvex sets and functions, positive (semi-)definite matrices
  
         Quiz 2: Background on linear program, semidefinite prgoram, linear time invariant control systems
  
         Quiz 3: Caclulus-of-variations problem formulation
  
         Quiz 4: Solution of calculus-of-variations problem
  
         Quiz 5: Solving Euler-Lagrange equation
  
         Quiz 6: Newtonian mechanics as special case of Euler-Lagrange equation
  
         Quiz 7: Using Euler-Lagrange equation to derive equations of motion, introduction to optimal control
  
         Quiz 8: Optimal control problem formulation, necessary conditions for optimality
  
         Quiz 9: Solving optimal control problem using PMP
  
         Quiz 10: Geometry of PMP, regulation vs. tracking, LQR formulation
  
         Quiz 11: Continuous-time finite horizon LQR solution
  
         Quiz 12: More on continuous-time LQR, discrete-time optimal control and LQR
  
         Quiz 13: Infinite horizon LQR
  
         Quiz 14: Additional constraints in continuous-time optimal control
  
         Quiz 15: Singular optimal control, bang-bang control
  
         Quiz 16: Ruling out singularity in nonlinear optimal control problems
  
         Quiz 17: Introduction to dynamic programming
  
         Quiz 18: Discrete-time finite horizon DP
  
         Quiz 19: Discrete-time infinite horizon DP
  
         Quiz 20: Bellman Operator in discrete-time infinite horizon DP
  
         Quiz 21: Continuous-time finite horizon DP
  
         Quiz 22: Solving HJB PDE
  
         Quiz 23: Stochastic DP in continuous-time
  
         Quiz 24: Second order HJB PDE IVP
  
         Quiz 25: Feynman-Kac path integral, infinite horizon HJB PDE, POMDP
  
  

• Practice Problems
  
         Problem Set 01: Calculus-of-variations in probability
  
         Problem Set 02: Equation of motion for pendulum with moving pivot
  
         Problem Set 03: Brachistochrone as optimal control problem
  
         Problem Set 04: Zermelo’s navigation problem [Code for PS04]
  
         Problem Set 05: Hamiltonian matrix
  
         Problem Set 06: Discrete-time finite horizon LQ tracking [Code for PS06]
  
         Problem Set 07: Singular optimal control problem
  
         Problem Set 08: Discrete-time finite horizon DP
  
         Problem Set 09: Continuous-time deterministic finite horizon DP
  
         Problem Set 10: Continuous-time stochastic infinite horizon DP