**D**eep **R**einforcement **L**earning

## 10-703 • Fall 2023 • Carnegie Mellon University

This course brings together many disciplines of Artificial Intelligence (including computer vision, robot control, reinforcement learning, language understanding) to show how to develop intelligent agents that can learn to sense the world and learn to act by imitating others, maximizing sparse rewards, and/or satisfying their curiosity.

**Course Goals:**

Upon completion of the course students should be able to:

- Implement and experiment with existing state-of-the-art methods for learning behavioral policies supervised by reinforcement, demonstrations and/or intrinsic curiosity.
- Evaluate the sample complexity, generalization and generality of these algorithms.
- Understand research papers in the field of robotic learning.
- Try out ideas/extensions on existing methods.

**Prerequisite Knowledge:**

Students should have a solid understanding of the following areas

- Algorithms: e.g., What problem does Dijkstra’s algorithm solve?
- Probability: e.g., What is Bayes rule? How do you normalize a distribution?
- Computer vision: convolutional networks, object detection architectures, LSTMs, attention models
- Deep Learning: familiarity with TensorFlow and/or Pytorch.
- Matrix Calculus: e.g., What are derivatives of matrix-matrix and matrix-vector products? What is the multivariate chain rule?
- Programming: e.g., What are classes and inheritance? How do you structure read data from files? How do you plot figures to visualize results?
- Numerical programming: e.g., How would you perform an elementwise product instead of an inner product? How do you invert a matrix?

**Prerequisites:**

- Prerequisites: 10601 introduction to machine learning
- Minimum Grades: B in 10601
- Corequisites: None
- Anti-requisites: None
- Anti-req Prohibits: None

**Lectures:**Monday, Wednesday 2:00 PM - 3:20 PM**Recitations:**Friday 2:00 PM - 3:20 PM**Lecture/Recitation Location:**GHC 4401**Discussion:**Piazza**HW submission:**Gradescope**Contact:**For external enquiries, personal matters or in emergencies, you can email Joshmin at joshminr@andrew.cmu.edu.

- Instructor Katerina Fragkiadaki
- Email: katef@cs.cmu.edu
- Office hours: Mon & Wed, 3:20pm - 3:50pm

- Education Associate Joshmin Ray
- Email: joshminr@andrew.cmu.edu
- Office hours:

- TA Youngseog Chung
- Email: youngsec@andrew.cmu.edu
- Office hours: Mon, Wed 10:00am - 11:00am
- Location: See Calendar Tab

- TA Athiya Deviyani
- Email: adeviyan@andrew.cmu.edu
- Office hours: Tues 4:00pm - 5:00pm, Wed 6:00pm - 7:00pm
- Location: See Calendar Tab

- TA Amy Lin
- Email: amylin2@andrew.cmu.edu
- Office hours: Mon 6:00pm - 8:00pm
- Location: See Calendar Tab

- TA Lawrence Jang
- Email: ljang@andrew.cmu.edu
- Office hours: Thurs 5:00pm - 7:00pm
- Location: See Calendar Tab

- TA Alex Kireeff
- Email: akireeff@andrew.cmu.edu
- Office hours: Sun 4:00pm - 5:00pm, Wed 7:00pm - 8:00pm
- Location: See Calendar Tab