Machine Learning Seminar

CS 570
Computer Science Department, Portland State University

Fall 2016

Time: Thursdays, 4:00-5:30pm

Location: FAB 88-03

Instructor: Melanie Mitchell, FAB 115-13, (503) 720-2412, e-mail
Office hours: Tuesdays and Thursdays, 2-3pm, or by appointment

Course Mailing list:

Course description: This course is a one-credit graduate seminar for students who have already taken a course in Machine Learning. Students will read and discuss recent papers in the Machine Learning literature. Each student will be responsible for presenting at least one paper during the term. This one-credit course will be offered each term, and students may take it multiple times. CS MS students who take this course for three terms may count it as one of the courses for the "Artificial Intelligence and Machine Learning" masters track requirement.

Prerequisites: CS 445/545 or permission of the instructor.

Textbook: None. We will read recent papers from the literature.

Course Work and Homework: One or more papers will be assigned per week for everyone in the class to read, along with a list of questions about the paper(s) that each student needs to answer before the following class. Each week one or more students will be assigned as discussion leaders for the week's papers.

Topic for Fall Term 2016: Deep Reinforcement Learning

Schedule for Fall Term 2016: This will be progressively filled in during the term.

No questions this week.



Discussion Leader(s)



Sept. 29

Introduction to deep reinforcement learning

Slides: pptx or pdf


For students not familiar with reinforcement learning, or who want a review, please read Chapter 1 of Reinforcement Learning: An Introduction, by Sutton and Barto, before the first class

For students not familiar with convolutional neural networks, or who want a review, this is a good summary.

Oct 6

Learning to play Atari video games Anthony Mnih et al., Human-level control through deep reinforcement learning Week 2 Questions: pdf

Oct 13

AlphaGo Erik and Mihir Silver et al., Mastering the game of Go with deep neural networks and tree search Week 3 Questions: pdf

Oct 20

Double Q-Learning and Dueling Network Architectures Caitlin and Casey

van Hasselt et al., Deep reinforcement learning with double Q-learing

Wang et al., Dueling network architectures for deep reinforcement learning

Week 4 Questions: pdf

Oct 27

Applications in computer vision Jordan and Joachim

Mnih et al., Recurrent models of visual attention

Caicedo and Lazebnik, Active object localization with deep reinforcement learning

Week 5 Questions: pdf

Nov 3

Natural language applications Ari and Wesley

Satija and Pineau, Simultaneous machine translation using deep reinforcement learning

Narasimhan et al., Language understanding for text-based games using deep reinforcement learning

No questions this week.

Nov 10

Visualizing Deep-Q Networks Sharad and Melanie

van der Maaten & Hinton, Visualizing data using t-SNE.

Note: This paper is fairly technical, so if you prefer, you can instead watch this very clear talk explaining the paper.

Short lecture on differences between Q-learning, policy-learning, and "actor-critic" models (Melanie)

Week 7 Questions: pdf

Nov 17

Multi-Agent Deep Reinforcement Learning Devin and Henry

He et al., Opponent modeling in deep reinforcement learning

Shalev-Shwartz et al., Safe, multi-agent, reinforcement learning for autonomous driving

Nov 24

Thanksgiving: No class

Dec 1

Continuous action spaces; symbolic deep reinforcement learning Mike and Sheng

Lilicrap et al., Continuous control with deep reinforcement learning

Garnelo et al., Towards deep symbolic reinforcement learning

Week 9 Questions: pdf

Dec 8

More DRL applications Roberto and Thomas

Mao et al., Resource management with deep reinforcement learning

Narasimhan et al., Improving information extraction by acquiring external evidence with reinforcement learning

Week 10 Questions: pdf