Machine Learning

CS 445/545
Spring Quarter 2017

Time : Tuesdays and Thursdays, 2:00-3:50pm

Location: ASRC 215

Instructor: Melanie Mitchell, FAB 115-13, (503) 725-2412,
Office hours: M,W 11am-noon, or by appointment.

Teaching Assistant: Mike Lane,
Office hours: T,Th Noon-1pm, FAB 115 in the "Data Lab" (across the hall from 115-13).

Course Mailing List:

Prerequisites: Undergraduate-level courses in calculus, linear algebra, and probability and statistics. Facility in at least one high-level programming language.

Main topics:: Perceptrons, neural networks, linear regression, logistic regression, evaluating classifiers, support vector machines, decision trees, ensemble learning, Bayesian learning, unsupervised learning, reinforcement learning, and deep learning.

Textbook: No textbook. Readings will be assigned from materials available on-line.

Relation to CS 441/541 (Artificial Intelligence): A couple of the same topics will be covered (e.g., neural networks), but these will be covered in more depth and at a more theoretical level in this course than in CS 441/541. Otherwise, the topics in this course differ from the topics covered in CS 441/541.

Homework: The class will have several homework assignments, involving writing code for and/or experimenting with various machine learning methods, along with written exercises.

Late homework policy: Students must request and be granted an extension on any homework assigment before the assignment is due. Otherwise, 5% of the assignment grade will be subtracted for each day the homework is late, up to a maximum of 25%.

Exams: The class will have an in-class final exam. There will also be several short in-class quizzes to test basic understanding of the material presented in class and in the readings. You are allowed to bring in one double-sided page of notes for each quiz, and four double-sided pages of notes for the final exam.

Grading: Homework 50%, Quizzes 20%, Final exam 30%.

Academic integrity: Students will be responsible for following the PSU Student Conduct Code.

Students with disabilities: If you are a student with a disability in need of academic accommodations, you should register with the Disability Resource Center and notify the instructor immediately to arrange for support services.

Syllabus (subject to change):



Homework and Reading

Tuesday April 4

Introduction to machine learning (pptx or pdf)

Class "To Do" list

Perceptrons (pptx or pdf)

Pre-test solutions

Chapters 1-3 of Michael Nielsen's online book on neural networks covers the basics of perceptrons and multilayer neural networks.

Thursday April 6

Perceptrons, continued

Homework 1 (Perceptrons), due Tuesday April 18

Assignment, mnist_train.csv, mnist_test.csv

Tuesday April 11

Multiclass classification (pptx or pdf)

Multilayer neural networks (pptx or pdf)

Thursday April 13

Multilayer neural networks, continued

Partial solutions to in-class exercises

Review for Quiz 1

Tuesday April 18

Quiz 1

Quiz 1 solutions

Evaluating classifiers (pptx or pdf)

In-class exercises solutions (partial)

Reading:: T. Fawcett, An introduction to ROC analysis, Sections 1-4, 7

Homework 2 (Neural Networks) , due Thursday April 27: Assignment

Update: Homework 2 deadline is now extended to Monday, May 1, 5pm

Thursday April 20

Support vector machines (pptx or pdf)

Reading: A. Ben-Hur and J. Weston, A User's Guide to Support Vector Machines

Tuesday April 25

Support vector machines, continued

Feature selection

Review for Quiz 2

Thursday April 27

Quiz 2

Quiz 2 solutions

Bayesian learning (pptx or pdf)

Reading: Tom Mitchell, Generative and discriminative classifiers: Naive Bayes and logistic regression, Sections 1-2.

Homework 3 (SVMs and feature selection) , due Tuesday May 9: Assignment

Tuesday May 2

Bayesian learning, continued

Thursday May 4

Bayesian learning, continued

"At home" exercises

Solutions to "At home" exercises

Tuesday May 9

Review for Quiz 3

Logistic regression (pptx or pdf)

Ensemble learning (pptx or pdf)

Solutions to in-class exercises

Homework 4 (Bayesian learning and logistic regression), due Thursday May 18: Assignment

Reading: X. Z. Fern, Notes on Logistic Regression

Thursday May 11

Quiz 3

Quiz 3 solutions

Ensemble learning, continued

Reading: R. Schapire, A brief introduction to boosting

Tuesday May 16

Unsupervised learning (pptx or pdf)

Reading: Chapter 8 from Introduction to Data Mining by Tan, Steinbach, and Kumar, pp. 487-515, 532-540, 546-550.

Thursday May 18

Unsupervised learning, continued

Entropy slides (pptx or pdf)

Review for Quiz 4

Homework 5 (K-means clustering), due Tuesday May 30: Assignment

Optdigits dataset for Homework 5

Tuesday May 23

Quiz 4

Quiz 4 solutions

Reinforcement learning (pptx or pdf)

Reading: pp. 278-280 of Reinforcement Learning for True Adaptive Traffic Signal Control (a brief and clear introduction to Q-learning)

Thursday May 25

Reinforcement learning, continued

Tuesday May 30

Review for Quiz 5

Deep learning (pptx or pdf)

Homework 6 (Reinforcement learning), due Thursday June 8 : Assignment

Thursday June 1

Quiz 5

Quiz 5 solutions

Deep learning, continued

Detecting visual situations (pptx or pdf)

Tuesday June 6

Review for final exam

Special Topics (pptx or pdf)

Thursday June 8

Guest lectures

Monday June 12

Final exam: 10:15am - 12:05pm