Machine Learning

CS 445/545
Winter Quarter 2017

Time : Tuesdays and Thursdays, 2:00-3:50pm

Location: University Pointe (PNT) 102

Instructor: Melanie Mitchell, FAB 115-13, (503) 725-2412,
Office hours: T,Th Noon-1pm, or by appointment.

Teaching Assistant: Mike Lane,
Office hours: M,W Noon-1pm, FAB 115 in the "Data Lab" (across the hall from 115-13).

Course Mailing List:

Prerequisites: Undergraduate-level courses in calculus, linear algebra, and probability and statistics. Facility in at least one high-level programming language.

Main topics:: Perceptrons, neural networks, linear regression, logistic regression, evaluating classifiers, support vector machines, decision trees, ensemble learning, Bayesian learning, unsupervised learning, reinforcement learning, and deep learning.

Textbook: No textbook. Readings will be assigned from materials available on-line.

Relation to CS 441/541 (Artificial Intelligence): A couple of the same topics will be covered (e.g., neural networks), but these will be covered in more depth and at a more theoretical level in this course than in CS 441/541. Otherwise, the topics in this course differ from the topics covered in CS 441/541.

Homework: The class will have several homework assignments, involving writing code for and/or experimenting with various machine learning methods, along with written exercises.

Late homework policy: Students must request and be granted an extension on any homework assigment before the assignment is due. Otherwise, 5% of the assignment grade will be subtracted for each day the homework is late.

Exams: The class will have an in-class final exam. There will also be several short in-class quizzes to test basic understanding of the material presented in class and in the readings. You are allowed to bring in one double-sided page of notes for each quiz, and four double-sided pages of notes for the final exam.

Grading: Homework 50%, Quizzes 20%, Final exam 30%.

Academic integrity: Students will be responsible for following the PSU Student Conduct Code.

Students with disabilities: If you are a student with a disability in need of academic accommodations, you should register with the Disability Resource Center and notify the instructor immediately to arrange for support services.

Syllabus (subject to change):



Homework and Reading

Tuesday Jan. 10

Introduction to machine learning (pptx or pdf)

Perceptrons (pptx or pdf)

Chapters 1-3 of Michael Nielsen's online book on neural networks covers the basics of perceptrons and multilayer neural networks.

Homework 1 (Perceptrons), due Tuesday Jan. 24: Assignment, mnist_train.csv, mnist_test.csv

Thursday Jan. 12

Snow day!

Tuesday Jan. 17

Snow day!

Thursday Jan. 19

Perceptrons, continued

Multiclass classification (pptx or pdf)

Tuesday Jan. 24

Quiz 1 (30 min)

Quiz 1 solutions

Multilayer neural networks (pptx or pdf)

Homework 2 (Neural Networks), due Thursday Feb 2: Assignment

Thursday Jan. 26

Multilayer NNs continued.

Backprop example (pptx or pdf)

Support vector machines (pptx or pdf)

Reading: A. Ben-Hur and J. Weston, A User's Guide to Support Vector Machines

Tuesday Jan. 31

Support vector machines, continued

Thursday Feb. 2

SVMs, continued

Feature selection

Evaluating classifiers (pptx or pdf)

Review for Quiz 2

Reading:: T. Fawcett, An introduction to ROC analysis, Sections 1-4, 7

Homework 3 (SVMs), due Tuesday Feb. 14: Assignment

Tuesday Feb. 7

Quiz 2 (30 min)

Quiz 2 solutions

Evaluating classifiers, continued

Thursday Feb. 9

Bayesian learning (pptx or pdf)

Reading: Tom Mitchell, Generative and discriminative classifiers: Naive Bayes and logistic regression, Sections 1-2.

Tuesday Feb. 14

Bayesian learning, continued

Solutions to Exercises, Part 3

Logistic regression (pptx or pdf)

Reading: X. Z. Fern, Notes on Logistic Regression

Homework 4 (Bayesian learning and logistic regression), due Thursday Feb. 23: Assignment

Thursday Feb. 16

Quiz 3 (45 min)

Quiz 3 solutions

Ensemble learning (pptx or pdf)

Reading: R. Schapire, A brief introduction to boosting

Tuesday Feb. 21

Boosting decision stumps (pptx or pdf)

Unsupervised learning (pptx or pdf)

Reading: Chapter 8 from Introduction to Data Mining by Tan, Steinbach, and Kumar, pp. 487-515, 532-540, 546-550.

Thursday Feb. 23

Unsupervised learning, continued

Homework 5 (K-means Clustering), due Tuesday March 7: Assignment

Tuesday Feb. 28

Quiz 4 (30 min)

Quiz 4 solutions

Unsupervised learning, continued

Reinforcement learning (pptx or pdf)

Reading: pp. 278-280 of Reinforcement Learning for True Adaptive Traffic Signal Control (a brief and clear introduction to Q-learning)

Thursday March 2

Reinforcement learning, continued

Tuesday March 7

Deep learning (pptx or pdf)

Homework 6 (Reinforcement Learning), due Thursday March 16 : Assignment

Thursday March 9

Quiz 5 (30 min)

Quiz 5 solutions

Deep learning, continued

Tuesday March 14

Guest lectures:

Will Landecker: Data Science in the Real World (pdf)

Sheng Lundquist:Using Tensorflow for Deep Learning (pdf)

Thursday March 16

Review for final exam

Deep reinforcement learning (pptx or pdf)

Monday March 20

Final exam: 10:15am - 12:05pm