Machine Learning

CS 445/545
Fall Quarter 2018

Time : Tuesdays and Thursdays, 10:00-11:50am

Location: Engineering Building 93

Instructor: Melanie Mitchell, FAB 115-13, (503) 725-2412,
Office hours: T,Th Noon-1pm, or by appointment.

Teaching Assistant: Li-Yun Wang
Office hours: M,W 3pm-4pm in CS Fishbowl.

Course Mailing List:

Prerequisites: Undergraduate-level courses in calculus and linear algebra. Facility in at least one high-level programming language.

Main topics:: Perceptrons, neural networks, logistic regression, evaluating classifiers, support vector machines, ensemble learning, Bayesian learning, unsupervised learning, deep learning, and reinforcement learning.

Textbook: No textbook. Readings will be assigned from materials available on-line.

Relation to CS 441/541 (Artificial Intelligence): A couple of the same topics will be covered (e.g., neural networks), but these will be covered in more depth and at a more theoretical level in this course than in CS 441/541. Otherwise, the topics in this course differ from the topics covered in CS 441/541.

Homework: The class will have several homework assignments, involving writing code for and/or experimenting with various machine learning methods.

Late homework policy: Students must request and be granted an extension on any homework assigment before the assignment is due. Otherwise, 5% of the assignment grade will be subtracted for each day the homework is late, up to a maximum of 25%.

Quizzes: The class will have several short in-class quizzes to test basic understanding of the material presented in class and in the readings. You are allowed to bring in one double-sided page of notes for each quiz. There will be no midterm or final exam.

Final Project: Students will work in small teams (2-4 people) on a final project. Details will be given during class.

Grading: Homework 50%, Quizzes 20%, Final project 30%.

Academic integrity: Students will be responsible for following the PSU Student Conduct Code.

Students with disabilities: If you are a student with a disability in need of academic accommodations, you should register with the Disability Resource Center and notify the instructor immediately to arrange for support services.

Syllabus (subject to change):



Homework and Reading

Tuesday September 25

Introduction to machine learning (pptx or pdf)

Class "To Do" list

Perceptrons (pptx or pdf)

Chapters 1-3 of Michael Nielsen's online book on neural networks covers the basics of perceptrons and multilayer neural networks.

Homework 1 (Perceptrons), due Thursday October 4

Assignment, mnist_train.csv, mnist_test.csv

Thursday September 27

Pre-test solutions

Perceptrons, continued

Multiclass classification (pptx or pdf)

Tuesday October 2

Multilayer neural networks (pptx or pdf)

Solutions to neural network exercises

Review for Quiz 1

Thursday October 4

Quiz 1

Quiz 1 Solutions

Multilayer neural networks, continued

Homework 2 (Neural Networks), due Tuesday October 16: Assignment

Tuesday October 9

Support vector machines (pptx or pdf)

Reading: A. Ben-Hur and J. Weston, A User's Guide to Support Vector Machines

Thursday October 11

Support vector machines, continued

Dimensionality reduction (pptx or pdf)

Tuesday October 16

Solutions to SVM Exercises

Feature selection

Evaluating classifiers (pptx or pdf)

Review for Quiz 2

Reading:: T. Fawcett, An introduction to ROC analysis, Sections 1-4, 7

Homework 3 (SVMs and feature selection), due Thursday October 25: Assignment

Thursday October 18

Quiz 2

Quiz 2 solutions

Final project instructions

Evaluating classifiers, continued

Tuesday October 23

Bayesian learning (pptx or pdf)

Reading: T. Mitchell, Chapter on Naive Bayes and Logistic Regression

Thursday October 25

Bayesian learning, continued

Solutions to "At home" exercises

Solution to "In class' exercises, part 2, number 3

Logistic regression (pptx or pdf)

Review for Quiz 3

Homework 4 (Bayesian learning and logistic regression), due Tuesday November 6: Assignment

Tuesday October 30

Quiz 3

Quiz 3 solutions

Unsupervised learning, Part I: K-Means Clustering (pptx or pdf)

Reading: Chapter 8 from Introduction to Data Mining by Tan, Steinbach, and Kumar, pp. 487-515, 532-540, 546-550.

Thursday November 1

Unsupervised learning, Part II: Gaussian Mixture Models (pptx or pdf)

One-page project proposal due

Tuesday November 6

Review for Quiz 4

Reinforcement learning (pptx or pdf)

Solutions to in-class exercices, problems 1 and 2

Reading: pp. 278-280 of Reinforcement Learning for True Adaptive Traffic Signal Control (a brief and clear introduction to Q-learning)

Homework 5 (K-means clustering), due Thursday November 15: Assignment

Optdigits dataset for Homework 5

Thursday November 8

Quiz 4

Quiz 4 solutions

Reinforcement learning, continued

Tuesday November 13

Ensemble learning (pptx or pdf)

Reading: R. Schapire, A brief introduction to boosting

Thursday November 15

Review for Quiz 5

Solutions to Exercises

Deep learning for vision (pptx or pdf) (apologies for large files!)

Tuesday November 20

Quiz 5

Quiz 5 Solutions

Deep learning for vision, continued

Thursday November 22

Thanksgiving holiday

Tuesday November 27

Deep learning for language (pptx or pdf)

Thursday November 29

Common Sense in AI (pptx or pdf)

Generative Adversarial Networks (pptx or pdf)

Friday December 7

Final projects due