Machine Learning

CS 445/545
Winter Quarter 2019

Time : Tuesdays and Thursdays, 2:00-3:50pm

Location: Cramer Hall 71

Instructor: Melanie Mitchell, FAB 115-13, (503) 725-2412,
Office hours: T,Th 10am-11am, or by appointment.

Teaching Assistant: Li-Yun Wang
Office hours: M,W 4pm-5pm in CS Fishbowl.

Course Mailing List:

Prerequisites: Undergraduate-level courses in calculus and linear algebra. Facility in at least one high-level programming language.

Main topics:: Perceptrons, neural networks, logistic regression, evaluating classifiers, support vector machines, ensemble learning, Bayesian learning, unsupervised learning, deep learning, and reinforcement learning.

Textbook: No textbook. Readings will be assigned from materials available on-line.

Relation to CS 441/541 (Artificial Intelligence): A couple of the same topics will be covered (e.g., neural networks), but these will be covered in more depth and at a more theoretical level in this course than in CS 441/541. Otherwise, the topics in this course differ from the topics covered in CS 441/541.

Homework: The class will have several homework assignments, involving writing code for and/or experimenting with various machine learning methods.

Late homework policy: Students must request and be granted an extension on any homework assigment before the assignment is due. Otherwise, 5% of the assignment grade will be subtracted for each day the homework is late, up to a maximum of 25%.

Quizzes: The class will have several short in-class quizzes to test basic understanding of the material presented in class and in the readings. You are allowed to bring in one double-sided page of notes for each quiz. There will be no midterm or final exam.

Final Project: Students will work in small teams (1-4 people) on a final project. Details will be given during class.

Grading: Homework 50%, Quizzes 20%, Final project 30%.

Academic integrity: Students will be responsible for following the PSU Student Conduct Code.

Students with disabilities: If you are a student with a disability in need of academic accommodations, you should register with the Disability Resource Center and notify the instructor immediately to arrange for support services.

Syllabus (subject to change):



Homework and Reading

Tuesday January 8

Introduction to machine learning (pptx or pdf)

Class "To Do" list

Perceptrons (pptx or pdf)

Chapters 1-3 of Michael Nielsen's online book on neural networks covers the basics of perceptrons and multilayer neural networks.

Homework 1 (Perceptrons), due Thursday January 17: Assignment, mnist_train.csv, mnist_test.csv

Thursday January 10

Perceptrons, continued

Perceptrons Exercises solutions

Tuesday January 15

Multiclass classification (pptx or pdf)

Multilayer neural networks (pptx or pdf)

Review for Quiz 1

Practice Quiz 1

Practice Quiz 1 Solutions

Thursday January 17

Quiz 1

Multilayer neural networks, continued

Homework 2 (Neural Networks), due Tuesday January 29: Assignment

Tuesday January 22

Support vector machines

Reading: A. Ben-Hur and J. Weston, A User's Guide to Support Vector Machines

Thursday January 24

Support vector machines, continued

Dimensionality reduction

Tuesday January 29

Feature selection

Review for Quiz 2

Reading:: T. Fawcett, An introduction to ROC analysis, Sections 1-4, 7

Homework 3 (SVMs and feature selection), due Thursday February 7

Thursday January 31

Quiz 2

Final project instructions

Evaluating classifiers, continued

Tuesday February 5

Bayesian learning

Reading: T. Mitchell, Chapter on Naive Bayes and Logistic Regression

Thursday February 7

Bayesian learning, continued

Logistic regression

Review for Quiz 3

Homework 4 (Bayesian learning and logistic regression), due Tuesday February 19

Tuesday February 12

Quiz 3

Unsupervised learning, Part I: K-Means Clustering

Reading: Chapter 8 from Introduction to Data Mining by Tan, Steinbach, and Kumar, pp. 487-515, 532-540, 546-550.

Thursday February 14

Unsupervised learning, Part II: Gaussian Mixture Models

One-page project proposal due

Tuesday February 19

Review for Quiz 4

Reinforcement learning

Reading: pp. 278-280 of Reinforcement Learning for True Adaptive Traffic Signal Control (a brief and clear introduction to Q-learning)

Homework 5 (K-means clustering), due Thursday February 28

Thursday February 21

Quiz 4

Reinforcement learning, continued

Tuesday February 26

Ensemble learning

Reading: R. Schapire, A brief introduction to boosting

Thursday February 28

Review for Quiz 5

Deep learning for vision

Tuesday March 5

Quiz 5

Deep learning for vision, continued

Thursday March 7

Deep learning for language

Tuesday March 12

Deep learning for language, continued

Common Sense in AI

Thursday March 14

Generative Adversarial Networks

Friday March 22

Final projects due