Machine Learning

CS 445/545
Winter Quarter 2012

Time : Mondays and Wednesdays, 2:00-3:50pm
Location: Fourth Avenue Building (FAB), Room 10.

Instructor: Melanie Mitchell, FAB 120-24, (503) 725-2412, mm-AT-cs.pdx.edu
Office hours: Tu,Th 3:00-4:00pm, or by appointment.

Teaching Assistant: Dona Hertel, herteld-AT-cs.pdx.edu
Office hours: M,F 1:00-2:00pm (in the CS Fishbowl), or by appointment.

Course Website: : http://www.cs.pdx.edu/~mm/MachineLearningWinter2012/index.html

Course Mailing List: MLWinter2012@cs.pdx.edu

Prerequisites: Undergraduate-level courses in calculus, linear algebra, and probability and statistics. Facility in at least one high-level programming language.

Main topics:: Linear classification, multi-level perceptrons, support vector machines, evaluating classifiers, decision trees, ensemble learning, probability and learning, unsupervised learning, dimensionality reduction, reinforcement learning, evolutionary learning.

Textbook: Machine Learning: An Algorithmic Perspective

Relation to CS 441/541 (Artificial Intelligence): A couple of the same topics will be covered (neural networks, support vector machines), but these will be covered in more depth and at a more theoretical level in this course than in CS 441/541. Otherwise, the topics in this course differ from the topics covered in CS 441/541.

Homework: The class will have bi-weekly homework assignments, involving writing code for and/or experimenting with various machine learning methods.

Late homework policy: Students must request and be granted an extension on any homework assigment before the assignment is due. Otherwise, 5% of the assignment grade will be subtracted for each day the homework is late.

Exams: The class will have a take-home open-book final exam. There will also be weekly short in-class quizzes (closed-book) to test basic understanding of the material presented in class and in the readings.

Grading: Homework 50%, In-Class Quizzes 30%, Final exam 20%.

Academic integrity: Students will be responsible for following the PSU Student Conduct Code.

Students with disabilities: If you are a student with a disability in need of academic accommodations, you should register with the Disability Resource Center and notify the instructor immediately to arrange for support services.

Syllabus (subject to change):

Date	Topics	Homework and Reading
Monday Jan. 9	Introduction (pptx or pdf) Linear discrimination, part 1 (pptx or pdf)	Reading for this week: Textbook, Chapters 1-2. T. Fawcett, An introduction to ROC analysis, Sections 1-4 Optional: J. Davis and M. Goadrich, The relationship between Precision-Recall and ROC curves Homework: HW 1, due Monday Jan. 23.
Wednesday Jan. 11	Linear discrimination, part 2 (pptx or pdf) Evaluating classifiers (pptx or pdf)
Monday Jan. 16	No class (Martin Luther King day).	Reading for this week: Textbook, Chapter 3.
Wednesday Jan. 18	Multi-layer perceptrons. Guest lecturer: Josh Hughes
Monday Jan. 23	Review sheet for Quiz 1. Evaluating classifiers, continued. Support vector machines, part 1 (pptx or pdf)	Reading for this week: Textbook: Chapter 5. A User's Guide to Support Vector Machines
Wednesday Jan. 25	Quiz 1: linear discriminantion, evaluating classifiers. Support vector machines, continued.	Homework: Homework 2 (SVMs), due Monday Feb. 6. spam.zip: spam data
Monday Jan. 30	Review sheet for Quiz 2. Support vector machines, continued (pptx or pdf) Decision trees, part 1 (pptx or pdf)	Reading for this week: Textbook: Chapter 6.
Wednesday Feb. 1	Quiz 2: Support vector machines. Decision trees, part 2 (pptx or pdf)
Monday Feb. 6	Ensemble learning ( pptx or pdf) Review sheet for Quiz 3.	Reading for this week: Textbook, Chapter 7. Homework: Homework 3 (decision trees and ensemble learning), due Monday Feb. 27. Homework 3 addendum Homework 3 Experiment 3 alternative Download gzipped tarball C4.5.tgz (code is adapted from that at http://www2.cs.uregina.ca/~dbd/cs831/notes/ml/dtrees/c4.5/tutorial.html). C4.5 documentation is here (documentation for C4.5) and here (documentation for "verbosity"). Spambase dataset is here. Spambase dataset for SVMs is here.
Wednesday Feb. 8	Quiz 3: Decision trees Ensemble learning, continued (pptx or pdf)
Monday Feb. 13	Probability and Learning, Part 1 (pptx or pdf) Review sheet for Quiz 4.	Reading for this week: Textbook, Chapter 8, except 8.3. Iyengar et al., Active learning using adaptive resampling
Wednesday Feb. 15	Quiz 4: Ensemble learning Probability and learning, part 2 (pptx or pdf) Active learning
Monday Feb. 20	Probability and learning, part 3 (Bayesian Networks) (pptx) or pdf) Review sheet for Quiz 5.	Reading for this week: S. Wooldridge, Bayesian belief networks Textbook, Chapter 9 (skip section 9.3)
Wednesday Feb. 22	Quiz 5: Probability and learning. Bayesian networks, continued (pptx) or pdf) Unsupervised learning, part 1 (pptx) or pdf)
Monday Feb. 27	Review sheet for Quiz 6. Bayes Nets review (pptx or pdf)	Reading for this week: Textbook, Chapter 10, Section 10.2 Smith, A Tutorial on Principal Components Analysis
Wednesday Feb. 29	Quiz 6: Bayesian networks, unsupervised learning Active learning: An Example (pptx or pdf) Unsupervised learning, part 2 (pptx or pdf)	Homework: Homework 4 (unsupervised learning), due Wednesday Mar. 14. Homework 4 Addendum
Monday Mar. 5	Principal components analysis (pptx or pdf) Evolutionary learning, part 1 (pptx or pdf)	Reading for this week: Chapter 12. Optional: Scholkopf et al, Kernel Principal Component Analysis Robby the Robot code page.
Wednesday Mar. 7	No quiz this week. Evolutionary learning, part 2 (pptx or pdf) Evolutionary learning, part 3 (pptx or pdf)
Monday Mar. 12	Review sheet for Quiz 7. Reinforcement learning (pptx or pdf)	Reading for this week: Chapter 13.
Wednesday Mar. 14	Quiz 7: Principal components analysis, evolutionary learning, reinforcement learning Recap of important topics (pptx or pdf)	Take-home final exam (open-book) handed out, due Wednesday Mar. 21.
Monday Mar. 19	No class (finals week).
Wednesday Mar. 21	No class (finals week).