Instructor: Jiayu Zhou [email]

Time: Monday and Wednesday 10:20am - 11:40am

Location: Engineering Building, EB 3400

Office Hours: Monday and Wednesday 11:40am-12:30pm, EB 2134

Machine Learning is concerned with computer programs that automatically improve their performance through experience (e.g., that learn to spot high-risk medical patients, recognize speech, classify text documents, detect credit card fraud, or drive autonomous robots). This course provides an in-depth understanding of machine learning and statistical pattern recognition techniques and their applications in biomedical informatics, computer vision, and other domains.

Topics: probability distributions, regression, classification, kernel methods, clustering, semi-supervised learning, mixture models, graphical models, dimensionality reduction, manifold learning, sparse learning, multi-task learning, transfer learning, and Hidden Markov Models.

Homework assignments include both theoretic derivation and hands-on experiments with various learning algorithms. Every student is required to finish a project that is either assigned by the intructor or designed by the student himself.

Announcements will be emailed to the course mailing list. A welcome note will be sent to the mailing list at the beginning of the semester. If you do not receive the welcome message before the first class, please send mail to me.

The grade will be calculated as follows:

• Assignments: 40%
• Project: 25%
• Exam: 30%
• Class participation: 5%

Lateness and Extensions

Homework is worth full credit at the beginning of class on the due date (later if an extension has been granted). It is worth at most 90% credit for the next 24 hours. It is worth at most 50% credit for the following 24 hours. It is worth 25% credit after that. If you need an extension, please ask for it (by sending email to the instructor) as soon as the need for it is known. Extensions that are requested promptly will be granted more liberally. You must turn in all assignments.

Collaboration among Students

The purpose of student collaboration is to facilitate learning, not to circumvent it. Studying the material in groups is strongly encouraged. It is also allowed to seek help from other students in understanding the material needed to solve a particular homework problem, provided no written notes are shared, or are taken at that time, and provided learning is facilitated, not circumvented. The actual solution must be done by each student alone, and the student should be ready to reproduce their solution upon request. Any form of help or collaboration must be disclosed in full by all involved on the first page of their assignment. In any case, you must exercise academic integrity.

Students are required to attend all classes and actively participate in discussions.

• Topic 1. Introduction
• Topic 2. Basics - Probability Theory
• Topic 3. Basics - Linear Algebra
• Topic 4. Basics - Linear Algebra - SVD
• Topic 5. Linear Models for Regression
• Topic 6. Linear Models for Classification
• Topic 7. Support Vector Machines
• Topic 8. Kernel Methods
• Topic 9. Ensemble
• Topic 10. Tree Methods
• Topic 11. Clustering/Mixture Models
• Topic 12. Deep Learning
• Topic 13. Dimensionality Reduction
• Topic 14. Graphical Model
• Topic 15. Sparse Learning
• Topic 16. Matrix Completion and Collaborative Filtering
• Topic 17. Transfer and Multi-Task Learning

Student leaders have recognized the challenging task of discouraging plagiarism from the academic community. The Associated Students of Michigan State University (ASMSU) is proud to be launching the Spartan Code of Honor academic pledge, focused on valuing academic integrity and honest work ethics at Michigan State University. The pledge reads as follows:

As a Spartan, I will strive to uphold values of the highest ethical standard. I will practice honesty in my work, foster honesty in my peers, and take pride in knowing that honor is worth more than grades. I will carry these values beyond my time as a student at Michigan State University, continuing the endeavor to build personal integrity in all that I do.

The Spartan Code of Honor academic pledge embodies the principles of integrity that every Spartan is required to uphold in their time as a student, and beyond. The academic pledge was crafted with inspiration of existing individual college honor codes, establishing an overarching statement for the entire university. It was formally adopted by ASMSU on March 3, 2016, endorsed by Academic Governance on March 22, 2016, and recognized by the Provost, President, and Board of Trustees on April 15, 2016. Student conduct that is inconsistent with the academic pledge is addressed through existing policies, regulations, and ordinances governing academic honesty and integrity: Integrity of Scholarship and Grades, Student Rights and Responsibilities, and General Student Regulations.

Sign Spartan Code of Honor here!

Article 2.3.3 of the Academic Freedom Report states that The student shares with the faculty the responsibility for maintaining the integrity of scholarship, grades, and professional standards. In addition, the (insert name of unit offering course) adheres to the policies on academic honesty as specified in General Student Regulations 1.0, Protection of Scholarship and Grades; the all University Policy on Integrity of Scholarship and Grades; and Ordinance 17.00, Examinations. (See Spartan Life: Student Handbook and Resource Guide and/or the MSU Web site: www.msu.edu.) Therefore, unless authorized by your instructor, you are expected to complete all course assignments, including homework, lab work, quizzes, tests and exams, without assistance from any source. You are expected to develop original work for this course; therefore, you may not submit course work you completed for another course to satisfy the requirements for this course. Also, you are not authorized to use the www.allmsu.com Web site to complete any course work in this course. Students who violate MSU academic integrity rules may receive a penalty grade, including a failing grade on the assignment or in the course. Contact your instructor if you are unsure about the appropriateness of your course work. (See also the Academic Integrity webpage.)

Essays, journals, and other materials submitted for this class are generally considered confidential pursuant to the Universitys student record policies. However, students should be aware that University employees, including instructors, may not be able to maintain confidentiality when it conflicts with their responsibility to report certain issues to protect the health and safety of MSU community members and others. As the instructor, I must report the following information to the Department of Police and Public Safety if you share it with me: Suspected child abuse/neglect, even if this maltreatment happened when you were a child, Allegations of sexual assault or sexual harassment when they involve MSU students, faculty, or staff, and Credible threats of harm to oneself or to others. These reports will trigger contact from the Department of Police and Public Safety who will want to talk with you about the incident that you have shared. In almost all cases, it will be your decision whether you wish to speak with that individual. If you would like to talk about these events in a more confidential setting you are encouraged to make an appointment with the MSU Counseling Center.

Michigan State University is committed to providing equal opportunity for participation in all programs, services and activities. Requests for accommodations by persons with disabilities may be made by contacting the Resource Center for Persons with Disabilities at 517-884-RCPD or on the web at rcpd.msu.edu. Once your eligibility for an accommodation has been determined, you will be issued a Verified Individual Services Accommodation (VISA) form. Please present this form to me at the start of the term and/or two weeks prior to the accommodation date (test, project, etc.). Requests received after this date may not be honored.

Article 2.III.B.4 of the Academic Freedom Report (AFR) for students at Michigan State University states: The student’s behavior in the classroom shall be conducive to the teaching and learning process for all concerned. Article 2.III.B.10 of the AFR states that The student has a right to scholarly relationships with faculty based on mutual trust and civility. General Student Regulation 5.02 states: No student shall ... interfere with the functions and services of the University (for example, but not limited to, classes ...) such that the function or service is obstructed or disrupted. Students whose conduct adversely affects the learning environment in this classroom may be subject to disciplinary action through the Student Judicial Affairs office.

Essays, journals, and other materials submitted for this class are generally considered confidential pursuant to the University's student record policies. However, students should be aware that University employees, including instructors, may not be able to maintain confidentiality when it conflicts with their responsibility to report certain issues to protect the health and safety of MSU community members and others. As the instructor, I must report the following information to other University offices (including the Department of Police and Public Safety) if you share it with me:

--Suspected child abuse/neglect, even if this maltreatment happened when you were a child,

--Allegations of sexual assault or sexual harassment when they involve MSU students, faculty, or staff, and

--Credible threats of harm to oneself or to others.

These reports may trigger contact from a campus official who will want to talk with you about the incident that you have shared. In almost all cases, it will be your decision whether you wish to speak with that individual. If you would like to talk about these events in a more confidential setting you are encouraged to make an appointment with the MSU Counseling Center.

• Textbook: Pattern Recognition and Machine Learning, Christopher M. Bishop, 2006. Webpage

• Reference book: The Elements of Statistical Learning: Data Mining, Inference, and Prediction (Second Edition) by Trevor Hastie, Robert Tibshirani and Jerome Friedman (2009) Book

• Gradient computation w.r.t. a vector/matrix

https://www.math.uwaterloo.ca/~hwolkowi/matrixcookbook.pdf

• Shorter materials

http://ai.stanford.edu/~paskin/gm-short-course/lec1.pdf

http://www.sci.utah.edu/~gerig/CS6640-F2010/prob-tut.pdf

• Longer books

http://mplab.ucsd.edu/tutorials/ProbabilityAndStats.pdf

https://www.dartmouth.edu/~chance/teaching_aids/books_articles/probability_book/amsbook.mac.pdf

• Lecture notes from Andrew Ng:

http://cs229.stanford.edu/notes/cs229-notes1.pdf

• If you are interested in systemtically studying the optimization knowledge, try reading the book Convex Optimization:

https://web.stanford.edu/~boyd/cvxbook/bv_cvxbook.pdf

The basic gradient descent is decribed in Page 463 (Page 477 of the PDF file).