B555: Machine Learning: Instructor: Roni Khardon What is this course about? Machine learning is the science of collecting and analyzing data and turning it into predictions, encapsulated knowledge, or actions. There are many variations of how data is obtained, many different models and algorithms for data analysis, and many potential applications. In recent years machine learning has attracted attention due to successful applications in science, commerce and the public sphere leading to widespread use. Much of modern machine learning relies on probabilistic models or other mathematical formulations of the "data mining problem" and derives algorithms that are based on such formulations. Course Structure and Objectives: The course provides an overview of machine learning with a strong focus on probabilistic models and algorithms, and especially on Bayesian models. We will study the main existing approaches and algorithms with an emphasis on rigorous mathematical development of the ideas and will illustrate some applications. Assignments are used to reinforce the ideas, with some programming assignments to help ground the ideas in a practical machine learning context. Tentative list of topics and rough schedule: 1: Basics: introduction to Bayesian Models, Regression and Classification (~4 lectures) 2: Mathematical Foundations and Bayesian Linear Regression (~5 lectures) 3: Bayesian models for Classification and other prediction problems (~5 lectures) 4: Kernel methods and neural networks (~6 lectures) 5: Probabilistic machine learning as inference in graphical models (~7 lectures) Prerequisites: Graduate standing or permission of the instructor. We assume background in probability, multi-variate calculus and linear algebra. Brief "refresher crash courses" will be integrated into lectures to bring students up to speed but these will not suffice without prior background. In addition, students should be able to program independently from high level specifications of machine learning methods. Programming assignments are typically "end-to-end" assignments where you implement all stages of the machine learning task (reading the data, preprocessing, analyzing, predicting) yourself rather than using existing machine learning libraries. Programming language: Python (with numpy and scipy) is (not required but) highly recommended. In general: you will need to be able to program in a system that supports text processing, linear algebra and probability functions. Textbooks: We will use [B] as the main text and will refer to other sources as needed. [B] Pattern Recognition and Machine Learning, by C. M. Bishop, Springer, 2006. https://www.microsoft.com/en-us/research/people/cmbishop/prml-book/