Machine Learning (Spring 2020)
Announcements
Final Project Term Paper (5 page, doublecolumn, IEEE style) is due by Wednesday, April 29, 2:00 pm.
Final Project Presentations to happen online during Monday, April 27 and Wednesday, April 29.
Lectures
(January 22, 2020) Lecture 1 : Introduction and Formal Model of Learning (cf. [R1], Chapter 1). Paper by Leo Breiman on the two cultures can be found here.
(January 27, 2020) Lecture 2 : PAC Learnability and Agnostic PAC Learnability (cf. [R1], Chapter 2)
(January 29, 2020) Lecture 3 : Learning via Uniform Convergence (cf. [R1], Chapter 3)
(February 3, 2020) Lecture 4 : NoFree Lunch Theorem and BiasComplexity Tradeoff (cf. [R1], Theorem 5.1, Corollary 5.2)
(February 5, 2020) Lecture 5 : VCDimension Theory (cf. [R1], Chapter 6)
(February 10, 2020) Lecture 6 : The Fundamental Theorem of PAC Learning (cf. [R1], Theorem 6.7, Lemma 6.10, Theorem 6.11)
(February 12, 2020) Lecture 7 : Convexity and Learning (cf. [R1], Chapter 12)
(February 17, 2020) Lecture 8 : Regularized Loss Minimization and Stable Rules (cf. [R1], Sections 13.1, 13.2, 13.3)
(February 24, 2020) Lecture 9 : OverfittingStability Tradeoff and Gradient Descent (cf. [R1], Corollary 13.8, Corollary 13.9, Section 14.1)
(February 26, 2020) Lecture 10 : Gradient Descent (contd.) (cf. [R1], Sectino 14.1, Section 14.2)
(March 2, 2020) Lecture 11 : Stochastic Gradient Descent (cf. Stochastic Gradient Descent and Its Variants in Machine Learning)
(March 4, 2020) Lecture 12 : Linear Predictors and Linear Regression (cf. [R1], Section 9.1, Section 9.2)
(March 9, 2020) Lecture 13 : Logistic Regresstion, Probability Interpretation of Regression, Regression Trees (cf. Andrew Ng's Notes; [R3], Section 9.2)
(March 11, 2020) Lecture 14 : Classification Trees, Ensemble Learning I : Bagging, Random Forests (cf. [R1], Chapter 18; [R3], Section 9.2, Section 15.1, Section 15.2, Section 15.3)
(March 23, 2020) Lecture 15 : Boosting, AdaBoost, Validation (cf. [R1], Chapter 10, Chapter 11)
(March 30, 2020) Lecture 16 : Support Vector Machines (cf. [R1], Chapter 15)
(April 1, 2020) Lecture 17 : Learning with Kernels (cf. [R1], Chapter 16)
(April 6, 2020) Lecture 18 : Nearest Neigbor Methods (cf. [R1], Chapter 19. Here is an interesting paper on the application of kNN methods in the estimation of information theoretic quantities.)
(April 8, 2020) Lecture 19 : Unsupervised Learning : kmeans Clustering (cf. [R1], Chapter 22)
(April 13, 2020) Lecture 20 : kmeans Clustering (contd.), Spectral Clustering, Information Bottleneck (cf. [R1], Chapter 22)
(April 15, 2020) Lecture 21 : Dimentionality Reduction (PCA), Compressed Sensing (cf. [R1], Chapter 23)
(April 20, 2020) Lecture 22 : Generative Models, Neural Networks (cf. [R1], Chapter 24, Andrew Ng's Notes)
(April 27, 2020) Lecture 23 : Final Project Presentations
(April 29, 2020) Lecture 24 : Final Project Presentations
Assignments
(Uploaded on February 12, 2020. Due by Wednesday, March 4, 2020, 2:00 pm) Assignment 1 (Solutions)
(Uploaded on March 5, 2020. Due by Monday, March 30, 2020, 4:00 pm) Assignment 2 (Solutions)
(Uploaded on March 30, 2020. Due by Monday, April 20, 2020, 2:00 pm) Assignment 3 (Solutions)
Final Project
Term Paper, due by Wednesday, April 29, 2020, 2:00 pm
Presentations on Monday, April 27, 2020 and Wednesday, April 29, 2020
Reference Books
[R1] Understanding Machine Learning : From Theory to Algorithms, Shai ShalevShwartz and Shai BenDavid
[R2] Pattern Recognition and Machine Learning, Christopher Bishop
[R3] Elements of Statistical Learning, Trevor Hastie, Robert Tibshirani, and Jerome Friedman
[R4] Introduction to Statistical Learning, Gareth James, Daniela Witten, Trevor Hastie, and Robert Tobshirani
[R5] A Brief Introduction to Machine Learning for Engineers, Osvaldo Simeone
General Information
Instructor : Himanshu Asnani
Venue : A201
Class Timings : Mondays, 4:00 pm  5:30 pm and Wednesdays, 2:00 pm  3:30 pm
Grading : Class Participation (10%), Homeworks (60%), Final Project (30%)
Assignments : We will be using Jupyter Notebook for the programming aspect of the course. Programming language will be Python and the installation guide can be located here.
Tentative Schedule
Module I [Theoretical Foundations] PAC Learning, BiasComplexity Tradeoff, VC Theory, Generalization, Gradient Descent, Stochastic Gradient Descent
Module II [Supervised Learning] Regression, Classification, Support Vector Machines, Decision Trees, Bagging, Boosting, Kernel Methods, Nearest Neighbor Methods, Neural Networks, Backpropagation
Module III [Unsupervised Learning] kmeans Clustering, Spectral Clustering, Dimensionality Reduction (PCA), Compressed Sensing, Information Bottleneck, Naive Bayes, Linear Discriminative Analysis, ExpectationMaximization Algorithm
