Introduction to Machine Learning (Spring 2020)

Announcements

Solutions to Assignment 2 are uploaded.

Assignment 3 uploaded. Due by Monday, April 20, 4:00 pm.

Final Project Term Paper (5 page, double-column, IEEE style) is due by Wednesday, April 29, 2:00 pm.

Final Project Presentations to happen online during Monday, April 27 and Wednesday, April 29.

  • Lectures

    • (January 22, 2020) Lecture 1 : Introduction and Formal Model of Learning (cf. [R1], Chapter 1). Paper by Leo Breiman on the two cultures can be found here.

    • (January 27, 2020) Lecture 2 : PAC Learnability and Agnostic PAC Learnability (cf. [R1], Chapter 2)

    • (January 29, 2020) Lecture 3 : Learning via Uniform Convergence (cf. [R1], Chapter 3)

    • (February 3, 2020) Lecture 4 : No-Free Lunch Theorem and Bias-Complexity Tradeoff (cf. [R1], Theorem 5.1, Corollary 5.2)

    • (February 5, 2020) Lecture 5 : VC-Dimension Theory (cf. [R1], Chapter 6)

    • (February 10, 2020) Lecture 6 : The Fundamental Theorem of PAC Learning (cf. [R1], Theorem 6.7, Lemma 6.10, Theorem 6.11)

    • (February 12, 2020) Lecture 7 : Convexity and Learning (cf. [R1], Chapter 12)

    • (February 17, 2020) Lecture 8 : Regularized Loss Minimization and Stable Rules (cf. [R1], Sections 13.1, 13.2, 13.3)

    • (February 24, 2020) Lecture 9 : Overfitting-Stability Tradeoff and Gradient Descent (cf. [R1], Corollary 13.8, Corollary 13.9, Section 14.1)

    • (February 26, 2020) Lecture 10 : Gradient Descent (contd.) (cf. [R1], Sectino 14.1, Section 14.2)

    • (March 2, 2020) Lecture 11 : Stochastic Gradient Descent (cf. Stochastic Gradient Descent and Its Variants in Machine Learning)

    • (March 4, 2020) Lecture 12 : Linear Predictors and Linear Regression (cf. [R1], Section 9.1, Section 9.2)

    • (March 9, 2020) Lecture 13 : Logistic Regresstion, Probability Interpretation of Regression, Regression Trees (cf. Andrew Ng's Notes; [R3], Section 9.2)

    • (March 11, 2020) Lecture 14 : Classification Trees, Ensemble Learning I : Bagging, Random Forests (cf. [R1], Chapter 18; [R3], Section 9.2, Section 15.1, Section 15.2, Section 15.3)

    • (March 23, 2020) Lecture 15 : Boosting, AdaBoost, Validation (cf. [R1], Chapter 10, Chapter 11)

    • (March 30, 2020) Lecture 16 : Support Vector Machines (cf. [R1], Chapter 15)

    • (April 1, 2020) Lecture 17 : Learning with Kernels (cf. [R1], Chapter 16)

  • Assignments

    • (Uploaded on February 12, 2020. Due by Wednesday, March 4, 2020, 2:00 pm) Assignment 1 (Solutions)

    • (Uploaded on March 5, 2020. Due by Monday, March 30, 2020, 4:00 pm) Assignment 2 (Solutions)

    • (Uploaded on March 30, 2020. Due by Monday, April 20, 2020, 2:00 pm) Assignment 3

  • Final Project

    • Term Paper, due by Wednesday, April 29, 2020, 2:00 pm

    • Presentations on Monday, April 27, 2020 and Wednesday, April 29, 2020

  • General Information

    • Instructor : Himanshu Asnani

    • Venue : A-201

    • Class Timings : Mondays, 4:00 pm - 5:30 pm and Wednesdays, 2:00 pm - 3:30 pm

    • Grading : Class Participation (10%), Homeworks (60%), Final Project (30%)

    • Assignments : We will be using Jupyter Notebook for the programming aspect of the course. Programming language will be Python and the installation guide can be located here.

    • Tentative Schedule

      • Module I [Theoretical Foundations (8 lectures)] PAC Learning, Bias-Complexity Trade-off, VC Theory, Generalization, Gradient Descent

      • Module II [Supervised Learning (8 lectures)] Regression, Classification, Support Vector Machines, Decision Trees, Bagging, Boosting

      • Module III [Unsupervised Learning (8 lectures)] Clustering, Dimensionality Reduction, Generative Models

      • Module IV [Deep Learning (3 lectures)] Neural Networks, Backpropagation, Generative Adversarial Networks (GANs)