Advanced Machine Learning

| Apr 23, 2021| Classes
Advanced Machine Learning

Machine learning studies algorithms that build models from data for subsequent use in prediction, inference, and decision making tasks. Although an active field for the last 60 years, the current demand as well as trust in machine learning exploded as increasingly more data become available and the problems needed to be addressed become literally impossible to program directly. In this advanced course we will cover essential algorithms, concepts, and principles of machine learning. Along with the traditional exposition we will learn how these principles are currently being revisited thanks to the recent discoveries in the field.

1. Introduction

Lecture Slides

Youtube Lectures

  1. Introductions 7:51
  2. Why Machine Learning 12:00
  3. What is Machine Learning 18:57
  4. History of Machine Learning 17:33
  5. Reinforcement Learning 10:32
  6. Course Overview 19:26
  7. The Project 20:03

2. Foundations of learning

Lecture Slides

Youtube Lectures

  1. Formalizing the Problem of Learning 24:19
  2. Inductive Bias 12:03
  3. Can We Bound the Probability of Error? 25:56

3. PAC learnability

Lecture Slides

Youtube Lectures

  1. Main Definitions from Lecture 2 13:52
  2. Agnostic PAC Learning 53:35
  3. Learning via Uniform Convergence 10:15

4. Linear algebra and Optimization (recap)

  1. 3Blue1Brown Playlist

5. Linear learning models

Lecture Slides

Youtube Lectures

  1. Linear Decision Boundary 34:10
  2. Perceptron 37:10
  3. Perceptron Extensions 14:09
  4. Linear Classifier for Linearly non Separable Classes 8:59

6. Principal Component Analysis

Lecture Slides

Youtube lectures

  1. Linear Regression 39:24
  2. Linear Algebra Micro Refresher 2:04
  3. Spectral Theorem 25:54
  4. Principal Component Analysis 22:29
  5. Demonstration 17:38

7. Curse of Dimensionality

Lecture Slides

Youtube lectures

  1. Curse of Dimensionality 1:16:27

8. Bayesian Decision Theory

Lecture Slides

Youtube lectures

  1. Bayesian Decision Theory 56:47

9. Parameter estimation: MLE

Lecture Slides

Youtube Lectures

  1. Independence 12:07
  2. Maximum Likelihood Estimation 50:35
  3. MLE as KL-divergence minimization 21:41

10. Parameter estimation: MAP & Naïve Bayes

Lecture Slides

Youtube Lectures

  1. MAP Estimation 56:00
  2. The Naïve Bayes Classifier 37:09

11. Logistic Regression

Lecture Slides

Youtube Lectures

  1. NB to LR 19:49
  2. Defining Logistic Regression 27:42
  3. Solving Logistic Regression 23:35

12. Kernel Density Estimation

Lecture Slides

Youtube Lectures

  1. Non-parametric Density Estimation 1:13:33

13. Support Vector Machines

Lecture Slides

Youtube Lectures

  1. Max Margin Classifier 35:53
  2. Lagrange Multipliers 32:45
  3. Dual Formulation of Linear SVM 10:34
  4. Kernel Trick and Soft Margin 27:28

14. Matrix Factorization

Lecture Slides

Youtube Lectures

  1. Matrix Factorization 1:24:22

15. Stochastic Gradient Descent

Lecture Slides

Youtube Lectures

  1. Stochastic Gradient Descent 1:06:57

16. k-means Clustering

Lecture Slides

Youtube Lectures

  1. Clustering 6:05
  2. Gaussian Mixture Models 16:34
  3. MLE recap 4:20
  4. Hard k-means Clustering 30:27
  5. Soft k-means Clustering 7:18

17. Expectation Maximization

Lecture Slides

Youtube Lectures

  1. Do we even need EM for GMM? 14:39
  2. A “hacky” GMM estimation 15:17
  3. MLE via EM 38:28

18. Automatic Differentiation

Lecture Slides

Youtube Lectures

  1. Introduction 25:10
  2. Forward Mode AD 26:46
  3. A minute of Backprop 2:26
  4. Reverse mode AD 17:26

19. Nonlinear Embedding Approaches

Lecture Slides

Youtube Lectures

  1. Manifold Learning 20:13

20. Model Comparison I

Lecture Slides

Youtube Lectures

  1. Bias Variance Trade-Off 36:52
  2. No Free Lunch Theorem 7:29
  3. Problems with using accuracy as performance indicator 12:39
  4. Confusion Matrix 25:15

21. Model Comparison II

Lecture Slides

Youtube Lectures

  1. Cross validation and hyperopt 29:08
  2. Expected Value Framework 22:48
  3. Visualizing Model Performance 1 31:02
  4. Receiver Operating Characteristics 22:34

22. Model Calibration

Lecture Slides

Youtube Lectures

  1. On Model Calibration 36:53

23. Convolutional Neural Networks

Lecture Slides

Youtube Lectures

  1. Building Blocks 39:22
  2. Skip Connection 38:46
  3. Fully Convolutional Networks 8:07
  4. Semantic Segmentation with Twists 23:40
  5. Special Convolutions 20:15

24. Word Embedding

Lecture Slides

Youtube Lectures

  1. Introduction 10:35
  2. Semantic Matrix 30:26
  3. word2vec 54:22
comments powered by Disqus