Lectures

You can download the lectures here. We will try to upload lectures prior to their corresponding classes.

• 

  Course Overview
  tl;dr: A weclome to the world of machine learning: the good, the bad, and the ugly.
  [slides]
  

  Suggested Readings:

  • CIML, Chapter 1
  • Atri’s 440 Pipeline Walkthrough
  • Quick “What is ML” Video
  • Important Alternative “What is ML” Video
• 

  Review - Syllabus, Probability, and some python
  tl;dr: Review - Syllabus, Probability, and some python
  [notes] [slides]
  

  Suggested Readings:

  • The course syllabus
  • Seeing Theory, Probability/Stats Review
  • Probability Cheatsheet
• 

  Stats and Intro to Linear Regression
  tl;dr:
  [notes (stat)] [notebook (regression)] [notebook (demo)] [COVID news data] [notes (regression)]
  

  Suggested Readings:

  • Installing Jupyterlab
  • Chapter 1, Shalizi Advanced Data Analysis
• 

  Linear Regression, continued
  tl;dr: Optimization, some evaluation, and some real-world practice
  [notebook (regression)] [notebook (demo)] [notes (regression)] [notes (demo)] [COVID news data]
  

  Suggested Readings:

  • Dr. Chandola’s Linear Regression Notes
  • Hunter Schafer’s CSE/STAT 416, Ch. 1
• 

  Linear Regression, Cont.; Model Eval. Pt 1
  tl;dr: Gradient Descent for Linear Regression, Intro to the Bias/Variance Tradeoff
  [UW 416 Eval Notes] [Raschka Bias/Variance (Slides 1-34)]
  

  Strongly Suggested Readings:

  • CIML Chapter 2, up to 2.6
  • Hunter Schafer’s CSE/STAT 416, Ch. 2

  Suggested Reading

  • sklearn regression tutorial
• 

  Model Eval, Pt. 2
  tl;dr: Interpreting Coefficients of linear models, understanding the bias/variance tradeoff in code
  [notebook] [notes] [Raschka's Lecture on Evaluation]
  

  Suggested Reading

  • Sebastian Raschka Model Eval Notes
• 

  Model Eval. Pt 3
  tl;dr: Selecting and Evaluating Models w/ and w/out hyperparameters using the Holdout method, Cross Validation, and/or the Bootstrap
  [notebook (cont. Real-world Example)] [notebook html (cont. Real-world Example)]
  

  Required Watching

  • StatQuest, Decision Tree Regression

  Suggested Reading

  • CIML, 3.1-3.2, 3.5
  • Reminder Lecture notes are on UBLearns
• 

  Regularization and Additive Models
  tl;dr: Regularization and generalized additive models
  [notebook (cont. Real-world Example)] [notebook html (cont. Real-world Example)]
  

  Suggested Reading/Watching:

  • StatQuest on the bootstrap
  • StatQuest on Ridge/Lasso
  • Armando Teixeira on GAMs
  • Reminder Lecture notes are on UBLearns
• 

  Probabilistic ML and Intro to Classification
  tl;dr: Revisiting Probabilistic ML + Maximum Likelihood
  [annotated_slides]
  

  Suggested Readings:

  • Cornell Intro ML Lecture 1
  • CIML, 9.1-9.4
  • Cornell Into ML Lecture 4
  • Reminder Lecture notes are on UBLearns
• 

  Classification and Logistic Regression
  tl;dr:
  [slides] [annotated slides]
  

  Suggested Readings:

  • CIML, Ch. 7
  • Tom Mitchell’s ML Book - Naive Bayes and Logistic Regression
  • Cornell Into ML Lecture 6
• 

  Logistic Regression (cont.), Decision Trees, and Ensemble Methods
  tl;dr:
  [slides] [Tree example code (Python Data Handbook)]
  

  Required Viewing

  • StatQuest on Decision Trees for Classification
  • StatQuest on Random Forests
  • StatQuest on AdaBoost

  Suggested Readings:

  • Cornell Into ML Lecture 5
  • StatQuest on MLE for Logistic Regression
  • L18/19 Cornell ML
• 

  Review Activity
  tl;dr:
  
  

  Details

  • The review will be Jeopardy-style
  • You will participate in your PA groups
  • Winners will receive prizes, but you must be present to receive your prize
  • The last 20 or so minutes will be open review; please bring questions if you have them.
• 

  Annotation and Evaluation
  tl;dr: Annotation and Evaluation
  [annotated slides]
  

  Required Readings

  • CIML, Section 5.5

  Suggested Readings

  • Data-centric AI
  • A python Library for Computing Krippendorf’s Alpha
  • Computing Krippendorf’s Alpha

  For the curious reader

  • Cheap and Fast, But is it Good?
  • Comparing Bayesian Models of Annotation
• 

  Clustering with K-Means
  tl;dr:
  [annotated slides] [notebook] [notebook html]
  

  Required

  • StatQuest K-Means

  Suggested Readings:

  • Bishop, Section 9.1
  • Visualizing K-Means
  • CIML Ch.15
• 

  Gaussian Mixture Models and the EM Algorithm
  tl;dr:
  [annotated slides] [notebook] [notebook html]
  

  Suggested:

  • Jordan Boyd-Graber on GMMs
  • Bishop, Section 9.3
• 

  Agglomerative Clustering, Missing Data, Overflow
  tl;dr:
  [annotated slides] [code+data zip]
  

  Required:

  • StatQuest, Hierarchical Clustering
• 

  Dimensionality Reduction with PCA
  tl;dr:
  [code] [annotated slides]
  

  Required:

  • StatQuest, PCA Basics

  Suggested:

  • Longer StatQuest, PCA
  • PCA, Visualized

  Curious:

  • 3Blue1Brown Eigenvectors
  • UMAP
• 

  Midterm Solutions
  tl;dr:
  
  

  See UBLearns for relevant documents

• 

  Bayes Theorem, Bayesian Stats, and Probabilistic Programming
  tl;dr:
  [annotated slides]
  

  Required:

  • StatQuest, Bayes Theorem

  Suggested:

  • BDA, Sections 1.3-1.4
  • Prob. Programming and Bayesian Methods For Hackers, Ch. 1: Example: Inferring behaviour from text-message data
• 

  Directed Graphical Models and Inference of Bayesian Models (briefly)
  tl;dr:
  [annotated slides]
  

  Suggested:

  • Cornell Intro ML, Lecture 4
  • MML, Section 8.5
• 

  Deep Learning 1: Overview
  tl;dr:
  [annotated slides]
  

  Required:

  • StatQuest, Neural Networks
  • StatQuest, Neural Nets, Multiple Input/Outputs

  Suggested:

• 

  Deep Learning 2: Convolutional Neural Networks
  tl;dr:
  
  

  Required:

  • Slides are on UBLearns
  • StatQuest, ReLU
  • StatQuest, Convolutional Nets

  Suggested

  • Chris Olah, ConvNets
• 

  Deep Learning 3: Backpropagation, Applications, and a Heavy Dose of Humility
  tl;dr:
  
  

  Required:

  • Slides are on UBLearns
  • StatQuest, BackProp

  Strongly Suggested

  • Chris Olah, Backprop
  • Sebastian Raschka, Backprop

  For the Curious

  • Why is training deep networks hard?
• 

  Bias and Justice in ML, Pt. 1: Overview and NLP Example
  tl;dr:
  [annotated slides]
  

  Required:

  • Fair Warning, Abebe Birhane

  Strongly Suggested

  • Bias in NLP
  • Emily Bender on the recent NYT GPT-3 article

  For the Curious

  • Reading/Watching/Listening List for ML&Soc UB Course
  • The Stochastic Parrots paper
• 

  Bias and Justice in ML, Pt. 2: Beyond "Fairness"
  tl;dr:
  [annotated slides]
  

  Required:

  • Fair Warning, Abebe Birhane

  For the Curious

  • Calculating the Souls of Black Folk
  • The Root Of Algorithmic Bias And How To Deal With It
• 

  Quiz reviews, A little causal inference, AMA
  tl;dr:
  [annotated slides]
  

  For the Curious

  • Causal Inference: The Mixtape
  • DoWhy/EconML: Library for computational inference
• 

  Review
  tl;dr:
  [annotated slides]