This list was used for the ILP lab’s reading group on Kevin Murpuy’s Probabilistic Machine Learning: An Introduction

  • Page 4: the importance of exploratory data analysis
  • Page 6: the difference between empirical risk and expected risk
  • Page 7: two types of uncertainty
    • model uncertainty: a model cannot perfectly predict the exact output given the input, due to lack of knowledge of the input-output mapping
    • data uncertainty: intrinsic (irreducible) stochasticity in the mapping
  • Page 8: the intuition of MLE
  • Page 8: the difference between \( \ell_2 \) loss and \( \ell_1 \) loss (in the footnote)
  • Page 9: when the error term follows a Gaussian distribution, then MLE is minimizing the sequared error
    • further: Bayesian version
  • Page 11: interpolation of data when the degree of polynomial \( D \) is \( D=N-1 \), where \( N \) is the number of examples — overfitting
  • Page 12:
    • the over-simplified viewpoint of deep nerual networks
    • overfitting and generalization
  • Page 15:
    • factor analysis vs. PCA
    • PCA vs. variational autoencoder
  • Page 16:
    • unspervised learning methods should be interpretable, if the goal is to gain understanding
  • Page 25:
    • equation 1.39 gives two different views of word embeddings
  • Page 27:
    • difference: Statistics vs. Machine Learning vs. AI
    • reward hacking