This list was used for the ILP lab’s reading group on Kevin Murpuy’s Probabilistic Machine Learning: An Introduction

• Page 4: the importance of exploratory data analysis
• Page 6: the difference between empirical risk and expected risk
• Page 7: two types of uncertainty
• model uncertainty: a model cannot perfectly predict the exact output given the input, due to lack of knowledge of the input-output mapping
• data uncertainty: intrinsic (irreducible) stochasticity in the mapping
• Page 8: the intuition of MLE
• Page 8: the difference between $$\ell_2$$ loss and $$\ell_1$$ loss (in the footnote)
• Page 9: when the error term follows a Gaussian distribution, then MLE is minimizing the sequared error
• further: Bayesian version
• Page 11: interpolation of data when the degree of polynomial $$D$$ is $$D=N-1$$, where $$N$$ is the number of examples — overfitting
• Page 12:
• the over-simplified viewpoint of deep nerual networks
• overfitting and generalization
• Page 15:
• factor analysis vs. PCA
• PCA vs. variational autoencoder
• Page 16:
• unspervised learning methods should be interpretable, if the goal is to gain understanding
• Page 25:
• equation 1.39 gives two different views of word embeddings
• Page 27:
• difference: Statistics vs. Machine Learning vs. AI
• reward hacking