__Chapter 01 Highlights for Discussion__

This list was used for the ILP lab’s reading group on Kevin Murpuy’s Probabilistic Machine Learning: An Introduction

- Page 4: the importance of exploratory data analysis
- Page 6: the difference between empirical risk and expected risk
- Page 7: two types of uncertainty
- model uncertainty: a model cannot perfectly predict the exact output given the input, due to lack of knowledge of the input-output mapping
- data uncertainty: intrinsic (irreducible) stochasticity in the mapping

- Page 8: the intuition of MLE
- Page 8: the difference between \( \ell_2 \) loss and \( \ell_1 \) loss (in the footnote)
- Page 9: when the error term follows a Gaussian distribution, then MLE is minimizing the sequared error
- further: Bayesian version

- Page 11: interpolation of data when the degree of polynomial \( D \) is \( D=N-1 \), where \( N \) is the number of examples — overfitting
- Page 12:
- the over-simplified viewpoint of deep nerual networks
- overfitting and generalization

- Page 15:
- factor analysis vs. PCA
- PCA vs. variational autoencoder

- Page 16:
- unspervised learning methods should be interpretable, if the goal is to gain understanding

- Page 25:
- equation 1.39 gives two different views of word embeddings

- Page 27:
- difference: Statistics vs. Machine Learning vs. AI
- reward hacking