[E] Explain supervised, unsupervised, weakly supervised, semi-supervised, and active learning.

Empirical risk minimization.

[E] What’s the risk in empirical risk minimization?

[E] Why is it empirical?

[E] How do we minimize that risk?

[E] Occam's razor states that when the simple explanation and complex explanation both work equally well, the simple explanation is usually correct. How do we apply this principle in ML?

[E] What are the conditions that allowed deep learning to gain popularity in the last decade?

[M] If we have a wide NN and a deep NN with the same number of parameters, which one is more expressive and why?

[H] The Universal Approximation Theorem states that a neural network with 1 hidden layer can approximate any continuous function for inputs within a specific range. Then why can’t a simple neural network reach an arbitrarily small positive error?

[E] What are saddle points and local minima? Which are thought to cause more problems for training large NNs?

Hyperparameters.

[E] What are the differences between parameters and hyperparameters?

[E] Why is hyperparameter tuning important?

[M] Explain algorithm for tuning hyperparameters.

Classification vs. regression.

[E] What makes a classification problem different from a regression problem?

[E] Can a classification problem be turned into a regression problem and vice versa?

Parametric vs. non-parametric methods.

[E] What’s the difference between parametric methods and non-parametric methods? Give an example of each method.

[H] When should we use one and when should we use the other?

[M] Why does ensembling independently trained models generally improve performance?

[M] Why does L1 regularization tend to lead to sparsity while L2 regularization pushes weights closer to 0?

[E] Why does an ML model’s performance degrade in production?

[M] What problems might we run into when deploying large machine learning models?

Your model performs really well on the test set but poorly in production.

[M] What are your hypotheses about the causes?

[H] How do you validate whether your hypotheses are correct?

[M] Imagine your hypotheses about the causes are correct. What would you do to address them?