An interview with Prof. Mikhail Belkin

In this exciting interview with Prof. Mikhail Belkin, we go over one of his publications entitled: “Reconciling modern machine-learning practice and the classical bias–variance trade-off”.

00:51 Prof. Belkin introduces himself

01:27 The research interests of Prof. Belkin

02:35 What is the story behind the paper and how did they even think about challenging the traditional single-descent U-shape plot of bias-variance trade-off?

05:40 The double-descent behavior of the interpolation models.

06:56 What if over-parametrized models did not have any regularization mechanism? Would they still generalize well (according to the double-descent behavior found in this paper)?

10:34 What is this idea of Interpolation Threshold and can it be computed mathematically?

14:32 What is Inductive Bias and how can we control it? (related to the ‘smoothness’ which is chosen as the inductive bias in the paper)

17:31 How did you determine that ‘smoothness’ is the right inductive-bias for the problems you have worked on in the paper? What if it was not? Would we still see the double-descent behavior?

25:57 The bottom of the second descent (for an over-parametrized model) does NOT necessarily correspond to less Test Error (better generalizability), compared to the bottom of the traditional U-shape model (for an under-parametrized model).

29:60 If we were not able to create highly rich models, maybe we would not have never questioned the U-shape model. What other well-founded theoretical foundation is out there (similar to the old regime and the U-shape behavior) that you think are ONLY half the picture and need us to investigate them more?

36:46 A lot of biological models out there are over-parametrized!