Overfitting is probably the most important concept that is missing from mainstream econometrics classes. Caltech prof Yaser Abu-Mostafa has an incredible intro lecture to overfitting here from his ML course:
If you take metrics with me, of course, you learn about overfitting. But often, this lesson is painful: it contradicts the powerful instinct that one should prefer complex, ideally nonparametric models to simple models that place strong structure on the data.
Overfitting affects all inference problems in economics because it arises when the true data-generating process is too complex for you to feasibly capture given the data and the tools that you have, and this is always the case in social science.
As the lecture shows, even using *the true DGP* as your statistical model can perform really badly if the sample you have is not up to the task (too small, too noisy, etc). You're sometimes better off using a woefully oversimplified model that you know is incorrect.
And the actual amount of information you have to estimate the structure you are interested in doesn't just depend on the sample size, but also the actual characteristics of the data itself as well as how much of the variation in the data you are willing or able to exploit.
You might think a linear regression model won't overfit, and it's true that you'd be more in danger with splines or whatever. But if you pump that linear model full of enough covariates (especially fixed effects) you're actually fitting a pretty complex model -- maybe too complex
a covariate is just a spline your data makes
*sung to the tune of "a dream is a wish your heart makes"*
