Discover and read the best of Twitter Threads about #divergences4lyfe
Most recents (1)
What a wonderful opportunity to talk about experimental design and the subtleties of what overfitting means in a Bayesian context! A "I've been consulting all day and my brain is too tired to do real work" thread.
Overfitting and identifiability are intimately related concepts. When you have a complex model but only small data sets then there will be _many_ model configurations consistent with the little data that you observed.
If your inferences are quantified by a point estimate then you will have to choose a single point amongst the entire subset of model configurations that are similarly consistent with any given observation. Any choice of a single point, however, is unlikely to generalize well.
