Thread by @betanalpha

It's the weekend so I'm going to rant a little bit about degenerate inferences, why they're so common in machine learning, and why they can't just be ignored. A cathartic thread.

https://twitter.com/ColinJMcAuliffe/status/1330014070016270336

A concept that's popular in machine learning and academic statistics is _universality_, or degrees thereof. As in universal function approximators or models that capture every possible data generating process over an observation space.

Mathematically the idea is to have models that are incredibly flexible -- lots and lots of parameters, little regularization -- so that with infinite data you can recover any signal/truth. It helps that this asymptotic limits facilitates a lot of mathematical analyses.

But the flexibility is a double-edged sword. A model that's flexible enough to fit almost any signal asymptotically will be _too_ flexible for almost any _finite_ observation. An outright profane amount of model configurations will be consistent with any finite data set.

The more flexible the model, and hence more universal it seems asymptotically, the worse this degeneracy will be preasymptotically. For example for any fixed data size the degeneracies will tend to cover more expansive areas of model configurations.

Alternatively the more flexible the model the more data will be needed to achieve any fixed concentration of consistent model configurations into a finite neighborhood.

Importantly these degeneracies don't manifest as spherical fuzziness concentrating around any particular point but rather complex, twisted surfaces that extend far across the model configuration space.

The stronger the degeneracy the less any single model configuration represents all of the model configurations compatible with any given, finite observation. Optimization will often find _a_ reasonable solution, but only by ignoring many others that are almost just as good.

Unfortunately quantifying _all_ of the model configurations consistent with the observed data, and any available domain expertise, is really really important for decision making as the optimal decision are often very sensitive to the model configurations.

This is fine when trying to caption images -- all you need is one good caption, not all of them -- but it becomes critical when informing policy -- a medical treatment that works for a few patients but not most of them is a disaster. Same for social and economic policy.

The frustrating thing is that none if this is particularly hard to see -- hell it's encoded in much of the computational folk lore of machine learning and statistics. Sensitivity to initial conditions? Multiple modes? The need for heuristic regularization?

Honestly I don't take anyone talking about Bayesian neural networks these days seriously. Try to run MCMC and pay attention to the recommended diagnostics and you'll start to see just how nasty, and hard to quantify, neural network likelihood functions are in practice.

Stop complaining that MCMC is slow when your model is ridiculously degenerate. MCMC is like a gas -- it will fill the available space, or at least try to. If there's too much space then this will take forever regardless of how efficient your Markov transitions are.

Oh, but variational Bayes is so much faster? Yeah, because variational Bayes implicitly projects your degenerate model onto a much simpler model space where all of those universally guarantees disappear. Not that anyone will ever actually acknowledge that.

Again all of that can be fine if you're dealing with applications with trivial consequences. But once your analysis will effect real people it's no longer ethical to pretend that data analysis is a fucking engineering task and completely ignore the assumptions you are making.

I once attended a workshop where a famous computer scientist boasted about how his topic model -- a notorious degenerate family of models -- identified a classification of medical pathologies that clinicians found reasonable, and that it was going to revolutionize medicine.

And all of those other model configurations that his variational Bayes fit ignored that the clinicians would also have found reasonable? The ones that would suggest very different interventions and treatments? No, those weren't important to consider.

Ultimately we get this perfect storm of models designed for asymptotic flexibility and model evaluations that focus on point estimates, which in practice manifests in extremely degenerate models and evaluation procedures that ignore that degeneracy entirely.

And now people in machine learning are patting themselves on the back for starting to talk about these problems while continuing to ignore the applied statisticians and practitioners screaming about these consequences forever? <fights growing aneurysm>

I know that due to dwindling resources and increasing competition academia, and some related industries, have become incredibly cutthroat and people have to fight for funding and recognition. I know that it's harder and harder to play that game and fulfill long held ambitions.

But when you start to externalize the costs of sloppy methodologies, pushing hype and ignoring important consequences to fight for your share of the increasing-meager pie? I know my personal opinion means little but I have zero respect for people who make that compromise.

Anyways, that's my weekend rant. I know that I shouldn't expect much better but I'm an idiot and even after all of this time I'm still frustrated, disappointed, and angry.

Shout out to the exceptional people on the front lines fighting against these trends, those refuse to ignore the consequent and toil under-funded and under-appreciated to do what is needed. I hope that my work helps improve your struggle, even if only marginally.