Thread by @mrdbourke

If machine learning projects were a relationship...

Data collecting and processing is the dating phase, fun, chaotic, up and down, tormenting and carefree, seeing if you're a good fit.

Modelling is the wedding day, takes forever to plan, over before you know it.

People using your model is the honeymoon.

Then comes the data drift.

Your data changes like the person you thought you married, maybe they're getting fat (distribution changes) or they're finding it hard to love you (your data features are no longer ideal).

So you bring in data monitoring, model evaluation (marriage counselling) and pull all the tricks.

Your marriage counsellor tells you to go back to what got you started.

The fun dates (collecting data), talking for hours learning about each other (processing data).

And you start to rebuild, you make a better model than ever before.

Mixing old data, new data, ideas built on experience, wisdom from seeing all the new SOTAs (other relationships) and realising the only real one you should focus on is your own.

You decide to produce some offspring in the hopes together they end up surpassing you (ensembles).

But you're always reminded.

Day in day out.

Tweaking your learning rate can help.

But nowhere near as much as better data (communication).

And even though you're older now, life feels like it's getting faster.

But that's just the new GPUs. They're faster but your model's bigger.

You remember when batch sizes were 32. The good old days.

No matter. Keep going. One epoch at a time.

It's worth it.