Thread by @samarth_b

Ever caught yourself anthropomorphizing your ML model's behaviour? It happens to me when discussing results with my colleagues. I admit my internal rigour police isn't happy doing it. There must be a more grounded statistical analysis to make the same point.

Usually, our lunch time discussion on this topic goes in the direction of Al vs ML. I refer you to Micheal Jordan (Berkeley prof)'s famous rant that highlights the dangers of calling neural network generalization behaviour "intelligence".

Calling an ML model "it" is perhaps a greater transgression, associating not only intelligence but also a biased personality 😂

I came across a new argument in an unlikely podcast guest Jordan Ellenberg while describing the beauty of prime numbers and why number theorists have been fascinated for millennia. As if anyone ever doubted primes are awesome!

He argues that several proofs and thought experiments are made intuitive by posing the occurrence of prime numbers as a random event. Even though it most definitely isn't random.

So as much as it hurts the purist to think that way, some intuitions are best conveyed by **agreeing** to embrace a distortion in facts!

I urge you to check out the link to the conversation here:
Where they delve into how the occurrence of primes is an analogic argument for the lack of free will in the universe!

To my original point, beyond conveying ideas informally around bias, are there other dangers to speaking about model behaviour in this way? Certainly we may induce our implicit biases to the model or experimentation.

So if possible it is always better to back a hypothesis with statistical analysis of behaviour. We often use a small less complicated model to analyse these ideas.

This is further reinforced by those of us who work in the areas of perception, i.e., vision, language and speech. Which is what my team does. Perhaps in more structured data like finance, inducing human bias might be a good thing!

There might be further debate if we are looking at model output on target data vs. ensuing explainability analysis. These are white/black box analyses designed to understand the models behaviour. It really depends on how they work, so it's difficult to make a general comment.

If the work of D. Kaheman and others tell us anything it is that humans can be extremely biased. So can a biased intelligent agent remove bias from other intelligent agents? If only ML models can update with few examples the way we do! Oops, I hit upon a fundamental ML problem!

O wonder what it means to the theory community of ML to embrace or assume that an ML model is "generally intelligent". Like in primes/number theory, I wonder if it can lead to provocative proofs or thought experiments?

The most fundamental view, assuming humans are also an ML model looking at the same data, is that we extract and augment features differently so comparison or any assumptions of "prediction rationale" is likely incompatible and best to avoid.

So as Jorden Ellenberg puts it, to the extent that it makes it easy to convey and digest each other's ideas, should be alright to call the model "it". But careful not to attribute intelligence or you will piss off his Berkeley namesake.