Thanks for the ping, @michaelbrundage
I don't think there's anything specific to LLMs here. Rather, this is endemic to the way ML is applied these days:
I don't think there's anything specific to LLMs here. Rather, this is endemic to the way ML is applied these days:
https://twitter.com/michaelbrundage/status/1434492389767278595
1. Someone creates a dataset & describes it as a benchmark for some skill. Sometimes, the skill is well-scoped & the benchmark represents a portion of it reasonably (eg ASR for news in X language). Others, the skill is bogus (all the physiognomy tasks; IQ from text samples; &c).
2. ML researchers use the benchmarks to test different approaches to learning. This can be done well (for well-scoped tasks): which algorithms are suited to which tasks and why? (Requires error analysis, and understanding the task as well as the algorithm.)
3. The focus shifts from understanding how learning algorithms relate to different kinds of data/tasks to leaderboardism. Many progress, much arXiv, wow!
4. In this research paradigm, no time is spent on critical analysis of benchmarks, including the data that make them up but especially also the task definitions.
5. #AIhype kicks in and the research is framed as "solving" such grand challenges as "language understanding" or "visual understanding" or...
6. That critical analysis work is happening, in at least two places: Within ML, like the work you point to, with typically a narrow lens ("hey look, the labels are bad!") but maybe getting some attention from the #AIhype crew.
But more important versions of the work, that critique the underlying conceptions of tasks and goals and claims tends to be marginalized: seen as lower on the 'hierarchy of knowledge' (see @timnitGebru's talk at Spelman ).
The folks who think of themselves as at the top of that hierarchy because they are the ones who create & apply the algorithms seem to largely dismiss (or just ignore) the work of people like Birhane, Gebru, Benjamin, Noble, Mitchell, Raji, Whittaker.
Note that some of this critical work is coming from people with deep training in ML, Gebru, Mitchell & Raji among them, but I think it's still an uphill battle to have it be taken as serious research by researchers who think of themselves as 'core ML'.
So we get:
MLbro: Look, I've solved language!
Critical researcher: No, you haven't. That task is bogus.
Rev2: But you didn't show what other task would prove the claim, so reject.
MLbro: Look, I've solved language!
Critical researcher: No, you haven't. That task is bogus.
Rev2: But you didn't show what other task would prove the claim, so reject.
Or:
MLbro: Look, I can predict criminality from faces!
Critical researcher: No you can't and claiming you can feeds into racism/other systems of oppression.
Rev2: Keep your politics out of our conference.
MLbro: Look, I can predict criminality from faces!
Critical researcher: No you can't and claiming you can feeds into racism/other systems of oppression.
Rev2: Keep your politics out of our conference.
But this works:
MLbro: Look I've solved language!
Other ML researcher: Well actually, the dataset in that benchmark is really messy and maybe you're just modeling noise.
Rev2: Okay, I guess we'll let this one through.
MLbro: Look I've solved language!
Other ML researcher: Well actually, the dataset in that benchmark is really messy and maybe you're just modeling noise.
Rev2: Okay, I guess we'll let this one through.
If ML/AI were just an obscure academic field this might not matter so urgently, but that's not the world we live in. The hierarchies of knowledge (see ⬆️) are closely modeled by hierarchies of funding and meanwhile techcos are pushing out AI snakeoil that is doing real harm.
Further reading/watching:
• • •
Missing some Tweet in this thread? You can try to
force a refresh
