I find this reporting infuriating, so I'm going to use it to create a mini-lesson in detecting #AIhype.

If you're interested in following this lesson, please read the article, making note of what you think sounds exciting and what makes you skeptical.

You read it and/or hit a paywall and still want my analysis? Okay, here we go:
First, let's note the good intentions. The journalist reports that mental health services are hard to access (because insufficient, but maybe not only that), and it would be good to have automated systems that help out.
Also, the reporter notes that it would be diagnostically helpful to have externally observable ("objective") indicators of mental health. Not my field, but this is believable to me.
And now we are squarely in the ML techno-solutionism danger zone: It's established that it would be beneficial to have something that can do X with only Y input, but not that it's actually possible to do X with only Y input.
On the other hand, you can always train an ML system that takes ys (elements of Y) as input and gives xs (elements of X) as output and thus LOOKS LIKE it's doing X with only Y input.
So what is the evidence that anything could do X (provide mental health diagnoses) with only Y (voice recordings) input? Our emotional state (depression, anxiety) can affect our speech: Screen cap from linked arti...
So, there might be some signal there. But is it enough to do anything reliable? Under what conditions? (Compare e.g. what needs to be true for accurate blood pressure readings, and the fact that even physiological medical tech is insufficiently tested on non-white, non-men.)
But never fear! The AI can pick up all the details! 🙃🙃🙃 Screen cap from linked arti...
So, we're being asked to believe, here, that not only is it possible to do the thing that we wish could be done, it's possible because "AI" is supposedly better than humans at doing this thing (modeling human emotional states).
Note also the jump from maybe-there's-evidence for anxiety and depression being observable via voice to also diagnosing schizophrenia and PTSD.
It makes me sad to see domain experts being drawn in in this way. I can't tell if Dr. Bentley is being quoted out of context, or if she actually believes the hype. Screen cap from linked arti...
Getting down to the level of individual sentences in this article for a bit, note the work that "perfectly" is doing here. This makes it sound like "AI" is some pre-existing natural phenomenon which just happens to be a good match for this problem. Screen cap from linked arti...
Also, that em-dash makes it hard to tell if this is a bald assertion or part of what some AI researchers believe or what they believe might be the case. I'm guessing the average reader will miss that nuance and read it as a bald assertion. Screen cap from linked arti...
Another type of #AIhype shows up subtly: the framing "little human oversight" which suggests autonomy on the part of the system. So-called "AI" systems are only artifacts, but the more they are hyped as autonomous agents, the easier it is to believe that they can do magic. Screen cap from linked arti...
The article does point out that for "mainstream" use, the technology would have to be tested to medical standards. Quoting Dr. Bentley again: Screen cap from linked arti...
But at the same time, the article is referring to apps that are already commercially available---the journalist tested two of them on herself. So I guess "mainstream" really means only "within the medical establishment" here?
And this brings me to my final point: dual use. If these systems are out there, purporting to measure aspects of mental health (one is called Mental Fitness, ffs) on the basis of short recordings, who else is going to use them, on whom, and to what ends?
I want to see all reporting on applications of so-called "AI" asking these questions. Can it be used for surveillance? Can it be used for stalking? How might the claims being made by the developers shape the way it could be used?


• • •

Missing some Tweet in this thread? You can try to force a refresh

Keep Current with Emily M. Bender

Emily M. Bender Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!


Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @emilymbender

Sep 19
This article in the Atlantic by Stephen Marche is so full of #AIhype it almost reads like a self-parody. So, for your entertainment/education in spotting #AIhype, I present a brief annotated reading:


Straight out of the gate, he's not just comparing "AI" to "miracles" but flat out calling it one and quoting Google & Tesla (ex-)execs making comparisons to "God" and "demons".

/2 Screencap from linked article: "Miracles can be perplex
This is not the writing of someone who actually knows what #NLProc is. If you use grammar checkers, autocorrect, online translation services, web search, autocaptions, a voice assistant, etc you use NLP technology in everyday life. But guess what? NLP isn't a subfield of "AI".
/3 Screencap, same article: "Early artificial intelligence
Read 25 tweets
Aug 25
This piece is stunning: stunningly beautifully written, stunningly painful, and stunningly damning of family policing, of the lack of protections against data collection in our country, & of the mindset of tech solutionism that attempts to remove "failable" humans decision makers
.@UpFromTheCracks 's essay is both a powerful call for the immediate end of family policing and an extremely pointed case study in so many aspects of what gets called #AIethics:

1. What are the potentials for harm from algorithmic decision making?

2. The absolutely essential effects of lived experience and positionality to understanding those harms.
3. The ways in which data collection sets up future harms.

Read 5 tweets
Aug 24
Just to recap the morning so far (admittedly, some of these news stories are from a couple of days ago):
Read 5 tweets
Aug 11
Read the recent Vox article about effective altruism ("EA") and longtermism and I'm once again struck by how *obvious* it is that these folks are utterly failing at ceding any power & how completely mismatched "optimization" is from the goals of doing actual good in the world.
Just a few random excerpts, because it was so painful to read...

"Oh noes! We have too much money, and not enough actual need in today's world."

First: This is such an obvious way in which insisting on only funding the MOST effective things is going to fail. (Assuming that is even knowable.)

>> Screencap reading: "EA...
Read 19 tweets
Jul 26
In Stochastic Parrots, we referred to attempts to mimic human behavior as a bright line in ethical AI development" (I'm pretty sure that pt was due to @mmitchell_ai but we all gladly signed off!) This particular instance was done carefully, however >>

@mmitchell_ai Given the pretraining+fine-tuning paradigm, I'm afraid we're going to see more and more of these, mostly not done with nearly the degree of care. See, for example, this terrible idea from AI21 labs:


@mmitchell_ai As Dennett says in the VICE article, regulation is needed---I'd add: regulation informed by an understanding of both how the systems work and how people react to them.


>> Screenshot from linked article, reading: "“Mainly it
Read 6 tweets
Jul 25
Thinking back to Batya Friedman (of UW's @TechPolicyLab and Value Sensitive Design Lab)'s great keynote at #NAACL2022. She ended with some really valuable ideas for going forward, in these slides:

Here, I really appreciated 3 "Think outside the AI/ML box".

>> Screenshot of slide, with t...
As societies and as scientific communities, we are surely better served by exploring multiple paths rather than piling all resources (funding, researcher time & ingenuity) on MOAR DATA, MOAR COMPUTE! Friedman points out that this is *environmentally* urgent as well.

Where above she draws on the lessons of nuclear power (what other robust sources of non-fossil energy would we have now, if we'd spread our search more broadly back then?) here she draws on the lessons of plastics: they are key for some use case (esp medical). >> Screenshot of slide, with t...
Read 7 tweets

Did Thread Reader help you today?

Support us! We are indie developers!

This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Don't want to be a Premium member but still want to support us?

Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal

Or Donate anonymously using crypto!


0xfe58350B80634f60Fa6Dc149a72b4DFbc17D341E copy


3ATGMxNzCUFzxpMCHL5sWSt4DVtS8UqXpi copy

Thank you for your support!

Follow Us on Twitter!