.@jeffdean is quoted here saying about Stochastic Parrots that “surveyed valid concerns with large language models, and in fact many teams at Google are actively working on these issues.”

>>
@JeffDean (His earlier comments that Stochastic Parrots “didn’t meet our bar for publication” are also cited --- nvm that it was published, after **anonymous** peer review, and that the PALM paper is just a preprint...)

>>
Anyway: "Google is actively working on these issues" is not a satisfactory response to the concerns that we & others have raised, esp when they come out with papers like the PaLM paper which are so utterly sloppy in how they handle ethical considerations.

... or when they're coming out with a steady stream of really unhinged techno-chauvinism, in education:



>>
... and in attempted (unregulated!) public health responses to mental health and domestic violence crises:



>>
Btw, "Google is aware and working on these issues" is also what @chirag_shah and I heard in response to our #CHIIR2022 paper,



>>
For those who'd like the tl;dr on that paper, here's our Q&A with @uwnews

Despite attempting to organize "all the world's knowledge", researchers at @Google are not in fact exempt from the processes of peer review if they want to be part of the scientific community.

>>
@Google "Google is aware of the issues and is working on it," especially when framed as a reason that critical work shouldn't be published or shouldn't be heeded, just doesn't cut it. If you're working on it, good: show us the results. Anything else just sounds defensive & petulant.

• • •

Missing some Tweet in this thread? You can try to force a refresh
 

Keep Current with Emily M. Bender

Emily M. Bender Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

PDF

Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @emilymbender

Apr 6
I find this reporting infuriating, so I'm going to use it to create a mini-lesson in detecting #AIhype.

If you're interested in following this lesson, please read the article, making note of what you think sounds exciting and what makes you skeptical.

nytimes.com/2022/04/05/tec…
You read it and/or hit a paywall and still want my analysis? Okay, here we go:
First, let's note the good intentions. The journalist reports that mental health services are hard to access (because insufficient, but maybe not only that), and it would be good to have automated systems that help out.
Read 19 tweets
Apr 4
This paper claims to follow best practice by including a datasheet and a model card, but really all it does is reinforce that 780B tokens is WAY too big to be sufficiently documented. >>

storage.googleapis.com/pathways-langu…
Dataset analysis done on a randomly sampled 1% of the data finds PII in >30% of the data points. No word about how to remove that. (p.66)

>>
Counts of third person pronouns across all of the English data finds 5.5x as many he/him as she/her (p.67) but whatevs, right?

>>
Read 15 tweets
Mar 16
Reading the linked blog post, it's all kinds of creepy. Just a couple of examples:
First, I'm super skeptical that learning math by doing problem sets is a good model for learning other kinds of things. And even if it were, the idea that LLMs would support that generalization seems super sketchy. "Imagine you’re a student stuck on a math problem. Wi
What, specifically, is the system doing to get the student "unstuck" in their non-math assignment? What role does the LLM play? How does the way that LLMs absorb various societal biases from their training data affect performance?
Read 7 tweets
Mar 15
Yes, this is great! I'm sorry we didn't find your paper while writing ours. (cc @chirag_shah)

A few favorite quotes & one quibble:
@chirag_shah Potthast et al (from @webis_de) suggest a standard disclaimer on direct answer responses, which is very well put:

“This answer is not necessarily true. It just fits well to your question.”

dl.acm.org/doi/abs/10.114…

>>

>>
@chirag_shah @webis_de Also Potthast et al: "As no actual conversations are currently supported by conversational search agents, every query is an ad hoc query that is met with one single answer."

No. Actual. Conversations.

There's a whole study to be done on the perils of aspirational tech names.>>
Read 6 tweets
Mar 15
LSA 2003, Newmeyer's keynote. I was a few years in to my academic job search and attending the conference with my 9 month old son and my mom to look after him.

>>
Mom was pushing the baby around the hotel ballroom level hallways in the stroller trying to keep him happy, but he was NOT happy and I could hear him.

>>
I got up from my seat in the standing-room-only crowd to go check on him, and when I came back, ended up standing at the back of the room, next to Andrew Garrett, who I knew from the one-year stint I did at Cal in 2000-2001.

>>
Read 9 tweets
Mar 14
Trying to attend "Conversational Information Seeking: Theory and Evaluation (Session 1)" at #CHIIR2022, but the Zoom link in the conference room in Gather isn't working. Anyone have a clue?
Also, there don't seem to be any papers listed in that session, nor "Conversational Information Seeking: Theory and Evaluation (session 2)" this afternoon. Maybe these are just phantom calendar entries? #CHIIR2022 what's going on?
I find it very nerve wracking when the interfaces to online conferences are unclear ... like I'm meant to be somewhere, but I can't figure out where, nor can I figure out why I can't figure it out. Also, no helpdesk that I can see, so no one to ask... #chiir2022
Read 4 tweets

Did Thread Reader help you today?

Support us! We are indie developers!


This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Don't want to be a Premium member but still want to support us?

Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal

Or Donate anonymously using crypto!

Ethereum

0xfe58350B80634f60Fa6Dc149a72b4DFbc17D341E copy

Bitcoin

3ATGMxNzCUFzxpMCHL5sWSt4DVtS8UqXpi copy

Thank you for your support!

Follow Us on Twitter!

:(