Some literary theory and AI person who's not fixated on autoencoders like I am should really write on how transformers (BERT, GPT and so on) never 'extract' meaning from a text but rather 'charge' the text's words with relational significance and then 'reread' the text
A little bit of Stein, a little bit of Derrida, a little bit of 'hermeneutic circle' -- could be a good time
I think there's legit interesting stuff here about the persistence of a text as text in our cognition of it, if transformer-like architectures really do prove indispensable in the long term
The underlying technical thing isn't fancy, btw: older architectures (RNNs) have a fixed-length 'the text so far' representation that they update after every word, but newer architectures (transformers) don't
Instead, the transformer assigns every word-token in the text a 'the significance of this word-token in light of the significance of the other word-tokens in the text' annotation, which it repeatedly updates in light of itself

• • •

Missing some Tweet in this thread? You can try to force a refresh

Keep Current with Peli Grietzer

Peli Grietzer Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!


Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @peligrietzer

22 Feb
In theory you can generate an infinite amounts of edge by cyclically adding and removing 'The' from the name of a superhero film
Can we use this to solve the energy crisis? I'm no physicist, but yes
I deleted and reposted OP 8 times to correct a typo, but the typo is still with us
Read 4 tweets
22 Feb
I love how characters in Frankenstein (1818) blow each other's minds with 'Western science is historically a spinoff of alchemy and Cabala' takes, just like we do today
A totally correct genre of takes that somehow kept its galaxy-brain oomph for centuries
I'd actually love to know if Shelley adapted that stuff from a known source or was an early contributor to that discourse or what
Read 4 tweets
21 Feb
I think that people like Scott Alexander come to scientific racism through faith that the rise of the nerds untwists a twisted world. Faced with the observation that the rise of the nerds isn't lessening racial disparities, they conclude that racial disparities aren't twisted
If there were data that Black people do much better in tech startups than in trad elite careers, people like Alexander would instead be all 'nerd culture overcomes the oppressive racial hierarchies of the normies that kept Black people down for hundreds of years'
Slate Star Codex is a single-issue ideology, and that issue is 'nerds good'
Read 8 tweets
20 Feb
If you aspire to do or embody something rare, and you're really hardcore about it, it's hard not to fall into a sense that most lives aren't worth living
There are at least two things here, maybe
One is that if you think about lives in a way that's like 'a life is good to the degree that it's a thread that makes the tapestry of human lives more good,' then you can reconcile a lot of contradictions in that neighborhood
Read 5 tweets
2 Jan
Can people just admit Diane di Prima is a fascist poet or nah. It's OK to be a fascist poet, lots of really good ones!
Come the fuck on
Read 18 tweets
1 Jan
The thing that's not emotional labor that people call emotional labor -- labor managing others' emotions -- is a good concept and does have a name: affective labor
If my boss has me put up Christmas decorations in the office to improve morale, that's affective labor but not (uniquely) emotional labor
If working alone in the office at night makes me want to start a fire but I grit my teeth through it, that's emotional labor but not (uniquely) affective labor
Read 14 tweets

Did Thread Reader help you today?

Support us! We are indie developers!

This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Too expensive? Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal Become our Patreon

Thank you for your support!

Follow Us on Twitter!