google bard Profile picture
May 25, 2021 25 tweets 6 min read Read on X
I think the reason I'm obsessed with CLIP is because it's hard evidence for unified meme theory
unified meme theory states that the "meme", defined as the transmissible unit of human thought, and defined has a picture with some words on it that gets copied on the internet, are not different.

a meme is picture+words because it represents a gradient in semantic space
semantic space is a much-theorized "embedding" for language concepts. Like if you think about a chair, and then you think about a chaise, and then you think about a silla... there's a region in semantic space that activates from all of those. in your mind and mine
the image provides a field, an area in semantic space. the words provide a vector. Your attention moves to that area of semantic space, and then updates in the direction that the vector points. a gradient in semantic space.
depending on how far the words send you, and how well developed your map of that semantic space, either it lands or it doesn't. if it works on you, you save it to send it to someone else later.

showed this one to gf, she made me send it to her so she can send it to her dad Image
but it has to be the right person: you have to simulate their whole mental state to understand whether this gradient will work on them. so memes follow social topologies as well.

this makes sense, though: semantic space is a product of social animals using mimetic calls
CLIP encodes visual image and text about that image into the same embedding space. You've probably seen this image, maybe with a caption that says "AI is too dumb to tell the difference between an apple and an iPod lol"

but this is actually amazing Image
This neural net learned to read. It learned to read handwriting! it's not very good at it, but it wasn't trained to do that. it was trained to match flashcards of images to text captions.

Is this not mostly a picture that says iPod? with a hint of apple and a dash of wood fence
The fact that CLIP can recognize letters in photographs Is a sign that they're encoded in the brain the same way as other visual data. they're just a bunch of weird squiggles, but they cause the meaning of the image to change.

in predictable ways!
there's a high dimensional space representing all these concepts and how they interact.

Your brain evolved to think about how other monkeys think about other other monkeys. CLIP is trained on contrastive pretraining objective. we are kinda the same
This is a great thread showing how CLIP vectors encode world knowledge and how you can add and subtract them

if you feel like you came up with these ideas independently, that's probably true! we're literally studying our own minds here
incidentally, this is why CLIP is perfect for meme search. I'd been wanting to make this program for a long time, but until now there wasn't a way to search for concepts embedded in both text and visual data. now there is

github.com/deepfates/meme…
I thought I was going to have to combine a bunch of things, OCR the text, object recognition on the images, combine a bunch of metadata and manual tags to make a classifier or something. but with CLIP it "just works" 😘
unified meme theory draws on this paper, "Embodiment vs. Memetics," by Joanna Bryson. she says humans have a special combination of temporal imitation and second order thinking that creates semantic space fertile for memetic vectors

preprint pdf here cs.bath.ac.uk/~jjb/ftp/memet…
Image
temporal imitation: like birds, we have the ability to repeat short, ordered phrases of sound. birds do their little mating calls and dances, humans do too.

most of the types of creature don't do this call and response. So they don't have imitatable units of action
second order thinking: our heritage as troop primates means we think not only about the internal state of our fellows, but about their internal models of our state! or even of third parties.

this creates a hall of mirrors effect. You can go fractally in any direction
The hall of mirrors is filled with the imitatable units of action, in all of their infinite slight variations.

it's a holographic construct, a multi-dimensional space intersecting with a smaller dimensional reality. that's semantic space.
You can move through it with your mind
Good question! memes don't have to be image macros. they could be visual, behavioral, sounds, motions. tiktok dances are memes. dabbing. imitatable, variational units of action

what used to be "image macros" became the most common definition for "memes" tho, because they are so easily shared. humans are visual creatures. we can parse an image instantaneously, and words almost as fast. and early internet could share them faster than video.
Now that video is cheap and fast to share, you can have all these other type of visual/behaviorism memes on tiktok (riding a skateboard + drinking cranberry juice + Fleetwood Mac * your_creative_addition_here)

but it still takes time to parse them, vs a screenshot or a meme.
this is why viral tiktoks use overlaid captions, btw. it's a memetic grappling hook that grabs you and reels you into their region of memespace where you will then enjoy the video
have to go back to selling prepackaged memes instead of theorizing about them, i'll leave you with this diagram i made ca. 2013 Image

• • •

Missing some Tweet in this thread? You can try to force a refresh
 

Keep Current with google bard

google bard Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

PDF

Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @deepfates

Mar 8
good thread, though I disagree with some of the fundamental assumptions.

strong AI is going to be compute bottlenecked. otherwise everyone wouldn't be racing for flops. there's a lot of capabilities that only make sense at low cost and those will happen in distilled models
there's also a lot of cases and what you do not want general intelligence for automation!

we already have a society where many operations are overseen by creatures that would rather be thinking about the nature of their own consciousness or whatever. Don't create billions more
what you end up with is: as much intelligence stuffed into every device as will fit (modulo battery life). this is already happening. XGboost, CNNs, Bert, CLIP, all run embedded on your device. your phone will run a 7B model, your laptop a 13B.

reverse Peter principle
Read 9 tweets
Nov 27, 2023
Asking DALL-E to make images of the archetypal resident of a city and then make it more and more archetypal. here's San Francisco


Image
Image
Image
Image
New York. That is indeed a big apple


Image
Image
Image
Image
Boston. interesting combination of academic and... juggalo elements?


Image
Image
Image
Image
Read 7 tweets
Mar 21, 2023
Serious moment: @OpenAI has decided to shut off access to code-davinci-002, the most advanced model that doesn't have the mode collapse problems of instruction tuning.

This is a huge blow to the cyborgism community
@OpenAI The instruct tuned models are fine for people who need a chat bot. But there are so many other possibilities in the latent space, and to use them we need the actual distribution of the data
@OpenAI They're giving 2 days notice for this project.

All the cyborgist research projects are interrupted. All the looms will be trapped in time, like spiderwebs in amber.
Read 17 tweets
Nov 22, 2021
Utopians are misunderstood
i mean, the word "utopian" is generally not understood to mean the same thing as the word Utopian when i use it. i use a specialized definition from a sci-fi novel. but also, the Utopians in that novel are misunderstood, by the societies around them, and they always have been
in the Terra Ignota series, Utopians are the smallest of seven voluntary societies, or Hives. in this post-scarcity society, most people work twenty hours a week max. yet Utopians commit to working as much as possible nonetheless, for their great goal:

Mars
Read 14 tweets

Did Thread Reader help you today?

Support us! We are indie developers!


This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Don't want to be a Premium member but still want to support us?

Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal

Or Donate anonymously using crypto!

Ethereum

0xfe58350B80634f60Fa6Dc149a72b4DFbc17D341E copy

Bitcoin

3ATGMxNzCUFzxpMCHL5sWSt4DVtS8UqXpi copy

Thank you for your support!

Follow Us!

:(