here's your DEEP DIVE into @grok's architecture!
I just went through the , for this 314B open source behemoth with *no strings attached*.

👇🧵 model.py
@grok 1. Basics:
314 B, mixture of 8 experts (2 active)
86B active parameters

It's using Rotary Embeddings #rope instead of fixed positional embeddings

📜👇👇

HOLY SH*T @grok IS 314 BILLION PARAMETERS

Mixture of 8 Experts, not RLHFd/moralized

THIS IS HUGE

https://twitter.com/grok/status/1769441648910479423

@grok Grok repo as a TXT for your prompting convenience:
drive.google.com/file/d/1YGKcT5…

ATTN: Bioinformatics/Comp Bio people

Save this thread for a walkthrough of #Devin AI designing primers for flu detection from scratch!

I asked "Help me identify DNA primer candidates for the most recent flu strain in the US"

Here's what happened! 🧵
#biotech #compbio 1.
I didn't give it much detail and my request implicitly required it to determine what the most recent flu strain in the US was, and how to get that data.

It intelligently used the NCBI api to get the H1N1 sequence data for 2023 (most recent flue season)

It wrote Python to retrieve the data and installed Biopython. Then, it looked at the FASTA file to see what it looked like.

There was an error in some of the code it wrote so it fixed it.

||
||
v

i never believe recorded demos so I reached out to the @cognition_labs team for early access to try for myself and got it!

will be sharing my unfiltered opinions on #devin here.

🧵🧵 1/n

https://twitter.com/cognition_labs/status/1767548763134964000

My first task I asked it for, was a website where you play chess against an LLM. You make a move, the move is communicated to GPT-4 via a prompt, and GPT-4 replies, and the reply is converted into a move that is reflected on the chessboard.

So quite a few moving parts.

I was curious to see if Devin:
1. Would figure out how to accurately use the GPT-4 API because most LLMs don't actually know how to use it, and there are conflicting versions online bc OpenAI changed the API in November
2. Would appropriately ask for an API key and securely handle it.
3. Would deal with package errors
4. Would understand how to prompt an LLM to make a chess move and return it in a precise notation (FEN in this case)

Here is what happened:
2/n

A 🧵 of peer reviewed published scientific research where the authors left out a key coauthor 😉:

“As an AI language model…”

@MicrobiomDigest This one is published on “” so who knows

Peer reviewed in the journal of internet banking and commerce

Stock price prediction is so tired 😴 predatory-publishing.com

Go to Google Scholar and look up ‘As an AI language model” -“ChatGPT”’ A thread:

https://twitter.com/itsandrewgao/status/1689634600086315008?s=20

🧵I had the opportunity to stay in South Africa over the summer.

They have a custom called"stokvel", which is
a friend-based credit union. It's genius + super interesting.

About half of South Africans take part and $2.5B USD is put in each year.

👇👇 You form a group of 12 with friends. Each month, you pitch in a set amount, such as $100. One out of 12 months, it's your turn to receive that month's money ($1200). The rest of the months, you put $100 in and one of your friends gets the money.

Why is this good?

WTF: Mind reading is here.

Researchers invented a new #AI method to convert brain signals into video. See the results for yourself

Published in Nature yesterday: nature.com/articles/s4158…

What are the implications? Is this the biggest paper of 2023?

#CEBRA Check out the homepage of CEBRA here: cebra.ai
And the abstract:

Offering FREE access to #GPT4 and comparing with #GPT3.

Retweet for instant access (follow so I can DM you).

💻🔮💫✨
#openai #ai #tech #chatgpt Is water wet?
GPT4 gives a direct answer.

GPT3 responds "As an AI language model..."

🧵Racial minorities are 40% of the US population but only 5% of jurors.

For @Stanford CS109, I built an interactive site that explores the probabilities of jury selection and race.

cs109.olafblitz.repl.co

👇 more info

https://twitter.com/itsandrewgao/status/1636611922429898754

@Stanford 1/n

In the #AhmaudArbery trial, only 1 in 12 jurors was Black, even though the local population is 27% Black.

The defendants struck out 11/12 jurors.

In another case, a jury pool of 105 people in Stockton, CA had 0 Black people.

The website teaches probability in the context… twitter.com/i/web/status/1…

for some reason, on desktop Twitter, it's always "load 34 tweets". Never 24 or 63. Just 34. twitter.com/i/web/status/1… @Scobleizer's tweet is caught in this screenshot lol

You can't do everything and be everything.

Someone who spends 100% effort on A will beat you who does 50% A, 50% B.

Jack of all trades is master of none.

🧵 Yes, synergy is a thing, many Nobel Prize winners have hobbies and are great musicians and artists. Not what I'm talking about here.

1/n

all my friends who read a lot when young are successful now

i think reading is probably the more important thing you can do in elementary school

if ur brain is like an LLM, you should maximize the # of tokens it sees while training i read so much in elementary school that writing is very easy for me. i liken it to next token prediction.

SAT/ACT grammar came very easy for me, i just went off of what sounded right. idioms as well, even though my family isn't native english speaker. but id read all them

“Money doesn’t buy happiness” is a coping mechanism.

Sure money isn’t 1:1 with happiness 💰 😆

But I’d rather be crying with 10M in the bank than 10 bucks.

Don’t settle for less. Caveats to this claim but I’m too lazy to write them out

🧵 EVERY new #AI feature @Google launched, and how YOU can take advantage!

This is going to save you hundreds of hours.👇👇👇 I have always been bullish on Google even though everyone says they’re behind. Google has the money and talent, but more importantly, existing users locked into their massive product ecosystem.

1: One click Powerpoint! Put in a topic. Seems to be Stable diffusion images 1/n #ai

looks like @openai bought the domain ai.com and hooked it up to chatgpt.

maybe everyone alr knew this and im just finding out lol...

accidentally found this out bc i was curious who owned the domain.

#ai #chatgpt #openai #gpt3 also open.ai
Is this a legacy openai site? cute

bookmark this 🧵:
5 of my favorite vids under 5 mins explaining AI/machine learning at a beginner level!

#1: What Is Artificial Intelligence?

#ai #tutorial #beginners #nocode #chatgpt #2: DALL-E Explained by @openai


This video is under three minutes and is made by the developers of DALL-E! It explains how DALL-E was trained and how it works to create cool images.

🧵 Thread of awesome AI projects by @Stanford students!

definitely gonna miss some projects so comment if i missed something!

(pic generated by stable diffusion) @Stanford From @bryanhpchiang: Symbiotic
Upload a document and ask questions!

https://twitter.com/bryanhpchiang/status/1620875782074802176?s=20&t=Ad2Dwz1TG_ATQdHOe_g4tA

see what the future looks like with AI!

generate scifi style images with an auto-complete interface to help you write prompts quicker.

try now: portalpix.app

made using Leap from @leap_api @fdotinc

#aiart #ai #ml #generativeai #stablediffusion #buildspace #fdotinc San Francisco

🧵If you're using GPT3 for multilingual applications, read this!

i took a 555 word/2928 char English text. It becomes 706 GPT tokens.

Chinese version of the same text: 2170 tokens. (3x 🇬🇧)

Hindi: 4480 tokens (6x 🇬🇧)

Implications 👇 First off, I only tested this on a very small amount of text so I'm not sure if this is a general trend. The actual token efficiency difference number will definitely be different than these numbers.
Also, the Chinese and English text are professionally translated parallel texts.