Interesting write-up of how @DeepMind 's #Alphafold2 works.

The takeaway from this is that a *lot* of human insight was required and there's a lot of highly specialized feature engineering going on here.

Why is that important?

blopig.com/blog/2021/07/a…
It's important because it shows us that @DeepMind is obviously nowhere near making any kind of general AI; they're doing good old-fashioned down-and-dirty hacks with great computational resources and a clever team.
If this paper had basically said something different like "Oh they just used a really big supertransducer on the raw data", then I would be scared.

But no, they didn't. They did a bunch of artisanal architecture engineering that's good for this specific task only.
That's not to say that that won't come in the future. The progression from AlphaGo to AlphaZero started with a lot of task-specific architecture engineering and gradually became more and more general-purpose and relied less and less on supplementary data sources.
It wouldn't surprise me if the next thing @DeepMind is working on is something like an "Alphafold Zero" - a generalized system for solving scientific problems without massive amounts of task-specific feature engineering (like Alphafold currently has)
Image
Image
"The main secret behind AlphaFold 2’s success is the superb deep learning engineering. A close look at the model reveals an architecture with a large amount of small details that seem fundamental for the performance of the network..."
"... As we admire the end product, we should not turn a blind eye to the enormous budget, and the large team of full-time, handsomely paid engineers that made it possible ..."
However, we should also bear in mind that alphafold and its successors will make drug discovery and computational biology much easier. This code could save people's lives in the future.

• • •

Missing some Tweet in this thread? You can try to force a refresh
 

Keep Current with Roko.eth - hereticalupdate.substack.com

Roko.eth - hereticalupdate.substack.com Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

PDF

Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @RokoMijicUK

22 Jul
Pro-tip: if you're smart, it's probably not worth studying hard at university.

The benefits are small and the costs can be devastating in terms of missing out on other more important opportunities (networking, dating etc).

I got a 1st from Cambridge but it's pretty worthless.
Really, nobody cares, nobody even asks, but I missed out on a network that lasts for a lifetime partly as a result of spending too much time & effort studying.
Don't get a 3rd or a fail though. It will mark you out as deviant. Just coast through to a 2.2 or a 2.1 and make the most of the nonacademic opportunities.
Read 8 tweets
21 Jul
The fundamental flaw in the Western Tradition that I come from is seeing humans, society etc through a lens of logic and principles, rather than through a lens of ecology, evolution, markets, mess.

You can see this in several key places.
(1) Absolutist political principles like "freedom", "consent" or "justice" come from seeing things through a logical lens.

The reality of all these things is messy and all the absolute versions of these are wrong.
(2) Early attempts at building AI were all based on logic. It failed.

Nowadays we're making progress by embracing the mess via Rich Sutton's "Bitter Lesson".

incompleteideas.net/IncIdeas/Bitte…
Read 11 tweets
20 Jul
Yes, Critical Race Theory should be banned from schools and Universities.

At the end of the day, memes are at the top of the food chain. If you don't want to live in hell, don't let these hellish memes proselytize themselves.
What about the marketplace of ideas? I've given up on it. (Political) ideas don't form a marketplace, they form an ecology. You have to think like a biologist, think about competition for energy and resources, niches, evolution and extinction.
It's not a marketplace of ideas, it's an ecosystem of ideas.
Read 5 tweets
20 Jul
I seem to remember that @spakhm had an excellent substack post on "efficient markets in self help" but I can't find it.

Does anyone remember it? Did someone else write it? Has it disappeared?
Anyway the point of the blog post (which I can't find) is that any kind of self-help that is widely known and claims to solve an important problem that many people have definitely doesn't work; if it worked, the problem wouldn't exist any more.
There can be exceptions to this if either (1) it's not widely known or (2) there's some weird exceptional reason why very few people do it.
Read 10 tweets
16 Jul
This is literally how women's brains work by the way

It sounds retarded but it really does work like this
The danger once you realize it works like this is you push it too far and become undateable but very sexy, which means you never actually settle down, which sucks.
Reposting this because it's such a succinct and accurate summary of how the human genders work and why they're fundamentally different pieces in the chess game of life:

Read 4 tweets
13 Jul
Tomas Pueyo on the #Delta Variant

This could be extremely bad.

Many people optimistically assumed that #Covid19 would evolve to become more infectious and less lethal. Unfortunately, it has become more infectious 𝘢𝘯𝘥 more lethal.

unchartedterritories.tomaspueyo.com/p/delta-varian…
The only good news here is vaccines seem to really help. Reminds me that I need to book a vaccination appointment.
Also, thanks to @akarlin88 for reminding me, @gcochran99 was on this as early as last year:

westhunt.wordpress.com/2020/03/19/evo…
Read 6 tweets

Did Thread Reader help you today?

Support us! We are indie developers!


This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Too expensive? Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal Become our Patreon

Thank you for your support!

Follow Us on Twitter!

:(