[🧠 collective intelligence 🧠] I've been intrigued by the cellular automata (CA) concept for a long time, and by the potential, mutually beneficial interaction between CAs and deep learning, so I decided to dig a bit deeper. Here are some interesting resources I found:

1/🧵
distill.pub/2020/growing-c… <- one and only Distill. A nice introduction to how neural CA works and how it may be a potentially useful model of morphogenesis and regeneration processes in developmental biology.

@zzznah

2/
(think how we humans are formed from a single egg cell - when and how does this multiplication of cells stop and stays stable?)

3/
arxiv.org/abs/2111.14377 "Collective Intelligence for Deep Learning" <- a recent paper that explores how ideas from collective intelligence (an umbrella term for CA, self-organization, swarm optimization, etc.) have influenced deep learning (and vice versa).

@hardmaru

4/
arxiv.org/abs/2111.13545 "μNCA: Texture Generation with Ultra-Compact Neural Cellular Automata" <- a recent paper that trains a neural CA (compare that to fixed rules of Conway's Game of Life!) to generate interesting textures with as little as 60 bytes of learnable weights!

5/
Finally check out this "upgraded" version of Conway's Game of Life: continuous time-space CA called "Lenia":

1) <- some quick visualizations so that you understand what it's all about (hint: amazing artificial "lifeforms" emerge)

@BertChakovsky

6/
2) <- Lenia explained in a bit more depth

7/

#collectiveintelligence #cellularautomata #deeplearning
Oh, and yeah - neural CA is basically a GNN (as all things should be haha).

In the discrete grid world, every cell (node) has a multidimensional state (feature vector), does some local communication with immediate "pixel neighbors", and outputs a novel cell state.

8/

• • •

Missing some Tweet in this thread? You can try to force a refresh
 

Keep Current with Aleksa Gordić

Aleksa Gordić Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

PDF

Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @gordic_aleksa

26 Dec 21
When successful people say they're not the best at anything, what they really mean is that they are not the best alongside any of the dimensions that we humans have a name for and that we can easily quantify (running 100m/chess/comp. programming/traveled the most/h-index...)

1/
But if we were to quantify "bestness" as the length of the resultant vector across all of the relevant dimensions - then the scoreboard changes.

They are usually, all in all, the most well-rounded professionals.

2/
That's why we don't have a global scoreboard for "the best entrepreneur", "the best leader", etc.

These roles have an immense breadth and it's impossible to quantify and rank those people.

3/
Read 7 tweets
7 Nov 21
Watch me code a Neural Network from Scratch in pure JAX! 🥳 in this 3rd video of the Machine Learning with JAX series.

YouTube:
GitHub: github.com/gordicaleksa/g…

@DeepMind @GoogleAI @jakevdp @froystig @SingularMattrix @cdleary

#jax
In this video, I build an MLP (multi-layer perception) and train it as a classifier on MNIST (although it's trivial to use a more complex dataset) - all this in pure JAX (no Flax/Haiku/Optax).

2/
I then add cool visualizations such as:
* Visualizing MLP's learned weights
* Visualizing embeddings of a batch of images in t-SNE
* Finally, we analyze the dead neurons

3/
Read 5 tweets
9 Oct 21
Wow. @MIT's course "Introduction to Computational Thinking" is a truly amazing initiative!

YT:

So glad they brought in @3blue1brown! + @JuliaLanguage!🤯By doing this MIT has shown once more that they are the true innovators.

1/
* Powerful interactive notebooks (like @PlutoJL and @ProjectJupyter) that promote active learning.

* High-quality animations/visualizations (created using open-source libs such as github.com/3b1b/manim).

* Contextual approach with lots of example.

2/
* + the best educators in the world, leveraging the scale that platforms such as @YouTube give us, is a glimpse into the future of education!

I know that @YouTube + @MIT shaped who I am - that much is certain. My self-education path heavily relied on them. Forever grateful.

3/
Read 7 tweets
6 Oct 21
[🔥 Learn ML for beginners 🥳] I recently said I'll be binge-watching fast.ai's Practical Deep Learning for Coders and I did, here are my final thoughts!

I'm mainly going to contrast it with @coursera's course as that's the course I took back in late 2018.

1/
Verdict:

If you're in high school or a student or more precisely somebody who still has difficulties creating your own learning program (no experience with self-education) I'd recommend you take @coursera's course - it's more streamlined.

2/
You'll know exactly when to read, watch, or code.

On the other hand, if you already have some experience (you had some tech internships/jobs) or you're considering switching careers (again you're experienced) or simply integrating deep learning into your own domain...

3/
Read 11 tweets
5 Oct 21
[landing a job at top-tier AI labs] It's officially published! 🥳The whole story of how I landed a job at @DeepMind is out!

Blog: gordicaleksa.medium.com/how-i-got-a-jo…

It took some time to write this one.😅
Again thanks to @PetarV_93, @relja_work, Cameron Anderson, Saima Hussain for being supportive throughout this journey!

2/
In this blog you'll find:
* The details on how @DeepMind's hiring pipeline is structured.

* Many tips on how to prepare for top-tier AI labs (like DeepMind, OpenAI, etc.) in the world (for research engineering roles but I guess many tips will apply for scientists as well).

3/
Read 6 tweets
4 Oct 21
I'll be binge-watching @jeremyphoward and @GuggerSylvain's @fastdotai "Practical Deep Learning for Coders" course today and tomorrow! 8 lectures, ~2h each. It's going to be fun! 😂 Why?

Well:

1/
* I want to update my blog on getting started with ML from 2019 where I only recommended @coursera (and I realized just how bad my writing was just 2.5 years ago!).

The blog is here: gordicaleksa.medium.com/get-started-wi…

2/
I recommend you bookmark it but don't read it just yet, it should be ready by the end of this week!

* I want to be able to give better advice to "younger folks" in general. I get a lot of questions on my Discord as well (join it if you haven't: discord.gg/peBrCpheKE).

3/
Read 18 tweets

Did Thread Reader help you today?

Support us! We are indie developers!


This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Too expensive? Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal

Or Donate anonymously using crypto!

Ethereum

0xfe58350B80634f60Fa6Dc149a72b4DFbc17D341E copy

Bitcoin

3ATGMxNzCUFzxpMCHL5sWSt4DVtS8UqXpi copy

Thank you for your support!

Follow Us on Twitter!

:(