Share this page!

Aleksa Gordić Profile picture
Twitter logo
7 Nov, 5 tweets, 3 min read
Watch me code a Neural Network from Scratch in pure JAX! 🥳 in this 3rd video of the Machine Learning with JAX series.

YouTube:
GitHub: github.com/gordicaleksa/g…

@DeepMind @GoogleAI @jakevdp @froystig @SingularMattrix @cdleary

#jax
In this video, I build an MLP (multi-layer perception) and train it as a classifier on MNIST (although it's trivial to use a more complex dataset) - all this in pure JAX (no Flax/Haiku/Optax).

2/
I then add cool visualizations such as:
* Visualizing MLP's learned weights
* Visualizing embeddings of a batch of images in t-SNE
* Finally, we analyze the dead neurons

3/
This is the first video of this kind (coding from scratch) on my YouTube channel - your feedback is much appreciated!

You'll be able to see how I think while I'm writing code + some messiness and the art of googling haha.

4/
If people find this useful I'll be pumping out more of these videos in the future. I enjoyed making this one.

As always notebooks are available here:
github.com/gordicaleksa/g…

5/

• • •

Missing some Tweet in this thread? You can try to force a refresh
　

Keep Current with Aleksa Gordić

Aleksa Gordić Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

PDF

Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @gordic_aleksa

Aleksa Gordić Profile picture
9 Oct
Wow. @MIT's course "Introduction to Computational Thinking" is a truly amazing initiative!

YT:

So glad they brought in @3blue1brown! + @JuliaLanguage!🤯By doing this MIT has shown once more that they are the true innovators.

1/
* Powerful interactive notebooks (like @PlutoJL and @ProjectJupyter) that promote active learning.

* High-quality animations/visualizations (created using open-source libs such as github.com/3b1b/manim).

* Contextual approach with lots of example.

2/
* + the best educators in the world, leveraging the scale that platforms such as @YouTube give us, is a glimpse into the future of education!

I know that @YouTube + @MIT shaped who I am - that much is certain. My self-education path heavily relied on them. Forever grateful.

3/
Read 7 tweets
Aleksa Gordić Profile picture
6 Oct
[🔥 Learn ML for beginners 🥳] I recently said I'll be binge-watching fast.ai's Practical Deep Learning for Coders and I did, here are my final thoughts!

I'm mainly going to contrast it with @coursera's course as that's the course I took back in late 2018.

1/
Verdict:

If you're in high school or a student or more precisely somebody who still has difficulties creating your own learning program (no experience with self-education) I'd recommend you take @coursera's course - it's more streamlined.

2/
You'll know exactly when to read, watch, or code.

On the other hand, if you already have some experience (you had some tech internships/jobs) or you're considering switching careers (again you're experienced) or simply integrating deep learning into your own domain...

3/
Read 11 tweets
Aleksa Gordić Profile picture
5 Oct
[landing a job at top-tier AI labs] It's officially published! 🥳The whole story of how I landed a job at @DeepMind is out!

Blog: gordicaleksa.medium.com/how-i-got-a-jo…

It took some time to write this one.😅
Again thanks to @PetarV_93, @relja_work, Cameron Anderson, Saima Hussain for being supportive throughout this journey!

2/
In this blog you'll find:
* The details on how @DeepMind's hiring pipeline is structured.

* Many tips on how to prepare for top-tier AI labs (like DeepMind, OpenAI, etc.) in the world (for research engineering roles but I guess many tips will apply for scientists as well).

3/
Read 6 tweets
Aleksa Gordić Profile picture
4 Oct
I'll be binge-watching @jeremyphoward and @GuggerSylvain's @fastdotai "Practical Deep Learning for Coders" course today and tomorrow! 8 lectures, ~2h each. It's going to be fun! 😂 Why?

Well:

1/
* I want to update my blog on getting started with ML from 2019 where I only recommended @coursera (and I realized just how bad my writing was just 2.5 years ago!).

The blog is here: gordicaleksa.medium.com/get-started-wi…

2/
I recommend you bookmark it but don't read it just yet, it should be ready by the end of this week!

* I want to be able to give better advice to "younger folks" in general. I get a lot of questions on my Discord as well (join it if you haven't: discord.gg/peBrCpheKE).

3/
Read 18 tweets
Aleksa Gordić Profile picture
3 Oct
[Cracking the AGI problem] Some beautiful ideas in this blog post: spectrum.ieee.org/how-deepmind-i…

Such as:

1/
* The problem of catastrophic forgetting (babies face it - that's (probably) why you don't have any memories of when you were really young - and AIs face it). How do we go about cracking it?

2/
* Embodied AI - does an "AI" have to have a body before it is actually intelligent? It seems our body is a distributed processing system.

As an example, they mention the shape of the ear canal and the fact it does some form of Fourier analysis in real-time ...

3/
Read 6 tweets
Aleksa Gordić Profile picture
3 Oct
Why do people care that much about being 1st on @kaggle?

What is the main motivation to compete there?

If you're trying to help people/learn a lot/get hired here is an alternative (way less competitive): build a useful and original ML project, open-source it, and maintain it.
@kaggle reminds me of the competitive programming world a lot.

Up to a certain point, the skills you acquire are super useful.

After that point, you start missing out on everything else it takes to be a good engineer/researcher or whatever it is you're trying to accomplish.
While you're learning shortcuts for std::vector::push_back, increasing your typing speed and some variations of the algorithms you already know, other people are learning how to design software that is maintainable, collaborate with other people, present their work, etc.
Read 8 tweets

Did Thread Reader help you today?

Support us! We are indie developers!

This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Too expensive? Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal Become our Patreon

Thank you for your support!

Follow Us on Twitter!

Like this author's thread?

Get emails when @gordic_aleksa has new unrolls!

Check out other threads from @gordic_aleksa!

Read all threads by @gordic_aleksa

:(