I'll be binge-watching @jeremyphoward and @GuggerSylvain's @fastdotai "Practical Deep Learning for Coders" course today and tomorrow! 8 lectures, ~2h each. It's going to be fun! 😂 Why?
Well:
1/
* I want to update my blog on getting started with ML from 2019 where I only recommended @coursera (and I realized just how bad my writing was just 2.5 years ago!).
I recommend you bookmark it but don't read it just yet, it should be ready by the end of this week!
* I want to be able to give better advice to "younger folks" in general. I get a lot of questions on my Discord as well (join it if you haven't: discord.gg/peBrCpheKE).
3/
Lots of smart people there!
IMPORTANT:
This is not the way to go through the course if your goal is to learn ML. I'm just skimming the course to get a feeling for what is covered and how. I'll be taking summaries along the way to improve my retention though.
4/
You should be going through the notebooks, asking questions on the forum, and in general spacing the lectures out (temporally) in order to learn more effectively.
Check out this awesome video BEFORE doing the course (the so-called "0th lecture"):
5/
I'll be sharing my summaries on this thread as I progress! And finally, I'll update my getting started blog with new info.
6/
My lecture notes:
* There is a book that goes along with the course. There is a free version (github.com/fastai/fastbook), but if you can buy the book to support them!
L1/
* There is no need for a math PhD, just get started!
* They give some historical context.
* They teach the course using the top-down approach (shown to be a much better approach for most people).
L2/
Check out this blog of theirs fast.ai/2016/10/08/tea… to learn more about their teaching methodology.
* The stack behind the course: fastai-PyTorch-Python + Jupyter, but as Jeremy Howard says - it doesn't matter that much. Focus on learning the concepts.
L3/
* A high-level overview of what ML is, what SGD is, etc.
General notes:
* The course seems to require more self-discipline compared to Coursera's course.
L4/
After the lecture, you need to go through the Jupyter, start Colab or a GPU server, play with code a bit, search through the docs. All in all, get comfortable before taking video lecture 2. Use forums to find peers (and check whether somebody created a Discord).
L5/
* Do the questionnaire, track what you didn't know in a todo list, and later get back to it!
* The code may have a weird coding style - it's a result of Jeremy mixing his understanding of various programming languages (like APL, etc.).
* Getting deeper into ML theory (why do we need validation/test sets, why does transfer learning work, diff between loss vs metric, etc.)
L21/
* Confidence intervals, p-value, etc. On a concrete example of whether COVID infections correlate with humidity & temperature.
* Going to production! Bear detector example (very funny). Shows you how you can create your own dataset (e.g. scrap data via Bing API).
L22/
General notes:
* The part on p-value may be very very overwhelming to beginners. Be comfortable with not understanding everything (that's hard if you only ever went through the formal educational system).
L23/
* In general younger folks (high-school kids, students) should probably take the @coursera route, IMHO. More experienced people who want to switch careers may find @fastdotai's course a better option.
* One part of the lecture was more about COVID than about ML per se 😅
L24/
* Videos are there only to help you better understand the notebooks but he also gives nice hands-on examples - so don't skip the videos.
Again thanks to @PetarV_93, @relja_work, Cameron Anderson, Saima Hussain for being supportive throughout this journey!
2/
In this blog you'll find:
* The details on how @DeepMind's hiring pipeline is structured.
* Many tips on how to prepare for top-tier AI labs (like DeepMind, OpenAI, etc.) in the world (for research engineering roles but I guess many tips will apply for scientists as well).
3/
* The problem of catastrophic forgetting (babies face it - that's (probably) why you don't have any memories of when you were really young - and AIs face it). How do we go about cracking it?
2/
* Embodied AI - does an "AI" have to have a body before it is actually intelligent? It seems our body is a distributed processing system.
As an example, they mention the shape of the ear canal and the fact it does some form of Fourier analysis in real-time ...
3/
Why do people care that much about being 1st on @kaggle?
What is the main motivation to compete there?
If you're trying to help people/learn a lot/get hired here is an alternative (way less competitive): build a useful and original ML project, open-source it, and maintain it.
@kaggle reminds me of the competitive programming world a lot.
Up to a certain point, the skills you acquire are super useful.
After that point, you start missing out on everything else it takes to be a good engineer/researcher or whatever it is you're trying to accomplish.
While you're learning shortcuts for std::vector::push_back, increasing your typing speed and some variations of the algorithms you already know, other people are learning how to design software that is maintainable, collaborate with other people, present their work, etc.
[🧠 GAN paper summary 🧠] "Eyes Tell All" <- An interesting short paper on how to detect fake images generated by GANs (at least the current GAN methods like StyleGAN v2!).
They use a simple heuristic: 1/
1. Crop the face and then eyes (they used dlib for this) 2. Segment out the pupils -> you get the predicted mask 3. Fit an ellipse -> you get the "GT" mask 4. Find BIoU between the 2) and 3)
2/
(BIoU is just a generalization of IoU where we focus only on the boundary pixels of the mask instead of the whole mask)
5. If BIoU is "too low" that means that the predicted pupil is deviating from the elliptical shape and thus the eyes i.e. the face is fake/generated!
3/
[big update 🥳] I'll be joining @Google@DeepMind later this year as a Research Engineer!!! 🤖😅❤
It feels surreal, I don't even know where to start. 1/
I didn't come from an "Ivy League" university. Not because I couldn't get in - but because when you're 19, from a less developed country, and you didn't have tech-savvy people around you growing up - you are not even aware that @Stanford/@MIT are a thing. 2/
I can deeply empathize with many because of that. Not all of us were lucky enough to get exposed to tech when we were 9. I "heard" about programming when I was 19! We had to compensate by working harder than everyone else. 3/