Thread by @0xsanny

In this thread, I want to compile a list of Deep Learning resources 📚🎬 that some people might not be aware of. It’s amazing how organizations like @fastai, @stanfordnlp and @OpenAI and people like @math_rachel ,@pieterabbeel and @lexfridman have made these freely available 🎉👇

I’ll be keeping a more comprehensive list on my GitHub. But I’m sure there are many that I missed, so I’d love to know, if you have any other suggestions!
github.com/sannykim/deep-…

Most people know about the great @fastai course, but did you know @mathrachel also published a Computational Linear Algebra course with textbook and videos, covering topics such as Singular Value Decomposition and Principal Component Analysis? /1
github.com/fastai/numeric…

Another great resource coming from fast.ai's @jeremyphoward and @the_antlr_guy is their paper “The Matrix Calculus You Need For Deep Learning”, which is very complementary to @math_rachel's course /2 arxiv.org/abs/1802.01528

One course that covers more of the basic statistics for Machine Learning is @NandoDF's UBC course, which offers one of the most intuitive introductions to ML and stats /3
youtube.com/playlist?list=…

For people, who want to dive a little deeper, Gibert Strang and @MITOCW just recently published a course that combines Linear Algebra with Deep Learning and a little bit of stats and calculus /4
youtube.com/playlist?list=…

Most people also know about @DeepLearningAI_ , but did you also know that @AndrewYNg, @kiankatan and @StanfordOnline made the complementary Stanford course "CS 230 Deep Learning" freely accessible? /5
youtube.com/playlist?list=…

Speaking of Stanford, the @stanfordnlp group around @chrismanning and @abigail_e_see recently published their new (!) 2019 version of course CS224n on NLP with amazing guest speakers like @huangcza, @ashVaswani and @RichardSocher /6
youtube.com/playlist?list=….

Very under-appreciated is @gneubig's CMU course, which provides another great intro NLP and Deep Learning and even covers topics such as Advanced Search Algorithms. Both courses are fantastic, so you can’t go wrong with either! /7
youtube.com/playlist?list=…

A slightly older but still invaluable resource is Oxford's and DeepMind’s 2017 Deep Learning for NLP course. Listen to @egrefen explain World Level Semantics or @redpony talk about Conditional Language Models. /8
youtube.com/playlist?list=…

Last NLP resource I'm gonna mention for now is @seb_ruder's PhD Thesis on Neural Transfer Learning for NLP, which provides an excellent overview of topics such as Multi-Task Learning and Pretrained representations in NLP. /9
ruder.io/thesis/neural_…

Jumping over to Reinforcement Learning, we have the amazing @EmmaBrunskill, who explains topics in RL in her CS234 course at Stanford. Also check out the assignments (+solutions) on the course website! /10
youtube.com/playlist?list=…

Another RL resource that's very well made is @jachiam0 and @OpenAI's "Spinning Up in RL" that contains both explanatory content and practical code examples. There's also a recorded webinar on OpenAI's YouTube channel! /11
spinningup.openai.com/en/latest/

@svlevine made another set of great RL lectures available on YouTube from his CS294-112 course at Berkeley. /12
youtube.com/playlist?list=…

His @berkeley_ai colleague @pabbeel also co-organized a Deep RL Bootcamp in 2017, from which all the videos are also public, including great lectures by @karpathy on policy gradients and @chelseabfinn on inverse RL! /13
youtube.com/playlist?list=…

One of the best courses recently also comes from @pabbeel's team on Deep Unsupervised Learning, which is the CS294-158 course at Berkeley. Particularly liked @AlecRad's guest lecture on Language Models (albeit the audio was sadly gone for a bit)! /14

@pabbeel has been quite amazing in contributing to online resources. So, another event he organizes with @josh_tobin_ + @sergeykarayev called Full Stack Deep Learning helps to "bridge training ML models to deploying AI systems in the real world" /15
youtube.com/playlist?list=…

Tip: There is currently a @twimlai study group on the contents of this bootcamp. Check out previous recordings and sign up for on of their regular online meetups. Props to @samcharrington and others for creating such an active community /16
youtube.com/playlist?list=…

Also give @twimlai podcasts a listen, @samcharrington delivers high quality podcasts week in and week out! My favorite shows so far have been conversations with @hannawallach, @JeffDean and @cbd /17
twimlai.com/shows/

When it comes to podcasts, @lexfridman has some of the best content I've listened to. His ability to facilitate a conversation is quite astonishing. The most memorable ones for me have been with @OriolVinyalsML, @gdb, @goodfellow_ian and @clattner_llvm /18
youtube.com/playlist?list=…

Another recent addition to great ML podcasts comes from @jeremiecharris, @russ_poll and @neutronsNeurons. Some really interesting conversations they had were with @mcleavey on MuseNet and @DavidDuvenaud on the state of academia /19
youtube.com/channel/UCDK-m…

Switching over to YouTube channels, shoutout to @ykilcher for producing some of the best paper overviews out there. He should have at least 10k subscribers! Check out his account, if you want to gain an overview of BERT, Neural ODEs etc. /20
youtube.com/playlist?list=…

When it comes to understanding papers, I love @rctatman + @kaggle's Reading Group on YouTube! She makes it really easy to get into a topic and appreciate how she addresses comments while reading these papers. Really enjoyed the videos on Transformers! /21
youtube.com/playlist?list=…

Probably the most comprehensive library of paper overviews comes from @AISC_TO (formerly TDLS), which goes through a diverse set of topics in ML. The interactive discussions, e.g. on PCGANs are super helpful to better understand a paper! /22
youtube.com/playlist?list=…

Another YouTube channel I'd like to highlight is @xsteenbrugge's Arxiv Insights! He creates very accessible introductions to concepts such as VAEs or PPO /23
youtube.com/channel/UCNIkB…

Next one on the list is @apsoleimany + @xanamini's MIT course on Intro to DL. Their Github Labs are especially nice! But even if you're familiar with these basic topics, I'd watch the guest lectures, e.g a fantastic one by @viegasf on ML visualization /24 youtube.com/playlist?list=…

One more paper reading group comes from Microsoft! Hosted by the great @ecsquendor, these new sessions offer a wealth of knowledge. If you ever wondered what the difference between a Transformer and a Transformer XL is, check out their channel! /25
youtube.com/playlist?list=…

Most people know @robertskmiles from his appearances on @computer_phile, but he also has his own YouTube channel! Take a look, if you you want to learn more about concrete problems in AI safety /26
youtube.com/playlist?list=…

That said, I don't know many resources when it comes to AI safety topics. Maybe @Miles_Brundage, @robertskmiles, @vkrakovna, @ch402 or @jachiam0 have some pointers?

For people interested in AI safety topics, have a look at @vkrakovna's great list of resources and @rohinmshah's AI alignment newsletter/podcast. Thanks for the recommendations @jachiam0, @robertskmiles and @vkrakovna! /27
vkrakovna.wordpress.com/ai-safety-reso…

When it comes to blogs and similar mediums, @gradientpub is doing an incredible job writing "accessible and technically informed" content about a diverse set of subjects in ML, from vision to language to ethics /28
thegradient.pub

Another website I think deserves a lot more attention is @DepthFirstLearn, an amazing initiative by @avitaloliver, @suryabhupa, @kumarkagrawal + @cinjoncin to create curricula for studying various topics/papers in depth such as WGANs and AlphaGoZero /29
depthfirstlearning.com

A blog that I would highly recommend is @JAlammar's. His posts are some of the clearest and nicely visualized I've ever seen. Definitely give his explanations of Word2Vec, BERT and the Transformer a read! /30