For 20 years I used a wide variety of machine learning and optimization algorithms to tackle predictive modeling challenges.

Nowadays, #deeplearning is part of nearly everything I do. In this @MelbCtrDataSci talk, I explain why I made this my focus.
Early on, I mainly just used GLMs with lots of feature engineering for predictive modeling. Just like Andy Beck and team did when they developed Cpath, the computational pathologist
Once I discovered the magic of restricted cubic splines, they became my main tool for handling continuous independent variables
But once random forests appeared on the scene, I started using them pretty much everywhere!

(Diagram from dtreeviz by @the_antlr_guy)
I was particularly fond of how much insight they gave into the data, such as through feature importance plots
Here's a talk I gave in 2011 about my pre-deep-learning toolbox, including a deep dive into random forests
But by 2012 deep learning was showing super-human performance on some vision tasks, and smashing the state of the art in numerous domains
I started using neural nets over 25 years ago, including writing my own Delphi implementation from scratch. I felt at the time that one day they'd become dominant. GPU libraries were the big change in 2012
In 2014, I explained why I thought deep learning would go far beyond just image recognition applications, to make a huge impact on society…
As I expected, deep learning is now becoming the best choice for a huge range of modeling applications
And that's why we created @fastdotai -- to help make this powerful technology available as widely as possible
PS: random forests are just bagged decision trees. Instead of bagging, you can use boosting with decision trees (GBDTs). Here's a book chapter I co-wrote with @the_antlr_guy about that approach:…

• • •

Missing some Tweet in this thread? You can try to force a refresh

Keep Current with Jeremy Howard

Jeremy Howard Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!


Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @jeremyphoward

26 May
Here's something really help from @amaarora - a thorough walk-thru of my recent talk "20 Years of Tech Startup Experiences in One Hour".

It's a great way to get the key messages of the talk, without watching the whole thing.…
My full talk is available here:
This approach, of writing up a talk that you like and posting it publicly, is great for everyone:
- The speaker gets a sharable text version of their talk
- The writer gets the appreciation of the speaker and the audience
- People can save time by reading instead of listening
Read 7 tweets
26 May
I've just uploaded a new "lesson 0" for Practical Deep Learning for Coders, which is an optional lesson of tips for how to get the most out of the course
The lesson contains a lot of insights from the new book from @radekosmulski, "Meta Learning", which describes his successful journey from non-coder to Kaggle winner and full-time deep learning scientist…
It finishes with a complete walk-thru showing how to set up a Linux GPU server from scratch on @awscloud EC2. For details on the course, and setup on many different cloud environments, see:
Read 4 tweets
4 Feb
I’m hearing comments that Grid AI (Lightning) seem to have copied fastai's API without credit, and claimed to have invented it.

We wrote a paper about our design; it's great it's inspiring others.

Claiming credit for other's work? NOT great 1/…
PyTorch Lightning is a new deep learning library, released in mid-2019. The same team launched "Flash", a higher level library, this week.

fastai was launched in 2017, based on extensive research.

As you can see, they look *very* similar.
The quote below is from the Flash launch post (h/t @tomcocobrico). It is very clearly not true.

fastai's focus has always been simple inference, fine-tuning, and customization of state of the art models on new data in a matter of minutes.

Read 20 tweets
11 Jan
Our paper, "An evidence review of face masks against COVID-19", written by a cross-disciplinary team of 19 international experts, was published in the Proceedings of the National Academy of Sciences today.

No time for the paper? Then read this thread!…
The paper, which includes 141 references (yes, I read every one of them!) argues that we should increase focus on a previously overlooked aspect of mask usage: mask wearing by infectious people ("source control"), rather than only mask wearing by susceptible people ("PPE") Image
Masks have been used to help control respiratory pandemics for at least 600 years. Wu Lien-Teh (the "Plague Fighter") showed the world the importance of masks nearly 100 years ago, doing detailed studies over many years.

Sadly, his work became largely forgotten in the west Image
Read 11 tweets
6 Jan
IIUC this "new technique" from Facebook is actually just a slight repackaging of bits of @wightmanr's brilliant timm library.

Great they wrote a paper that documented how well it works, but they should at *minimum* have cited timm, and really should have made him senior author
I've read thru their code, and it's basically calling out to timm (which is based on years of research from @wightmanr), along with a standard Pytorch training loop and data munging.…
There are clearly some pretty deeply rooted issues surfaced here regarding how @facebookai works with the open source community.

Hopefully this turns out to be a productive learning experience.
Read 6 tweets
18 Dec 20
Excited to launch "ghapi" today in partnership with @GitHub. ghapi provides complete access to the entire GitHub API, using a consistent interface with many nice touches.

See thread below for a demo and summary, or read the post for details: 1/…
ghapi has both a Python and a CLI interface. The operations and parameters of them are identical, so once you know one, you know them both! Here's a demo of the CLI interface, which includes help for all API operations and tab-completion.
Here's a demo of ghapi's Python interface, which even includes links to the official @GitHub documentation for every endpoint.
Read 11 tweets

Did Thread Reader help you today?

Support us! We are indie developers!

This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Too expensive? Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal Become our Patreon

Thank you for your support!

Follow Us on Twitter!