One of my favorite papers recently questions a fundamental building block of machine learning: cross entropy loss. (Surprised it took until 2017 to discover focal loss, and 2020 to apply it to DNN.)

🔗 arxiv.org/abs/2002.09437
📝 Calibrating Deep Neural Networks using Focal Loss
No matter how well you understand something mathematically, you still might be missing something that your current models just don't show you...

Intuitive understanding comes first, the math follows!
The code is #PyTorch (of course :-) and available here:
github.com/torrvision/foc…

If you drop this into your codebase and see improvements, let me know!
In the introduction they mention multi-class classification, so I presume the whole thing works fine with multi-class! (cc. @kcimc)

• • •

Missing some Tweet in this thread? You can try to force a refresh
 

Keep Current with Alex J. Champandard

Alex J. Champandard Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

PDF

Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @alexjc

24 Nov
#DeepFake Tutorial.

In this thread I'm going to post some tips & tricks to identify Deep Fakes using these examples I found online.

📊Vote in this other thread first:
There are many ways to detect deep fakes. Here are three of them:
1) Impossible static poses.
2) Impossible movements.
3) Technology artefacts.

The first category seems to be easier to detect, but the second is more reliable. The third may go away soon!
VIDEO #1

This one seems easy enough...

TIP: Use the speed controls in your favorite video player to slow it down while it's playing.
Read 19 tweets
24 Nov
#DeepFake Alert!

I've been digging through various propaganda and conspiracy websites (so you don't have to) and finding a surprisingly large number of deep fake appearances of Mr. Bіden.

Here is a thread with videos+polls to test your skills at discerning what's real or not... Image
VIDEO #1

Original clip at higher quality (mp4/vp9):
📺 ipfs.io/ipfs/Qmb9ekFmw…
Is this real Bіden or fake Bіden?
Read 19 tweets
16 Jun
Next in our literature survey in Texture Synthesis, a personal favorite and under-rated paper by Li et Wand. 💥

An illustrated review & tutorial in a thread! 👇

📝 Combining Markov Random Fields & Convolutional Neural Networks for Image Synthesis
🔗 arxiv.org/abs/1601.04589 #ai
Here's the nomenclature I'm using.

✏️ Beginner-friendly insight or exercise.
🕳️ Related work that's relevant here!
📖 Open research topic of general interest.
💡 Idea or experiment to explore further...

See this thread for context and other reviews:
🕳️ The paper of Li & Wand is inspired by Gatys' work from 2015. It explores a different way (sometimes better) to use deep convolution networks to generate images...
Read 19 tweets
16 May
Let's start our tour of research papers where #generative meets deep learning with this classic by Gatys, Ecker and Bethge from 2015.✨

A multimedia tutorial & review in a thread! 👇

📝 Texture Synthesis Using Convolutional Neural Networks
🔗 arxiv.org/abs/1505.07376 #ai
Here's the nomenclature I'll be using.

✏️ Beginner-friendly insight or exercise.
🕳️ Related work that's relevant here!
📖 Open research topic of general interest.
💡 Insight or idea to experiment further...

See this thread for context and other reviews:
The work by Gatys et al. is an implementation of a parametric texture model: you extract "parameters" (somehow) from an image, and those parameters describe the image — ideally such that you can reproduce its texture.

I'll be using these textures (photos) as examples throughout:
Read 24 tweets
4 May
It appears that the experiment known as creative.ai will be ending soon, roughly four years after it began. Since I got permission to share, I thought I'd write more about my part in particular and what I could have done differently.

Thread < 1 /🧵>
2/ I say "experiment" because on some level everything is just an experiment, with lessons to be learned from the outcomes. Experiments never fail!

With hindsight I feel more able to discuss this rationally...
3/ However, when you're in the trenches, everything can feel like a critical moment — with the full weight of potential failure behind them.

𝑁𝑜𝑏𝑜𝑑𝑦 𝑠𝑎𝑖𝑑 𝑖𝑡 𝑤𝑎𝑠 𝑒𝑎𝑠𝑦, 𝑏𝑢𝑡 𝑛𝑜 𝑜𝑛𝑒 𝑒𝑣𝑒𝑟 𝑠𝑎𝑖𝑑 𝑖𝑡 𝑤𝑜𝑢𝑙𝑑 𝑏𝑒 𝑡ℎ𝑖𝑠 ℎ𝑎𝑟𝑑.
Read 21 tweets
17 Jan 19
Impressive! There has been significant progress in GANs over the past few years, but that's not really what we're seeing here... [1/8]

(Thread. Buckle up! 👇🔥)
If you were to include images in a Tweet that promoted progress specifically in GAN research (relativistic discriminators, wasserstein constraint, unrolled steps, spectral norm, etc.), it'd look like this... Not quite so catchy! 🤓 [2/8]
It's like a different field of research.

The progress in the latest NVIDIA paper was made 100% with domain-specific changes independent of GANs. The authors say so themselves in the introduction: [3/8]
Read 9 tweets

Did Thread Reader help you today?

Support us! We are indie developers!


This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Too expensive? Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal Become our Patreon

Thank you for your support!

Follow Us on Twitter!