I recently started using both @CohereAI and @pinecone: 2 powerful libraries

I recommended them to my mentees and the rewards were both impactful and instantaneous

@gunaratne_rahel won first place in a hackaton, and @alexbakr shared his POC and it generated 80K+ impressions

🧵 Rahel won first place at the LabLab Transformers AI Hackathon.

He used Cohere and Annoy to cluster @arxiv papers for semantic search

https://twitter.com/gunaratne_rahel/status/1590359886600208384

eDiffi: Text-to-Image Diffusion Models with Ensemble of Expert Denoisers

It uses:
• An ensemble of expert denoising networks
• CLIP Text , T5 Text embeddings, CLIP Image encoders
• CLIP Image: Only used for style transfer

eDiffi has two capabilities: 🧵 1- Style transfer: controls the style of the generated sample using a reference style image

2- "Paint with words": The user can generate images by painting segmentation maps on canvas. Each segment has its description which controls the image layout

🤔 Interested in Leveraging Few-Shot Learning in your projects?

💡 Here is a library that covers both Image Classification and Object Detection Few-Shot Learning Here is also an awesome survey about Low-Shot Learning I already shared:

https://twitter.com/ai_fast_track/status/1569868300535975936

Getting familiar with time series forecasting foundation and breakthroughs isn't an easy task

Here is the survey you were looking for ✔️

It covers time series conventional statistical models and deep learning ones

📰Time Series Analysis and Modeling to Forecast: a Survey It covers:

1- Introduction
2- Stationarity
3- Time Series Decomposition
4- Linear Time Series Models
5- Nonlinear Time Series Models
6- Deep Learning
7- Time Series Model Evaluation
8- Available Implementations
9- Future Directions of Research
10- Glossary

💡 Do you want to reshape your resume/profile and attract more attention?

🔥 🚀 Here is an anwesome resource written by the people who do the resume screening: engineering managers and recruiters working at tech companies. @GergelyOrosz

🎁 Free for developers out of a job! What's Included

• 215 pages
• PDF, EPUB and MOBI formats
• 17 contributing tech industry experts
• 3 resume templates crafted for the book
• 5 resume improvement case studies
• 10 popular resume templates analyzed
• Bonus chapter on COVID-19

Drawing thousands of bounding boxes (bboxes) to annotate your dataset is a pain 🙈

Automagically creating bboxes was a dream until ... 🤖

Zero-Shot Object Detection models appeared: They can annotate most of your dataset for free and in a few minutes 😍

Let's dive in🧵 🤔 Context

First, let's anwer this burning question:

Why would you train a dataset if a Zero Shot (ZD) model can already predict your bounding boxes?

• Very often, the ZD model is unable to detect all the objects (see image: only few bboxes)

• ZD models are genealist ones

Data imbalance can crash your model performance any day of the week 😨

💡 Here is a brief description of each one of them and some potential solutions

• Scale imbalance
• Objective imbalance
• Class imbalance
• Spatial imbalance
🧵 🔸 Scale imbalance
It happens when the objects have different sizes with different numbers of objects: e.g. small objects vs. big objects.

✅ Potential Solution
• Oversample small objects using the Copy&Paste data augmentation

• Use higher resolution images

🤔 You probably never heard of MMRotate

Here are some reasons why you should be familiar with:

• MMRotate is an open-source toolbox for rotated object detection based on PyTorch

• Like the awesome MMDetection library, It is part of the @OpenMMLab project (18 libraries 🤯) • it provides strong baselines and SOTA methods in rotated object detection

• It makes it easy enough to build a new model by combining existing modules

😨 Training an Object Detection Model is a very challenging task and involves tweaking so many knobs

Here is an exhaustive 🎁 tips & tricks list 🎁 that you could use to boost your model performance

🧵 👉 Data Labeling

• Use representative data for each class

• Avoid adding low-quality data

• Small dataset size for pre-trained models

• Bigger dataset size when training from scratch

• Identify and fix incorrect classes

If you are preparing for a Data Science Interview 😰 then this resource is a real gem 💎

A repo with 5.7K ⭐ created by @Al_Grigor and has 72 contributors!

It's divided in 2 parts: Theoretical and Technical Questions

github.com/alexeygrigorev… @Al_Grigor • Part 1: Theoretical Questions

github.com/alexeygrigorev…

Video Panoptic Segmentation: VPSNet

VPSNet is built on MMDetection.

Video Panoptic Segmentation =

Video Instance Segmentation
[things: countable objects, e.g., person, car]
+
Video Semantic Segmentation
[stuff: amorphous regions, e.g., the sky, road] Paper: arxiv.org/abs/2006.11339
Repo: github.com/mcahny/vps

Someone asked me if classification would be good for detecting small objects.

The answer is yes, but not what you might think. Let's see why:

A pure classification model won't probably be the best option for that task. Let's suppose your objects are 32 x 32 px: 👇 - Your feature map for those objects will be 4 x 4 px at C3 layer (subsampling = 2^3. So, 32px / 8. 8 = 4px)

- At the C5 layer (last one), your feature map for those objects will be 1 x 1 px (subsampling = 2^5 = 32)

Those features are too tiny to capture those small objects.

Here is a summary of summaries about:

• Creating a Deep Learning Pipeline
• Deploying Models on AWS Lambda
• Deploying Models on Edge Devices
• Showcasing Models Hugging Face Spaces
👇 1- Creating a Deep Learning Pipeline

https://twitter.com/ai_fast_track/status/1482759861519761409

🎉 Celebrating 100 days of sharing Visual Summaries in Computer Vision

I plan to continue sharing:
• More content
• Summaries of summaries by topic

Follow me for more threads on advanced computer vision techniques used in industry-level applications → @ai_fast_track - 12 visual summaries on OD Modeling:

https://twitter.com/ai_fast_track/status/1463177309381402634

🌟 VFNet: IMHO, is the best anchor-free single-stage model, and it's not under the radar.

VariFocalNet: An IoU-aware Dense Object Detector

🧊 Background:
📌 Accurately ranking candidate detections is crucial for dense object detectors to achieve high performance.
... 📌 Prior work uses the classification score or a combination of classification and predicted localization scores (centerness) to rank candidates.

📌 Those 2 scores are still not optimal.

🧊 Novelty:
📌 VFNet proposes to learn an IoU-Aware Classification Score (IACS)

4 types of imbalance issues in object detection that you should know:

Here is a brief description of each one of them and some potential solutions.

• Scale imbalance
• Objective imbalance
• Class imbalance
• Spatial imbalance 🔸 Scale imbalance
It happens when the objects have different sizes with different numbers of objects: e.g. small objects vs. big objects.

✅ Potential Solution
• Oversample small objects using the Copy&Paste data augmentation
• Use higher resolution images

How do you use transfer learning with images with 3+ (or 1) channel(s)?

Timm library, developed by @wightmanr, has an elegant way to handle that:

You can specify any input channel number (e.g. in_chans=1 or in_chans=8) using timm.create_model() function like this: @wightmanr m = timm.create_model('resnet34', pretrained=True, in_chans=8)

How does it work?

• Case 1: number of input channels is 1
timm simply sums the 3 channel weights into one single channel

Here is a mega-summary of my YOLO-Series Visual Summaries:

1- YOLO Family Real-Time Performance

https://twitter.com/ai_fast_track/status/1474969748626673666

2- IA-YOLO improves object detection in adverse weather conditions using a hybrid task.

Image improvement combined with object detection.

https://twitter.com/ai_fast_track/status/1472070080557113344

🔥 ZSD-YOLO: Zero-Shot YOLO Detection using Vision-Language Knowledge Distillation

Heads up: I’m preparing a visual summary on ZSD-YOLO.

So, what is Zero-Shot Detection? • Zero-shot detection allows a model to detect something in an image even if the model has never seen that thing before

• So, if you have an image of a Chimpanzee and the model has never seen a Chimpanzee before, you can use your zero-shot detector to locate it in the image

Many open-world applications require the detection of novel objects.

but state-of-the-art object detection and instance segmentation models are unable to do so.

• It’s because models learn to suppress any unannotated objects by treating them as background • To address that issue, the authors propose a simple yet surprisingly powerful data augmentation and training scheme they call Learning to Detect Every Thing (LDET)

❓ What is Multi-Scale Training (MST)?

💡 MTS helps your model to be robust to image sizes, an get better performance

• Training on small images is faster

• Training on large images increases your model performance

How is MST done? Every N (e.g., 10) epochs, we randomly chooses a new image dimension from a range of sizes [640, 768, 800], and train our model

This means the same network becomes better at predicting at different resolutions.