Akshay 🚀 Profile picture
Simplifying LLMs, MLOps, Python & Machine Learning for you! • AI Engineering @LightningAI • Lead DataScientist • BITS Pilani • 3 Patents
Ella Sanders Profile picture Rodolpho Gurgel Profile picture Shashi Kumar Profile picture Learning in Public - Coding - DataSci Profile picture Heroeroo Profile picture 9 subscribed
Feb 21 • 4 tweets • 3 min read
Vector Embeddings, clearly explained: Embeddings are one of the most powerful ideas in the world of Machine Learning.

They are the building blocks of the powerful LLMs that we see today!

So, let's understand what embeddings are and how they came into the picture! 🚀

❗️Computers, particularly neural networks, are good with numbers & when it comes to LLMs, they are trained on large corpus of words!

So, Instead of teaching them with the dictionary definition of each word, we allow them to learn a mathematical representation, which we call embeddings (a series of numbers)

Now these numbers i.e. embeddings are more powerful than you think, they capture:

1️⃣ Meaning: Words come with a myriad of meanings, contexts, and nuances. Embeddings pack all this complex information into a dense vector, compressing vast amounts of information into a small space.

2️⃣ Context: Words are understood by their company. "Apple" in the context of "orchard" is different from "Apple" in the context of "MacBook". Embeddings capture these contextual nuances.

3️⃣ Semantic Relationship: Embeddings can capture relationships between words, words which have similar meaning or say are closer to each other will be closer in the embedding space as well!

Let’s understand these points with a very famous analogy (you can now refer the image below as you read)!

King : Queen :: Man : Woman

The word king might be close to the man because they're both male figures. Similarly, the queen might be close to the woman.

But here's the magical part, let’s say we are in the embedding space.

🔹And, if we know the direction and distance from the queen to the king (Gender vector in the image below), we can use the same direction and distance starting from the Woman to find where the Man should be in this embedding space.

🔸Similarly if we know the direction & distance from Man to King (Royalty vector in the image below), we can use the same direction and distance starting from the woman to find where the queen should be in this embedding space.

🔹When we perform these vector arithmetic operations on properly trained embeddings, we get results that capture these relationships.

This makes the embeddings truly magical & powerful!✨

Today we can convert a large corpus of text, documents, images & audio into embeddings.

We have sophisticated databases to store and index embeddings known as vector data bases.

With these databases, we can efficiently perform various operations on the embeddings and build powerful systems on top of them.

If you're interested in LLMs & AI Engineering, understanding the concept of embeddings and being able to work with them is crucial!

In the next tweet I've shared two links:

1. A FREE AI Engineering Newsletter that I write.

2. @LightningAI's LLM Learning Lab, FREE world class learning material on training, fine-tuning & deploying LLMs at scale.

That’s a wrap!

I love breaking down complex ideas in AI and Machine Learning to their fundamentals.

If you enjoyed reading this, follow me → @akshay_pachaar, so you don't miss my updates.Image
Feb 19 • 5 tweets • 2 min read
Data Classes in Python, clearly explained: Data classes in Python are amazing!

They provide:
- Quick initialisation
- Easy comparison
- Concise representation
- And more ...

Check this out 👇 Image
Feb 14 • 7 tweets • 2 min read
K-Nearest Neighbours (KNN), clearly explained: KNN involves 4 simple steps:

1️⃣ Selecting the the value of K

2️⃣ Calculate the distance between the new & existing data points

3️⃣ Find K closest points (neighbours)

4️⃣ For classification, vote for the majority class, for regression, compute the average.

Let's code it ...👇 Image
Feb 13 • 8 tweets • 3 min read
LoRA: Low-Rank Adaptation of LLMs, clearly explained: LoRA (Low-Rank Adaptation) is one of the most powerful techniques when it comes to Fine-Tuning LLMs.

Today I’ll explain:

• What is Lora❓
• How it works ❓
• Followed by a hands-on coding tutorial❗️

But before we do that, let's understand what finetuning means...👇
Feb 10 • 8 tweets • 3 min read
Precision, Recall clearly explained: Let's say there are 10 people in a Town.

2 of them have committed a crime, so in reality:

- 8 are innocent
- 2 are guilty

This is how it looks 👇 Image
Feb 9 • 12 tweets • 5 min read
10 great Python packages for Data Science not known to many: 1️⃣ Mitosheets

Write Python code by editing a spreadsheet in your Jupyter Notebook, so you can easily explore and transform your data with Excel spreadsheet formulas, pivot tables, and graphs

Check this out👇 | cc: @tryMito
trymito.io/?source=pc&utm…
Feb 7 • 10 tweets • 3 min read
How LLMs work, clearly explained: Before diving into LLMs, we must understand conditional probability.

Let's consider a population of 14 individuals:

- Some of them like Tennis 🎾
- Some like Football ⚽️
- A few like both 🎾 ⚽️
- And few like none

Here's how it looks 👇 Image
Feb 2 • 6 tweets • 2 min read
Python *args & **kwargs clearly explained: *args allows you to pass a variable number of non-keyword arguments to a function.

It collects all non-keyword arguments passed to the function and stores them as a tuple.

Consider the following example: Image
Jan 29 • 5 tweets • 2 min read
PCA from scratch using Python 🔥

Principal Component Analysis is a powerful technique for dimensionality reduction and data visualisation.

-- step by step explanation with code --

Let's go! 🚀 Image We'll use wine_dataset which has 13 features (dimensions) originally!!

Check this out 👇 Image
Jan 28 • 7 tweets • 2 min read
PyTorch 101: Dataset Class clearly explained!

Today, we learn how to build a PyTorch Dataset for:

- Text
- Tabular
- and Imagery Data

Let's go! 🚀 Image First things first, let's look at a simple example of a `Dataset` class.

Here's what it generally looks like: Image
Jan 27 • 9 tweets • 3 min read
Self-attention clearly explained: Before we start a quick primer on tokenization!

Raw text → Tokenization → Embedding → Model

Embedding is a meaningful representation of each token (roughly a word) using a bunch of numbers.

This embedding is what we provide as an input to our language models.

Check this👇 Image
Jan 25 • 9 tweets • 3 min read
Run Mistral-7B almost 3x faster on a single GPU!

Introducing AWQ: Activation-aware Weight Quantization! 🚀

A hands-on tutorial with code...👇 Image In this tutorial we'll be using vLLM, a fast and easy-to-use library for LLM inference and serving.

The tutorial has 3 parts:

- Part 1: Inference on 1 GPU!
- Part 2: Distributed Inference on 4 GPUs
- Part 3: Inference on 1 GPU + AWQ!⚡️

Continue reading... 📖 Image
Jan 24 • 12 tweets • 4 min read
I started my career in Data Science back in 2016 ⏳

Being self-taught, YouTube has been a great source of learning for me!

Here are some of the channels I've ardently followed: 1️⃣ StatQuest:

BAM!!

Joshua Stammer's fun take on teaching Stats & Mathematics for ML to masses is an absolute gold mine 🥇

Check this out 👇
youtube.com/c/joshstarmer
Jan 23 • 6 tweets • 2 min read
PyTorch 101: Understanding Backpropagation: Backpropagation is all about optimizing our neural network by adjusting it's weights.

And it's done by calculating gradients of the loss function w.r.t. these weights!

Today, we learn how to do this using PyTorch...👇 Image
Jan 22 • 8 tweets • 3 min read
List comprehension in Python clearly explained: Simply put, list comprehension are a concise way to replace multi-line for loops with a single line of code!

And, they are reasonably faster 🔥

Let's take a quick look at the Syntax before we break it down with examples!👇 Image
Jan 19 • 9 tweets • 3 min read
Retrieval Augmented Generation (RAGs) clearly explained!

Here's an example of how you can create a ChatGPT-like interface using your own knowledge base.

Let's understand each component one-by-one...👇 Retrieval Augmented Generation (RAG) is a powerful tool for enhancing the performance of Large Language Models (LLMs) by incorporating external knowledge into the generation process.

Let's explore the key components of RAG...👇
Jan 18 • 9 tweets • 3 min read
I started my career in Data Science back in 2016 ⏳

Here are 7 tips for those starting out today: 1️⃣ Become a strong programmer 🔥

It helps you to bridge the gap between theory and building stuff that matters!

If you're new to programming, Harvard's CS50: Intro to AI with Python is a great starting point!

It's FREE & self-paced!

Check this out👇
pll.harvard.edu/course/cs50s-i…
Image
Jan 17 • 6 tweets • 2 min read
Lambda functions in Python clearly explained: What are lambda functions ?

Simply put, they are small anonymous functions that are defined without a name.

Check out the syntax 👇 Image
Jan 16 • 10 tweets • 3 min read
Important statistical techniques that you should know!

Explained with examples...👇 Image 1️⃣ Counterfactual Causal Inference:

Compares what actually happened with what could have happened under different circumstances.

Eg: A new drug trial: Helps understand impact by comparing patient outcomes with a scenario where the drug wasn't administered. Image
Jan 15 • 6 tweets • 2 min read
Cosine similarity clearly explained!

A geometric intuition behind the most commonly used similarity metric in modern Vector Databases!

Let's go! 🚀 Image Quick recap of how a Vector DB works!

A vector database indexes and stores vector embeddings for fast retrieval and similarity search!

Check this out👇 Image
Jan 14 • 10 tweets • 4 min read
Autoencoders are one of my favourite Neural Networks!

They are simple yet so powerful!

Today, We'll understand:

- What Autoencoders are❓
- How they are trained
- The applications of Autoencoders
- And how to implement them from scratch using PyTorch Lightning!⚡️

Let's go! 🚀 Image Autoencoders have two main parts:

1️⃣ Encoder: Compresses the input into a dense representation (latent space)

2️⃣ Decoder: Reconstructs the input from this dense representation.

The idea is to make the reconstructed output as close to the original input as possible:👇 Image