Akshay πŸš€ Profile picture
Apr 13, 2024 β€’ 9 tweets β€’ 3 min read β€’ Read on X
LEGB rule in Python, clearly explained:
Every Python developer must know the LEGB rule!

It's crucial for avoiding major bugs!

And today, I will provide a clear explanation of the same!

Let's go! πŸš€ Image
Let's start by understanding meaning of scope❗️

The scope of a variable defines the area of a program from where you can access it.

The name will only be visible to and accessible by the code in its scope.

Here's an illustration of the same...πŸ‘‡ Image
1️⃣ Local Scope:

Local scope refers to variables defined within a function.

These variables are only accessible within the function where they are defined.

They are created at the function's call and destroyed when the function ends.

Example...πŸ‘‡ Image
2️⃣ Enclosing Scope:

Enclosing scope is relevant in the context of nested functions.

If a variable isn't in the local scope but is defined in an outer function, it's in the enclosing scope.

It is accessible from the inner function but not from the global scope!

Example...πŸ‘‡ Image
3️⃣ Global scope

Global scope encompasses variables defined at the top level of a script or a module.

Global variables are accessible from anywhere within the module or script, including inside any functions.

Created when the script starts and last until it ends.

Example...πŸ‘‡ Image
4️⃣ Built-in scope

Built-in scope includes names that are pre-defined in the Python language.

This scope contains functions like `print()`, ` len()`, and types like `int`, `float`, which are always available without the need for any imports.

Example ...πŸ‘‡ Image
What if scopes collide❓

When scopes collide in Python, the LEGB rule plays a crucial role in determining how variables are resolved.

Python always looks for a variable in the order:

Local ➝ Enclosing ➝ Global ➝ Builtin

Here's a good example: Image
That's a wrap!

If you interested in:

- Python 🐍
- ML/MLOps πŸ› 
- CV/NLP πŸ—£
- LLMs 🧠
- AI Engineering βš™οΈ

Find me β†’ @akshay_pachaar βœ”οΈ
Everyday, I share tutorials on above topics!

Cheers!

β€’ β€’ β€’

Missing some Tweet in this thread? You can try to force a refresh
γ€€

Keep Current with Akshay πŸš€

Akshay πŸš€ Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

PDF

Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @akshay_pachaar

Jun 7
Google just dropped a new LLM!

You can run it locally on just 8GB RAM.

Let's fine-tune this on our own data (100% locally):
Google released Gemma 4 12B, a multimodal model that runs text, images, and audio on 8GB VRAM!

We'll fine-tune it to master chess and predict the exact next move.

Tech stack:
- @UnslothAI for efficient fine-tuning.
- @huggingface transformers to run it locally.

Let's go! πŸš€
1️⃣ Load the model

We start by loading Gemma 4 12B and its tokenizer using Unsloth.

Check this πŸ‘‡ Image
Read 10 tweets
Jun 3
You're in a Research Scientist interview at OpenAI.

The interviewer asks:

"How would you expand the context length of an LLM from 2K to 128K tokens?"

You: "I will fine-tune the model on longer docs with 128K context."

Interview over.

Here's what you missed:
Extending the context window isn't just about larger matrices.

In a traditional transformer, expanding tokens by 8x increases memory needs by 64x due to the quadratic complexity of attention. Refer to the image below!

So, how do we manage it?

continue...πŸ‘‡ Image
1) Sparse Attention

It limits the attention computation to a subset of tokens by:

- Using local attention (tokens attend only to their neighbors).
- Letting the model learn which tokens to focus on.

But this has a trade-off between computational complexity and performance. Image
Read 12 tweets
Dec 18, 2025
Turn any Autoregressive LLM into a Diffusion LM.

dLLM is a Python library that unifies the training & evaluation of diffusion language models.

You can also use it to turn ANY autoregressive LM into a diffusion LM with minimal compute.

100% open-source.
Here's why this matters:

Traditional autoregressive models generate text left-to-right, one token at a time. Diffusion models work differently - they refine the entire sequence iteratively, giving you better control over generation quality and more flexible editing capabilities.
dLLM GitHub:

(don't forget to star 🌟)github.com/ZHZisZZ/dllm
Read 4 tweets
Dec 6, 2025
You're in a Research Scientist interview at Google.

Interviewer: We have a base LLM that's terrible at maths. How would you turn it into a maths & reasoning powerhouse?

You: I'll get some problems labeled and fine-tune the model.

Interview over.

Here's what you missed:
When outputs are verifiable, labels become optional.

Maths, code, and logic can be automatically checked and validated.

Let's use this fact to build a reasoning model without manual labelling.

We'll use:

- @UnslothAI for parameter-efficient finetuning.
- @HuggingFace TRL to apply GRPO.

Let's go! πŸš€
What is GRPO?

Group Relative Policy Optimization is a reinforcement learning method that fine-tunes LLMs for math and reasoning tasks using deterministic reward functions, eliminating the need for labeled data.

Here's a brief overview of GRPO before we jump into code:
Read 11 tweets
Dec 5, 2025
I have been training neural networks for 10 years now.

Here are 16 ways I actively use to optimize model training:

(detailed explanation ...🧡)
First, lets look at some basic techniques:

1) Use efficient optimizersβ€”AdamW, Adam, etc.

2) Utilize hardware accelerators (GPUs/TPUs).

3) Max out the batch size.

4) Use multi-GPU training through Model/Data/Pipeline/Tensor parallelism.

Check the visualπŸ‘‡
5) Bayesian optimization for hyperparameter optimization:

This technique takes informed steps based on the results of the previous hyperparameter configs.

This way, the model converges to an optimal set of hyperparameters much faster.

Check these results πŸ‘‡ Image
Read 9 tweets
Nov 23, 2025
You’re in an ML Engineer interview at Google.

Interviewer: We need to train an LLM across 1,000 GPUs. How would you make sure all GPUs share what they learn?

You: Use a central parameter server to aggregate and redistribute the weights.

Interview over.

Here’s what you missed:
One major run-time bottleneck in multi-GPU training happens during GPU synchronization.

For instance, in multi-GPU training via data parallelism:

- The same model is distributed to different GPUs.
- Each GPU processes a different subset of the whole dataset.

Check this πŸ‘‡
This leads to different gradients across different devices.

So, before updating the model parameters on each GPU device, we must communicate the gradients to all other devices to sync them.

Let’s understand 2 common strategies next!
Read 14 tweets

Did Thread Reader help you today?

Support us! We are indie developers!


This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Don't want to be a Premium member but still want to support us?

Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal

Or Donate anonymously using crypto!

Ethereum

0xfe58350B80634f60Fa6Dc149a72b4DFbc17D341E copy

Bitcoin

3ATGMxNzCUFzxpMCHL5sWSt4DVtS8UqXpi copy

Thank you for your support!

Follow Us!

:(