Discover and read the best of Twitter Threads about #manim
Most recents (1)
You may know that @huggingface Accelerate has big-model inference capabilities, but how does that work?
With the help of #manim, let's dig in!
Step 1:
Load an empty model into memory using @PyTorch's `meta` device, so it uses a *super* tiny amount of RAM
With the help of #manim, let's dig in!
Step 1:
Load an empty model into memory using @PyTorch's `meta` device, so it uses a *super* tiny amount of RAM
Step 2:
Load a single copy of the model's weights into memory
Load a single copy of the model's weights into memory
Step 3:
Based on the `device_map`, store the checkpoint weights using @numpy or move it to a device for each group of parameters, and reset our memory
Based on the `device_map`, store the checkpoint weights using @numpy or move it to a device for each group of parameters, and reset our memory
