NVIDIA's $7B Mellanox acquisition was actually one of tech's most strategic deals ever.
The untold story of the most important company in AI that most people haven't heard of
1/12
The untold story of the most important company in AI that most people haven't heard of
1/12
Most people think NVIDIA = GPUs. But modern AI training is actually a networking problem.
A single A100 can only hold ~50B parameters. Training large models requires splitting them across hundreds of GPUs.
2/12
A single A100 can only hold ~50B parameters. Training large models requires splitting them across hundreds of GPUs.
2/12
Enter Mellanox.
They pioneered RDMA (Remote Direct Memory Access) which lets GPUs directly access memory on other machines with almost no CPU overhead. Before RDMA, moving data between GPUs was a massive bottleneck.
3/12
They pioneered RDMA (Remote Direct Memory Access) which lets GPUs directly access memory on other machines with almost no CPU overhead. Before RDMA, moving data between GPUs was a massive bottleneck.
3/12
The secret sauce is in Mellanox's InfiniBand.
While Ethernet does 200-400ns latency, InfiniBand does ~100ns. For distributed AI training where GPUs constantly sync gradients, this 2-3x latency difference is massive.
4/12
While Ethernet does 200-400ns latency, InfiniBand does ~100ns. For distributed AI training where GPUs constantly sync gradients, this 2-3x latency difference is massive.
4/12
Mellanox didn't just do hardware.
Their GPUDirect RDMA software stack lets GPUs talk directly to network cards, bypassing CPU & system memory. This cuts latency another ~30% vs traditional networking stacks.
5/12
Their GPUDirect RDMA software stack lets GPUs talk directly to network cards, bypassing CPU & system memory. This cuts latency another ~30% vs traditional networking stacks.
5/12
NVIDIA's master stroke: Integrating Mellanox's ConnectX NICs directly into their DGX AI systems.
The full stack - GPUs, NICs, switches, drivers - all optimized together. No one else can match this vertical integration.
6/12
The full stack - GPUs, NICs, switches, drivers - all optimized together. No one else can match this vertical integration.
6/12
The numbers are staggering:
- HDR InfiniBand: 200Gb/s per port
- Quantum-2 switch: 400Gb/s per port
- End-to-end latency: ~100ns
- GPU memory bandwidth matching: ~900GB/s
7/12
- HDR InfiniBand: 200Gb/s per port
- Quantum-2 switch: 400Gb/s per port
- End-to-end latency: ~100ns
- GPU memory bandwidth matching: ~900GB/s
7/12
Why it matters: Training SOTA scale models requires:
- 1000s of GPUs
- Petabytes of data movement
- Sub-millisecond latency requirements
Without Mellanox tech, it would take literally months longer.
8/12
- 1000s of GPUs
- Petabytes of data movement
- Sub-millisecond latency requirements
Without Mellanox tech, it would take literally months longer.
8/12
The competition is playing catch-up:
- Intel killed OmniPath
- Broadcom/Ethernet still has higher latency
- Cloud providers mostly stuck with RoCE
NVIDIA owns the premium AI networking stack
9/12
- Intel killed OmniPath
- Broadcom/Ethernet still has higher latency
- Cloud providers mostly stuck with RoCE
NVIDIA owns the premium AI networking stack
9/12
Looking ahead: CXL + Mellanox tech will enable even tighter GPU-NIC integration.
We'll see dedicated AI networks with sub-50ns latency and Tb/s bandwidth. The networking advantage compounds.
10/12
We'll see dedicated AI networks with sub-50ns latency and Tb/s bandwidth. The networking advantage compounds.
10/12
In the AI arms race, networking is the silent kingmaker.
NVIDIA saw this early.
The Mellanox deal wasn't about current revenue - it was about controlling the foundational tech for training next-gen AI.
11/12
NVIDIA saw this early.
The Mellanox deal wasn't about current revenue - it was about controlling the foundational tech for training next-gen AI.
11/12
Next time you hear about a new large language model breakthrough, remember: The GPUs get the glory, but Mellanox's networking makes it possible.
Sometimes the most important tech is invisible.
12/12
Sometimes the most important tech is invisible.
12/12
• • •
Missing some Tweet in this thread? You can try to
force a refresh
