In January, I speculated how "the most impactful outcome from DeepSeek's rise may ultimately be closer collaboration with Huawei and other chip designers".
We now have direct evidence of this collaboration, with potential standardization around UE8M0 the first major tangible result.
While some may dismiss this as technical or esoteric jargon relevant only to AI, computer science, or math enthusiasts ... I will try to explain here in plain language some key market and geopolitical implications of this development.
We now have direct evidence of this collaboration, with potential standardization around UE8M0 the first major tangible result.
While some may dismiss this as technical or esoteric jargon relevant only to AI, computer science, or math enthusiasts ... I will try to explain here in plain language some key market and geopolitical implications of this development.
First I want to acknowledge others who are much closer to DeepSeek and AI for both raising, highlighting and explaining these recent developments, particularly @teortaxesTex @zephyr_z9 and @Compute_King
In this 🧵 I am merely synthesizing the insights and knowledge gained from following their timelines and trying to add value by layering on market and geopolitical insights.
In this 🧵 I am merely synthesizing the insights and knowledge gained from following their timelines and trying to add value by layering on market and geopolitical insights.
In particular I highly recommend first digesting this post on the technical and strategic implications of UE8M0:
https://x.com/Compute_King/status/1959245865328984144
Okay, what is UE8M0 and why does it matter?
UE8M0 is a custom 8-bit floating-point number format (FP8) proposed by DeepSeek, designed to improve memory efficiency + hardware compatibility with nextgen AI chips.
(How numbers are represented is critical in AI training due to the gazillion calculations that are performed over the course of a training run. Each operation represents some quanta of time and energy, which ultimately translates to cost.)
One of DeepSeek's more notable innovations was relying on FP8 to train its v3 model released last December instead of 16-bit format FP16. They found that the increase in efficiency was worth the trade-off in accuracy (which could be mitigated through other algorithmic optimizations).
UE8M0 is a custom 8-bit floating-point number format (FP8) proposed by DeepSeek, designed to improve memory efficiency + hardware compatibility with nextgen AI chips.
(How numbers are represented is critical in AI training due to the gazillion calculations that are performed over the course of a training run. Each operation represents some quanta of time and energy, which ultimately translates to cost.)
One of DeepSeek's more notable innovations was relying on FP8 to train its v3 model released last December instead of 16-bit format FP16. They found that the increase in efficiency was worth the trade-off in accuracy (which could be mitigated through other algorithmic optimizations).
You may recall the ridiculous debate over the "real" cost of training DeepSeek v3 (and how many chips they really used to train the model 👇).
Well, relying on FP8 was a key reason why DeepSeek's training costs were so much lower than other models.
Well, relying on FP8 was a key reason why DeepSeek's training costs were so much lower than other models.
https://x.com/GlennLuk/status/1902675419461005714
Notably, to implement FP8, DeepSeek had to use nVidia chips.
Domestic alternatives like Huawei's latest Ascend chips did not natively support FP8 (aside: which is why claims that reliance on Huawei is what delayed DeepSeek made little sense 👇).
Domestic alternatives like Huawei's latest Ascend chips did not natively support FP8 (aside: which is why claims that reliance on Huawei is what delayed DeepSeek made little sense 👇).
https://x.com/GlennLuk/status/1956690547575325045
But while nVidia provides native support for two variants of FP8, DeepSeek is now moving to UE8M0 with v3.1, which nVidia does not (yet) support.
By announcing model standardization around UE8M0, this creates a new space for China's domestic AI chip industry to fill, who now find themselves in a novel and unfamiliar position of having first-mover advantage instead of constantly having to play catchup.
By announcing model standardization around UE8M0, this creates a new space for China's domestic AI chip industry to fill, who now find themselves in a novel and unfamiliar position of having first-mover advantage instead of constantly having to play catchup.
If standardization around UE8M0 proves to deliver key technical benefits — and we won't know until we see it widely implemented/deployed — China will have taken a rather significant step to "fork" AI development paradigm away from a hardware-dominated ecosystem.
This standards-setting has historical precedent:
It reminded me of the debate on "polar coding" & "low-density parity check" during the 3GPP-led 5G standards debate in the 2010s.
(note: both methods were eventually incorporated in 5G standards)
It reminded me of the debate on "polar coding" & "low-density parity check" during the 3GPP-led 5G standards debate in the 2010s.
(note: both methods were eventually incorporated in 5G standards)
https://x.com/GlennLuk/status/1329026825247281152
Indeed, the 5G wars merely presaged continued future "forking" and bifurcation in global technology standards, as I also speculated on in this 2018 piece.
https://x.com/GlennLuk/status/1775204246469681655
DeepSeek's move to UE8M0 has significant market and geopolitical implications.
We have already seen notable market reaction e.g. AI chip company Cambricon's stock surging to an all-time high.
ft.com/content/3bae71…
We have already seen notable market reaction e.g. AI chip company Cambricon's stock surging to an all-time high.
ft.com/content/3bae71…
While the original "DeepSeek moment" had clear implications on (the shift to) AI inference 👇 and downstream use cases, the implication of a forked, Chinese-centered AI development ecosystem now directly impacts the training stage too.
https://x.com/GlennLuk/status/1884220643614572726
IOW while DeepSeek's v3 release already made it quite clear that nVidia would not be able to maintain a dominant monopoly position in AI inference, its CUDA-driven moat seemed like it might hold (e.g. DeepSeek had to continue using nVidia chips to train) and allow it to continue to maintain near-monopoly global market share in training (including in China) for the foreseeable future.
By potentially shifting the "standards-setting" power from hardware to the software (model layer), this development strikes directly at its moat in training.
By potentially shifting the "standards-setting" power from hardware to the software (model layer), this development strikes directly at its moat in training.
P.S. This does not mean nVidia is "toast"!
First, it is not yet clear whether the technical benefits of this new UE8M0 standard are realized.
Second, nVidia is still likely to maintain its dominant market position in Western tech ecosystem.
Third, driving further innovation in AI will only increase demand for nVidia chips. Any declines in market share can be outweighed by increasing TAM.
Last and perhaps most significantly, nVidia got to this point by being a rockstar company with elite technical capabilities. If UE8M0 delivers those technical benefits, nVidia will rapidly move to support it.
First, it is not yet clear whether the technical benefits of this new UE8M0 standard are realized.
Second, nVidia is still likely to maintain its dominant market position in Western tech ecosystem.
Third, driving further innovation in AI will only increase demand for nVidia chips. Any declines in market share can be outweighed by increasing TAM.
Last and perhaps most significantly, nVidia got to this point by being a rockstar company with elite technical capabilities. If UE8M0 delivers those technical benefits, nVidia will rapidly move to support it.
From a geopolitics perspective, the Chinese AI ecosystem is now taking an increasing leadership role and driving standards.
While the Western tech ecosystem is likely to remain mostly protected and "safe" from Chinese competition, Chinese and Western AI ecosystems will compete directly in "neutral" battleground states around the world.
While the Western tech ecosystem is likely to remain mostly protected and "safe" from Chinese competition, Chinese and Western AI ecosystems will compete directly in "neutral" battleground states around the world.
This map — now-defunct Biden-era U.S. diffusion rules tiering — provides a decent proxy on levels of this neutrality — at least from the U.S. perspective
(notably, India might have also recently changed colors in recent weeks)
(notably, India might have also recently changed colors in recent weeks)
https://x.com/GlennLuk/status/1910046703610396707
These recent developments are also yet another blow to the original rationale for U.S. export controls — whose main/explict goal was to contain the progress of Chinese AI development.
Export controls on advanced chip fabrication merely forced the Chinese tech ecosystem to pursue optimizations elsewhere in the technology stack.
We saw that with DeepSeek selecting FP8 for v3 and getting "low level" with PTX to unlock untapped horsepower in export-restriction-crippled nVidia H800s; we saw it in Huawei leveraging core expertise in optical networking with CloudMatrix384.
And now with DeepSeek’s close collaboration with China’s AI chip ecosystem, export controls have provided the missing impetus to transform a previously chaotic and fragmented sector into a more coordinated, unified effort.
The last one is analogous to a similar coordinating effect that export controls had on accelerating China's SME sector.
Export controls on advanced chip fabrication merely forced the Chinese tech ecosystem to pursue optimizations elsewhere in the technology stack.
We saw that with DeepSeek selecting FP8 for v3 and getting "low level" with PTX to unlock untapped horsepower in export-restriction-crippled nVidia H800s; we saw it in Huawei leveraging core expertise in optical networking with CloudMatrix384.
And now with DeepSeek’s close collaboration with China’s AI chip ecosystem, export controls have provided the missing impetus to transform a previously chaotic and fragmented sector into a more coordinated, unified effort.
The last one is analogous to a similar coordinating effect that export controls had on accelerating China's SME sector.
Chinese AI development has not been impeded in any apparent way and it may even be leading now in downstream use.
Export controls have not made AI usage more expensive or materially less capable in China. Pricing is an order-of-magnitude lower:
Export controls have not made AI usage more expensive or materially less capable in China. Pricing is an order-of-magnitude lower:
https://x.com/GlennLuk/status/1958594201421324416
With such low pricing, not surprisingly commercialization and token usage (as a proxy for downstream use cases) is skyrocketing and may even exceed the U.S. now.
https://x.com/GlennLuk/status/1958959799606854125
It is looking increasingly likely that history is going to look back on the export controls as the 21st century version of France's Maginot Line.
https://x.com/GlennLuk/status/1877341873720930479
Export controls traded dubious short-run benefits — the fanciful theory that achieving AGI/ASI would create some insurmountable long-term technology/economic advantage ...
... for long-term erosion in some of the most powerful economic and technology moats that American companies like nVidia had built up through decades of industry leadership.
... for long-term erosion in some of the most powerful economic and technology moats that American companies like nVidia had built up through decades of industry leadership.
IMO a massive self-own, and possibly a consequence of technology-related decisions with massive LT strategic implications being made in a society dominated lawyers instead of engineers 👇.
(h/t to @danwwang and his upcoming new book "Breakneck")
(h/t to @danwwang and his upcoming new book "Breakneck")
https://x.com/GlennLuk/status/1766171034334396836
• • •
Missing some Tweet in this thread? You can try to
force a refresh
