Releasing the model weights and technical report of Kimi K3.
Kimi K3 is our most capable model: a 2.8T MoE model with native visual understanding and a 1M-token context window.
New model architecture: 2.5x the intelligence per unit of compute, not just more params.
Alongside Kimi K3, we're opening up more of the stack behind it — high-performance attention kernels, MoE communication library, and infrastructure for running agent environments at scale.
🔹 2.8 Trillion Parameters, 1 Million Context, Native Multimodal
🔹 Kimi Delta Attention enables up to 6.3x faster decoding in million-token contexts
🔹 Attention Residuals deliver ~25% higher training efficiency at <2% additional cost
🔹 Built for long-horizon agentic coding and self-evolving workflows
Kimi K3 is now live on on Kimi.com, Kimi Work, Kimi Code, and the Kimi API.
Open Weights by July 27, 2026.
K3 is built on Kimi Delta Attention (KDA) and Attention Residuals (AttnRes), two architectural updates designed to improve how information flows across sequence length and model depth.
We have also scaled up Mixture of Experts (MoE) sparsity, effectively activating 16 out of 896 experts when paired with a Stable LatentMoE framework.
Together with refined training and data recipes, these structural changes yield an approximate 2.5× improvement in overall scaling efficiency compared to K2, allowing the model to convert compute into intelligence more effectively.
Internal knowledge work bench
Beyond public benchmarks, Kimi K3 Max also shows consistent gains on our internal benchmarks, which are built from recurring patterns and challenges in real-world user-agent workflows.
It scores 75.5 on Online Exp Bench, 73.5 on DECK-Bench, and 62.6 on Finance-Bench, outperforming Claude Opus 4.8 (max) and GPT-5.5 (xhigh) across all three.
These results reflect broad improvements in Kimi K3's agentic knowledge work capabilities, enabling more capable and reliable performance in real-world use cases.
Meet Kimi K2.6 agent - Video hero section, WebGL shaders, real backends. From one prompt.
🔹 Video hero sections - cinematic aesthetic, auto-composited
🔹 WebGL shader animations - native GLSL / WGSL, liquid metal, caustics, raymarching
🔹 Motion design - GSAP + Framer Motion
🔹 Backend database: Kimi wires up auth + database + backend in one pass.
🔹 Website stack - React 19 + TypeScript + Vite + Tailwind + shadcn/ui
🔹 3D w/ physically-based lighting - Three.js + React Three Fiber
Video hero sections, built right in.
K2.6 agent calls video generation APIs to create real cinematic footage for your hero, not stock placeholders. Composited into the page, synced to scroll, with shader overlays.
Speaks fluent WebGL shader.
Writes GLSL / WGSL directly - fragment shaders, vertex shaders, noise, SDF, raymarching. Prompt: "a liquid-metal hero with soft caustics."
Residual connections have long relied on fixed, uniform accumulation. Inspired by the duality of time and depth, we introduce Attention Residuals, replacing standard depth-wise recurrence with learned, input-dependent attention over preceding layers.
🔹 Enables networks to selectively retrieve past representations, naturally mitigating dilution and hidden-state growth.
🔹 Introduces Block AttnRes, partitioning layers into compressed blocks to make cross-layer attention practical at scale.
🔹 Serves as an efficient drop-in replacement, demonstrating a 1.25x compute advantage with negligible (<2%) inference latency overhead.
🔹 Validated on the Kimi Linear architecture (48B total, 3B activated parameters), delivering consistent downstream performance gains.
Scaling law experiments reveal a consistent 1.25× compute advantage across varying model sizes.
Analysis of training dynamics demonstrates how AttnRes naturally mitigates hidden-state magnitude growth and yields a more uniform gradient distribution across depth.