Introducing 𝑨𝒕𝒕𝒆𝒏𝒕𝒊𝒐𝒏 𝑹𝒆𝒔𝒊𝒅𝒖𝒂𝒍𝒔: Rethinking depth-wise aggregation.

Residual connections have long relied on fixed, uniform accumulation. Inspired by the duality of time and depth, we introduce Attention Residuals, replacing standard depth-wise recurrence with learned, input-dependent attention over preceding layers.

🔹 Enables networks to selectively retrieve past representations, naturally mitigating dilution and hidden-state growth.
🔹 Introduces Block AttnRes, partitioning layers into compressed blocks to make cross-layer attention practical at scale.
🔹 Serves as an efficient drop-in replacement, demonstrating a 1.25x compute advantage with negligible (<2%) inference latency overhead.
🔹 Validated on the Kimi Linear architecture (48B total, 3B activated parameters), delivering consistent downstream performance gains.

🔗Full report:
github.com/MoonshotAI/Att… Scaling law experiments reveal a consistent 1.25× compute advantage across varying model sizes.

Meet Kimi Agentic Slides!
Now with Nano Banana Pro 🍌

🎁 Thanksgiving Gift: 48H FREE & UNLIMITED ACCESS

🔸 Agentic search (Kimi K2)
🔸 Files → Slides (PDFs, images, docs+)
🔸 Fully editable + PPTX export
🔸 Designer-level visuals (infographics, illustrations)

Try now: kimi.com/slides Here's a quick guide. 👇

🚀 Hello, Kimi K2! Open-Source Agentic Model!
🔹 1T total / 32B active MoE model
🔹 SOTA on SWE Bench Verified, Tau2 & AceBench among open models
🔹Strong in coding and agentic tasks
🐤 Multimodal & thought-mode not supported for now

With Kimi K2, advanced agentic intelligence is more open and accessible than ever. We can't wait to see what you build!

🔌 API is here: platform.moonshot.ai
- $0.15 / million input tokens (cache hit)
- $0.60 / million input tokens (cache miss)
- $2.50 / million output tokens

🔗 Tech blog: moonshotai.github.io/Kimi-K2/
🔗 Weights & code: huggingface.co/moonshotai
🔗 Github: github.com/MoonshotAI/Kim…
Try it now at Kimi.ai or via API! Here are some vibe tests we ran:

1. Interactive 3D Mountain Scene

Meet Kimi-Researcher - an autonomous agent that excels at multi-turn search and reasoning. Powered by k 1.5 and trained with end-to-end agentic RL.

Achieved 26.9% pass@1 on Humanity's Last Exam, 69% pass@1 on xbench.

🔗 Tech blog： moonshotai.github.io/Kimi-Researche… Benchmarks aside, It thinks:
→ 23 reasoning steps per task (avg.)
→ 200+ URLs explored
→ Multi-turn tool use of search, browser, and code
→ Inline citations

Beta access is rolling out at kimi.com — get on the waitlist 👉 [docs.google.com/forms/d/e/1FAI…]