Finally shipping Truffle-1: a modern computing stack
Truffle is driven by:
> A Mixture of Models (MoM) acting like a CPU
> An application stack that looks like OpenAI Gym Environments
> A 64GB Orin in a glowing case
🧵 inbound (1/N)
"But what does it do Simp .."
Here's an example of an app that we quickly built on Truffle to buy or sell crypto using tools:
In the video, the agent on device auto-regresses over tools we wrote to accomplish the prompt instead of yapping endlessly.
> An Action Transformer
It runs on MacOS as a little widget right now, and has all the standard features you'd expect when prompting.
Chat however, isn't a thing in our stack, tasks are.
A task is a short term objective that can be accomplished through multi-hop tool use
We're on the agency arc.
The core innovation we're pursuing is that interacting with AI should be and interface to loop over actions, not chat.
We think AGI is downstream of agency - the ability to take actions, ground them in reality, and accomplish medium to longer term tasks.
A wordcel could never.
With Truffle-1 applications become a single Python file containing an array of tools.
Tools are then tokenized and added to the vocabulary of our action transformer
This makes applications like OpenAI gym environments where an agent steps through tools to accomplish an objective
Local AI also means Infinite Inference Time Compute.
Imagine being able to think for 6 months on a hard prompt but paying $0 in inference cost.
At $115/month, why pay for O1 pro when you get a private, agentic computer on a glowing orb you can write and share apps for?
We're officially taking orders and shipping in January.
Initial run limited to 333 units.
To make it even more special, we're hand-delivering 50 units in the U.S.
If selected, we'll show up to your home in an RV with your Truffle.
The Truffle World Tour (the US is the world)
To recap:
> $500 deposit authorized today
> $115/month for 12 months
> Infinite inference time compute
> Writing & Sharing your own apps
> A glowing orb with a 64GB Orin
> We're actually shipping units
> Arriving at your doorstep in late January
Finally, excited to launch Truffle-1 — a $1299 inference engine designed to run OSS models using just 60 watts preorder.itsalltruffles.com
Truffle is not peak $/flop on the market nor is it designed to be. Instead, we built a compiler stack from the ground up for an Nvidia Orin device, a custom carrier board, and a nice shell.
It'll run Mixtral at 22+ tokens/s without speculative decoding, which is 2x faster than GGML on the Nvidia Orin architecture because of our software stack.
Our compiler is not open source. It is optimized for our boards, and thus wouldn't be valuable to OSS anyway
This tweet will cost me followers, make people think I’m a religious nut job, and likely blacklist me from being funded by some VCs but it’s important
> Sam Altman is probably the anti christ
1. He’s setting up a totalitarian system, and knows how to play the charisma card.
Worldcoin: An Iris scan for some useless token. UBI via the world app, and we’re not even allowed to talk about the technology the Orb is built off to make sure it isn’t nefariously storing data
> He did a bait and switch with OpenAI. Nobodies asking for gpt4 weights. But he used OSS research (transformers) to get this far, and won’t even contribute back with knowledge.
He says it’s for “SaFEty” but where have we heard that before? Oh yeah — financial regulation after 9/11, forced vaccinations during Covid. Etc.
The crypto bill should pass with the strictest regulations possible.
A contrarian 🧵 (1/7)
Whitepapers have been published on every single protocol, explaining how their protocols are censorship resistant, decentralized, etc etc. All the buzzwords you'd expect from a crypto project,
None have been battle-tested.
If you've been in crypto for a long time, you knew that the day would come when governments would stop laughing at it, but start fighting it. The day is now here.
What use is crypto if it's unable to withstand even the strictest regulation?