jeff.hl Profile picture
Apr 30, 2023 12 tweets 3 min read Read on X
1/ There's a lot of edge in building a production crypto HFT system the right way.

The abstractions should be rich enough to prevent careless mistakes but not too rigid to work with.

I converged on these after countless hours of building, debugging, and rewriting code 🧵
2/ Let's focus on a concrete example: How do you deal with continuous quantities?

Doing math in code with raw numeric types is error prone. Multiplying the price of one symbol by the size of another is sure way to blow up your strategy.

Just be careful? Nah, we can do better.
3/ A pervasive feature of our trading system is the concept of "units."

The first idea is to have separate types for Price, Size, Notional, etc.

Define relations:
Price * Size = Notional
Size + Size = Size
etc.

But Size * Size? Nonsensical, and now you get a compiler error.
4/ This is already safer, but these quantities still all wrap raw floats.

We can do even better: It doesn't make sense to add 1 BTC to 2 ETH.

So sizes should be Size(2, ETH), and you can check that the units agree in each of your arithmetic implementations.
5/ To continue this type extension, prices would be Price(2000, USDT/BTC). That is, the price of one BTC is in units of USDT *per BTC*.

The units work out like:
Price(2000, USDT/BTC) * Size(2, BTC) = Notional(4000, USDT).
6/ To sanity check the progress so far, we can still do sensical computations like computing total exchange volume. You add a bunch of Price * Size = Notional(..., USDT) quantities.

However, we're still not done. Assets across exchanges are not fungible.
7/ When shit hits the fan, networks can be congested, or deposits and withdrawals can be closed by the exchange.

To ensure robustness, it's helpful to include the exchange as part of the symbol too, so that you can't directly compare BTC on Binance with BTC on Coinbase.
8/ Some seemingly benign things like computing the global "mid price" across BTC/USDT listed on all exchanges or setting USD=USDT are now more annoying.

But this is a feature, not a bug. We should think twice about these dangerous assumptions that are true almost always.
9/ There is a lot more to do here, for example fitting derivatives into this system. But this should be enough to get the idea.

The beauty of building a system from scratch is the freedom to rethink simple things. Use it wisely to help with the trading you're trying to do.
10/ Implementation-wise #rustlang makes it super ergonomic to create complex unit systems.

Define macros for type definitions, relationships, and constructors.

You can even do things like compile out the symbol checks in release mode, but check them in unit tests. Image
11/ By the way dimensional analysis is a super common tool in physics. When you do a big computation, the first sanity check on the answer is to check the units are correct.

This is just a way to do that automatically in your trading code.
12/ We've even extended these principles to Hyperliquid. Many concepts from HFT carry over naturally to building robust and performant exchanges.

Often principles transcend the specific application. When building complex systems, it's always worth setting strong foundations.

• • •

Missing some Tweet in this thread? You can try to force a refresh
 

Keep Current with jeff.hl

jeff.hl Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

PDF

Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @chameleon_jeff

Jun 30, 2023
1/ Cool to see people trying market making and documenting the progress in public!

Usually it's an opaque space and hard to learn from others. Most professionals want to protect their alpha at all costs.

Here's my advice on dealing with these practical issues 🧵
2/ First of all in HFT there are "strict" problems that you must get right and "loose" problems that you just want to 80/20 at the beginning.

More problems than you think will fall into the second bucket.

Problems will migrate to the first bucket, but slower than you think.
3/ When getting started, only model, strategy, and latency should be part of the first problem class.

Some important sounding things that I would recommend the 80/20 principle on:
1) inventory management
2) sizing
3) hedging

I.e. almost all of the issues @pedma7 mentioned
Read 8 tweets
Jun 24, 2023
1/ Order book perp DEXs are a hot "narrative" these days.

Most protocols run order books offchain. What does this mean for MEV?

Answer: it's far worse than sandwiching and frontrunning on AMMs.

A thread on why onchain order books are the only viable protection for end users 🧵
2/ I'm going to focus on problems, ideas, and approaches in this thread.

I'm not going to call out any specific projects.

But hopefully after reading this thread you more objectively assess what each project is building.
3/ First off, the mission statement.

Perps on CEXs have found the most product-market fit out of any business in crypto.

But many users want to move off CEXs.

So the problem is to build something truly decentralized but offering the same UX as CEXs.
Read 26 tweets
May 15, 2023
1/ Many in crypto HFT rely heavily on historical simulation, while A/B testing is an afterthought.

In my experience there is real alpha in live testing well.

With proper experimental setup and sufficient market share, you can glean causal insights not available in tick data 🧵 Image
2/ First some background:

A/B testing is a useful tool in consumer tech.

Classic example: Google optimized their shade of blue for an additional $200M in yearly ad revenue.

When controlled experiments are cheap, A/B testing is a great way to juice marginal gains.
3/ Crypto HFT is an interesting domain for a few reasons though:

1) Your experiment has market impact at scale
2) You can simulate with historical data

So it's less clean to A/B test, and there is a viable alternative. So, is it still worth the trouble?
Read 15 tweets
Apr 25, 2023
1/ Building a crypto HFT system is a challenging and rewarding journey.

A mega-thread covering:
Trading strategies ("what")
The infrastructure and latency optimization ("how")
Exchange selection ("where")
Market dynamics ("who")

What I wish I knew when getting started 🧵
2/ Starting out with the core of the trading system: the strategy itself.

First some general HFT strategy selection advice before specific alpha:
3/ A specific strategy on locked books with maker rebates.

This is a case study in the alpha research process, going from raw data to an idea to a strategy:
Read 16 tweets
Apr 22, 2023
1/ We've talked before about how latency is an important component of every crypto HFT strategy.

The optimization process is fun detective work because the exchange's tech is often a black box.

A thread on what to measure, graph, and analyze for your production HFT system 🧵 Image
2/ The first important number to look at is the "exchange timestamp" field of websocket messages and REST responses.

If multiple time fields exist, use the one corresponding to matching engine time. The other fields are edge server times, which add more noise to your analysis
3/ Matching engine time is crucial because network delay and the exchange's internal machine topology add noise to the outgoing timestamps on your end.

Reconcile the matching engine and your outgoing timestamps and you can quantify and optimize the factors in between
Read 9 tweets
Apr 9, 2023
1/ Been getting good questions as people try the hyperliquid API/SDK.

A few wondered if it's really possible to do crypto HFT in python. The answer is undoubtedly yes!

A thread on how to build a fully automated HFT system, and why python is the ideal language to get started 🧵
2/ A bit of background: our strategy began as a single file python script.

We now use an optimized rust system in production, but the journey to get there was crucial.

If I had started with rust or C++ from day 1, there would've been no trading. Start where you're comfortable.
3/ Even today, you can write a large class of profitable crypto HFT strategies in python.

The fundamental reason is that network latency dominates compute latency.

Doesn't matter what language you're using, make sure to measure latencies as early as possible.
Read 13 tweets

Did Thread Reader help you today?

Support us! We are indie developers!


This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Don't want to be a Premium member but still want to support us?

Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal

Or Donate anonymously using crypto!

Ethereum

0xfe58350B80634f60Fa6Dc149a72b4DFbc17D341E copy

Bitcoin

3ATGMxNzCUFzxpMCHL5sWSt4DVtS8UqXpi copy

Thank you for your support!

Follow Us!

:(