Markets14 min read
One book, many venues
Building a perpetual futures aggregator at Print.World that merges 15 live venues into one order book, routes against real depth, and never places the same order twice.
From Print.World · Aug 2025 – Present
For the last stretch at Print.World I've been working on perps: an aggregator that pulls 15 live perpetual futures venues into one unified order book, routes orders across them, and shows the result in the terminal. The idea is simple to say. A user wants to buy some amount of BTC perp, and we should find them the best real execution across every venue we support.
Most of the work is in the word "real".
Venues don't agree on what a market looks like
Some venues publish a full order book. Others only give you a quote: here's a price for this size, take it or leave it. A unified book has to merge both, so quote-only venues get represented as levels they'll actually honour rather than pretending to be something they're not.
There are two kinds of quote-only venue. Oracle venues fill at an oracle price up to some capacity, so the price doesn't move with size until you hit the cap. RFQ ("request for quote") venues give a quoted band in fixed notional steps that goes stale quickly. An oracle venue becomes exactly one level per side, sized to its capacity divided by price. If the venue doesn't tell me its capacity, it isn't treated as infinite: it's refused, with the reason recorded. And these levels are never walked like real depth. A quote from one is marked as having no modelled price impact, so nothing downstream mistakes "one big level at one price" for a deep book.
Each venue gets its own adapter, and all adapters sit behind a single staleness policy and a single error boundary. That was a deliberate choice. With 15 venues, something is always slightly broken somewhere, and I wanted one place that decides "this data is too old to route against" instead of 21 slightly different opinions. If one adapter throws, it drops out of the book; it doesn't take the rest down.
The policy is one number, 5 seconds, used by the published book, the router, and the health monitor alike, and tunable live between 1 and 60 seconds. Snapshot age is judged on the reader's clock, with up to 2 seconds of clock skew tolerated between the machine that wrote a snapshot and the one reading it. A venue that goes stale isn't silently dropped: it's excluded with the reason stale, so a quote can say which venues it didn't consider and why. Health is counted in book ticks: a venue is degraded after 8 dark ticks (2 seconds) and ineligible after 40 (10 seconds), or after 3 order errors.
Five seconds is a trade-off. Books on liquid markets update many times a second, so routing on a 5-second-old book can mean a worse fill than quoted. But a tighter window would exclude a venue on every hiccup in its feed, and a missing venue is a worse quote too, just a less visible one. The window bounds how stale the routing input can be; the price you actually pay is bounded separately, at submission, by the order's slippage tolerance.
Walking the book
The router walks book depth level by level, which tells you three things a top-of-book price can't: the average fill price for your size, the price impact, and whether a venue runs out of depth before your order is filled.
Each venue's walk produces a volume-weighted average price (VWAP) for the requested size, and price impact is measured against that venue's own best level, not a cross-venue mid: impact_bps = |vwap - best| / best x 10,000. If the venue runs out of depth before the size is filled, the walk says so (insufficient_depth) but keeps the partial result, because a partial fill on one venue is exactly what a split across venues needs.
Fees are a separate step. Depth walking answers "what price would I get", and a fee-aware quote step answers "what does that actually cost me". Keeping them apart made both easier to test, and it meant that when a venue changed its fee schedule, the depth logic didn't have to move.
The fee step is one line: all_in = vwap x (1 ± (taker_bps + builder_bps) / 10,000), plus for a buy, minus for a sell. A venue whose fee I don't know isn't quoted with a guess of zero; it's excluded with the reason fee_unknown. Fees matter more than they look, because taker fees across the fleet span 0 to 6 bps. A venue with a 4.5 bps worse VWAP than Hyperliquid on a given size still wins if its taker fee is 0, which is why ranking on the book alone would get the answer wrong.
Show data
| venue | Taker fee (bps) |
|---|---|
| Lighter | 0.00 |
| Ondo | 2.50 |
| Extended | 2.50 |
| Derive | 3.00 |
| Orderly | 3.00 |
| Bulk | 3.50 |
| Nado | 3.50 |
| Phoenix | 3.50 |
| Aster | 4.00 |
| Pacifica | 4.00 |
| StandX | 4.00 |
| Hyperliquid | 4.50 |
| Avantis | 4.50 |
| Ostium | 6.00 |
When one venue isn't enough, the router splits. It fills greedily from the best marginal level across every venue's book, up to 3 legs. A split only replaces the best single venue if it beats it by more than 2 bps and more than $5, with another 0.5 bps of buffer for each extra leg. The dollar floor sets the minimum order size where splitting is even considered: a 2 bps improvement is worth $5 at 5 / 0.0002 = $25,000 of notional, so below that the router keeps orders on one venue, where there's one fill to reconcile instead of three. The per-leg buffer exists because every extra leg is another venue that can reject, fill partially, or answer late.
Route state is frozen per tick. Each tick takes a consistent snapshot of the book and computes routes from it, so any replica can serve a quote and two replicas given the same tick will agree. That made horizontal scaling boring, which is what you want.
The tick is 250 ms, four publishes a second, and the next tick is scheduled for the interval minus however long this one took, so ticks never overlap. One lock-holding process publishes; the API tier (2 to 20 tasks in production) reads the snapshot from Redis, so any task can quote. A snapshot keeps 200 levels per side per venue. It's written to Redis at most once a second per symbol, or immediately if the set of venues in it changes, which caps the Redis write rate at one snapshot per symbol per second instead of four, and means a replica's route input is at most about a second plus the 5-second staleness window old. Agreement between replicas also needs a deterministic merge: levels are ordered by price, same-price levels from different venues are kept separate rather than summed (the router has to know whose liquidity it's taking), and ties go to the larger level, then the venue name.
Never place an order twice
This is the part I spent the most time on, because it's where money can go wrong.
Every order is placed at most once per client order ID. If a request is retried, by the user, a network blip, or our own code, the second attempt finds the first and doesn't submit again.
The mechanics, in the order they happen:
- The caller supplies the client order ID (8 to 64 characters). It's scoped to the user, so two users can't collide.
- A per-key lock on user plus client ID serializes concurrent attempts, and the idempotency record lives in Redis for 24 hours. If that Redis read fails, the order is refused with a 503. This is the opposite of the rate limiter below, on purpose: a limiter that can't reach its store should step aside, but a dedupe check that can't reach its store can't prove the order is new, so it must not send it.
- A row is written to Postgres with status
sendingbefore the venue is called. If the process dies mid-request, the evidence that we might have sent something survives it. - The venue gets a deterministic ID derived from ours, so the venue itself can dedupe too. Hyperliquid's client order ID is 128 bits, so it gets the first 16 bytes of SHA-256 of our ID; at 128 bits, a collision is a birthday problem at around 2^64 orders, which isn't a real risk. Split legs and failover hops get derived suffixes (
:leg<n>,:h<hop>), and a caller ID may not end in either pattern, so a user's own ID can never be mistaken for one of ours.
Take-profit and stop-loss legs were trickier. Sometimes you send one, the connection drops, and you genuinely don't know whether the venue accepted it. The tempting move is to resend. That's how people end up with two stop-losses. Instead, an ambiguous leg is stored with an explicit status. The outcomes are:
filled | resting | cancelled | rejected | unknown
unknown is reconciled against the venue's actual state before anything else happens. It's never resent blindly. Inline, that means up to 3 position reads, a second apart, before the request returns, and the order is never re-routed to another venue while it's unknown. In the background, any row still sending after 2 minutes becomes unknown (a live send is bounded by a 60-second order lock, so a row that old belongs to an attempt that died mid-send), and reconciliation matches rows to venue orders by the venue's order ID first, then by our client ID. It's bounded so it can't become its own load problem: at most one sweep per user every 5 minutes, and at most 200 unresolved rows per history read.
The same rule covers splits. Legs go out concurrently, and if one is refused, the others stand: a leg that filled is never automatically unwound, because unwinding is a second trade with its own price and its own chance of failing. The plan is stored before any leg is sent, so a retry resumes the plan instead of starting a new one. A single-venue order that's refused can fail over to the next venue, up to 3 hops, but only on a clear verdict given before anything was accepted (rejected, insufficient balance, below minimum size, market closed, and so on). A gateway error or a "status unknown" body is treated as ambiguous: it's reconciled, never re-routed, because re-routing an order that may have filled is how you double a position. A partial fill re-quotes only the remainder, a runner-up venue that's worse than the accepted quote by more than the order's slippage tolerance isn't taken, and no new hop starts after 40 seconds, which keeps the whole loop inside the 60-second order lock.
Rate limits across a fleet
One venue enforces its rate limit per IP, and we run several instances. Each instance tracking its own budget would collectively blow through the limit. So there's a shared token bucket in Redis that every instance draws from.
Two rules on that bucket mattered a lot. It fails open: if Redis is unreachable, we don't stop trading because a rate limiter is down. And it only gates reads, never order submission. A limiter that could block a user's order or a stop-loss is a limiter that could cause real damage.
The venue is Hyperliquid, whose info API allows 1,200 units of request weight per minute per IP. Production runs 2 to 20 API tasks behind one NAT IP, so splitting the budget statically would give each task as little as 1,200 / 20 = 60 weight a minute, one unit a second, even when the other 19 are idle. A shared bucket, keyed by egress IP, lets whichever task needs the budget use it.
The bucket's shape is chosen so no 60-second window can ever exceed the limit, whatever the timing: a burst capacity of a sixth of the budget (200), refilled at the remaining 1,000 per minute (about 16.7 a second), so 200 + 60 x 16.7 = 1,200. Of that burst, 100 is reserved: routine reads can't drain the bucket below it, so the reads an order needs (balances, positions) never queue behind a background refresh. A caller waits at most 2.5 seconds for a token, and a 429 from the venue closes the bucket for 2 seconds by default (at most 60), because a venue that's already rejecting us won't start accepting if we keep asking. The take and refill is a single Lua script, so two tasks can't both read "one token left" and both spend it.
Funding carry
One feature I enjoyed was ranking funding-rate carry pairs, where you're long on one venue and short on another to collect the difference in funding. The naive version ranks by funding spread. That's misleading, because entering two legs costs money. So pairs are ranked by net carry after both legs' entry costs, with a break-even time ("this pays for itself after roughly this long"), and sized to the depth that's actually available on both sides.
A great-looking spread on a venue with no depth isn't an opportunity. It's a screenshot.
Comparing funding across venues first needs a common unit, because venues don't settle on the same clock. Hyperliquid publishes an hourly rate, Nado a 24-hour one, and some venues set the interval per market (Aster has markets at 8, 4 and 1 hours). Worse, the published number isn't always the settlement interval: Lighter's funding API reports an 8-hour-equivalent rate for every market, even though it pays hourly. So each venue's spec records what its published number means, every rate is normalised to a daily rate, rate_per_day = raw_rate x 24 / interval_hours, and only then compared. A venue whose interval I can't pin down is excluded rather than assumed, because reading an hourly rate as eight-hourly is an 8x error.
Break-even is then simple: break_even_hours = entry_cost / net_carry_per_day x 24, where entry cost is both legs' all-in price against their own mids, fees included. As an illustration, not a real pair: 20 bps to get into both legs against a net carry of 10 bps a day breaks even after 48 hours. Funding rates move, so a two-day break-even is a bet that the spread survives two days, and that's the number a user should be looking at, not the spread.
The same normalisation feeds ordinary routing, but only when the user says how long they plan to hold. Funding is a cost of holding, and over an eight-hour hold the funding difference between two venues can easily exceed the whole 0 to 6 bps spread in taker fees, so a router that only looks at entry price can send a holder to the most expensive place to sit. With a stated hold, each venue is charged funding_per_day x hold_hours / 24 on the side you're taking (a short that gets paid funding nets more). Without one, funding stays out, because charging a scalper eight hours of funding would be just as wrong.
Pushing the book to users
The terminal gets the book over a websocket push layer. Sending the full book on every update was wasteful, so updates carry deltas: only the levels that changed. On the BTC book that took traffic from 132 KB/s to 6.4 KB/s. Choosing the server for that push layer was its own debate, which I wrote up in fix our websocket server, or adopt one?.
The wire carries a small book on purpose: the 25 best merged rows plus each venue's own top 10, not the 200 levels per side the router walks. At four frames a second, the full-book figure works out to 132 / 4 ≈ 33 KB per frame; with deltas, about 1.6 KB.
For state streams, where the newest message fully replaces the previous one, a slow client doesn't need every intermediate state; it needs the latest one. So the book channel keeps a history of exactly one message: a client that reconnects gets the current book, not a backlog of every delta it missed. Upstream, the venue feeds are the opposite: they're sequenced, and a delta that doesn't chain onto the last one marks that venue's book stale until it re-snapshots, because applying an update past a gap silently serves a wrong book. Where a feed needs buffering during a resync, the buffer is bounded (1,000 updates on Aster) and drops the oldest. Unbounded queues on a slow consumer are just a memory leak with extra steps.
What I took from it
The pattern across all of this is being explicit about what you don't know. A stale venue is marked stale. An ambiguous order leg is UNKNOWN, not guessed. A rate limiter that can't reach its store says so and steps aside. Every time I was tempted to pick the optimistic default, the honest one turned out to be simpler to reason about.
The principle I took from it: build the reconciliation path alongside the happy path, not after it. The interesting engineering in a trading system lives in the space between "we sent it" and "we know what happened". The same lesson showed up again in bridging.