Lobsters benchmark results

1. Roundhouse vs Rails — runtime × JIT

5 lanes, three of them the claim. Rails YJIT is the baseline; Roundhouse YJIT is the like-for-like comparison against it — same runtime, same JIT, so the only variable is the code; Roundhouse AOT is the same emit through a different runtime, and is not at full content parity. The 2 no-JIT rows size the JIT's contribution and are not the headline.

1.94× faster than Rails — 266.8 ms/iter (avg) vs Rails' 518.4, both stacks CRuby with YJIT. The full field:

Every lane replays the same frozen sequence in-process, in-memory, single-threaded — no HTTP server, no load generator — so the only variable across rows is the interpreter × JIT under test (the ruby-bench lobsters-for-YJIT shape). Lower is better; whiskers span each lane's min–max.

• Rails gains 1.68× from YJIT (871.1 → 518.4 ms/iter)
• Roundhouse gains 1.45× from YJIT (386.4 → 266.8 ms/iter)
• Roundhouse is 1.94× faster than Rails with YJIT (518.4 → 266.8 ms/iter)
• Roundhouse is 2.25× faster than Rails without a JIT (871.1 → 386.4 ms/iter)

Every lane queried SQLite 3.50.3 at runtime.

The AOT lane marks generationally — spinel's default collector since d3b1400d, attested by the binary's own counters rather than by what the harness exported. Roundhouse AOT: 711 collections in the timed loop, 608 of them whole-heap.

Two ratios, because one lane is not rendering what Rails renders. Roundhouse YJIT is at 25/26 content parity; over the 22 routes it renders equivalently it is 1.88× (243.6 ms/iter vs 459.0), against 1.94× over the whole sequence; Roundhouse no JIT is at 25/26 content parity; over the 22 routes it renders equivalently it is 2.21× (354.2 ms/iter vs 782.3), against 2.25× over the whole sequence; Roundhouse AOT is at 20/26 content parity; over the 19 routes it renders equivalently it is 4.46× (75.4 ms/iter vs 336.3), against 4.49× over the whole sequence. Both sides of the restricted division cover the same routes. The unrestricted number is not wrong, but part of it is earned by rendering less.

“vs Rails” is a speed ratio — Rails' ms/iter ÷ this row's, at the same JIT level — so 3× means three times faster. Memory is a separate section below. CV = per-lane standard deviation ÷ mean across the timed iterations.

Lanes measured 2026-10-01T09:09:29Z · host showcase.party — one process per lane, JIT pinned per lane.

2. Per-endpoint breakdown

Where the overall 1.94× win comes from — each endpoint's median latency over every timed visit of the frozen sequence, per lane. Bars are log-scaled; the ratio at the right is Rails ÷ roundhouse (both CRuby +YJIT) (green = roundhouse faster). The chart draws rails, roundhouse, roundhouse AOT; the no-JIT lanes appear as extra columns in the table below.

Both headline ratios include the transpile. Isolating the two: transpiling accounts for the bulk of the win, and AOT compilation adds a further 2.31× over the same emit running on CRuby+YJIT — in aggregate only, since AOT is faster on the heavy routes and slower on the cheap ones, where per-request fixed costs dominate.

A diagnostic breakdown, not a re-derivation: per-endpoint medians don't sum to the sequence median (GC and ordering don't decompose linearly), and the sub-millisecond rows sit near clock resolution — trust the ranking, not the third decimal.

3. Memory footprint

Peak resident memory of each lane's replay process — lower is better. Because the lanes replay in-process, each one measures itself (VmHWM from /proc/self/status), so this is the whole stack: interpreter, framework, the in-memory fixture DB, and whatever the timed run retains on top.

The bar splits at the point the sequence starts. The solid segment is the baseline — everything loaded and seeded, before a single visit. The translucent segment is growth across the timed run. The two say different things: baseline is what the stack costs to stand up, growth is what serving retains.

Roundhouse AOT is at 20/26 content parity; Roundhouse no JIT is at 25/26 content parity; Roundhouse YJIT is at 25/26 content parity. Peak memory is measured over the same sequence as the timings, so a lane that renders less also allocates and retains less — read its bar the way you read its ms/iter, not as a like-for-like footprint.

• Rails pays 40 MB (+13%) in peak memory for YJIT, buying 1.68× the speed
• Roundhouse pays 16 MB (+14%) in peak memory for YJIT, buying 1.45× the speed
• Roundhouse peaks 2.70× lower than Rails with YJIT (345 → 128 MB), and starts 1.84× lower before serving anything (183 → 99 MB)
• Roundhouse peaks 2.73× lower than Rails without a JIT (306 → 112 MB), and starts 1.79× lower before serving anything (176 → 99 MB)
• Across the timed run Rails retains 5.7× more than Roundhouse (163 MB of growth against 28 MB) — a separate fact from the baseline gap, and the one a peak-only number hides

“vs Rails (memory)” is peak RSS ÷ peak RSS at the same JIT level — not the speed ratio in the matrix table above; the two differ. Baseline is sampled after the in-memory fixture DB is seeded, and it is not a common floor across lanes — each stack loads its own framework, so subtracting one lane's baseline from another's peak is meaningless.

4. Coverage — HTTP 200 (26/26 · spinel 25/26)

Incomplete coverage: 1 route(s) fail on the spinel AOT lane — the failing routes' visits short-circuit cheaply, so treat the affected lane's timing rows above as optimistic. Details in the table.

5. Environment & provenance

Generated 2026-10-01T14:18:45Z from summary.json.