Lobsters benchmark results

1. Roundhouse vs Rails — runtime × JIT

4 lanes, three of them the claim. Rails YJIT is the baseline; Roundhouse YJIT is the like-for-like comparison against it — same runtime, same JIT, so the only variable is the code; the AOT lane did not run. The 2 no-JIT rows size the JIT's contribution and are not the headline.

3.63× faster than Rails — 126.0 ms/iter (avg) vs Rails' 457.4, both stacks CRuby with YJIT. The full field:

Every lane replays the same frozen sequence in-process, in-memory, single-threaded — no HTTP server, no load generator — so the only variable across rows is the interpreter × JIT under test (the ruby-bench lobsters-for-YJIT shape). Lower is better; whiskers span each lane's min–max.

• Rails gains 1.66× from YJIT (759.9 → 457.4 ms/iter)
• Roundhouse gains 1.34× from YJIT (168.3 → 126.0 ms/iter)
• Roundhouse is 3.63× faster than Rails with YJIT (457.4 → 126.0 ms/iter)
• Roundhouse is 4.52× faster than Rails without a JIT (759.9 → 168.3 ms/iter)

Every lane queried SQLite 3.50.3 at runtime.

“vs Rails” is a speed ratio — Rails' ms/iter ÷ this row's, at the same JIT level — so 3× means three times faster. Memory is a separate section below. CV = per-lane standard deviation ÷ mean across the timed iterations.

Lanes measured 2026-08-21T07:00:44Z · host showcase.party — one process per lane, JIT pinned per lane.

2. Per-endpoint breakdown

Where the overall 3.63× win comes from — each endpoint's median latency over every timed visit of the frozen sequence, Rails vs roundhouse. Bars are log-scaled; the ratio at the right is Rails ÷ roundhouse (green = roundhouse faster).

A diagnostic breakdown, not a re-derivation: per-endpoint medians don't sum to the sequence median (GC and ordering don't decompose linearly), and the sub-millisecond rows sit near clock resolution — trust the ranking, not the third decimal.

3. Memory footprint

Peak resident memory of each lane's replay process — lower is better. Because the lanes replay in-process, each one measures itself (VmHWM from /proc/self/status), so this is the whole stack: interpreter, framework, the in-memory fixture DB, and whatever the timed run retains on top.

The bar splits at the point the sequence starts. The solid segment is the baseline — everything loaded and seeded, before a single visit. The translucent segment is growth across the timed run. The two say different things: baseline is what the stack costs to stand up, growth is what serving retains.

• Rails pays 35 MB (+12%) in peak memory for YJIT, buying 1.66× the speed
• Roundhouse pays 22 MB (+20%) in peak memory for YJIT, buying 1.34× the speed
• Roundhouse peaks 2.61× lower than Rails with YJIT (342 → 131 MB), and starts 1.94× lower before serving anything (183 → 94 MB)
• Roundhouse peaks 2.82× lower than Rails without a JIT (306 → 109 MB), and starts 1.86× lower before serving anything (175 → 94 MB)
• Across the timed run Rails retains 4.3× more than Roundhouse (159 MB of growth against 37 MB) — a separate fact from the baseline gap, and the one a peak-only number hides

“vs Rails (memory)” is peak RSS ÷ peak RSS at the same JIT level — not the speed ratio in the matrix table above; the two differ. Baseline is sampled after the in-memory fixture DB is seeded, and it is not a common floor across lanes — each stack loads its own framework, so subtracting one lane's baseline from another's peak is meaningless.

4. Coverage — HTTP 200 (26/26)

What the numbers above are numbers of: every distinct route the benchmark exercises returns 200 on the CRuby emit target — the timing sections measure a complete application, not a working subset. A failing cell here would invalidate that lane's rows above.

Content was not compared for this run — no Rails lane ran, so there was no oracle to compare against. These cells attest that each route answered, not that it answered correctly.

5. Environment & provenance

Generated 2026-08-21T17:48:27Z from summary.json.