Lobsters is a real, community-run Rails application — the link-aggregation site, not a scaffold. It's Roundhouse's proving lane: a substantial, un-contrived codebase that pushes ingestion and inference past the tidy shape of the reference blog.
Status: work in progress — most of it transpiles; the constructs not yet modeled surface as diagnostics you can read live, rather than as a hard stop.
Why Lobsters
A transpiler that only handles a generated blog hasn't proven much. Real apps carry accumulated idiom: hand-rolled scopes, concerns, callbacks, view helpers, and query patterns that no scaffold produces. Lobsters is large enough to expose those, small enough to reason about whole — the right size to drive Roundhouse's coverage toward real-world Rails and keep an honest ledger of what's left.
It's also a known quantity in the Ruby world. The Ruby core team turned Lobsters into a representative benchmark for YJIT precisely because it stresses a real Rails request the way a microbenchmark can't — see We turned Lobsters into a Rails benchmark for YJIT.
Two Lobsters
There are two of them here, measured differently on purpose.
The benchmark runs the
capture the Ruby core team froze into
ruby/ruby-bench,
pinned to a fixed commit. A benchmark has to freeze its input or its
numbers stop being comparable between runs — so that lane stays deliberately
fixed, and the trade is that it describes the app as it was when the
snapshot was taken.
The conformance lane
answers the other question: does this handle the code they have
now? It tracks
upstream HEAD,
records the commit it ran against rather than pinning it, and runs
Lobsters' own RSpec suite against a Roundhouse emit of that same
checkout. Their tests are a much finer oracle than a route returning 200:
each failure names a specific Rails construct not yet modeled, which makes
the suite a worklist generator. That page publishes the ranked list of what
fails and why, including what it excludes and what the harness papers over.
The honest ledger
Open it in the IDE and every unmodeled construct is a diagnostic on the exact line — the coverage gap made legible. Open it in the Playground and it transpiles to any target, emitting what it can and annotating what it can't. This is the same bar the blog meets perfectly and Mastodon is climbing toward: show the work in progress with the gap described, don't pretend the gap isn't there.
Further reading
- We turned Lobsters into a Rails benchmark for YJIT — why Lobsters is the reference real-app workload
- The Ruby JRuby Was Built to Run — the 2×2 performance experiment
- Numbers Without Conclusions — benchmark methodology and honest caveats
Source: ruby/ruby-bench
(upstream: lobsters/lobsters).