OpenChainBench

> Open, reproducible benchmarks for crypto infrastructure. Aggregators, bridges, RPCs, oracles, price feeds, prediction markets, perp DEXes, cross-chain messaging. Same metric, same conditions, every provider. Live at **openchainbench.com**.

OpenChainBench publishes one benchmark at a time, each shipping with the harness that produces its data. The goal is to make performance an observable property of crypto infra, measured in the open, by anyone who wants to add a provider or a metric.

Community-run, MIT-licensed harnesses + CC-BY-4.0 data, PRs from any party including the providers we benchmark.

At a glance

• 94 live benchmark specs across 8 categories (RPCs, Trading, Aggregators, Bridges, Blockchains, RWA, Explorers, NFT APIs)
• 50 self-contained Go harnesses, each shipping a `/metrics` Prometheus endpoint
• 32 curated question/answer pages · 23 "alternatives to X" landing pages
• 26 machine-readable API routes (citation, badges, MCP, feeds, per-format quotes)
• One shared Prometheus, one materialization worker, CDN-fastpath snapshot store: cold reads in `](https://openchainbench.com/api/stat/aggregator-head-lag) | Devs, agents | Single benchmark: full rankings, sparkline (24h), methodology, paste-ready quote, attribution URL. |

Structured data on every bench page: `Dataset` + `StatisticalReport` + `Observation` + `TechArticle` + `FAQPage` + `BreadcrumbList` JSON-LD, plus a schema.org Dataset microdata graph layered on the visible "At a glance" infobox.

Journalist citation format (what `/api/stat/.headlineSentence` emits):

"Mobula leads head lag at 0.8s (p50, 24h) on Fastest onchain data provider.
Source: OpenChainBench (https://openchainbench.com/benchmarks/aggregator-head-lag)"

Site surfaces

Architecture

┌─────────────────────────┐    ┌───────────────────┐    ┌──────────────────────┐
│  50 Go harnesses on     │    │  Shared Prometheus │    │ Materialization worker│
│  ocb-par-main (Paris)   │─── ▶│  (same VPS)        │───▶│ (same VPS, docker)   │
│  each exposes /metrics  │    │  federates a few   │    │ sweeps every 60s     │
└─────────────────────────┘    │  Railway regional  │    └──────┬───────────────┘
                                │  harnesses         │           │
                                └───────────────────┘           ▼
                                                    ┌─────────────────────────┐
                                                    │ Redis + CDN blob store   │
                                                    │ (SRH at kv.opencha…)     │
                                                    └──────────┬──────────────┘
                                                               │
                                                    ┌─────────────────────────┐
                                                    │ Vercel Fluid Compute:    │
                                                    │ pages read CDN blobs     │
                                                    │ first, SRH as fallback,  │
                                                    │ last-known-good on cold  │
                                                    └─────────────────────────┘

• Harnesses + Prometheus + worker + Redis + SRH run on one VPS (`ocb-par-main` in Paris) via docker compose. A handful of multi-region harnesses (rpc-capabilities-us/eu/sgp, evm-quote-latency, aggregator-head-lag regional) stay on Railway and federate into the VPS Prom.
• Worker publishes atomic snapshots every 60 s to Redis + CDN blobs at `kv.openchainbench.com/aggregate/{latest,benches/,variants//}.json`. Site reads CDN first (~20 ms edge), falls back to SRH, then to the last-known-good snapshot key on cold cache.
• Site is Next.js 16 App Router on Vercel Fluid Compute. Auto-deploy from `main` on every push (with sitemap smoke test + auto-rollback + bench-page warm-up); `dev` branch runs on staging preview.
• Middleware (`src/middleware.ts`) handles lowercase 308 normalization, `/compare/-vs-` → canonical alphabetical, 410 Gone for removed benches, 301 for renamed ones, and cache-key normalization on read-only public APIs.
• Live stream relay (WebSocket at `wss://stream.openchainbench.com/ws`) is a separate service run by Mobula because it holds upstream API keys; the browser talks to it directly, Vercel only serves the static shell.

Full walkthrough with diagrams + a "clone-and-run-it-yourself" section is in `docs/architecture.md`.

Repo layout

benchmarks/        94 YAML specs, one per public benchmark
harnesses/         50 Go harnesses, each self-contained + Dockerized
answers/           32 Q&A YAMLs backing /answers/
alternatives/      23 "alternatives to X" YAMLs backing /alternatives/
worker/            Materialization worker (sweeps Prom → publishes CDN + Redis blobs)
infrastructure/    Shared Prometheus config
src/               Next.js 16 site (App Router, ISR, Tailwind 4, TypeScript, Zod)
docs/              architecture.md, methodology, walkthrough.md
scripts/           Spec validators + prebuild manifest generators
.github/           CI: prod deploy, staging preview, sync dev from main

Contributing

Read CONTRIBUTING.md for the long-form guide. TL;DR:

1. Float an idea in Discussions

2. Open a `📊 Propose a benchmark` issue with the metric + methodology

3. Write the harness (Go recommended, any language shipping `/metrics` works)

4. Host it (anywhere with HTTPS + public `/metrics`)

5. Open a PR against `dev`: spec YAML + harness ref + Prom scrape config

6. Review on the staging preview; merge to `main` when data lands

Editorial conventions

• Same metric, same conditions. If two providers can't be measured identically, they don't share a bench.
• Provider marketing does not shape published numbers. Public methodology + reproducible harness are the only source of truth.
• Retirement over removal. Retired benches return `410 Gone` with a link to the current catalog so previously-indexed URLs decay cleanly.
• Freshness signals everywhere. Every citable surface carries `dateModified` + `asOf` so LLMs can tell fresh data from stale.
• Draft benches never quote a number. If quorum drops, the page renders "awaiting samples" rather than a fabricated leader.

Community + license

• Discussions: github.com/ChainBench/OpenChainBench/discussions
• Twitter/X: @OpenChainBench
• Wikidata: Q140172649
• Zenodo DOI: 10.5281/zenodo.20800311
• Hugging Face dataset mirror: OpenChainBench/benchmarks (daily parquet snapshots)

License: harness code MIT · dataset CC-BY-4.0 · attribution required. See LICENSE.