The moat

Validated against published numbers.

Credibility in this field is built on reproducing published results. Every code, decoder, noise model, and estimator in Glean traces to a peer-reviewed source and ships with a benchmark that reproduces that source’s number — on the identical problem the paper used.

10validated benchmarks

3decoder paradigms

2rare-event estimators

10⁻⁷logical-error-rate regime reached

The validated-numbers table

Each row is a reproduction, not a leaderboard entry. Rerun the whole suite in one command: python benchmarks/run_all.py.

ID	What it checks	Source	Published	Glean	Status
`B1`	Toric BP+OSD threshold (code-capacity) OSD-0 gives ≈9.5%; the two orders bracket the central 9.9%.	Roffe 2020	9.9 ± 0.2%	≈10.0–10.1% crossing (OSD-CS-10)	in-band
`B2`	DEM ingest + surface decode vs PyMatching d=3 low-p lags MWPM — the known BP+OSD-vs-matching surface-code gap, not a bug.	Stim · PyMatching	integration test	d=5 agrees ≤1.3σ	pipeline validated
`B3`	Gross [[144,12,12]] circuit-level vs ldpc Hypergraph DEM (max detector degree 6 — MWPM does not apply).	Bravyi 2024	0.8% threshold	== ldpc 99.8% / <0.4σ; threshold ≈0.80–0.85%	validated
`B4`	relay-BP vs reference crate; accuracy; real-time budget	Müller 2025	“factor of three”; real-time	== relay_bp crate <0.95σ; 2.6× @ p=4e-3; p90 = 212 < 600	validated
`B5`	Decode speed on the identical gross DEM	Roffe ldpc · relay_bp	incumbent baselines	on par with relay_bp; out-runs ldpc; rayon 4–5× on 10 cores	validated
`B6`	glean.SinterDecoder conforms to sinter.Decoder	sinter (de-facto std)	interface conformance	bit-exact vs direct path; reproduces B3/B4 LER via sinter.collect	validated
`B7`	BP+OTF vs BP+OSD-0 oracle No public BP+OTF repo — oracle is internal (Glean BP+OSD-0 == ldpc, B3).	deMarti iOlius 2024	“on par with BP+OSD-0”	ensemble BP+OTF == BP+OSD-0, 0.33σ (99.4% agree)	Fig-2 reproduced
`B8`	Dynamical Subset Sampling vs direct MC Deep Bravyi point approached (1.3×10⁻⁶, ~6×), not pinned — splitting (B9) pins it.	Heußen 2024	~2 OOM fewer shots	== direct MC 0.0σ; whole p_L(p) curve from one campaign	method validated
`B9`	Splitting + Bennett ratio (the deep-number pin) vs direct MC (×2 depth)	Bravyi–Vargo 2013 · Mayer 2025	reaches <10⁻²⁰	== MC 0.07σ @ 3e-3 & 0.30σ @ 2e-3; deep = 7.78×10⁻⁷ ± 13%	method validated
`B10`	Per-cycle normalization of the Bravyi deep point Consistent with — slightly below — the Nature headline once both use the per-cycle, combined-sector convention.	Bravyi 2024	2×10⁻⁷ / cycle	1.30×10⁻⁷ / cycle = 0.65× the headline	reproduced within uncertainty

The curves

What the numbers look like.

Gross-code error suppression to the rare-event regime

B3 + B9 · [[144,12,12]], memory Z, 12 cycles. Direct Monte-Carlo where it is samplable; the splitting estimator carries the same decoder ≈5 orders deeper.

B3 · B9

BP+OSD-CS-10 — direct Monte-Carlo
same decoder — splitting estimator

Direct MC bottoms out near 10⁻³ (≈10⁹ shots would be needed to go deeper); splitting reaches 7.78×10⁻⁷ ± 13% at p = 10⁻³ from a single MCMC chain — and is proven unbiased against direct MC two decades down (B9).

Toric threshold

B1 · code-capacity, BP+OSD-CS-10. The per-distance curves cross inside Roffe’s published band.

L = 12 (n = 288)
L = 16 (n = 512)
L = 20 (n = 800)

Crossing at p ≈ 10.0–10.1% — inside Roffe’s 9.9 ± 0.2%.

relay-BP vs BP+OSD

B4 · gross code, 12 cycles. relay-BP sits below BP+OSD-CS-10 across the sub-threshold band.

BP+OSD-CS-10
Relay-BP-1
Relay-BP-5

2.6× more accurate than BP+OSD-CS-10 at p = 4×10⁻³ — the paper’s “factor of three.”

The deep point, honestly

Reproducing the Nature headline.

Bravyi 2024’s headline is a per-syndrome-cycle rate; Glean measures a per-experiment rate over 12 cycles. Comparing them requires the paper’s own normalization — p_L = 1 − (1 − P_L)^(1/N_c), N_c = d = 12, combined-sector. Once both are on that convention (B10):

Glean, per experiment 7.78×10⁻⁷ ± 13% B9 splitting

→ per cycle

Glean, per cycle (combined) 1.30×10⁻⁷ B10

Bravyi 2024 headline 2×10⁻⁷ Nature 627, 778

0.65× the headline — consistent with, slightly below. This is a reproduction within mutual uncertainty (Bravyi’s number is extrapolated below their per-cycle floor; Glean’s carries ±13% plus the per-cycle assumption, validated at p = 3×10⁻³). It is not a claim to beat the headline. Two-digit precision wants tighter depth and the authors’ exact BP+OSD tuning.

The Iron Rule

No code without a citation.

Every implemented decoder, code, and estimator records its authoritative source, what was taken from it, and the benchmark that validates Glean against its published numbers.

gross code B3 · B10

High-threshold and low-overhead fault-tolerant quantum memory

Bravyi, Cross, Gambetta, Maslov, Rall, Yoder · 2024 · Nature 627, 778 · arXiv:2308.07915

Took Bivariate-bicycle [[144,12,12]] construction + the exact distance-preserving syndrome-extraction schedule.

Construction + circuit-level decode == ldpc 99.8% / <0.4σ; 0.8% threshold bracketed.

relay-BP B4

Improved belief propagation is sufficient for real-time decoding of quantum memory

Müller, Alexander, Beverland, Bühler, Johnson, Maurer, Vandeth · 2025 · arXiv (IBM) · arXiv:2506.01779

Took Disordered-memory min-sum BP + relaying + ensembling, OSD-free, transcribed from the reference crate.

== the authors’ relay_bp crate (<0.95σ); real-time iteration budget met (p90 212 < 600).

BP+OTF B7

An almost-linear time decoding algorithm for quantum LDPC codes under circuit-level noise

deMarti iOlius, Etxezarreta Martinez, Roffe, Etxezarreta Martinez · 2024 · arXiv · arXiv:2409.01440

Took Ordered Tanner Forest post-processor (reliability-ordered modified-Kruskal forest) — inversion-free, O(n log n).

Ensemble BP+OTF == Glean’s validated BP+OSD-0 oracle (0.33σ) on the gross-code DEM.

BP+OSD B1 · B5

Decoding across the quantum LDPC code landscape

Roffe, White, Burton, Campbell · 2020 · Phys. Rev. Research 2, 043423 · arXiv:2005.07016

Took Normalized min-sum BP + OSD; the toric-threshold target and the ldpc speed baseline.

Toric threshold ≈10.0–10.1%, inside the 9.9 ± 0.2% band; BP+OSD out-runs ldpc on the gross DEM.

OSD-CS B1

Degenerate quantum LDPC codes with good finite-length performance

Panteleev, Kalachev · 2021 · Quantum 5, 585 · arXiv:1904.02703

Took OSD-0 + the higher-order combination sweep (OSD-CS) on the BP reliability order.

Valid-syndrome + never-worse-than-OSD-0 tested; closes the toric-threshold gap into the band.

DSS B8

Dynamical subset sampling of quantum error correcting protocols

Heußen, Winter, Rispler, Müller · 2024 · Phys. Rev. Research 6, 013177 · arXiv:2309.12774

Took Weight-stratified subset sampling: p_L = Σ_w A_w(p)·p_fail^(w), with A_w exact and decoder-independent.

Unbiased on Glean’s pipeline (0.0σ vs direct MC); whole p_L(p) curve from one campaign.

splitting B9

Simulation of rare events in QEC · Rare-event simulation of QEC circuits

Bravyi–Vargo · Mayer et al. · 2013 · 2025 · Phys. Rev. A 88, 062308 · arXiv · arXiv:2509.13678

Took Telescoping product of Bennett-acceptance-ratio steps over a Metropolis-in-faults chain — no resolution wall at depth.

Unbiased vs direct MC two decades into the rare regime (0.07σ, 0.30σ); pins the gross-code deep point to ±13%.

Stim DEM B2

Stim: a fast stabilizer circuit simulator

Gidney · 2021 · Quantum 5, 497 · arXiv:2103.02202

Took Flattened detector-error-model parser → check/observable matrices + per-mechanism priors (the upstream pairing).

Pipeline validated vs PyMatching on a surface-code DEM (d=5 ≤1.3σ).

HGP / toric B1

Quantum LDPC codes with positive rate and minimum distance proportional to √n

Tillich, Zémor · 2014 · IEEE Trans. Inf. Theory 60(2) · arXiv:0903.0566

Took Hypergraph-product construction → the toric warm-up code.

Parameters validated (n=2L², k=2, CSS).

Rerun the moat.

The reproducible suite is one command with a PASS/FAIL table. Optional dependencies (ldpc, relay_bp, pymatching, sinter) are auto-detected and skipped when absent.

python benchmarks/run_all.py --tier quick