3. Schemas and compatibility
Every structured artifact is checked against its bundled JSON Schema. Breaking public-schema changes require an explicit site-contract update.
Browse leaderboard schemaDownload and verify
A versioned, answer-safe release bundle for independent analysis, citation, and reproducible site builds.
Results 0.1.14Data through Oct 8, 2026
Published release
1. Release bundle
| File | Format | SHA-256 | Size | Download |
|---|---|---|---|---|
| category-summary.json | JSON | 299fdc36c0…1c36da8e | 3.4 KB | |
| CHANGELOG.md | Text | 174921c591…664f1f09 | 717 B | |
| CITATION.cff | Text | 58687d0539…8cf75a2d | 888 B | |
| configuration-manifests.json | JSON | 19c3f86977…869982ce | 653.4 KB | |
| coverage-gaps.json | JSON | 1b4ffa9dcb…c171f2e5 | 5.6 KB | |
| leaderboard.csv | CSV | 82e2a86c3a…a142055e | 99.4 KB | |
| leaderboard.json | JSON | dba390b2b9…2ebf8f8d | 249.2 KB | |
| LICENSES.md | Text | 917a6c517e…7589a893 | 1.4 KB | |
| methodology.json | JSON | 5411753da2…6428113a | 4.2 KB | |
| README.md | Text | f5e5941b0a…f0cbe841 | 3.3 KB | |
| schemas/category-summary.schema.json | JSON | 4c39481af7…33409f20 | 1.8 KB | |
| schemas/configuration-manifests.schema.json | JSON | 4645d08a5c…b5f42150 | 44.1 KB | |
| schemas/coverage-gaps.schema.json | JSON | fa141ea846…24d1b712 | 1.4 KB | |
| schemas/leaderboard.schema.json | JSON | 6cdfe7692e…57551543 | 4.3 KB | |
| schemas/methodology.schema.json | JSON | 63ab847693…2096b790 | 6.8 KB | |
| schemas/release-manifest.schema.json | JSON | a814944ad3…5c061968 | 7.2 KB | |
| schemas/score-changes.schema.json | JSON | 27b238c82b…8e321f3b | 3.8 KB | |
| schemas/uncertainty.schema.json | JSON | 5c7cec3ced…96a3bac3 | 2.6 KB | |
| score-changes.json | JSON | 789e93b79a…f6fbb8e6 | 116.1 KB | |
| uncertainty.json | JSON | 0971a2b080…73fd3dea | 99.8 KB |
Manifest SHA-256: 2f99dac7c610ad439ec00525e1d3b6cafb417caf185d941867c3d88043f2d69c
Immutable published storage revision: v0.1.14. Downloads on this path are checksum-bound and are never replaced in place.
Cost dataset
A separate dated, answer-safe aggregate provides Cost for 252 of 268 configurations. It is bound to release gpqa-clean-results-v0.1.14, release manifest 2f99dac7c6…43f2d69c, and Cost file SHA-256 c44243db77…294359ed. The immutable score bundle above is unchanged.
Prices dated 2026-10-10 · USD for the full 189-question Clean run, one attempt each · 16 unavailable.
Download cost data Explore Cost–score frontier Read Cost methodology2. Verify locally
cd data/releases/v0.1.14
shasum -a 256 -c checksums.sha256
cd ../../..
node scripts/verify-release.mjs --release v0.1.14The release timestamp is pinned; reproducible builds do not use wall-clock time.
4. Cite this release
- Title
- GPQA Diamond Clean Results
- Bundle version
- 0.1.14
- Data through
- 2026-10-08
- Results DOI
- Not assigned
- Clean membership
- gpqa-diamond-clean 1.2.2
5. Distribution status
- This website serves the published, checksum-bound results bundle.
- The corrected benchmark is available from GitHub and Hugging Face.
- A separate results DOI has not yet been assigned.
- The paper DOI remains separate from the dataset identity.
6. Licensing and attribution
- Release data and documentation: CC BY 4.0.
- Website dependencies: notices bundled locally.
- GPQA benchmark: upstream attribution retained.
- Evaluation evidence: provenance and source attribution retained.
Coverage gaps
These are evidence gaps, not model-quality judgments.
| Source rank | Configuration | Organization | Status | Note |
|---|---|---|---|---|
| 1 | gpt-5.4-pro-2026-03-05_xhigh | OpenAI | no_public_source_artifact | No completed exact-configuration project run or accepted equivalent evidence was available by the data-through cutoff. |
| 8 | claude-opus-5_max | Anthropic | no_public_source_artifact | The completed provider-default or high-equivalent Opus 5 run is not an Opus 5 max score. |
| 10 | grok-4.5_high | xAI | no_public_source_artifact | No completed exact-configuration project run or accepted equivalent evidence was available by the data-through cutoff. |
| 13 | kimi-k3_max | Moonshot | no_public_source_artifact | Skipped for this release candidate because a full Kimi K3 run is too expensive. |
| 16 | qwen3.8-max_xhigh | Alibaba | no_public_source_artifact | No completed exact-configuration project run or accepted equivalent evidence was available by the data-through cutoff. |
| 19 | glm-5.2_max | Z.ai (Zhipu AI) | no_public_source_artifact | Deferred until GLM-5.3 is available through the API. |
| 22 | claude-opus-4-8_max | Anthropic | no_public_source_artifact | Completed Opus 4.8 low or default runs are distinct from this max configuration. |
| 23 | deepseek-v4-flash-0731_max | DeepSeek | no_public_source_artifact | The fresh DeepSeek V4 Flash max result is not bound to the dated 0731 configuration. |
| 30 | claude-sonnet-5_xhigh | Anthropic | no_public_source_artifact | The xhigh configuration was skipped; the completed Sonnet 5 max run was the selected comparison configuration. |
| 35 | muse-spark | Meta AI | no_public_source_artifact | No completed exact-configuration project run or accepted equivalent evidence was available by the data-through cutoff. |
| 38 | grok-4.20-0309-reasoning | xAI | no_public_source_artifact | Grok 4.20 was skipped; Grok 4.3 was the selected xAI run. |
| 46 | Inkling_xhigh | Thinking Machines | no_public_source_artifact | Skipped; Inkling-small is not a GPQA run and does not cover this exact configuration. |