One combined rank.
Every accepted mean@N protocol shares the ranking. Repeat count remains explicit and is not normalized away.
How ranking worksGPQA Diamond Clean leaderboard
Combined ranking 268 configurations 189 retained questions
Rein et al. introduced GPQA (opens in a new tab), a 448-question graduate-level science benchmark; GPQA Diamond (opens in a new tab) is the paper's 198-question highest-quality subset. GPQA Diamond Clean (opens in a new tab) is this independent project's corrected overlay for Diamond, retaining 189 questions after nine audited exclusions—not an official update from the GPQA authors. The leaderboard credits Epoch AI's public GPQA Diamond evaluations (opens in a new tab) where used, while derived, recovery, and project-native evidence remain explicitly labelled.
Cost available for 252 of 268 configurations · 16 unavailable · prices dated 2026-10-10. USD for the full 189-question Clean run, one attempt each. Explore the Cost–score frontierDownload cost data
| Rank Rank movement Compares this configuration's standard competition rank on Clean 189 with its paired rank on Original 198.
Ranks use exact scores. Tied ranks list lower Cost first, with unavailable Cost last. Filters never recompute them. | Model | Clean score | Cost | Clean change | Benchmark source Benchmark source classes How each score entered the combined ranking.
|
|---|---|---|---|---|---|
| 1— |
GPT 6 AstraOpenAI · mean@1 · maxNative transport composite
|
Clean score98.9% | Cost$16.72 | Clean change+3.0 ppfrom 96.0% | Benchmark sourceNative transport composite |
Evidence detailExpand this record to load its answer-safe evidence detail. | |||||
| 2↑2 |
GPT 6.1 SolOpenAI · mean@1 · maxNative transport composite
|
Clean score98.4% | Cost$3.58 | Clean change+3.5 ppfrom 94.9% | Benchmark sourceNative transport composite |
Evidence detailExpand this record to load its answer-safe evidence detail. | |||||
| 2— |
GPT 6 SolOpenAI · mean@1 · maxNative transport composite
|
Clean score98.4% | Cost$4.61 | Clean change+3.0 ppfrom 95.5% | Benchmark sourceNative transport composite |
Evidence detailExpand this record to load its answer-safe evidence detail. | |||||
| 2— |
GPT 6 Astra ProOpenAI · mean@1 · maxComplete native primary
|
Clean score98.4% | Cost$21.81 | Clean change+3.0 ppfrom 95.5% | Benchmark sourceComplete native primary |
Evidence detailExpand this record to load its answer-safe evidence detail. | |||||
| 5↑2 |
Gemini 3.7 FlashGoogle · mean@1 · highNative transport composite
|
Clean score97.4% | Cost$2.02 | Clean change+2.9 ppfrom 94.4% | Benchmark sourceNative transport composite |
Evidence detailExpand this record to load its answer-safe evidence detail. | |||||
| 5↓1 |
Gemini 3.8 FlashGoogle · mean@1 · highNative transport composite
|
Clean score97.4% | Cost$4.33 | Clean change+2.4 ppfrom 94.9% | Benchmark sourceNative transport composite |
Evidence detailExpand this record to load its answer-safe evidence detail. | |||||
| 5↑5 |
Gemini 3.6 FlashGoogle · mean@1 · highNative transport composite
|
Clean score97.4% | Cost$6.10 | Clean change+3.4 ppfrom 93.9% | Benchmark sourceNative transport composite |
Evidence detailExpand this record to load its answer-safe evidence detail. | |||||
| 5↑5 |
GPT 5.5 Pro pre-releaseOpenAI · mean@7 · xhighDerived uniform
|
Clean score97.4% | Cost$196.29 | Clean change+3.4 ppfrom 93.9% | Benchmark sourceDerived uniform |
Evidence detailExpand this record to load its answer-safe evidence detail. | |||||
| 9— |
GPT 5.5 pre-releaseOpenAI · mean@8 · xhighStrict
|
Clean score97.2% | Cost$31.35 | Clean change+3.2 ppfrom 94.0% | Benchmark sourceStrict |
Evidence detailExpand this record to load its answer-safe evidence detail. | |||||
| 10↓4 |
Gemini 3.1 Pro PreviewGoogle · mean@8 · defaultTargeted recovery
|
Clean score97.0% | Cost$14.14 | Clean change+2.5 ppfrom 94.5% | Benchmark sourceTargeted recovery |
Evidence detailExpand this record to load its answer-safe evidence detail. | |||||
| 11↑1 |
GPT 5.6 SolOpenAI · mean@1 · maxComplete native primary
|
Clean score96.8% | Cost$7.44 | Clean change+3.4 ppfrom 93.4% | Benchmark sourceComplete native primary |
Evidence detailExpand this record to load its answer-safe evidence detail. | |||||
| 12↑1 |
GPT 5.4OpenAI · mean@8 · xhighTargeted recovery
|
Clean score96.7% | Cost$15.98 | Clean change+3.4 ppfrom 93.2% | Benchmark sourceTargeted recovery |
Evidence detailExpand this record to load its answer-safe evidence detail. | |||||
| 13↑7 |
Qwen3.8 Max 0902Qwen · mean@1 · xhighComplete native primary
|
Clean score96.3% | Cost$13.58 | Clean change+3.9 ppfrom 92.4% | Benchmark sourceComplete native primary |
Evidence detailExpand this record to load its answer-safe evidence detail. | |||||
| 13↓6 |
Gemini 3.1 Pro PreviewGoogle · mean@1 · highStrict
|
Clean score96.3% | Cost$14.97 | Clean change+1.9 ppfrom 94.4% | Benchmark sourceStrict |
Evidence detailExpand this record to load its answer-safe evidence detail. | |||||
| 13↑1 |
Grok 4.7xAI · mean@1 · xhighNative transport composite
|
Clean score96.3% | Cost$22.94 | Clean change+3.4 ppfrom 92.9% | Benchmark sourceNative transport composite |
Evidence detailExpand this record to load its answer-safe evidence detail. | |||||
| 16↑2 |
Gemini 3.5 FlashGoogle · mean@8 · highStrict
|
Clean score96.1% | Cost$9.76 | Clean change+3.3 ppfrom 92.8% | Benchmark sourceStrict |
Evidence detailExpand this record to load its answer-safe evidence detail. | |||||
| 17↓3 |
DeepSeek V4 FlashDeepSeek · mean@1 · maxComplete native primary
|
Clean score95.8% | Cost$2.06–$4.11 | Clean change+2.8 ppfrom 92.9% | Benchmark sourceComplete native primary |
Evidence detailExpand this record to load its answer-safe evidence detail. | |||||
| 17↓3 |
Claude Sonnet 5.5Anthropic · mean@1 · highDirect API run
|
Clean score95.8% | Cost$2.10 | Clean change+2.8 ppfrom 92.9% | Benchmark sourceDirect API run |
Evidence detailExpand this record to load its answer-safe evidence detail. | |||||
| 17↑3 |
Grok 4.6xAI · mean@1 · highComplete native primary
|
Clean score95.8% | Cost$6.51 | Clean change+3.3 ppfrom 92.4% | Benchmark sourceComplete native primary |
Evidence detailExpand this record to load its answer-safe evidence detail. | |||||
| 20↓1 |
Gemini 3 Pro PreviewGoogle · mean@8 · defaultStrict
|
Clean score95.5% | Cost$18.87 | Clean change+2.9 ppfrom 92.6% | Benchmark sourceStrict |
Evidence detailExpand this record to load its answer-safe evidence detail. | |||||
| 21↑12 |
GPT 6 LunaOpenAI · mean@1 · maxComplete native primary
|
Clean score95.2% | Cost$0.94 | Clean change+4.3 ppfrom 90.9% | Benchmark sourceComplete native primary |
Evidence detailExpand this record to load its answer-safe evidence detail. | |||||
| 21↓1 |
DeepSeek V4.1 FlashDeepSeek · mean@1 · maxComplete native primary
|
Clean score95.2% | Cost$1.36–$2.73 | Clean change+2.8 ppfrom 92.4% | Benchmark sourceComplete native primary |
Evidence detailExpand this record to load its answer-safe evidence detail. | |||||
| 21↓1 |
DeepSeek V4 ProDeepSeek · mean@1 · maxComplete native primary
|
Clean score95.2% | Cost$6.54–$13.08 | Clean change+2.8 ppfrom 92.4% | Benchmark sourceComplete native primary |
Evidence detailExpand this record to load its answer-safe evidence detail. | |||||
| 21↓7 |
Claude Opus 5Anthropic · mean@1 · noneStrict
|
Clean score95.2% | Cost$7.32 | Clean change+2.3 ppfrom 92.9% | Benchmark sourceStrict |
Evidence detailExpand this record to load its answer-safe evidence detail. | |||||
| 21↑4 |
Ember-1Fireworks · mean@1 · maxNative transport composite
|
Clean score95.2% | Cost$10.02 | Clean change+3.3 ppfrom 91.9% | Benchmark sourceNative transport composite |
Evidence detailExpand this record to load its answer-safe evidence detail. | |||||
No configurations match these filters.
What changed
Nine defective questions moved scores and ranks.
Cleaning did not make models more capable. It removed questions that could not fairly distinguish a correct answer, then recomputed every accepted configuration on the same retained membership.
- Scores that rose
- 260 / 268
- Crossed 95%
- 26
- Observed ranks changed
- 195
- Unresolved evidence
- 0
The broader answer-key audit
When the Answer Key Is Wrong
This leaderboard is the GPQA Diamond results layer of a broader calibrated audit of GPQA, MMLU-Pro, and MMMU-Pro. The paper explains how questionable items were found, how over-calls were stripped away, and why a single disagreement pass can miss benchmark defects.
Latest published paper10.5281/zenodo.21613590 (opens in a new tab)
Every score carries evidence.
5 accepted evidence pathways are reconciled into 268 canonical configurations without hiding their source class.
Use the evidence keyThree things to cite.
- Paper
- Latest published paper (opens in a new tab) · Cite this paper version (opens in a new tab) · Source (opens in a new tab)
- Benchmark
- GPQA-Diamond-Clean 1.2.2 · GitHub (opens in a new tab) · Hugging Face (opens in a new tab)
- Results
- Published 0.1.14 bundle · results DOI not yet assigned