Organic Chemistry is the hardest reportable subcategory.
61.4% mean correctness.
Where GPQA remains difficult
Aggregate question difficulty across 268 accepted configurations,
without publishing prompts, answer keys, or a model-by-question matrix.
Results 0.1.14Data through Oct 8, 2026
189 retained questions · 3 domains · 9 reportable subcategories · Aggregate-first publication boundary
These controls preview a future, separately reviewed question-analysis release. This release intentionally publishes one aggregate cohort only.
Displayed statistics require ≥5 questions and ≥5 configurations.
1. Difficulty by domain
Each configuration is weighted equally. Repeated attempts reduce to one per-question mean before aggregation.
2. Subcategories with enough questions to report
4 subcategories suppressed: fewer than 5 questions| Subcategory | Questions | Mean correctness | Value |
|---|---|---|---|
| Astrophysics | 13 | 74.0% | |
| Chemistry (general) | 19 | 72.1% | |
| Electromagnetism and Photonics | 6 | 74.1% | |
| High-energy particle physics | 14 | 84.3% | |
| Molecular Biology | 14 | 71.6% | |
| Organic Chemistry | 68 | 61.4% | |
| Physics (general) | 18 | 79.9% | |
| Quantum Mechanics | 25 | 83.4% | |
| Relativistic Mechanics | 6 | 84.7% |
3. Consistently difficult patterns
Difficulty remains broad.
The release does not expose Record IDs or per-model correctness.
Descriptive, not causal.
These aggregates do not identify why a configuration failed a question.
4. Question disclosure boundary
Included in this release
Domain/subcategory aggregates, counts, definitions, cohort scope, and suppression markers.
Held back pending dedicated review
Record IDs, per-model correctness, prompt text, choices, keys, completions, and reasoning.
5. How to read this page
- Attempts per configuration
- Per-question configuration mean
- Aggregate configurations equally
- Suppress small cells before rendering