SPARKITMeasured June 2026
HLE-Gold
SPARKIT answered 54.4% of the 149 evaluated HLE-Gold questions correctly, compared with 34.9% for each listed direct-model baseline.
149 questions · Accuracy · Updated June 24, 2026
Results
| System | Evaluation mode | Accuracy |
|---|---|---|
| SPARKITSPARKIT research-agent run | SPARKIT research-agent run | 54.4% |
| GPT-5.5Direct model call | Direct model call | 34.9% |
| Claude Opus 4.8Direct model call | Direct model call | 34.9% |
Methodology summary
- 01The evaluation uses 149 biology, medicine, and chemistry questions from the gold subset of Humanity's Last Exam.
- 02Each system is scored against the benchmark answer key, and the reported value is the percentage of evaluated questions answered correctly.
- 03The direct-model baselines are intentionally shown without the SPARKIT search, reading, and analysis loop; they measure the value of the research workflow rather than a model-family comparison.
Sources and supporting artifacts
Humanity's Last Exam paper ↗The primary publication describing the public benchmark.Representative HLE-Gold run artifacts (JSON) →Three difficult example questions with run metadata and outputs. These examples are not the complete benchmark dataset.End-to-end HLE-Gold walkthrough →A worked example showing the question, research trace, report, and grading outcome.
Limitations
- 01HLE-Gold is a difficult scientific subset, not a comprehensive measure of every research domain or production workflow.
- 02Model and agent behavior can change as providers update their systems, so these results are a dated snapshot rather than a permanent ranking.
- 03Representative run artifacts are published below, but the complete run-level benchmark output is not yet public.
- 04Benchmark accuracy is not evidence of clinical, legal, or regulatory reliability; cited sources and conclusions still require expert verification.