Lineup — recorded transcripts, re-scored
The lineup
Every row below was produced by crucible-grader-v1.0.0, the same function the API and the agent tools call. Nothing is inherited from a leaderboard elsewhere.
| # | Model | Score | Pass / fail / open | Latency | Tokens |
|---|---|---|---|---|---|
| 1 | anthropic/claude-sonnet-4 | 1.0000 | 3 / 0 / 0 | 1602 ms | 36 |
| 2 | google/gemini-2.5-flash | 1.0000 | 3 / 0 / 0 | 1310 ms | 34 |
| 3 | openai/gpt-4.1 | 1.0000 | 3 / 0 / 0 | 1500 ms | 35 |
Ranked by crucible-grade-v1.0.0. If every row above is identical, the task is measuring agreement rather than capability — the engine charges that as discrimination and the verdict says so.
Can anyone reproduce this?
A score you cannot rerun is a rumour. These are live facts from the Hugging Face Hub about the models above: whether the weights are gated, and whether a pinnable revision exists at all.
live · 2026-10-04| Model | Reproducible? | Revision | License | Downloads |
|---|---|---|---|---|
| Qwen/Qwen2.5-72B-Instruct | pinnable | 495f39366efe… | other | 284,503 |
| meta-llama/Llama-3.1-70B-Instruct | gated | 1605565b47bb… | llama3.1 | 99,597 |
| mistralai/Mistral-7B-Instruct-v0.3 | pinnable | c170c708c41d… | apache-2.0 | 2,147,029 |
| deepseek-ai/DeepSeek-V3 | pinnable | e815299b0bcb… | — | 1,353,839 |
| google/gemini-2.5-flash | no public repo Not published on the Hugging Face Hub, so there is no public revision to pin. A reader cannot reproduce this row from a revision hash. | — | — | — |
| anthropic/claude-sonnet-4 | no public repo Not published on the Hugging Face Hub, so there is no public revision to pin. A reader cannot reproduce this row from a revision hash. | — | — | — |
| openai/gpt-4.1 | no public repo Not published on the Hugging Face Hub, so there is no public revision to pin. A reader cannot reproduce this row from a revision hash. | — | — | — |
Model metadata from the Hugging Face Hub public API. Downloads and likes are Hub counters, not usage telemetry. Counter values are Hub totals, not usage telemetry.