Dataset cleanup v2

A full read of every row across the three scored benchmarks turned up duplicate questions, one broken multiple-choice item, and a taxonomy that didn't match between tiers. This is the changelog for the fixes. Nothing about the tier design changed — the questions that remain are the same questions, minus the redundant and broken ones.

bench-effortless-6-2026

• Relabeled the six coarse categories to the fine-grained taxonomy used by the other tiers: Math→Math-arithmetic, Logic→Logic-deduction, Language→Language-comprehension, Knowledge→Knowledge-basic, Commonsense→Commonsense-reasoning, PatternRecognition→Pattern-recognition.
• Removed stray blank-line separators; the file is now clean one-object-per-line JSONL.
• Row count unchanged: 240.

bench-easy-6-2026  269 → 238

• Dropped 10 rows in Logic-consistency that were a verbatim copy-paste of the other 10 (the category was double-counted at n=20).
• Removed questions duplicated verbatim in the Effortless tier (e.g. "What is 1 + 1?", "What is the capital of Romania?"); the two generative tiers are now disjoint.
• Removed a handful of reworded twins that share the same answer (e.g. two phrasings of the freezing-point-of-water question).
• Metric label in models.json corrected: exact_match → hybrid_score (the score was always hybrid; only the label was wrong).

bench-mid-6-2026  155 → 143

• Fixed Logic-consistency-0003 (barber paradox): it scored two options at 1.0. The restatement "the barber does not exist" is now partial credit (0.5); every item now has exactly one correct answer.
• Removed the same numeric/letter sequences reused across Math-pattern, Pattern-generation, and Pattern-recognition (e.g. 1,4,9,16, 1,1,2,3,5,8, A,C,F,J), keeping one copy in the most specific category.
• Removed reworded twins that share an answer (e.g. "Who painted the Mona Lisa?" / "Which artist painted the Mona Lisa?").
• Metric label in models.json corrected: acc_norm → soft_score_norm (matches the value actually reported).

Leaderboard

• All existing model scores are flagged "stale": true and pinned to the pre-v2 dataset revision they were computed against. They stay visible but should not be trusted until re-evaluated against these files.
• Reminder: scores are not comparable across tiers. Effortless and Easy are free-form generation (exact / hybrid scoring); Mid is multiple-choice log-likelihood. A higher Mid score reflects an easier evaluation protocol, not easier questions.

Re-run any model against the cleaned data with script.py from the leaderboard Space.