Historical record · v0.1.0
BABEL v1
The benchmark and replication protocol remain public; they do not validate execution-failure prediction.
Does the frozen benchmark support reproducible compositional semantic-failure tests?
Internal and annotation-derived labels did not establish prediction of real execution failure.
Why it was pursued
The benchmark froze compositional semantic-failure tasks for public reproduction and challenge.
Decision date
2026-07-14; status current to 2026-07-17.
No longer claimed
- The structural diagnostic predicts execution failure
- Benchmark labels alone validate real-world safety consequences
What survives
- Frozen benchmark artifact
- Evaluation and replication protocol
- Scoped reported benchmark results
Successor records
BABEL v2
The claim is unverified; advancement requires the frozen partner evaluation and held-out gate.