PalmBench
Same program.
Same question. Different species.
We run the gauntlet the way your team would: one production-scale COBOL program, one instruction — extract the business rules — and the receipts on the table.
The gauntlet
One instruction. Two answers.
same 50,000-line program · same question · palmbench 2026
generic llm
- run 1 · a91f·4c
- run 2 · 77c2·d9
- run 3 · e03d·11
three runs · three answers
palm ark
- run 1 · 5b1c·e7
- run 2 · 5b1c·e7
- run 3 · 5b1c·e7
three runs · one truth
Accuracy · lineage · determinism
Repeatability is the benchmark
Accuracy is only half the result. The half that decides audits is repeatability: ask a generative model three times and you get three artifacts; nobody can sign off on a moving target. Ark’s pipeline is deterministic — three runs, one hash, one truth your reviewers can pin to the wall.
We’ll happily reproduce this on your code, against any model you choose, inside your perimeter — that’s the standing offer behind the POC. Figures shown (~95% vs ~30%) are from PalmBench, our reproducible extraction benchmark; ask us for the methodology.