研究Research2026.11.15

Would Your Agent Benchmark Notice a Missing Run?

A valid chain of receipts can still leave an experiment incomplete. ClaimReceipt asks what the evidence supports, and whether it covers what we committed to run.

BB
Blossom Builders Team
Blossom AI
·Nov 15, 2026·2 min read

Chinese ink-wash illustration of a scroll of receipts with one missing link

Would your agent benchmark still pass if one run quietly disappeared?

A clean audit log can preserve every recorded byte without showing whether the experiment is complete. Before trusting a result, we need to know which runs were expected and what evidence each claim requires.

Two questions for the evidence

In ClaimReceipt: Verifying Evidence Sufficiency and Coverage in Agent Evaluations, we separate sufficiency from coverage. Sufficiency asks whether retained evidence lets an auditor recompute a claim. Coverage asks whether the records cover the experiment committed to before outcomes were observed.

These questions require different checks. A transcript may omit the private inputs needed to recompute an economic outcome. A signed chain may contain authentic records while leaving out one expected run.

What happens when a receipt disappears?

Our prospective test committed 30 assignments before inference. With all required evidence available, coverage and accounting passed. Removing one terminal receipt left a valid chain of 29, but coverage became inconclusive and descriptive reporting was blocked.

The missing record mattered even though the remaining records were authentic. The experiment-level commitment made the omission visible.

Keep the conclusion within the evidence

The prototype was evaluated in one hotel-transaction testbed. It cannot reveal transactions that were never committed or establish that synthetic profiles represent real people. Our exploratory specification-legibility probe also found that the frozen specification was not yet unambiguous to an independent reader.

Before trusting an agent benchmark, ask what was supposed to be recorded, what is missing, and which conclusions the retained evidence permits.

Peiying Zhu and Sidi Chang. ClaimReceipt: Verifying Evidence Sufficiency and Coverage in Agent Evaluations. arXiv:2609.01992, 2026.

Read the paper · Download PDF