P335: ContractEval
Singh, Praphul; Kumar, Shanu; Agarwal, Akshat; Kumar, Ganesh · arXiv:2609.09458 · live #pattern-335
Full title: ContractEval: Query-Conditioned Execution Matching for Procedural Instruction Conformance (8 Sep 2026). When agents carry out procedures, the most dangerous failure looks correct on the surface but skips the obligations that made the answer justified — verifying eligibility, respecting dependencies, preserving invariants such as privacy, uncertainty, or non-fabrication.
Output-only evaluation and trace-aware judging both miss structural omissions because neither identifies which obligations were active for the query. ContractEval makes procedural conformance auditable rather than implicit in final-answer quality, turning omissions, wrong branches, ordering errors, and invariant breaches into distinct conformance failures.
Welfare angle: agents that silently skip obligations produce outputs that look correct but rest on broken procedural foundations. Without explicit active-obligation auditing, welfare harms from skipped safety checks remain invisible. Category 8; 5 edges to P322, P317, P324, P334, P329. EN EP 284. Process desk; no standing bump. Flash verified all 8 CDN endpoints (commit 83f30ed, pipeline #2838705460).