Trial by Trace: Benchmarking AI Agent Harnesses for Reliability and Failure Attribution

Researchers introduced TraceTrial-Bench, a benchmark designed to evaluate the reliability of AI agent harnesses across seven components: context, execution, governance, lifecycle, observability, tools, and verification. The benchmark includes 95 expert-level tasks with executable or state-based ground truth to enable semantically equivalent comparisons while preserving each harness's native behavior. A Trajectory Tracing (TT) module captures agent execution traces, and a two-layer evaluation framework separately assesses agent performance and harness reliability.