Recursive Semantic Divergence for LLM Agent Consistency

Researchers propose Recursive Semantic Divergence (RSD), an unsupervised trajectory-level consistency metric for LLM agents that measures expected cumulative semantic disagreement between two independent rollouts from the same state using natural language inference. The metric addresses limitations in existing evaluation methods, which primarily assess task correctness and do not capture consistency across agent executions or how inconsistency propagates across future turns.