AI auditors catch most agent misbehavior but flood reviewers with false alarms, new study finds

A study by Anthropic researcher Andy Hall found that AI models can often spot problematic behavior by other AI agents in logs. While GPT-5 found about 98 percent of reward-hacking cases, more exhaustive searches resulted in more false alarms, which is not yet enough for reliable safety checks.