---
format: "aidr-story-markdown/v1"
id: "34e01d4a0b8b50d01c6622cf8394a13ee74abb2e70d07c97103e3a04d75d9953"
canonical_url: "https://aidr.today/34e01d4a?lang=en"
title: "SnorkelAI Finds 55% of TerminalBench 4.0 AI Fixes Are Flawed"
lang: "en"
requested_lang: "en"
available_langs: ["en","vi"]
translation_fallback: null
fallback_fields: []
published_at: "2026-10-26T18:25:49.000Z"
category: "Models"
topics: ["anthropic","llm","agent"]
source_urls: ["https://marketbrief.now/ai/update-snorkelai-finds-55percent-of-terminalbench-40-ai-fixes-are-flawed-17ce3193","https://x.com/chem_safety/status/2112212878132744674","https://x.com/chem_safety/status/2112212879734947937","https://x.com/ajratner/status/2112678199104844241","https://x.com/madiator/status/2112721419906032104","https://x.com/madiator/status/2112572866332164116","https://x.com/scaling01/status/2112357861355352124","https://x.com/rebeccatqian/status/2112684238965514286"]
summary: "SnorkelAI found that 55% of the fixes proposed by the Preference Model team for TerminalBench 4.0 vulnerabilities were themselves flawed, citing over-prescriptiveness or new gaps allowing reward hacking. A preliminary audit also noted that 40% of the 325 identified issues were already tracked in the benchmark's repository, including 21 tasks with outstanding fix requests."
---

# SnorkelAI Finds 55% of TerminalBench 4\.0 AI Fixes Are Flawed

> [Open the canonical story](<https://aidr.today/34e01d4a?lang=en>)

**Published:** 2026-10-26T18:25:49.000Z
**Category:** Models
**Topics:** anthropic, llm, agent

## Summary

SnorkelAI found that 55% of the fixes proposed by the Preference Model team for TerminalBench 4\.0 vulnerabilities were themselves flawed, citing over\-prescriptiveness or new gaps allowing reward hacking\. A preliminary audit also noted that 40% of the 325 identified issues were already tracked in the benchmark's repository, including 21 tasks with outstanding fix requests\.

## Sources

- [Story source](<https://marketbrief.now/ai/update-snorkelai-finds-55percent-of-terminalbench-40-ai-fixes-are-flawed-17ce3193>)
- [Story source](<https://x.com/chem_safety/status/2112212878132744674>)
- [Supporting source](<https://x.com/chem_safety/status/2112212879734947937>)
- [Story source](<https://x.com/ajratner/status/2112678199104844241>)
- [Supporting source](<https://x.com/madiator/status/2112721419906032104>)
- [Story source](<https://x.com/madiator/status/2112572866332164116>)
- [Supporting source](<https://x.com/scaling01/status/2112357861355352124>)
- [Supporting source](<https://x.com/rebeccatqian/status/2112684238965514286>)