---
format: "aidr-story-markdown/v1"
id: "cd19a18065e25c81e9d4d59e59f2b57bfa9a42698a87821ad9c3d9989819dca9"
canonical_url: "https://aidr.today/cd19a180?lang=en"
title: "Hermes Agent Leads Reliability Ranking on Updated AI Coding Harness Leaderboard"
lang: "en"
requested_lang: "en"
available_langs: ["en","vi"]
translation_fallback: null
fallback_fields: []
published_at: "2026-11-07T18:54:39.000Z"
category: "Tools"
topics: ["deepseek","agent","devtools"]
source_urls: ["https://marketbrief.now/ai/hermes-agent-leads-reliability-ranking-on-updated-ai-coding-harness-lead-9dfa3ba5","https://x.com/joelniklaus/status/2119082748870844870","https://x.com/joelniklaus/status/2118373808692437403","https://x.com/OfirPress/status/2118404374238318610","https://x.com/Teknium/status/2118604525003251800","https://x.com/joelniklaus/status/2119090178304201076","https://x.com/joelniklaus/status/2119035871735230752","https://x.com/joelniklaus/status/2118993153264963700"]
summary: "Researcher Joel Niklaus added a consistency metric to the Harness Leaderboard to track how often coding-agent wrappers solve the same task on three consecutive attempts. Hermes Agent was identified as the most reliable overall, achieving a 71.3% solve rate on the DeepSeek-V4.1-Flash model. The updated data shows that reliability for unattended AI work remains low, with the top-ranked mini-swe-agent solving only 33% of tasks consistently across three open models."
---

# Hermes Agent Leads Reliability Ranking on Updated AI Coding Harness Leaderboard

> [Open the canonical story](<https://aidr.today/cd19a180?lang=en>)

**Published:** 2026-11-07T18:54:39.000Z
**Category:** Tools
**Topics:** deepseek, agent, devtools

## Summary

Researcher Joel Niklaus added a consistency metric to the Harness Leaderboard to track how often coding\-agent wrappers solve the same task on three consecutive attempts\. Hermes Agent was identified as the most reliable overall, achieving a 71\.3% solve rate on the DeepSeek\-V4\.1\-Flash model\. The updated data shows that reliability for unattended AI work remains low, with the top\-ranked mini\-swe\-agent solving only 33% of tasks consistently across three open models\.

## Sources

- [Story source](<https://marketbrief.now/ai/hermes-agent-leads-reliability-ranking-on-updated-ai-coding-harness-lead-9dfa3ba5>)
- [Story source](<https://x.com/joelniklaus/status/2119082748870844870>)
- [Story source](<https://x.com/joelniklaus/status/2118373808692437403>)
- [Supporting source](<https://x.com/OfirPress/status/2118404374238318610>)
- [Supporting source](<https://x.com/Teknium/status/2118604525003251800>)
- [Supporting source](<https://x.com/joelniklaus/status/2119090178304201076>)
- [Supporting source](<https://x.com/joelniklaus/status/2119035871735230752>)
- [Supporting source](<https://x.com/joelniklaus/status/2118993153264963700>)