ResearchSep 2, 2026 · 23:14
Show HN: FrontierHarness Eval – 9 harness, same model, cost per pass varies 17x
From frontierharness.org
A benchmark tested 9 harnesses across 12 configurations on identical software engineering tasks using the same model and runtime. All 360 trials started from the same fresh checkpoint restore to prevent warm-cache bias. The evaluation used Kimi K3 and was executed on Runta with identical vCPU, memory, disk size, disk contents and memory state.
Read the sourceSign in to suggest
Sources
+ 1 more
- DISCUSSIONshiqimeinews.ycombinator.com
#harness#benchmark#inferenceMarkdown
Content history (2)
- 2026-10-16 · Summary · vi · Replaced stored text9 harness trên 12 cấu hình, được thử nghiệm trên các nhiệm vụ kỹ thuật phần mềm giống hệt…
- 2026-10-16 · Title · vi · Replaced stored textFrontierHarness Eval: 9 harness khác nhau trên cùng một model, chi phí chênh lệch tới 17 …Show HN: FrontierHarness Eval – 9 harness, cùng một mô hình, chi phí mỗi lần vượt qua khá…
One ranked edition, every morning
The stories that matter today. Free, cancel anytime.
Telegram @aidr_today