ResearchSep 2, 2026 · 23:14

Show HN: FrontierHarness Eval – 9 harness, same model, cost per pass varies 17x

From frontierharness.org

AI;DR

A benchmark tested 9 harnesses across 12 configurations on identical software engineering tasks using the same model and runtime. All 360 trials started from the same fresh checkpoint restore to prevent warm-cache bias. The evaluation used Kimi K3 and was executed on Runta with identical vCPU, memory, disk size, disk contents and memory state.

Read the sourceSign in to suggest

Sources

+ 1 more
  1. DISCUSSIONshiqimeinews.ycombinator.com
#harness#benchmark#inferenceMarkdown
Content history (2)
  • 2026-10-16 · Summary · vi · Replaced stored text
    9 harness trên 12 cấu hình, được thử nghiệm trên các nhiệm vụ kỹ thuật phần mềm giống hệt…
  • 2026-10-16 · Title · vi · Replaced stored text
    FrontierHarness Eval: 9 harness khác nhau trên cùng một model, chi phí chênh lệch tới 17 …
    Show HN: FrontierHarness Eval – 9 harness, cùng một mô hình, chi phí mỗi lần vượt qua khá…

One ranked edition, every morning

The stories that matter today. Free, cancel anytime.

Telegram @aidr_today