1

Independent Tester's 8.6% Score Contradicts Cantina's 66.7% AI Claim

huggingnews.com

An independent tester recorded an 8.6% pass rate for Cantina Security's apex-flash-1 model on the CyberGym benchmark, contradicting the developer's claimed 66.7% pass rate. The evaluation found issues including infinite retry loops, tool-call confusion, and context decoherence, and noted that apex-flash-1 is a post-trained version of glm-5.3-flash released as a small run to refine its training process.

Sign in to suggest edits

Key sources

  1. SOURCEmarketbrief.now
Markdown