Same Output, Different Gold: Measuring How Reference Choice Moves a Multilingual Benchmark Score

A six-language benchmark study found that changing the reference annotation while keeping system output fixed can shift scores by up to 4.95 F1 points, with the effect attributed to an undocumented aggregation default that made the shipped reference partially match one of the outputs being scored.