---
format: "aidr-story-markdown/v1"
id: "7c4aec239d9fdd911fbf2d16366df88c5408cb9c5bca9e62b5ecd905913e1013"
canonical_url: "https://aidr.today/7c4aec23?lang=en"
title: "The Social World Model Evaluation Gap"
lang: "en"
requested_lang: "en"
available_langs: ["en","vi"]
translation_fallback: null
fallback_fields: []
published_at: "2026-10-27T04:00:00.000Z"
category: "Research"
topics: ["agent"]
source_urls: ["https://arxiv.org/abs/2610.33014"]
summary: "An audit of 113 social world model evaluations published from 1999 to 2026 finds that only 2 of 9 primary evaluations directly measure a real person's response to a system-selected action, and none of 25 checked language-model works meet the proposed evaluation conditions. The study proposes a 1-step negotiation experiment to separate response prediction from other evaluation approaches, noting the findings describe the audited sample rather than field-wide prevalence."
---

# The Social World Model Evaluation Gap

> [Open the canonical story](<https://aidr.today/7c4aec23?lang=en>)

**Published:** 2026-10-27T04:00:00.000Z
**Category:** Research
**Topics:** agent

## Summary

An audit of 113 social world model evaluations published from 1999 to 2026 finds that only 2 of 9 primary evaluations directly measure a real person's response to a system\-selected action, and none of 25 checked language\-model works meet the proposed evaluation conditions\. The study proposes a 1\-step negotiation experiment to separate response prediction from other evaluation approaches, noting the findings describe the audited sample rather than field\-wide prevalence\.

## Sources

- [Story source](<https://arxiv.org/abs/2610.33014>)