BIBIMBAP: A Benchmark for Instructional BIM-Based Automated Programming

BIBIMBAP, a new Text-to-BIM benchmark, evaluates large language models' ability to edit Industry Foundation Classes (IFC) through 100 curated tasks covering Create-Read-Update-Delete operations across spatial, geometric, topological, numeric, and conceptual categories. Each task includes a natural-language prompt, an IFC model, expected structured outputs, and test scripts for automated evaluation. Baseline results show limited performance with a best score of 50.2% and frequent failures in constraint preservation and spatial reasoning.