---
format: "aidr-story-markdown/v1"
id: "3a474d73c07650bcabb4521a937b5c9483554198dd26c3ede05e140dec801109"
canonical_url: "https://aidr.today/3a474d73?lang=en"
title: "Mix, Don't Tune: Bilingual Pre-Training Outperforms Hyperparameter Search in Data-Constrained Settings"
lang: "en"
requested_lang: "en"
available_langs: ["en","vi"]
translation_fallback: null
fallback_fields: []
published_at: "2026-10-13T00:00:00.000Z"
category: "Research"
topics: ["llm"]
source_urls: ["https://machinelearning.apple.com/research/mix-dont-tune"]
summary: "A systematic comparison across four model scales shows that mixing data from a high-resource auxiliary language yields larger improvements than hyperparameter tuning in data-constrained language model pre-training. Using Arabic as the low-resource target and English as the auxiliary, mixing boosts performance by an amount equivalent to 2–3× the unique target data on validation loss and 2–13× on downstream task accuracy."
---

# Mix, Don't Tune: Bilingual Pre\-Training Outperforms Hyperparameter Search in Data\-Constrained Settings

> [Open the canonical story](<https://aidr.today/3a474d73?lang=en>)

**Published:** 2026-10-13T00:00:00.000Z
**Category:** Research
**Topics:** llm

## Summary

A systematic comparison across four model scales shows that mixing data from a high\-resource auxiliary language yields larger improvements than hyperparameter tuning in data\-constrained language model pre\-training\. Using Arabic as the low\-resource target and English as the auxiliary, mixing boosts performance by an amount equivalent to 2–3× the unique target data on validation loss and 2–13× on downstream task accuracy\.

## Sources

- [Story source](<https://machinelearning.apple.com/research/mix-dont-tune>)