On the Opportunities and Challenges of Prefill-to-Autoregressive Cross-Model Transfer

Prefill-to-Autoregressive Cross-model Transfer (PACT) combines a larger model's prompt representations with a smaller model's efficient generation. PLatour is an open-source PACT framework that connects frozen models through lightweight, zero-initialized linear bridges. PLatour significantly outperforms open-source baselines on generation tasks and surpasses Mostik's reported results on shared mathematics and coding benchmarks using a substantially smaller sender.