● ResearchNov 6, 2026 · 12:001 source

BlockLock: Reproducible LLM Inference with Fixed Batch Geometry

From arxiv.org

AI;DR

Reproducible LLM inference requires a request to produce the same output regardless of other requests in the batch. BlockLock keeps shared batched prefill and pins only the numerical path of the protected request. BlockLock has two modes: BlockLock-Rows uses fixed-width rows and retains batched decode of all requests, and BlockLock-Pack uses terminal-anchored packing with an isolated dummy segment.

Read originalSign in to suggest

Sources

#inferenceMarkdownScore 3.26.4k tokens
Image: arxiv.org
Read original