Exploring the Design Space of Permutation Diffusion LLMs

Researchers studied three diffusion-LLM parameterizations for permutation generation: hard-assignment prediction over shuffled candidate-index sequences, soft-assignment prediction over doubly stochastic matrices, and direct ordering-score prediction followed by sorting. The hard- and soft-assignment variants achieved Recall@1 comparable to a constrained masked-diffusion baseline at one function evaluation, while direct ordering-score prediction guarantees valid permutations without Hungarian assignment or Sinkhorn normalization. Additional function evaluations did not consistently improve ranking quality across three Amazon domains with candidate sets of size 10 and 20.