1

Fixing Tool-Call Accuracy in vLLM

ubicloud.com

A vLLM fork achieved 100% tool call schema accuracy with Kimi and DeepSeek models, and vLLM 0.31's constrained decoding support now enables the same result with newer versions of those models.

Sign in to suggest

Key sources

  1. DISCUSSIONcoffeemugnews.ycombinator.com
Markdown