---
format: "aidr-story-markdown/v1"
id: "5b66b471dda6a30666311ba511cd1fdf12103b40b78d16de1a4ce79691288b20"
canonical_url: "https://aidr.today/5b66b471?lang=en"
title: "Does the Ranking Hold? A Practitioner-Facing Audit of Hallucination Detectors on Small, Quantized, On-Device LLMs"
lang: "en"
requested_lang: "en"
available_langs: ["en","vi"]
translation_fallback: null
fallback_fields: []
published_at: "2026-11-16T05:00:00.000Z"
category: "Research"
topics: ["llm"]
source_urls: ["https://arxiv.org/abs/2611.17339"]
summary: "A one-GPU audit examined four hallucination detectors on small, quantized, on-device LLMs including Phi-3-mini and Llama-3.2-3B-Instruct. Cross-model ranking transfer was poor, with Spearman rho at -0.80 and Kendall tau-b at -0.667. The cheap lexical baseline was statistically indistinguishable from the sampling-based consistency detectors on both models."
---

# Does the Ranking Hold? A Practitioner\-Facing Audit of Hallucination Detectors on Small, Quantized, On\-Device LLMs

> [Open the canonical story](<https://aidr.today/5b66b471?lang=en>)

**Published:** 2026-11-16T05:00:00.000Z
**Category:** Research
**Topics:** llm

## Summary

A one\-GPU audit examined four hallucination detectors on small, quantized, on\-device LLMs including Phi\-3\-mini and Llama\-3\.2\-3B\-Instruct\. Cross\-model ranking transfer was poor, with Spearman rho at \-0\.80 and Kendall tau\-b at \-0\.667\. The cheap lexical baseline was statistically indistinguishable from the sampling\-based consistency detectors on both models\.

## Sources

- [Story source](<https://arxiv.org/abs/2611.17339>)

