---
format: "aidr-story-markdown/v1"
id: "c92eee09f8eb0137235f3d18ecc531c9d9984d8733b2b91ce8c9e1ba1e867720"
canonical_url: "https://aidr.today/c92eee09?lang=en"
title: "Can we trust LLMs to judge whether a research idea is novel?"
lang: "en"
requested_lang: "en"
available_langs: ["en","vi"]
translation_fallback: null
fallback_fields: []
published_at: "2026-10-27T08:00:00.000Z"
category: "Research"
topics: ["llm"]
source_urls: ["https://allenai.org/blog/novelty-bench","https://huggingface.co/blog/allenai/novelty-bench"]
summary: "The study Old Ideas, Novel Problems examines how consistently LLMs assess the novelty of research ideas, finding that minor changes to evaluation setups can yield significantly different judgments."
---

# Can we trust LLMs to judge whether a research idea is novel?

> [Open the canonical story](<https://aidr.today/c92eee09?lang=en>)

**Published:** 2026-10-27T08:00:00.000Z
**Category:** Research
**Topics:** llm

## Summary

The study Old Ideas, Novel Problems examines how consistently LLMs assess the novelty of research ideas, finding that minor changes to evaluation setups can yield significantly different judgments\.

## Sources

- [Story source](<https://allenai.org/blog/novelty-bench>)
- [Story source](<https://huggingface.co/blog/allenai/novelty-bench>)