Can we trust LLMs to judge whether a research idea is novel?

The study Old Ideas, Novel Problems examines how consistently LLMs assess the novelty of research ideas, finding that minor changes to evaluation setups can yield significantly different judgments.