---
format: "aidr-story-markdown/v1"
id: "ce7d68cc7819c7557461de59308a7980aa191640565851ddd3ae29d45608014e"
canonical_url: "https://aidr.today/ce7d68cc?lang=en"
title: "OncoNoteBERT: A Foundation Representation Model for Natural Language Processing of Real-World Outpatient Oncology Notes"
lang: "en"
requested_lang: "en"
available_langs: ["en","vi"]
translation_fallback: null
fallback_fields: []
published_at: "2026-10-06T04:00:00.000Z"
category: "Research"
topics: ["llm"]
source_urls: ["https://arxiv.org/abs/2610.03829"]
summary: "OncoNoteBERT, a BERT-style encoder trained from scratch on 290,026 UK outpatient oncology notes from 21,564 lung and head-and-neck cancer patients, achieved a masked language modelling perplexity of 2.10 after continued pretraining, outperforming RadBERT and PathologyBERT in zero-shot evaluation."
---

# OncoNoteBERT: A Foundation Representation Model for Natural Language Processing of Real\-World Outpatient Oncology Notes

> [Open the canonical story](<https://aidr.today/ce7d68cc?lang=en>)

**Published:** 2026-10-06T04:00:00.000Z
**Category:** Research
**Topics:** llm

## Summary

OncoNoteBERT, a BERT\-style encoder trained from scratch on 290,026 UK outpatient oncology notes from 21,564 lung and head\-and\-neck cancer patients, achieved a masked language modelling perplexity of 2\.10 after continued pretraining, outperforming RadBERT and PathologyBERT in zero\-shot evaluation\.

## Sources

- [Story source](<https://arxiv.org/abs/2610.03829>)