Cancer segmenters can hang a clean-looking mask that is still wrong. Missed lesions and false positives that invite unnecessary biopsies both look plausible on a worklist. Mengmeng Ma, Yunxiang Peng, Tang Li, Lu Lin, Binsheng Zhao, Oguz Akin, and Xi Peng (University of Virginia, University of Delaware, Memorial Sloan Kettering) ask whether the model already carries an internal Feeling of Error (FOE) before the mask ships. They posted arXiv:2609.08879 around 8 September 2026. The paper is Medical AI Encodes a “Feeling of Error”: Verifying Cancer Segmentation via Internal Concepts. Code is at github.com/deep-real/CancerSegFailure.
Viewer shops already know the hang: overlay on, radiologist still on the hook. Output confidence and entropy can warn that a mask is risky, but they rarely say why, and pushing detection sensitivity often costs segmentation quality. FOE leaves the segmenter alone and reads a Sparse Autoencoder concept signature from its activations instead.
Fewer, weaker concepts when the mask is wrong
The backbone is MedSAM (ViT). Sparse Autoencoders train on activations from layers {1, 3, 5, 7, 9, 11} with BatchTopK sparsity (dictionary size 1,536, L0 sparsity 8 in the best ablation), then concatenate those sparse concept vectors into one failure representation. On PI-CAI prostate MRI, success cases light more concepts at higher magnitude than failure cases (p < 1e-5, Cohen’s d about 1.8). Shared anatomy concepts such as peripheral zone and transition zone still fire on both sides. Tumor-specific concepts fire strongly only when the mask is right. That gap is the machine FOE the authors chase.
A lightweight classifier, XGBoost in the paper’s best setup, scores that concept vector for failure. Feature importance maps back to named concepts (tumor core, gland margin, extra-gland tissue), so the flag comes with a short “why” rather than a bare confidence number. Deep layers carry the strongest failure signal, mid layers help segmentation quality, and early layers add low-level visual cues. Using all six layers beats any single depth band. Patch-level scores also support stripping false-positive patches while keeping confident true-positive regions, which is how they correct some masks without retraining MedSAM.
Where it was measured
Main cohorts are PI-CAI prostate MRI (1,200 / 60 / 240 train / val / test), PanTS pancreatic CT, and a supplementary MSD-Brain MRI split (310 / 78 / 96) for complete, core, and enhancing tumor. Prostate158 is held out for zero-shot transfer of a detector trained only on PI-CAI. Concepts localize to clinically named anatomy (peripheral zone, pancreas head, enhancing tumor) when IoU against anatomical masks clears 0.5.
Output baselines (max probability, mean probability, entropy, energy, direct thresholding) sit on a Pareto curve: raise failure-detection F1 and Dice falls. The paper’s own framing is that a roughly 20-point F1 gain via those proxies can cost about 8 to 10 Dice points. FOE sits above that front on prostate and pancreas in their Fig. 5, with about +8.7 Dice and +6.2 F1 on PI-CAI and about +2.3 Dice and +9.2 F1 on PanTS relative to the baseline front. Ablation F1 peaks at 63.5 when all six layers feed XGBoost (deep alone 57.6; mid 50.4; early 44.5). Zero-shot on Prostate158, MeanProb AUROC is 52.6 while the concept detector reaches 73.6. Full pipeline latency on an A6000 is about 176 ms versus 126 ms for MedSAM alone; encode-only SAE overhead for routine flagging is closer to +4%.
How this lands in a viewer
If you already hang oncology MRI or CT in a DICOM viewer, treat FOE as a second rail beside the mask, not a replacement segmenter. Run MedSAM (or your frozen vendor ViT) as usual, hang the tumor overlay, then run the six SAE encoders plus classifier and show a failure flag with the top contributing concepts next to the scan. Keep human confirm before any automatic false-positive strip. Fail closed when the series is outside prostate MRI, pancreatic CT, or the MSD-Brain style stack the SAEs saw, when you need open-set non-cancer organs, when concept labels have not been validated for your site, or when intervention would rewrite masks without a reader in the loop.
SAT3D feeds voxel-wise uncertainty back as dense prompts inside the segmenter. This paper is the opposite trade: keep the cancer overlay path intact, read FOE from internal SAE concepts, and flag quiet misses with an explanation.
Rebuild from arXiv:2609.08879. As of 9 September 2026 the abstract and PDF respond (HTTP 200). The preprint is the source of record until a camera-ready version exists. Public code is linked in the paper.
Sources
- Mengmeng Ma, Yunxiang Peng, Tang Li, Lu Lin, Binsheng Zhao, Oguz Akin, Xi Peng. Medical AI Encodes a “Feeling of Error”: Verifying Cancer Segmentation via Internal Concepts. arXiv:2609.08879, posted ~8 September 2026. https://arxiv.org/abs/2609.08879 (HTTP 200 on 9 September 2026). PDF: https://arxiv.org/pdf/2609.08879 (HTTP 200 on 9 September 2026). Code: https://github.com/deep-real/CancerSegFailure.