Hang an abdominal CT next to a T1W or T2W MRI of the same patient and ask for one pancreas overlay language. On CT the gland is a soft-tissue stripe against fat and vessels. On MRI contrast flips with sequence, and the boundary against bowel can vanish. Training a separate whole-pancreas network per modality is expensive when labels are scarce. Asking for head, body, and tail on top of that usually means a second annotation campaign, and CT subregion labels are often the missing piece. Ziliang Hong, Hongyi Pan, Halil Ertugrul Aktas, Andrea Bejar, Elif Keles, Frank H. Miller, Michael B. Wallace, Rajesh N. Keswani, Gorkem Durak, and Ulas Bagci (Northwestern University Radiology with Mayo Clinic Florida and Northwestern Gastroenterology) posted arXiv:2609.13043 on 11 September 2026. The paper is Unified CT–MRI Pancreas Segmentation for Label-Efficient Cross-Modality Subregion Transfer.
They train one shared nnU-Net encoder-decoder on 4,604 heterogeneous CT and MRI scans with a domain discriminator that pushes the latent features toward modality-invariant anatomy. After that whole-pancreas stage, they freeze the encoder and train a head-body-tail decoder using limited MRI-only subregion labels from Cyst-X. No CT subregion annotations enter that transfer step. On the in-distribution whole-pancreas test set the average Dice is 87.31%. Downstream subregion Dice averages 80.53% on MRI and 83.05% on CT.
What hangs on the viewer
Upstream, mixed CT and MRI volumes go into one encoder. Latent features feed the decoder for whole-pancreas Dice and cross-entropy loss, and also pass through a gradient reversal layer into a modality discriminator (CT vs MRI). The adversarial branch is meant to strip modality-specific appearance so the encoder keeps pancreatic shape rather than windowing tricks. Training uses a warm-up on segmentation alone, then raises the domain-loss weight up to 0.5. Patches are 80×160×192 with sliding-window inference. The paper reports stable whole-pancreas Dice from about 84% to 88% on external OOD sets (AMOS CT/MRI, BTCV CT, U-Mamba MRI) without extra fine-tuning of that primary model.
Downstream is the part a DICOM shop cares about when CT part labels never arrive. The shared encoder stays frozen. A new decoder learns head, body, and tail from the Cyst-X MRI training and validation subset only. CT head-body-tail is then evaluated on the Cyst-X CT holdout path plus a small expert-labeled OOD CT subset (7 AMOS and 10 BTCV scans). Yellow, green, and red mark head, body, and tail in their Fig. 2 qualitative panels; MRI sequences and CT share that overlay vocabulary even though CT never supplied part labels for training.
Why one overlay pipeline matters
If your viewer already shows SEG overlays on CT and MRI series in the same hanging protocol, this paper is a concrete blueprint for keeping one pancreas mask family across modalities instead of shipping two modality-locked models. Whole organ first, then head-body-tail when surgical planning or cyst localization needs the parts. The label-efficiency claim is specific: MRI-only subregion supervision is enough to drive CT part predictions under their transfer setup. That matches a common hospital constraint where whole-organ masks exist on both modalities but fine anatomy labels live on one.
Use the published numbers as deployment checks, not as a product claim. Whole-pancreas ID Dice 87.31% and subregion averages near 81–83% tell you the overlay is in a usable band for assistive display. The pancreatic tail stays hardest on MRI in their table (lower Dice than head), which is the slice where a reader should still zoom and edit. Distance errors on CT subregions stay low in the paper (average HD95 around 4.3 mm), which is useful context for how far a contour may drift before a QC flag.
Where it still fails
Domain alignment is modality-level (CT vs MRI). Scanner, protocol, and center shifts inside MRI or CT are left for later work. OOD whole-pancreas Dice stays above 84% in their external sets, but that is still a different distribution, so fail closed when the mask fragments, when the series is not an abdominal pancreas-relevant acquisition, or when the hanging mixes unusual MRI contrasts the training mix never saw. Subregion transfer assumes the frozen encoder already carries modality-invariant anatomy; if whole-pancreas QC fails, do not trust head-body-tail on the same case. CT part evaluation used a small hand-labeled OOD sample for AMOS and BTCV, so treat CT subregion claims as promising transfer evidence, not a large multi-center parts trial. Code and weights are promised upon acceptance and are not linked in the preprint as of this writing.
For a viewer or clinic AI shop
Treat this as a cross-modality pancreas rail: CT or MRI series in, one whole-pancreas overlay out from the shared network, then optional head-body-tail overlays when MRI subregion labels (or a frozen encoder plus MRI-finetuned decoder) are available. Hang whole-organ green next to the source slices first. Only then hang the three-color part mask so a reader can audit head versus body versus tail against vessels and bowel. Fail closed when whole-organ overlap against a quick QC contour is poor, when the tail mask jumps into spleen or bowel, when the study is not CT/MRI abdomen, or when someone wants CT part metrics published from a model that never saw CT part labels without a human check. If your platform already supports multi-modality SEG and a second overlay layer, this paper is a practical pattern for sharing one anatomical encoder across CT and MRI while keeping the expensive part labels on the modality that actually has them.
Rebuild from arXiv:2609.13043. As of 16 September 2026 the abstract and PDF respond. Public code and weights are not linked in the preprint.
Sources
- Hong, Z., Pan, H., Aktas, H. E., Bejar, A., Keles, E., Miller, F. H., Wallace, M. B., Keswani, R. N., Durak, G., Bagci, U. Unified CT–MRI Pancreas Segmentation for Label-Efficient Cross-Modality Subregion Transfer. arXiv:2609.13043, posted 11 September 2026. https://arxiv.org/abs/2609.13043. PDF: https://arxiv.org/pdf/2609.13043.
- Isensee et al. nnU-Net (cited segmentation backbone).
- Cyst-X, AbdomenCT-1K, Peri-Pancreatic Edema, AMOS, BTCV, U-Mamba (cited datasets).
- Ganin et al. Domain-adversarial training of neural networks (cited DANN / GRL setup).