Anyone who has shipped automatic segmentation in a viewer knows the pattern. The liver and spleen come back clean, and then the adrenal gland is split into three islands, the esophagus has a gap in the middle, and the gallbladder looks chewed. Radiologists notice these masks first, and they are the ones that make a whole product look careless. Koushik Swarna, Pedro R. A. S. Bassi, Lam Ngo, Wenxuan Li, Jiancheng Yang, Alan Yuille and Zongwei Zhou (Johns Hopkins University, Aalto University, the ELLIS Institute Finland and Johns Hopkins Medicine) target those structures with DART, posted as arXiv:2610.27605 on 20 October 2026. DART stands for Diffusion for Anatomical Reshaping with Text. It is one 3D diffusion model that takes the probability map from any segmentation model, plus a short text description of the target structure, and returns a corrected mask. It never sees the CT or MRI.
Why a viewer or platform team should care
Most correction steps we see in production are hand-written rules: keep the largest connected component, fill holes, close small gaps. They are fast, and they treat a vessel the same way they treat a kidney. Learned shape models do better, but each one usually covers one organ or a fixed list. DART is a single post-processing model with a vocabulary of 377 anatomical classes, and because it only reads probabilities, it sits behind whatever engine you already run. In the paper that means TotalSegmentator, AtlasNet, WholeBodySeg, TotalSegmentator MRI and MedSAM. You keep your segmentation model and add one correction pass after it.
How it works
Training uses no images and no manual masks as diffusion input. The authors ran existing segmentation models on 3,066 CT, cone-beam CT and CT angiography volumes from eight public datasets, and kept the predicted probability maps. For each structure, they threshold the map at 0.5 only to find a bounding box, add a margin, crop the original probability values, and resize the crop to 128 x 128 x 128. A 3D U-Net then learns the usual diffusion task of removing Gaussian noise from these crops.
The text is how one model handles hundreds of classes. Each class has a description of its name, shape and location, for example “colon, long convoluted haustrated hollow tube, framing the peripheral abdomen from cecum to sigmoid”. A large language model drafted the descriptions from anatomy references and the authors reviewed them. A frozen sentence encoder (all-MiniLM-L6-v2) turns the description into a 384-dimensional vector, a small trainable projection maps it to the network’s conditioning width, and it is added to the timestep embedding in every residual block. Swapping the descriptions for plain class indices cut the average gain from 5.49 to 0.39 Dice points in their ablation.
Correction borrows the SDEdit trick. DART takes the cropped probability map from your model, adds noise up to an intermediate step t*, and denoises it back under the description. A small t* allows small edits and a large t* allows bigger reshaping. That step is the only DART-specific setting. It is picked once per class on validation data from 50 to 650 and then frozen, and the chosen values follow anatomy: solid organs get the least noise, glands and hollow organs more, vessels and muscles the most. The denoised crop is resampled to the native grid and thresholded at 0.5.
What it hangs on a scan
The output is an ordinary binary mask per structure, on the original voxel grid, so it drops into the same label map or DICOM SEG your viewer already renders. Fig. 2 of the paper shows the intended effect on a colon: a fragmented prediction from a segmentation tool goes in, and a continuous colon comes out. The authors report that on held-out WORD cases DART halves the number of connected components per mask, from 3.33 to 1.67. That count is a fair proxy for how many stray islands a reader would have to clean up by hand.
The paper also suggests a review flag that needs no extra model. The change between the input mask and the corrected mask, 1 minus their Dice, can be computed without any reference annotation. A large change means DART rewrote the structure heavily, which is a reasonable trigger to send that structure to a human.
What the evaluation shows
The authors test on 13 public benchmarks, ten of them external to training. In the controlled comparison, every correction method starts from the same MedSAM predictions on held-out TotalSegmentator cases, and DART had the highest Dice on 77 of 90 classes and was the only method whose class average never dropped below its input. The biggest gains landed on thin and small structures such as the autochthonous back muscles, spinal cord, right adrenal gland and esophagus. Other learned correctors in the comparison, SegRefiner and Post-DAE, made Dice worse on most classes.
For us the most useful result is the transfer from CT to MRI. No MRI volume was used in training, yet the same frozen model, with the same t* values, improved TotalSegmentator MRI predictions on all four MRI-only datasets, for example the prostate from 87.6 to 91.9 mean Dice. DART reads probability maps and never image intensities, so MRI contrast never enters its input.
On KiTS23, where the reference kidney includes the tumour, DART improved TotalSegmentator kidneys without erasing the tumour to make the kidney look typical. And when the authors trained fifteen segmentation model configurations on DART-corrected labels instead of the raw automatic labels, all fifteen improved. They used the same pipeline to produce DART-45K, annotations for 45,000 public CT scans, where readers checked 10,000 scans against per-structure guidelines and 98.5% passed.
Where it fails or has not been tested
DART only works inside the bounding box of the thresholded input plus a margin. If your model missed half the pancreas, DART cannot bring it back from outside the crop, and if the input mask is empty it returns an empty mask. The gain shrinks as the input gets better: in the appendix it ranges from 8.2 points on an input of 74.1 Dice to 0.4 points on inputs above 94. The portal vein, pulmonary vessels and colon gain less than one point in the controlled comparison, and thin branching vessels have the lowest ceiling in the scaling study. Because it learns from predictions, it can inherit the mistakes of the models that produced its training data. Shapes far from typical anatomy, beyond tumour-bearing kidneys, have not been tested, and the training datasets that report demographics skew male and oncologic. The ethics statement says DART is not validated for clinical decision-making.
The authors measure 0.59 s per probability map on one AMD MI250X GPU, so a scan with 50 structures adds about half a minute of GPU time on top of segmentation.
Code, weights and license
The paper says the code, model weights, every anatomical description and the per-class t* values will be released. The abstract points to a Hugging Face repository, Koushik45/DART-Seg, but when we checked on 24 October 2026 it was not publicly accessible, and we found no public DART code repository yet. DART-45K is released as an annotation layer for each source dataset under that dataset’s own terms, and the authors note that several of those terms are non-commercial or share-alike, so check each source before using the labels in a product. The preprint itself is under CC BY-NC-ND 4.0. Treat this as research code once it appears, not a cleared medical device.
How we would use it
We build DICOM viewers and segmentation pipelines, and we would try DART first in annotation work. Run TotalSegmentator or an in-house nnU-Net, pass the probability maps for the structures that are known to break (adrenals, esophagus, gallbladder, small glands) through DART, and load both versions into the annotation viewer with the 1 minus Dice change shown per structure. Annotators then start from cleaner masks and spend their time on the structures DART changed the most. For a clinical feature we would wait for the weights and license, then validate the correction on the site’s own scans before any mask reaches a report. For related reading, see our posts on ReSeg, abdominal organ masks trained on synthetic volumes, on scoring segmentation batches without labels, and on synthetic whole-body CT label maps.
Sources
- Swarna, K., Bassi, P. R. A. S., Ngo, L., Li, W., Yang, J., Yuille, A., Zhou, Z. DART: Scaling a Shape Representation Across Anatomical Classes and Imaging Modalities. Preprint, 2026. arXiv:2610.27605: https://arxiv.org/abs/2610.27605. PDF: https://arxiv.org/pdf/2610.27605.
- Model link given in the paper (not public when checked on 24 October 2026): https://huggingface.co/Koushik45/DART-Seg.
- TotalSegmentator: https://github.com/wasserth/TotalSegmentator. MedSAM: https://github.com/bowang-lab/MedSAM.
- Featured image: Fig. 1 (3D renderings of multi-class CT segmentations) and Fig. 2 (inference row: imperfect colon prediction to refined mask) of arXiv:2610.27605, real paper pixels, cropped and resized only. The preprint is licensed CC BY-NC-ND 4.0.