In adaptive radiotherapy, every new CT taken during the course raises the question of whether the planning contours still fit. The usual answer is to register the planning CT to the new scan, warp the contours across, and have someone fix what the warp got wrong. A group led by Wei Liu at Mayo Clinic Radiation Oncology, with ShanghaiTech University, MD Anderson and the New York Proton Center, posted arXiv:2610.18926 on 13 October 2026 with CAP-DIR, a deformable registration model that does not look at the two CTs alone. It also reads what the treatment team already knows: the planning contours, the dose plan, a structured summary of the case, and how this patient’s anatomy moved at earlier fractions.
Why this matters for a viewer or a clinical platform
Generic registration networks optimise image similarity over the whole body. Radiotherapy cares mostly about the target and the organs near the high dose. A field that lines up the skin and bones well but drifts at the parotid or the rectum still produces contours that a dosimetrist has to redraw. In the paper, propagated organ-at-risk contours reached a Dice of 92.07% with CAP-DIR against 82.28% with CorrMLP, the strongest learning-based baseline. The blinded review is more useful for judging practice. Two radiation oncology reviewers scored 20 cases inside ARIA, the clinic’s own treatment planning environment, and rated target contours clinically acceptable (minor edits at most) in 82.5% of reads with CAP-DIR against 45.0% with CorrMLP. For contour review or replanning tools, that difference decides how much redrawing lands back on the dosimetrist.
What goes in and what comes out
The input is a pair of CTs from the same patient: the planning CT (or an earlier treatment CT) as the source, and a later treatment or verification CT as the target. On the source side the model also takes the organ and target contours, the planned dose distribution and a structured text prompt covering site, diagnosis, plan and prescription. If the patient already has earlier registrations in the course, it takes a compact summary of those too. Nothing from the target side is needed except the image, so the target contours never enter the model. The output is a dense displacement field. Contours and dose are then warped with it, which gives you propagated structures to review and a mapped dose that can feed dose accumulation. Scans are resampled to 1 mm isotropic and cropped or padded to 512 x 512 x 256 voxels, and the paper only covers CT to CT.
How it works
The backbone is a familiar design. A shared 3D convolutional encoder reads both CTs at five scales, and a Swin transformer decoder predicts the displacement from coarse to fine, adding a correction at each level. What changes is how the clinical information is injected. In the first stage, contours are turned into signed distance maps and proximity bands, and the dose is turned into isodose bands and gradient maps. These act as a spatial gate on the features, so regions near organs at risk and steep dose falloff get more attention. The text prompt goes through a frozen BioMedBERT-large encoder, and the resulting embedding scales and shifts the decoder features (FiLM conditioning). Training uses local normalised cross-correlation over the body plus an extra term restricted to organ regions, with a smoothness penalty on the field.
The second stage adds memory. For each earlier registration of the same patient, the stage-one model stores a 128-value image feature, four statistics on the deformation (mean and 95th-percentile displacement, fraction of folded voxels, spread of the Jacobian) and five quality scores. A small recurrent network (a GRU) reads this sequence in order and produces a patient embedding that modulates the decoder again. The effect is that a patient whose bladder filling or weight has been shifting in a consistent direction gets a registration informed by that trend, without any per-patient fine-tuning at inference. When there is no history yet, the model falls back to stage-one behaviour.
What the evaluation shows
The data set is large for registration work, with 3,473 patients, 5,005 plans and 16,361 CT exams across seven anatomical sites, giving 11,986 registration pairs. Training and internal testing used Mayo Clinic Arizona and Minnesota, and external testing used MD Anderson, the New York Proton Center and five public longitudinal CT collections (LongCT-Lung, NLST, Pancreas-CT, 4D-Lung and RIDER). The comparison set covers VoxelMorph, TransMorph, CorrMLP, MambaMorph, DiffuseMorph, uniGradICON and classic Demons. The ablations separate the parts. Text alone improved the organ-region match, spatial context from contours and dose improved it more, and patient history added gains at every history length the authors tested. The deformation fields also stayed well behaved, with fewer folded voxels in organ regions than CorrMLP. A controlled dose benchmark with known synthetic deformations showed lower error in accumulated dose as well. In the featured figure, taken from Fig. 3e of the paper, the green contour is what you get by copying the planning contours across without registration, the blue one is CAP-DIR, and the red one is the clinician’s reference on the new scan.
Where it breaks
Everything was developed at two Mayo centres, so the clinical text template, contouring conventions and dose plan formats reflect one institution. The external sites help, but for the public collections the clinical inputs were partly reconstructed from TotalSegmentator organ masks, proxy dose maps and a standardised text template, which is not the same as real plan data. Only CT to CT is covered. Cone-beam CT, which most daily adaptive workflows acquire, and MR are not tested. The memory module feeds on earlier registrations, so a bad early result can carry forward, and the authors say they have not evaluated how errors propagate through the history. The dose result comes from a synthetic benchmark, not prospective delivered-dose accumulation. The reader study is 20 cases with two reviewers, and editing time was estimated by the reviewers rather than measured. The paper does not report inference time or GPU memory for a full 512 x 512 x 256 volume, which matters for an online adaptive slot.
Code, weights and license
As of 22 October 2026 there is no public repository. The paper states that preprocessing, training, baseline and evaluation code will be released on GitHub upon publication, without a license named yet. The trained weights will not be released, because they come from institutional data under Mayo Clinic data governance, and the institutional cohorts are not public either. The public validation sets are available from their original sources under their own terms. In practice that means you can expect to retrain on your own planning data once the code appears, and training needs contours, dose and plan records for every pair, on top of the CTs. The text encoder is BioMedBERT-large, which is publicly available.
How we would use it
We build DICOM viewers and clinical imaging platforms, and we would put this at the contour review step in an adaptive workflow. The inputs already exist in standard objects: the planning CT, the RT Structure Set, the RT Dose and the plan. A service would pull those with the new CT, run the registration, and return propagated contours as a new RT Structure Set plus a registration object, so the dosimetrist reviews them side by side with the reference in the viewer and edits only what needs it. The history feature fits naturally in a platform that already stores each fraction’s registration, since keeping a few summary numbers per fraction is cheap. Until there is code and a CBCT evaluation, we would treat the idea, conditioning registration on the plan, as the part worth borrowing now. Weighting organ regions in the loss and feeding contour and dose maps to an existing registration network is something you can test on your own data today. For related reading, see our posts on cleaning up radiotherapy CBCT without paired scans, on uncertainty-guided RTSTRUCT output on thoracic planning CT, and on what a DICOM viewer does.
Sources
- Jiang, C., Zhao, S., Ding, Y., Mi, J., Patel, S. H., et al., Shen, D., Liu, W. Personalising Deformable Image Registration with Clinical Context and Deformation History for Adaptive Radiotherapy. arXiv:2610.18926, 2026. https://arxiv.org/abs/2610.18926. PDF: https://arxiv.org/pdf/2610.18926.
- Featured image: Fig. 3e of arXiv:2610.18926 (head and neck, abdomen and pelvis columns, OAR and CTV rows; reference, no registration and CAP-DIR contours drawn by the authors), real paper pixels, cropped and resized only.