Hang a preoperative pelvic CT next to a live fluoro frame and you want a usable 2D/3D pose for image-guided work, not a twelve-hour patient-specific retrain before the case starts. XPos3R takes that live X-ray and the preop volume, regresses an intraoperative pose with one anatomy-specific pretrained model, then optionally spends a few seconds of test-time optimization. Shiyan Su, Ruyi Zha, Hongdong Li, Xuelian Cheng, and Zongyuan Ge (Monash University and The Australian National University) posted arXiv:2609.10733 on 9 September 2026 (eess.IV, ECCV 2026). No public code URL appears in the paper.
Viewer teams already sell the hang: preop CT on one viewport, live C-arm fluoro on another, and a pose that lets overlays and guidance land on the right anatomy. Prior regression pipelines often force a per-patient retrain to get that pose. This paper is about skipping that prep for pelvic fluoro while keeping an optional seconds-scale refine.
Why patient-specific retrain hurts
Live X-ray is fast and portable, but a single fluoro frame leaves depth ambiguous. Preoperative CT has the 3D anatomy, yet you still need the extrinsic pose that aligns that volume to the current C-arm view. Classical intensity methods render digitally reconstructed radiographs (DRRs) from the CT and optimize against the real X-ray. They are sensitive to a bad start pose.
Recent feedforward regressors predict an initial pose from the X-ray alone, then refine. DiffPose and related 2D-only models usually overfit that regressor to one patient anatomy. On the numbers in this paper, DiffPose needs about 12 hours of preoperative per-patient training from scratch. That clock is a non-starter for emergency pelvic trauma and many same-day image-guided cases. XVR can cut finetuning to minutes after a patient-agnostic pretrain, but it still adds a per-patient step. Until this line of work, the authors argue no regression model applied across patients without extra preparation.
What XPos3R actually does
XPos3R is a feedforward cross-modal transformer inspired by DUSt3R-style geometry models, adapted for heterogeneous inputs: one 2D X-ray and one 3D CT. The design is asymmetric and image-centric.
The image branch uses a pretrained RayDINO vision transformer on the X-ray. The volume branch is a four-stage 3D CNN trained from scratch so it learns dense geometry rather than sparse organ tags. A unidirectional cross-attention decoder lets image tokens attend into volume tokens as key-value memory (not the reverse), then a regression head predicts rotation and translation. Inputs are resized to 224×224 for the X-ray and 96×96×96 for the CT.
Training is anatomy-specific at pelvic scale. The authors take 441 CTPelvic1K volumes (20 held out for validation), preregister each to a canonical bone template, sample 5,000 C-arm poses per volume, and render DRRs with DiffDRR plus contrast augmentation. That yields about 2.3 million volume-DRR-pose triplets. One pretrained checkpoint is meant to serve new patients in that anatomy without another overnight retrain.
Optional test-time optimization (TTO) starts from the feedforward pose and refines with DiffDRR using multiscale and gradient normalized cross-correlation (NCC) against the real X-ray. The paper reports that refine finishes in seconds on their eval GPU.
Where it was measured
Evaluation uses DeepFluoro: six cadaver subjects, each with one CT and 24 to 111 Siemens CIOS Fusion X-rays, calibrated intrinsics, and fiducial landmarks. Metrics in the paper are support for the clinic claim, not the headline.
In the feedforward setting (no TTO), a single pretrained XPos3R beats DiffPose and patient-finetuned XVR on 3D fiducial accuracy (TRE about 24 mm mean with lower variance than the patient-specific baselines). With feedforward plus DiffDRR TTO, XPos3R reaches about 0.24° rotation error, about 3.3 mm TRE, and about 0.45 mm projection error, with intraoperative time under 3 seconds per X-ray on their H100 setup. DiffPose still needs the 12-hour patient retrain to get into that TTO ballpark, and shows larger variance. Pure DiffDRR from a default PA start is far worse without a strong init.
Limits are explicit. Results are pelvic / DeepFluoro. Feedforward alone is not millimeter-accurate; TTO is what brings TRE near 3 mm. Inference is slower than tiny 2D CNN regressors because the model reads a 3D volume. Fail closed if your anatomy is outside the pretrained region, if you cannot run differentiable rendering for TTO, or if the site forbids any intraoperative optimize step.
How this lands in a viewer
If you already hang preop CT and live fluoro in a DICOM viewer, treat XPos3R as a pose service on that hang, not a new worklist. Load the pelvic CT, ingest the current fluoro frame, run the anatomy-specific checkpoint to get an initial extrinsic, optionally run DiffDRR NCC refine for a few seconds, then rehang overlays, DRR compare, or guidance layers with that pose. Keep one pelvic checkpoint per deployment. Budget roughly 1.5 seconds for feedforward and under 3 seconds when TTO is on, on hardware in the paper’s class.
SCoPE-Reg solves a related six-DoF pose from untracked 2D ultrasound onto a 3D volume. XPos3R is the fluoro-plus-CT cousin for intraoperative pelvic registration without patient-specific retrain.
Rebuild from arXiv:2609.10733. As of 11 September 2026 the abstract and PDF respond (HTTP 200). The paper does not list a public code URL yet.
Sources
- Shiyan Su, Ruyi Zha, Hongdong Li, Xuelian Cheng, Zongyuan Ge. XPos3R: Cross-Modal Transformer for Intraoperative 2D/3D Registration. arXiv:2609.10733, posted 9 September 2026. https://arxiv.org/abs/2609.10733 (HTTP 200 on 11 September 2026). PDF: https://arxiv.org/pdf/2609.10733 (HTTP 200 on 11 September 2026). Code: not listed in the paper.