Garcia-Henao and colleagues at Balgrist University Hospital and ZHAW submitted arXiv:2608.00195 on 31 July 2026. The paper is MedSAM2-Anatomy: Training-Free Inference-Time Optimization for Musculoskeletal Segmentation. It is a routing policy over two frozen public checkpoints. Frozen TotalSegmentator writes a prior. That prior becomes N=3 equidistant slice prompts. Frozen MedSAM2 propagates each prompt in 3D. The three masks are union-fused (τ=1). A 15% contralateral-volume check can reject a sliver prior before it is prompted. No weights move. No human clicks.
On Balgrist-V0 hip MRI (12 FAI cases), cohort-average median Dice goes from 0.7105 for TotalSegmentator to 0.9242, and median HD95 from 22.04 mm to 4.99 mm. Shoulder CT (12 PSI cases) moves from 0.8851 to 0.9216. Left acetabulum is the ugly cell: 0.5096 to 0.9119. The public TotalSegmentator CT test set still belongs to the expert CNN (0.9865 versus 0.9566). Primary evidence is 24 institutional subjects. The paper points at github.com/BAL-DMU/medsam2-anatomy-inference-framework. That path 404’d when we checked. Do not plan a clone from it. Balgrist-V0 stays access-restricted.
Frozen TotalSegmentator as a prompt factory
The expert CNN runs once per scan and is used for localisation. Its surface is discarded. The z-extent of the prior sets three seed slices, equally spaced, with the two end slices skipped because those cross-sections are small and the prior is weakest there. Each seed yields a 2D mask plus a box. MedSAM2 (SAM2.1-Tiny, MedSAM2_latest.pt) then does bidirectional memory propagation from that slice. Cost scales linearly with N per anatomy.
Union (τ=1) keeps any voxel any pass claimed. Majority vote erodes thin bone: on the ablation set the acetabulum kept 0.865 of its reference volume under majority and 0.985 under union. Soft-mean and distance-weighted logit fusion both lost to the vote baseline. Hole filling after fusion hurt (p=0.003). Largest-component cleanup did nothing. The retained recipe is union plus the laterality guard: drop a paired prior if its volume is under 15% of the contralateral side. Observed slivers sat at 0.0003-0.047 of the partner; real pairs sit near 1. The guard fired on 3 of 30 ablation instances and left the other 27 untouched. On the one collapse case it took left acetabulum from 0.458 to 0.954 Dice and left femur from 0.221 to 0.944. It never fired on the public TotalSegmentator CT or MRI sets. Versus majority vote the selected arm is p=0.058 on median Dice. Overlaps are last-writer: later anatomies overwrite earlier ones, which is why a drifted contralateral mask has to be stopped at the prior, not after fusion.
They run on the native voxel grid. CT is windowed to [0, 255]. MRI is clipped at the 0.5-99.5 percentiles. PyTorch inference_mode, bfloat16. Cached priors plus pinned flags reproduced the 30 ablation Dice values exactly.
N=3 was the budget they kept
Policy selection used 15 Balgrist-V0 subjects (8 CT, 7 MRI, 30 paired instances), the full institutional set they had at that time. Public test sets were not used to pick N or τ. N=2 was skipped because an even budget can tie under majority vote. N=5 did not raise median Dice versus N=3 (Friedman χ²=3.20, p=0.20; pairwise p=0.14). It did take the shoulder CT cohort from 16.6 minutes to 21.6 minutes, about five extra minutes, on a laptop RTX 4090 16 GB (Intel Core Ultra 9 185H, 32 GB LPDDR5). N=3 is the number in the rest of the tables.
If you still mix the coefficient with the training loss, read the note on Dice versus Dice loss.
Balgrist-V0: 12 hip MRI, 12 shoulder CT
Balgrist-V0 is 24 planning scans with independent expert contours. Hip MRI is FAI work: acetabulum and femoral head, six left and six right, labrum also drawn. Shoulder CT is PSI work: scapula and humerus, 7 left and 5 right. TotalSegmentator’s “hip” (os coxae) label is mapped onto the institutional acetabulum sub-region so the four-structure label space lines up. The labrum is in the hip reference and in none of the three methods, so there is no labrum number.
Table 2 medians, same 272 paired instances across methods after QC:
- Hip MRI cohort average: TotalSegmentator 0.7105 / 22.04 mm, MedSAM2 0.8423 / 9.26 mm, MedSAM2-Anatomy 0.9242 / 4.99 mm. Median ASSD 3.84 mm to 1.04 mm.
- Acetabulum L: 0.5096 to 0.9119. Acetabulum R: 0.6930 to 0.8909. Femur L: 0.8248 to 0.9464. Femur R: 0.8145 to 0.9477.
- Shoulder CT cohort average: 0.8851 / 7.65 mm to 0.9216 / 6.07 mm. Median ASSD 1.20 mm to 0.89 mm.
- Scapula L: 0.8104 to 0.8859. Scapula R: 0.8252 to 0.8481 (MedSAM2 0.8498, a tie). Humerus L: 0.9548 to 0.9791. Humerus R: 0.9498 to 0.9735.
The authors treat DSC 0.85 as a conventional bone-overlap bar on the planning cohorts only. Both promptable methods clear it on all four Balgrist structures. TotalSegmentator clears it only on the humerus. Unguided MedSAM2 is lopsided on the femora (0.9156 left, 0.8351 right) and returns no mask on one MRI hip instance, scored as Dice 0. The fused prior-seeded runs put both femora above 0.94. Right scapula HD95 drops from 9.78 mm to 4.50 mm even though Dice barely moves.
Read the Balgrist rows as in-sample for the policy. Fusion and N were chosen on a 15-subject subset of the same institutional data, then applied unchanged to the full 24. That is the bound on the 0.71-to-0.92 headline.
Public TotalSegmentator tests still favor the expert CNN
Secondary eval is the public TotalSegmentator CT hold-out (62 subjects with the four MSK labels) and the MRI internal test (31 subjects). Those references were model-assisted then expert-refined on the same protocol the CNN was trained on. On that ground the frozen expert stays first: CT cohort-average Dice 0.9865 versus 0.9566 for MedSAM2-Anatomy (MedSAM2 alone 0.9485). MRI: 0.8534 versus 0.7389 (MedSAM2 0.7361). HD95 on the CT set sits on the 1.50 mm voxel-spacing floor for almost every cell, so they do not rank it. The MRI public structures are 8-12 instances even after pooling sides; the ranking is thin. The laterality guard never activated there. Union-plus-reject is built for a prior that localises and then draws a blocky surface. On the expert’s own test set those surfaces are the better estimate, and throwing them away costs Dice.
Instance QC started at about 360 candidates. Under 50% of the anatomy in FOV was dropped by hand, leaving 288. Sixteen more were held out: 14 because a prior-cache bug fed seven public shoulder CT scans another subject’s prior, and 2 because most methods returned empty masks. Scoring is on the remaining 272, paired. Empty predictions count as Dice 0. The cache bug is a limitation they report; those 14 are not charged against any method.
How this lands in a viewer
If you already hang AI in a DICOM viewer, treat this as a no-click batch refine on hip MRI and shoulder CT when the site protocol has walked away from TotalSegmentator’s training distribution. Keep the expert CNN as the first pass on in-distribution CT. RadYOLO is still the faster first-pass box. SAT3D is tumour plus uncertainty in Slicer. SLIP is click latency and undo. This paper is automatic prompts from a frozen organ prior, then three MedSAM2 tracks and a union.
Hardware in the paper is one laptop 4090. N=3 already multiplies foundation-model passes per anatomy. There is no measured inter-reader variability on Balgrist-V0, only a single contour and a 0.85 convention. Scope is hip and shoulder, CT and MRI. The labrum is unmeasured. Retrospective overlap is not a planning-measurement study and not a device claim. The preprint is the source of record until a venue version exists. Rebuild from the paper and the public TotalSegmentator and MedSAM2 checkpoints; do not wait on the 404 GitHub path.
Sources
- Garcia-Henao, J., Bünger, N., Herzog, B., Guerrero Toro, C., Vella, B., Biner, M., Brütsch, R., Castroviejo Fernandez, C., Öttl, F., Juchler, N., Hoch, A., Hochreiter, B., Hirsch, S., Caprara, S. MedSAM2-Anatomy: Training-Free Inference-Time Optimization for Musculoskeletal Segmentation. arXiv:2608.00195. Submitted 31 July 2026. https://arxiv.org/abs/2608.00195 (HTML: https://arxiv.org/html/2608.00195)
- Claimed implementation (404 when checked): https://github.com/BAL-DMU/medsam2-anatomy-inference-framework
- Ma, J., et al. MedSAM2: Segment Anything in 3D Medical Images and Videos. arXiv:2504.03600. https://arxiv.org/abs/2504.03600. Code: https://github.com/bowang-lab/MedSAM2
- Wasserthal, J., et al. TotalSegmentator: Robust segmentation of 104 anatomic structures in CT images. Radiol. Artif. Intell. 2023. DOI: 10.1148/ryai.230024