If you hang a follow-up chest CT in a DICOM viewer, the useful layout is already two viewports. Prior on one side, current on the other. The line the referring doctor needs is what changed in the interval: a new nodule, a smaller effusion, a resolved consolidation, a stable node.
That is the job CT-ΔBench scores. Tang, Wang, Ren, and Wang (University of Tennessee at Chattanooga) posted it on 12 August 2026 as arXiv:2608.11534. arXiv comments list it as accepted at COLM 2026. The Hugging Face package is JSONL manifests plus download and eval scripts. The volumes stay in CT-RATE.
The input is a pair (It1, It2) from the same patient. The output is a difference report RΔ with Findings and Impression written around interval change. After full supervised fine-tuning, the authors’ own DeltaMed sits at Change-F1 0.1980, Missing Rate 0.8301, Hallucination Rate 0.8057. The zero-shot medical VLMs they tried sit near zero on those event metrics.
The pair is the sample
They start from CT-RATE, keep patients who have more than one CT, and form one earlier / follow-up pair per patient. The split is patient-level: 2,638 training pairs and 169 validation pairs, each pair from a different patient, so a volume cannot leak across the cut.
CT-RATE already has a report at each time point. It does not have a longitudinal difference report. The authors keep only Findings and Impression from the prior and follow-up reports, drop ClinicalInformation and Technique, and ask Gemini-2.5-Flash to write a change-focused Findings_EN and Impressions_EN. The prompt tells the model not to invent details, and to phrase a finding that appears in only one report as mentioned or not mentioned rather than as true clinical absence.
The training target is report-to-report synthesis. The model you evaluate still has to look at the two volumes. A finding that a radiologist wrote once and omitted the next time becomes a change event even if the pixels did not move.
Each released row is five fields: patient_id, VolumeName_A, VolumeName_B, Findings_EN, Impressions_EN. The download script pulls the matching nii.gz files from CT-RATE. Use of the pair has to follow the original CT-RATE license and access terms. The Hugging Face viewer currently fails to preview because the event JSONL schema does not match the pair schema. The files themselves are there: train.jsonl, validation.jsonl, and validation_change_events_qwen14b.jsonl.
They score events, then text
ROUGE-L, BERTScore, and BLEURT are in the tables. Two difference reports can share wording and still miss the new node, or invent a resolved effusion. The authors extract atomic change events from both the prediction and the Gemini reference with Qwen2.5-14B-Instruct, then match those events with canonicalized text, laterality and anatomy constraints, and a token-F1 gate of 0.5.
Each event is a pair (type, text). The five types are NEW, RESOLVED, INCREASED, DECREASED, and STABLE. From the matched sets they report:
- Change-F1: agreement between predicted and reference events
- Missing Rate: reference events the model omitted
- Hallucination Rate: predicted events with no reference match
- Change Type Accuracy: among matched events, how often the type agrees
Read Change Type Accuracy with the match count. Merlin-RadLLaMA-7B posts type accuracy 1.0000 in zero-shot with Change-F1 0.0034. Almost nothing matched, and the few that did had the right verb.
The validation event file is a Qwen silver reference, 169 rows aligned by patient_id. The authors say so on the dataset card. Treat it as a scoring layer. PatchChestCT is the spatial layer on single CT-RATE studies. This release has no boxes or patches to hang on either volume.
Two physicians read the reports
Because both the Gemini references and the Qwen events are model-written, the authors sampled 50 validation cases and asked two physicians from different hospitals to score them independently. The packet was the prior report, the follow-up report, the synthesized difference report, and the extracted events. Nobody in that review sat down with the two CTs.
On the synthesized reports: overall acceptability 4.82/5, correctness 4.83/5, completeness 4.84/5. Across 100 physician evaluations, 99 were judged clinically acceptable. Severe hallucination 0/100. Severe omission 0/100. On the events: overall correctness 4.83/5, erroneous events 3/100, missed important events 3/100. Positive-rating agreement (both 4 or 5) was 195/200 on the Likert items and 243/250 on the binary calls.
That stays a report-to-report check. If you ship a compare tool, a reader still has to look at both series before you treat a number from this benchmark as a clinical claim. The paper’s ethics note says the same: intended for controlled research evaluation. Clinical use would need a larger prospective read of the references and the model outputs, on the images.
Zero-shot paired prompting
Five medical VLMs get both volumes and a single instruction: write a difference report about interval changes only. No in-context examples. Change-F1 on the validation set:
- MedGemma-1.5-4B: 0.0175 (Missing Rate 0.9849, Hallucination Rate 0.9791)
- Med3DVLM-Qwen2.5-7B: 0.0138
- M3D-LaMed-Phi-3-4B: 0.0051
- Merlin-RadLLaMA-7B: 0.0034
- RadFM-13B: 0.0000
Merlin is a single-study CT VLM that fails this paired task. It also wins zero-shot BERTScore at 0.8059, next to that Change-F1 of 0.0034. Med3DVLM wins ROUGE-L (0.0980) and BLEURT (0.3822) the same way.
RADAR is draft review against one study.
Writing two reports, then a diff
The two-stage setup is closer to how a lot of stacks will try this first. Each model writes a report for It1 and a report for It2, then the language side is asked to produce RΔ from those two texts. If the first-stage report drops a finding, the interval change is gone. If it invents one, the differencer can promote it into a fake NEW or RESOLVED.
Med3DVLM-Qwen2.5-7B is the best of this group, Change-F1 0.0614, Missing Rate 0.9500, Hallucination Rate 0.9204. RadFM-13B moves from 0.0000 to 0.0542. M3D-LaMed-Phi-3-4B goes to 0.0212. MedGemma gets worse (0.0078). Merlin goes to 0.0000.
Textual differencing helps the models that already write a usable single-study report. RadYOLO and AMPLIFAI are still single-visit tools (a box on one volume, or feature masks on one multiphase exam). A follow-up hang needs a second series in the protocol before any of those overlays mean interval change.
What DeltaMed trains
DeltaMed encodes prior and follow-up with a shared MedSigLIP encoder, builds zt2 - zt1, concatenates that with zt1 and zt2, runs a linear projection plus normalization, and decodes with Gemma 3 4B. The encoder, the original multimodal projector, and the base Gemma weights stay frozen. Training updates the fusion layer and LoRA adapters on the language model. Loss is next-token negative log-likelihood on the difference report. Hardware in the paper is two 80 GB A100s.
They fine-tune DeltaMed and a direct paired-CT MedGemma-1.5-4B baseline with LoRA at 1%, 10%, and 100% of the training set. Change-F1:
- 1%: DeltaMed 0.0909 vs MedGemma 0.0010
- 10%: 0.1313 vs 0.0649
- 100%: 0.1980 vs 0.1577
On the full split, DeltaMed’s Missing Rate is 0.8301 against MedGemma’s 0.8856. MedGemma has the lower Hallucination Rate (0.7462 vs 0.8057). Change Type Accuracy is 0.4902 for DeltaMed and 0.4671 for MedGemma, on a much larger matched set than the zero-shot rows. ROUGE-L favors DeltaMed in every regime. BERTScore and BLEURT favor MedGemma once more data is available, and those gains do not show up in Change-F1.
0.1980 is the best event number in the paper. Most reference change events are still missing, and most predicted events do not match the reference. A compare viewport that auto-wrote “new right-lower-lobe opacity” from this checkpoint would be a draft you audit.
What you can hang
The useful object in a viewer is the pair, plus a structured change list. Load Volume A and Volume B in a two-viewport protocol. Keep spacing honest when you convert the CT-RATE NIfTI back to a series; a drifted prior is a fake interval change. Then render the Qwen events as a worklist next to the hang: NEW / RESOLVED / INCREASED / DECREASED / STABLE, each with the short finding text. Clicking an event cannot jump to a box, because this set has no spatial label. It can scroll you to the series and put the sentence on screen.
That is enough to QA the benchmark before you train. Open five validation pairs. Read Findings_EN against the two original CT-RATE reports. If Gemini called something NEW because the second report mentioned it and the first did not, mark that row. You will see the mentioned-versus-absent rule in the prompt show up as noise in the event list.
If you later train a localizer on PatchChestCT or a detector such as RadYOLO, you can try to ground those event strings on each time point. CT-ΔBench will not do that join for you.
Limits you should design around
The references are synthesized from reports, then scored by another LLM. The physician check never opened the images. The val set is 169 pairs. The volumes inherit CT-RATE’s hospital, language, and acquisition mix. There is no public test split in this release. The authors originally used train plus validation, and the released names match that.
DeltaMed subtracts in feature space. It does not register the volumes or output a lesion track. If your product already does rigid or deformable compare, this paper does not replace that step. It asks whether the language side can name the interval change after the two volumes are in front of it.
What to do next
Request CT-RATE, accept its terms, then pull train.jsonl, validation.jsonl, and validation_change_events_qwen14b.jsonl from tangkg/CT-DeltaBench. Download a handful of pairs with download_ct_rate_from_jsonl.py, hang A and B side by side, and read the Gemini Findings against the two source reports before you attach a loss.
When you score a model, use evaluate_change_events.py against the released silver events. Report Change-F1 next to Missing Rate and Hallucination Rate, on a pair you can actually open.
Sources
- Tang, K., Wang, J., Ren, S., Wang, Z. CT-ΔBench: A Benchmark for Longitudinal 3D Medical Imaging Difference Reporting with Vision-Language Models. arXiv:2608.11534 (submitted 12 August 2026; accepted COLM 2026). https://arxiv.org/abs/2608.11534 (HTML: https://arxiv.org/html/2608.11534v1)
- Dataset and scripts: https://huggingface.co/datasets/tangkg/CT-DeltaBench
- COLM 2026 poster: https://colm.eventhosts.cc/virtual/2026/poster/2296
- CT-RATE volumes (gated): https://huggingface.co/datasets/ibrahimhamamci/CT-RATE