AgentCT: A CT Agent That Picks the Right Restoration Model

Fig. 2 of arXiv:2611.11503 by Chen, Gao, Niu, Liu, Zhang and Shan (Fudan University and co-authors): abdominal CT slices with simulated low-dose noise, sparse-view streaks and metal artifacts, comparing the degraded input, the all-in-one AMIR model, the AgentCT routed specialist and the clean reference, with zoomed regions and PSNR/SSIM.

Most CT restoration networks are trained for one kind of damage: low-dose noise, sparse-view streaks or metal artifacts. The paper’s starting point is that someone has to look at the series and pick the matching model, and that all-in-one networks that skip this step use one image-domain architecture for three problems with different physics. AgentCT, from Fudan University with co-authors at Harbin Institute of Technology, Nanchang University and Sichuan University, hands that choice to a small vision-language model. It is described in arXiv:2611.11503 (8 November 2026).

What AgentCT does

AgentCT is a router with three specialists behind it. A controller looks at a CT slice, decides whether it shows low-dose noise (LDCT), sparse-view streaks (SVCT), metal artifacts (MAR) or nothing at all, grades the severity as mild, moderate or severe, and calls the matching restoration network. The restored slice goes back to the same controller, which checks it again. If it still sees damage it may make one more call; if the input was clean to begin with it returns it untouched.

The authors call this a perceive, route, restore, verify loop. Only the controller is a language model. The restoration is done by task-specific networks, and the paper argues they should differ per degradation because the physics differs.

How it works

The controller is Qwen3-VL-4B with LoRA adapters (rank 16) in its attention and feed-forward layers. Each slice is shown to it in a fixed window of -350 to 350 HU, chosen to make noise and streaks visible, together with an instruction. It answers with one sentence describing what it sees and a JSON action with three fields: degradation, severity and tool. The tool is one of denoise_net, svct_net, mar_net or none. The authors simulated every degradation, so the labels for training come with the data.

The three specialists are built differently:

  • Low dose. CoreDiff, a diffusion denoiser from the same group, warm-started from its public checkpoint and fine-tuned on three dose levels. It works on the image only. CoreDiff expects three neighbouring slices, so the authors copy the single slice into all three channels.
  • Sparse view. A dual-domain network in the DuDoNet and DuDoTrans line. The sparse sinogram is zero-filled onto a 720-view grid, a sinogram network fills the gaps, the views that were really measured are written back unchanged (data consistency), a differentiable filtered back-projection makes an image, and an image network refines it.
  • Metal. Also dual domain. The degraded image is re-projected, the metal mask is projected to find the metal trace, and a network replaces only the sinogram values inside that trace before back-projection and image refinement. At inference the metal mask is estimated by thresholding high HU values and a little morphology.

Because the sparse-view and metal specialists work in the sinogram, they can keep the result consistent with the measured projections, which an image-only all-in-one network cannot do.

What they tested it on

All experiments use the public Mayo 2020 low-dose CT and projection data from The Cancer Imaging Archive, abdominal scans only, split by patient. The authors simulated nine settings from the full-dose images: 1/2, 1/4 and 1/10 dose; 120, 60 and 30 views out of 720 in a parallel-beam geometry; and titanium, iron and gold implants as mild, moderate and severe metal, simulated with a polychromatic 120 kVp model and 100 metal masks placed in the body. That gives about 15,600 training pairs and 4,740 test pairs.

On 2,000 test images the controller named the degradation type correctly 95.7% of the time and the severity 93.9% of the time, and every answer parsed as valid JSON. Most mistakes were between mild low dose and clean images.

The restoration comparison is against AMIR, an all-in-one medical restoration model retrained on the same data, and TAMP, a CT foundation model adapted with one LoRA per degradation. Each method was given the correct degradation, so routing errors do not enter these numbers. AgentCT had the best PSNR in all nine settings, and the gap was largest where the sinogram matters. On 30-view sparse CT its error was 26.0 HU RMSE against 34.6 to 36.0 HU for the two image-domain baselines. On low dose, TAMP was close.

The featured image is Fig. 2 of the paper: one slice per degradation, showing the degraded input, AMIR, AgentCT and the clean reference, with the authors’ zoomed regions and per-slice PSNR and SSIM. On these slices the authors print 42.28 dB PSNR for AMIR and 44.34 dB for AgentCT in the metal row, and their error maps (Fig. 3) show the image-domain baselines leaving most residual error along the streaks and in the tissue next to the metal. An ablation checks whether the gain comes from the architecture or from training separate models: AMIR trained on sparse view alone did better than the all-in-one AMIR, but the 4.2 million parameter dual-domain specialist beat both, against 23.5 million parameters for AMIR.

Why a viewer or clinic team should care

A CT post-processing service with a denoiser, a metal artifact tool and a sparse-view model has the same routing problem, and metal or streaks are not something you can read from a header field. A small controller that looks at the pixels, writes a one-line reason and emits a JSON action is easy to plug into a queue: log the action, run the tool, show the result as a derived series next to the original, and let the controller decline clean studies. The written observation, for example “radial streaks from undersampling” in the paper’s Fig. 5, gives a reviewer something to check at a glance.

The design is modular. To swap the low-dose specialist for a better one, or add a new tool, you add a routing label and retrain the controller’s adapter instead of retraining one universal network.

The authors also report that the dual-domain specialists keep CT numbers closer to the reference: in their line profiles the image-domain baselines undershoot soft tissue by roughly 25 HU for sparse view and 100 HU for metal. The paper names attenuation-based tissue characterization and radiotherapy dose calculation as the uses that depend on this.

What it does not tell you yet

  • The restoration numbers use ground-truth routing. The controller and the specialists were evaluated separately, and the full loop with automatic routing has no reported metrics. The authors say so themselves.
  • All degradations are simulated from one public abdominal dataset, one degradation per image. Real scans can have low dose and metal at the same time, and the paper leaves multi-step tool planning to future work.
  • The sparse-view and metal specialists use a 2D parallel-beam geometry. Clinical scanners are fan or cone beam and helical, and none of it has been run on raw clinical projection data from different scanners or sites.
  • The specialists see single 2D slices, and the low-dose model gets one slice copied three times instead of real neighbours.
  • At inference the metal mask comes from an HU threshold plus light morphology, and the paper does not evaluate how often that mask is wrong.
  • Metrics are PSNR, SSIM and HU RMSE, with no significance tests, no reader study and no downstream task.

How to try it

The paper says code is at github.com/hao1635/AgentCT and that hyperparameters and trained weights are released with it. On 15 November 2026 that repository returned a 404, so code, weights and licence are announced but not yet available. The building blocks are public: CoreDiff has its own released checkpoint, the Mayo 2020 data are on TCIA under its data use terms, and Qwen3-VL-4B has open weights. Until the repo appears, the paper gives enough detail on the prompt format, LoRA settings and specialist designs to rebuild the controller on your own degradation labels. For related work on the blog, see a low-dose CT denoiser trained without paired normal-dose scans, EUReCA for sparse-view linac CBCT and Dx-Rx, an MRI agent that measures artifacts before repairing them.

Source and license

Paper: Zhihao Chen, Qi Gao, Chuang Niu, Qiegen Liu, Yi Zhang and Hongming Shan, “AgentCT: A Vision-Language Agent for Universal CT Imaging”, arXiv:2611.11503 (cs.CV, 8 November 2026), preprint. Affiliations: Institute of Science and Technology for Brain-inspired Intelligence, Fudan University (corresponding author Hongming Shan); Harbin Institute of Technology; Nanchang University; Sichuan University. Code and weights: announced at github.com/hao1635/AgentCT, not reachable on 15 November 2026, licence not stated. Data: Mayo 2020 low-dose CT and projection data (TCIA), with simulated degradations. Featured image: Fig. 2 of the paper, four of the five columns, cropped and scaled.

We build custom medical imaging platforms — advanced DICOM viewers, AI segmentation, and the clinical systems around them.

Copyright © 2026 PYCAD. All Rights Reserved.