A CT segmentation model usually does well on the typical patient and worse on the cases its training set barely has, such as a small pancreatic tumor, a very large liver or a patient with a BMI of 40. Can Zhao, Pengfei Guo, Daguang Xu and colleagues at NVIDIA, with Benjamin Simon, Stephanie Harmon and Baris Turkbey from the National Institutes of Health and Mason Belue from the University of Arkansas for Medical Sciences, posted arXiv:2610.24153 on 16 October 2026 with a model for producing those cases on request. NV-Generate-Body (the paper also calls it NV-Generate-BodyMask) is a 3D latent diffusion model that generates whole-body CT label maps, the segmentation masks without a CT image, and lets you set organ sizes, tumor burden and patient demographics before it draws them. The code is on GitHub under Apache 2.0 and the weights are on Hugging Face.
Why a segmentation or viewer team should care
Labeled CT is the expensive part of every segmentation product. A mask generator you can steer gives you a way to ask for exactly the cases you are short of, then hand those masks to a CT image synthesizer to get image and label pairs. NVIDIA’s NV-Generate-CTMR repository already contains that second step, a ControlNet that paints a CT volume from a label map, so the two stages run from one codebase. For a team that builds viewers and clinical tools, the same masks also make handy test fixtures. Synthetic patients with a known liver volume or a known bone lesion let you check volumetry, overlay rendering and label lookups without touching patient data.
What goes in and what comes out
The input is a 19-number conditioning vector. Nine slots hold size quantiles for liver, spleen, stomach, pancreas, right kidney, left kidney, lung, gallbladder and the whole body, each mapped to 0 to 1 against the training population. Five slots hold tumor burden quantiles for lung tumors, pancreatic tumors, hepatic tumors, colon cancer primaries and bone lesions. The last five hold age, sex, weight, BMI and height. Any slot can be set to -1 for unknown or absent, so you only specify what you care about. The output is a 256 x 256 x 256 integer label map. In the current repository it comes out as a NIfTI file in the MAISI 132-label vocabulary, with label 200 marking the body envelope, and the command-line tool also generates the paired CT image in the same run. One mask takes about 4 seconds on a single H100.
How it works
A frozen 3D autoencoder compresses each 256-cube label map into a 64-cube latent with four channels. The generator is a 3D U-Net that works in that latent space. The conditioning vector enters every normalization layer through Adaptive Group Normalization, which shifts and scales the features according to the requested sizes and demographics, and through cross-attention at the two coarsest levels. A voxel-spacing embedding lets the same model generate at different isotropic spacings. Sampling uses rectified flow with 100 steps and classifier-free guidance at a scale of 2.0.
Most of the training work went into the tumors. Annotated tumors are scarce, and the tumor library behind the model holds only 63 lung tumor crops, 281 pancreatic, 615 hepatic, 126 colon and 112 bone lesion crops. During training, half of all samples get a tumor grafted into the right organ, preferring crops from the same patient so the anatomy stays consistent, and the graft’s volume is written into the matching tumor slot as a quantile. A plain diffusion loss barely notices a small tumor inside a large organ, so the authors add a size-adaptive loss that weights each tumor and some small organs more heavily, with a weight of 150 for lung tumors. A three-ring halo around each region penalizes voxels placed just outside the boundary, which otherwise leads to tumors that grow too large.
The body envelope label is computed from the original CT, and naive thresholding pulls the scanner table into the body. The authors remove the exterior air, the table and any air-density pockets left by table foam or side rails before labeling the body. In their Fig. 1 example, three table components made up 41% of the first body mask.
What the evaluation shows
Training used 8,571 CT masks from 26 public and institutional cohorts, led by NLST with 2,447 cases, TCIA Colon with 1,141 and StonyBrook-CT with 1,103, all segmented into the VISTA 132-label space. The test is a sweep. For each organ or tumor type, the authors request sizes from the 10th to the 90th percentile, generate 50 masks per setting, and check whether the generated volumes follow the request in rank order. For large organs the control is good, with a Spearman correlation of 0.92 for lung and 0.80 for spleen. For small or deformable organs it is weaker, 0.47 for pancreas and 0.49 for gallbladder, and the abstract’s 0.70 to 0.92 range leaves those two out. With sex set to male, every generated mask had a prostate, and with sex set to female, none did.
The figure
The featured image takes two panels from Fig. 4 and two from Fig. 5 of the paper. The top row shows the liver requested at the 20th and the 80th percentile with the same random seed, so the only change is the size request, and the paper outlines the liver in white. The bottom row shows the BMI slot set to 18 and to 40. Each panel shows a coronal, sagittal and axial view, and colors are organ labels. No CT image is shown, only label maps, and the authors picked the cases.
Limits
The paper measures whether the masks follow the request, and that is all it measures. There is no experiment showing that a segmentation model trained on these masks, or on CT images synthesized from them, performs better on real patients. There is no reader study of realism, and the only baseline is NVIDIA’s own earlier mask model. Hepatic tumors are the weak spot. At the 10th percentile only 42% of generated masks contained a tumor above 1 mL. Colon cancers come out too large at small requests, about 14 mL generated against 6 mL in the reference at the 10th percentile. Demographic labels existed for only part of the training data, from 7.8% of cases for weight to 23.2% for sex. Where age was recorded it ran from 19 to 87 with a mean of 64, so there are no children in that subset. The training crops span thoracic, abdominal and pelvic anatomy, which leaves head and neck out. The largest cohort is NLST, a lung screening trial, which pulls the population toward older adults.
Code, weights and license
The code is in NVIDIA-Medtech/NV-Generate-CTMR under Apache 2.0, where the mask model is the rflow-mask variant. The weights are in the nvidia/NV-Generate-CT Hugging Face repository, and the first run downloads about 10 GB of weights and data. The README and the model card list the CT weights under the NVIDIA Open Model License, which NVIDIA describes as commercially usable. The repository root also carries a LICENSE.weights file with an NVIDIA license limited to non-commercial use, so have someone read both before you put generated data into a product. Some of the 26 training cohorts are institutional, so you cannot rebuild the training set yourself. NVIDIA also runs a live demo on Hugging Face.
How we would use it
We would treat it as a data tool, outside the viewer. Start from your own real data and find the organ size quantiles, tumor types and BMI range where your segmentation model does worst. Generate masks for those settings, map the MAISI labels to your own label schema, synthesize CT with the paired ControlNet, and mix the result into training at a modest ratio. Validate only on real held-out scans, and tag every synthetic case so it can be pulled back out if it hurts. For viewer and platform work, synthetic cases make good fixtures for testing DICOM SEG or RT-STRUCT export, overlay colors and volume measurements, as long as they are clearly marked synthetic in the patient name and series description so they never mix with clinical studies. For background, see what a DICOM viewer does, our post on training a microbleed detector on synthetic lesions, and our older guide to generating synthetic medical images with medigan.
Sources
- Zhao, C., Guo, P., Yang, D., He, Y., Tang, Y., Harouni, A., Simon, B., Belue, M., Harmon, S., Turkbey, B., Xu, D. NV-Generate-Body: 3D Latent Diffusion Model for Controllable Anatomy, Tumor, and Demographic Body Mask Synthesis. arXiv:2610.24153, 2026. https://arxiv.org/abs/2610.24153. PDF: https://arxiv.org/pdf/2610.24153.
- Code: NVIDIA-Medtech/NV-Generate-CTMR (Apache 2.0), https://github.com/NVIDIA-Medtech/NV-Generate-CTMR. Weights: https://huggingface.co/nvidia/NV-Generate-CT.
- Featured image: Figs. 4 and 5 of arXiv:2610.24153, real paper pixels, two panels from each, cropped and resized only.