Medical image processing is the set of computational steps that turn a raw CT, MRI, PET, or ultrasound into something you can measure or fuse. It is the field, not a product list. The software that runs these steps is image analysis software. Segmentation as its own job is medical image segmentation.
Three techniques do most of the work: enhancement, segmentation, and registration. Multi-modality (PET-CT, PET-MRI, SPECT-CT) is those three applied across scanners. The rest of this page is a map of the methods and algorithms people actually name — not a tutorial and not a fake accuracy table.
The three techniques
Enhancement
Make the thing you care about easier to see or easier for the next algorithm. Histogram equalization (and adaptive variants — see AHE) spreads intensity so a faint lesion is not stuck in one bin. Noise filters (Gaussian, median, anisotropic) trade grain for blur. Enhancement can invent edges. It is a prep step, not a diagnosis.
Segmentation
Label every pixel or voxel that belongs to an organ, a lesion, or a vessel. Heart MRI volume, tumor staging, an aorta you can measure — all start as a mask. How you draw that mask (threshold, snakes, U-Net) is below. A walkthrough lives elsewhere: ImageJ clicks at image segmentation in ImageJ; a train loop at nnU-Net; the generic methods page at medical image segmentation.
Registration
Put two images of the same anatomy into one coordinate frame. Same patient, two visits (longitudinal). CT bone + MRI soft tissue (multi-modal). A labeled atlas onto a new brain. Rigid is rotate/translate. Deformable is the hard one. Bad registration makes a fused PET-CT lie about where the uptake is.
| Technique | Question | Breaks when |
|---|---|---|
| Enhancement | Can I see / can the next step see? | You amplify noise or invent contrast |
| Segmentation | Which voxels are the thing? | Weak edges, odd anatomy, no labels for a net |
| Registration | How do these two images line up? | Motion, missing correspondences, a bad deformable |
Multi-modality
One scanner is one contrast. PET-CT puts uptake on anatomy. PET-MRI does the same with soft tissue and less dose. SPECT-CT is the nuclear-medicine version for perfusion and some bone work. The processing problem is the same: resample, register, display. The clinic name is “fusion.” Preprocessing the volume before a net is a different page: what is image preprocessing.
Eight methods people mean
The old “methods” URL was this list. Same job as this page. Complexity is “how much you have to get right,” not a score.
| Method | What it does | Use when | Cost |
|---|---|---|---|
| CNN analysis | Learns features; classifies or segments | You have labeled scans and a GPU | Data + compute. Black box unless you add maps. |
| Active contours / snakes | A curve that shrinks to an edge (Kass 1988) | Smooth organ walls, you can initialize | Bad init, weak gradients |
| Atlas-based registration | Warp a labeled template onto the patient | Normal anatomy (brain structures) | Tumors and resections break the atlas |
| Texture analysis | Stats on local intensity (radiomics-adjacent) | Tissue look you cannot threshold | Scanner / protocol drift |
| Classical ML classification | Hand features → SVM / random forest / etc. (scikit-learn) | Small data, you want an inspectable model | Feature engineering is the job |
| Morphological processing | Erode / dilate / open / close on a mask | Clean a threshold; fill holes | Wrong structuring element |
| Multi-scale analysis | Look at the image at several resolutions | Vessels or lesions that change size | Compute; which scale to trust |
| Graph-based analysis | Voxels or regions as nodes; cut or walk the graph | Irregular, connected structures | Memory on 3D volumes |
Eight algorithms (the other list)
The old “algorithms” URL overlapped the methods list on CNN and snakes. The six that are not already above:
| Algorithm | Family | Typical use |
|---|---|---|
| CNN | Learning | Nodule / tumor / retinopathy classification or seg |
| Active contours | Segmentation | Chamber or tumor boundary when you can seed it |
| Watershed | Segmentation | Touching cells / particles; over-segments without markers |
| SURF | Features / registration | Landmarks for matching two images; faster than SIFT |
| Random forest (on voxels) | Classical ML seg | Tissue class from hand features; uncertainty per voxel |
| Histogram equalization | Enhancement | Contrast on X-ray / MRI / mammo; watch noise |
| U-Net | Learning (seg) | The biomedical mask net. Paper: arXiv 1505.04597 (Ronneberger, Fischer, Brox 2015) |
| Gabor filter | Texture | Oriented texture (vessels, tissue); a filter bank, not a diagnosis |
U-Net is an encoder–decoder with skip connections so the mask keeps its edges. 3D U-Net is the same idea on a volume. A paper Dice on one hospital’s scanner is not a number you can paste onto the next vendor. How you actually train a modern 3D net is nnU-Net for medical image segmentation.
Public datasets (names, not a tutorial)
The old “tutorial” URL named datasets and then never opened a notebook. Keep the names: NIH ChestX-ray (order of 100k frontal films) and TCIA (The Cancer Imaging Archive — oncology collections, usually DICOM). They are starting points for a methods paper, not a clinic install. DICOM itself is a different page.
What this page is not
- Not a software list. Named products sit on image analysis software.
- Not a Fiji walkthrough. Image segmentation in ImageJ.
- Not a train loop. nnU-Net.
- Not the generic seg explainer. Semantic vs instance, threshold-to-U-Net map: medical image segmentation.
- Not preprocessing-as-its-own-job. What is image preprocessing.
PYCAD builds the imaging side of this when the mask or the fusion has to live in a clinic app. Case studies.