Medical image registration puts two (or more) scans of the same patient into one coordinate frame. Same person, two visits. CT bone next to MRI soft tissue. A PET uptake map on the anatomy it belongs to. Rigid vs deformable is the first split. Intensity vs feature vs hybrid is how the algorithm decides the two volumes match.
If you meant Python / SimpleITK (or OpenCV on 2D), that is image registration in Python. If you meant processing techniques as a field (enhancement, segmentation, registration as one chapter), that is medical image processing. Satellite / Earth observation is not this page.
What it is
You have a fixed image (the reference) and a moving image (the one you warp). A transform maps every moving voxel onto the fixed grid. A similarity metric scores the overlap. An optimizer walks the transform until the score stops improving. That is the whole machine. The clinic names are longitudinal (time), multi-modal (different scanners), and atlas-based (a labeled template onto this patient).
History, one row: manual landmarks in the 1960s–70s, intensity methods in the 1980s–90s (mutual information made CT–MRI possible), feature detectors in the 2000s, learned / deep registration in the 2010s. Rigid in the 1970s, affine in the 1980s, non-rigid in the 2000s. The names stuck because the jobs stuck.
Rigid vs deformable
Rigid is rotate and translate. Bones, a head in a frame, anything that did not change shape. Fast. If a rigid fit is already wrong, a deformable will happily warp the wrong thing.
Deformable (non-rigid) lets local regions bend. Soft tissue, a lung that inhaled, a liver that shifted after a resection. Affine sits in the middle: global scale and shear, still one matrix, still not a local warp.
Start rigid. Add affine if the FOV or pixel size is off. Open a deformable only when the residual after that is local tissue motion. A bad deformable is worse than no registration — the fused PET-CT will lie about where the uptake is.
Intensity, feature, hybrid
Intensity-based compares voxel values. Mutual information is the usual multi-modal metric (CT ↔ MRI, PET ↔ CT). Normalized cross-correlation (or mean squares) when both scans share a look. No landmarks required. Sensitive to FOV mismatch and a bad initializer.
Feature-based finds landmarks — corners, edges, implanted fiducials, a segmented surface — and aligns those. Strong on bone and anything with a clear landmark. Weak on a brain MRI with no corners.
Hybrid does a feature (or surface) pass for a coarse pose, then an intensity refine. That is the usual clinical stack when anatomy varies and you still need sub-voxel polish.
Which method
The old “registration of images” page was this box. Same job as this page.
| Method | Best use | Accuracy | Time | Limit |
|---|---|---|---|---|
| Rigid | Bone, skull, a frame that did not deform | High when the assumption holds | Fast (<1 min) | Soft tissue that changed shape |
| Deformable | Soft tissue, longitudinal organs | High when initialized well | Minutes | Compute; a bad warp looks plausible |
| Hybrid | Multi-organ, messy anatomy | High | Variable | More knobs, more ways to miss |
| Surface-based | External anatomy, a segmented organ wall | Moderate | Fast | Surface only — inside the organ is free |
| Intensity-based | Internal structure, multi-modal | High | Slower | Image quality and a sensible metric |
What clinics actually use it for
- Oncology. Same tumor, two visits. Growth or shrink is a registration problem before it is a measurement problem. Radiation therapy needs the planning CT and the daily image in one frame so the dose hits the target and not the cord.
- Surgery. Multi-modal planning: CT bone + MRI soft tissue (+ fMRI / DTI in neuro). The fused volume is the map. Intra-op, the same idea is “where is the tool relative to that map.”
- Cardiology. Temporal registration across the cycle — wall motion, a cine loop that has to line up before you measure. CT / ultrasound / MRI of the same heart, not three different hearts.
- Ortho. Bone is rigid. Implant sizing and a pre-op plan are the use. Soft-tissue MRI next to CT is the multi-modal case.
Multi-modal
Each scanner measures a different physical thing. CT is attenuation (bone is easy). MRI is soft tissue. PET / SPECT is function. Ultrasound is live and operator-dependent. Registration is how those become one hanging. Mutual information is the default metric when the intensities are not comparable. Normalized cross-correlation when they are.
| Specialty | Modalities | Why you register |
|---|---|---|
| Oncology / rad-onc | CT, PET, MRI | Tumor location, dose on the right voxels |
| Neurosurgery | MRI, CT, fMRI | Map + guidance; stay off eloquent cortex |
| Cardiology | CT, ultrasound, MRI | Function on anatomy; stent / wall motion |
| Orthopedics | CT, MRI | Joint / implant plan |
Multi-temporal is the same math on two dates instead of two scanners. Soft tissue will have moved. That is why a rigid-only pipeline on a liver follow-up is the usual miss.
What actually breaks a deploy
- Compute. A 3D deformable on a high-res series is not a laptop job. Size the box for the study you actually run, not the demo.
- Workflow. If the registered series does not land back on the PACS / viewer hanging, nobody will use it. The product is the hanging, not the algorithm.
- QA. A checkerboard or a fused overlay a human signs. Automated metrics lie. Two operators should not get two different warps on the same pair without a flag.
- Edge cases. Resection cavities, contrast that was not there last time, a FOV that clips the organ. Those need a fallback (rigid-only, or a human landmark), not a more aggressive deformable.
Small clinic: one well-supported rigid + MI pair on the studies you already hang, plus a person who can reject a bad overlay. Large center: the same, plus a written protocol and a second reader on the deformable cases.
PYCAD builds the imaging side of this when the registered volume has to live in a clinic app. Case studies.