Image registration in Python

Image registration in Python is lining up a moving image to a fixed one so the same anatomy (or the same scene) sits in one coordinate frame. For 2D photos you will usually use OpenCV: ORB keypoints, a matcher, a homography with RANSAC, then warpPerspective. For 3D medical volumes you will usually use SimpleITK: ImageRegistrationMethod, Mattes mutual information or correlation, a rigid or affine transform, Execute, then resample.

If you meant what medical image registration is (rigid vs deformable, intensity vs feature), that is medical image registration. If you meant the processing field (enhancement / segmentation / registration as one chapter), that is medical image processing.

What you are aligning

Every registration run has the same four parts. The libraries only change how you set them.

Fixed / moving. The fixed image does not move. The moving image is the one you transform.
Transform. Rigid is rotate + translate. Affine adds scale and shear. A deformable (B-spline) is for soft tissue that actually changes shape — do not start there.
Similarity metric. How “aligned” is scored. Mean squared error for same-modality 2D. Mutual information when CT and MRI do not share a look.
Optimizer. Walks the transform until the metric stops improving.

Environment

Use a venv. Install the four packages this page actually calls:

python -m venv .venv
source .venv/bin/activate
pip install opencv-python SimpleITK scikit-image numpy

Optional later, not required for the snippets below: antspyx (ANTsPy) and itk-elastix.

2D with OpenCV — ORB, homography, RANSAC, warpPerspective

This is the feature-based path. It works when both images have corners you can lock onto (a skyline, a printed page, a bone X-ray with clear edges). It fails on a low-contrast MRI where ORB finds nothing. Then you switch to SimpleITK.

import cv2
import numpy as np

fixed = cv2.imread("fixed.jpg", cv2.IMREAD_GRAYSCALE)
moving = cv2.imread("moving.jpg", cv2.IMREAD_GRAYSCALE)

orb = cv2.ORB_create(nfeatures=5000)
kp_fixed, des_fixed = orb.detectAndCompute(fixed, None)
kp_moving, des_moving = orb.detectAndCompute(moving, None)

matcher = cv2.BFMatcher(cv2.NORM_HAMMING, crossCheck=False)
raw = matcher.knnMatch(des_moving, des_fixed, k=2)
good = [m for m, n in raw if m.distance < 0.75 * n.distance]

src = np.float32([kp_moving[m.queryIdx].pt for m in good]).reshape(-1, 1, 2)
dst = np.float32([kp_fixed[m.trainIdx].pt for m in good]).reshape(-1, 1, 2)
H, inliers = cv2.findHomography(src, dst, cv2.RANSAC, 5.0)
aligned = cv2.warpPerspective(moving, H, (fixed.shape[1], fixed.shape[0]))

findHomography with RANSAC drops the bad matches. H is the 3×3 homography. warpPerspective applies it. Lowe’s 0.75 ratio test is the usual first filter; tighten it if you still have outliers.

Detector	Use when	Speed	Scale / rotation
ORB	Default. Fast, patent-free, good enough for most 2D jobs	Fast	Good
SIFT	You need more invariance and can wait	Slow	Excellent
SURF	A faster SIFT-class detector (patented; avoid in products)	Medium	Very good
AKAZE	Blur or nonlinear scale. Still slower than ORB	Medium	Excellent

Swap the detector, keep the rest of the pipeline. SIFT descriptors want NORM_L2, not Hamming.

3D medical volumes with SimpleITK

OpenCV is a 2D library. A CT or MRI is a 3D volume. SimpleITK is the library that actually registers those. Intensity-based: it never looks for corners. It scores the whole volume with a metric and walks a rigid or affine transform until the score stops moving.

Mattes mutual information when the two volumes do not share a look (CT ↔ MRI, PET ↔ CT). Correlation (or mean squares) when they do (MRI ↔ MRI, two CTs of the same protocol).

import SimpleITK as sitk

fixed = sitk.ReadImage("fixed.nii.gz", sitk.sitkFloat32)
moving = sitk.ReadImage("moving.nii.gz", sitk.sitkFloat32)

reg = sitk.ImageRegistrationMethod()
reg.SetMetricAsMattesMutualInformation(numberOfHistogramBins=50)
# same-modality alternative:
# reg.SetMetricAsCorrelation()
reg.SetOptimizerAsGradientDescent(
    learningRate=1.0,
    numberOfIterations=200,
    convergenceMinimumValue=1e-6,
    convergenceWindowSize=10,
)
reg.SetOptimizerScalesFromPhysicalShift()
reg.SetInterpolator(sitk.sitkLinear)

# rigid (Euler3D). Affine: sitk.AffineTransform(3)
initial = sitk.CenteredTransformInitializer(
    fixed,
    moving,
    sitk.Euler3DTransform(),
    sitk.CenteredTransformInitializerFilter.GEOMETRY,
)
reg.SetInitialTransform(initial, inPlace=False)

final_transform = reg.Execute(fixed, moving)

resampled = sitk.Resample(
    moving,
    fixed,
    final_transform,
    sitk.sitkLinear,
    0.0,
    moving.GetPixelID(),
)
sitk.WriteImage(resampled, "moving_on_fixed.nii.gz")
print(final_transform)
print(reg.GetMetricValue())

That is the checklist the old page named and never showed: load two 3D volumes → ImageRegistrationMethod → metric + optimizer + transform → Execute → resample onto the fixed grid. Start rigid. If the overlap is still wrong after a clean rigid, switch the initializer to sitk.AffineTransform(3). A B-spline deformable is a later step, not the first run.

ANTsPy and elastix — one-liners, not a second article

If SimpleITK’s rigid/affine is not enough and you already know you want SyN or a parameter-file elastix run:

import ants
reg = ants.registration(
    fixed=ants.image_read("fixed.nii.gz"),
    moving=ants.image_read("moving.nii.gz"),
    type_of_transform="Affine",  # or "SyN" for deformable
)

import itk
parameter_object = itk.ParameterObject.New()
parameter_object.AddParameterMap(parameter_object.GetDefaultParameterMap("rigid"))
result = itk.elastix_registration_method(fixed, moving, parameter_object)

Those are entry points, not a tutorial. Stay on this page for OpenCV + SimpleITK.

Did it work?

Checkerboard. Alternate squares from fixed and resampled. Edges that jump at the seams are a miss.
Metric. Mutual information up, or mean squares down, is necessary but not sufficient. A brightness shift can fake a good MSE.
Wrong model. Rigid will not fix a warped liver. Affine will not fix a resection. If the residual is local, that is when you name a B-spline — after the rigid/affine is already close.
No keypoints. ORB on a brain MRI is the usual dead end. Use SimpleITK.

FAQ

Feature-based or intensity-based?

Corners you can name → OpenCV. Soft tissue / multi-modal volumes → SimpleITK. That is the split. Not “which is better.”

Can Python do 3D?

Yes. SimpleITK (and ANTsPy / elastix) take 3D and 4D volumes. OpenCV does not.

Lighting or contrast does not match

Drop MSE. Use normalized cross-correlation (same modality) or mutual information (different modality). Histogram equalization as a preprocess can help a 2D OpenCV run; do not hist-eq a CT before an intensity-based medical registration — you will change the metric.

PYCAD builds the imaging side of this when the registered volume has to live in a clinic app. Case studies.

We build custom medical imaging platforms — advanced DICOM viewers, AI segmentation, and the clinical systems around them.