Machine learning model deployment is packaging a trained model so a clinic system can call it — Docker image, REST API, then monitoring — and get a mark, a mask, or a score back. It is not training. It is not evaluation. It is not the service page.
The same job used to live here as “deploy AI model.” That URL was this explainer with a shorter slug. How you train the net (data, annotation, ResNet / U-Net, learning rate) is a different article. How you know the model is good — metrics, k-fold, leakage — is machine learning model evaluation. The product page at /model-deployment/ is a service blurb, not this guide. The MONAI / nnU-Net service is a clinic engagement, not a second explainer.
What deployment actually is
A .pkl or .pt on a laptop is not a product. Deployment is the system around that file: a pinned environment, a container that boots the same way on a GPU box and a staging VM, an API that accepts a study and returns JSON, and a watch on latency, errors, and drift. Notebooks hide all of that. Production does not.
The sequence is short:
- Package. Pin every dependency. Serialize the weights. Put the inference code and the runtime in one image.
- Place. Cloud GPU, on-prem next to PACS, or a hybrid (train in the cloud, infer inside the hospital network).
- Expose. A REST endpoint the viewer or orchestrator already talks to. A separate portal is how tools die.
- Watch. Latency, 5xx, input shape, and whether the score still matches a held-out set. Retrain is a decision, not a cron joke.
| Stage | What you actually do |
|---|---|
| Package | Docker image, pinned requirements.txt, model artifact |
| Infrastructure | VM / Kubernetes / on-prem GPU; network to PACS |
| Deploy | Roll the container; keep a rollback tag |
| Expose | REST API (FastAPI is the usual Python door), gateway, auth |
| Monitor | Latency, errors, drift; a retrain trigger you can defend |
Pin the environment before you containerize
The silent failure is a CUDA or NumPy pin that differs between the scientist’s workstation and the server. The DICOM model that “worked on my machine” and then returned garbage is almost always a driver or a floating-point library, not the weights.
- Pin exact versions:
torch==2.3.1, nottorch. - Isolate with
venvor conda so two projects do not share a site-packages. - Version the code, the config, and the artifact. The weights are not a side file on someone’s desktop.
Docker: one image, same inference
A container is the shipping box: model file, Python, CUDA-compatible runtime, and the API process. Medical volumes make fat images. A multi-stage build keeps the compiler and the download cache in a builder stage and copies only the runtime into a slim final image (python:3.10-slim, not the full tag).
Two ways to ship the weights:
- Bake them in. Simple and reproducible. Slow to update. Image is huge.
- Load at boot from object storage. Faster model swaps. You now own a fetch path and a checksum.
A slim FastAPI image looks like this — not as <h1> lines, which is how the old “deploy AI model” post rendered it:
FROM python:3.9-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY . .
EXPOSE 8000
CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]
Build and run it locally before you push: docker build -t medical-imaging-api . If it does not boot on your laptop, it will not boot in the hospital.
The API is the product
FastAPI is the usual door for a Python model. Flask and Django work. TorchServe is a serving layer, not a reason to keep a second URL. The endpoint should reject a JPEG when it asked for DICOM, return 400 with a reason, and never crash the worker on a bad header.
A response a viewer can use:
{
"prediction_id": "a1b2c3d4-e5f6-7890-1234-567890abcdef",
"label": "likely_ich",
"confidence": 0.92,
"regions": [{"box": [150, 200, 350, 450], "label": "candidate"}]
}
That is a candidate, not a signed report. Auth, rate limits, and a request-id in the logs are part of the same job. Secrets (API keys, DB URLs) go in the host’s secret store — Vault, Secrets Manager — not in the image.
Cloud, on-prem, or both
A 200 MB CT series sent across the internet for a two-second inference is a latency and a HIPAA conversation. On-prem next to PACS wins on those two. Cloud wins on a GPU you do not want to buy. Hybrid is common: train where the cards are cheap, infer where the pixels must stay.
| Factor | Cloud (SageMaker / Azure ML / Vertex) | On-prem |
|---|---|---|
| Scale | Spin a GPU up or down | Buy another box |
| Upfront cost | Pay as you go | Capex for hardware |
| Data control | Region and BAA; still a vendor | Your network, your disks |
| Latency | Depends on the hop | Local; needed next to the scanner |
| Ops | Provider owns the metal | Your team owns the metal |
Named platforms are options, not the article. AWS SageMaker, Azure Machine Learning, and Vertex AI wrap the same loop (package → endpoint → log). Pick one because your hospital already has a BAA there, not because a listicle ranked them.
Watch it after it ships
A live model goes stale. A new scanner, a new protocol, a new population, and the score moves. That is drift. Log the input hash, the version tag, the latency, and a periodic check against a frozen hold-out. Alert on 5xx and on a metric you already defined on the evaluation page — do not invent a second “is it good” article here.
HIPAA is not a footer. PHI in logs is a breach. De-identify before a study leaves the premises. The weights do not need a patient name.
What this page is not
- Not training. Data, annotation, ResNet / U-Net, and learning-rate sits on machine learning model training.
- Not evaluation. Precision, Dice, k-fold, and leakage sit on machine learning model evaluation.
- Not the service page. /model-deployment/ and the MONAI / nnU-Net service are engagements, not explainers.
- Not a market forecast. Vendor CAGRs are not a reason to ship.
If you are scoping a first clinic deploy, use the medical AI deployment checklist. PYCAD builds the imaging side of this — custom pipelines and viewers when the study has to live in a clinic app. Case studies.