Computer vision in healthcare is software that reads medical pixels — X-ray, CT, MRI, ultrasound, whole-slide pathology, fundus photos, endoscope video — and returns a mark, a mask, or a score a clinician then uses or ignores. It is not a robot doctor. It does not sign the report.
The same job gets sold as “computer vision in medicine,” “computer vision in medical imaging,” and a pile of numbered listicles. Those are keyword wraps of this page. A general what is computer vision explainer (pixels, classification, detection, segmentation, cars and farms) is a different article. So is AI in healthcare (the broad stack), AI for medical diagnosis (CAD as a specialty), and AI in radiology.
What it actually does
Most of what ships is a second reader. After the exam, images go into a model. The model flags candidates. The clinician reviews the flags with the originals and signs. The usual engine is a convolutional net — sometimes a transformer sitting on the same pixels. That is the imaging form of computer-aided diagnosis, not a new specialty.
Three outputs show up again and again:
- Classification. One label for the study or the region: “referable diabetic retinopathy,” “likely ICH,” “this slide is prostate adenocarcinoma.”
- Detection. A box or a worklist bump: nodule on chest CT, polyp on colonoscopy video, a bleed on non-contrast head CT.
- Segmentation. A pixel mask: organ or lesion outline, tumor bed, vessel tree. The number you then compute (volume, diameter) is a measurement, not a diagnosis.
| Area | Input | Typical output |
|---|---|---|
| Diagnostic imaging | X-ray, CT, MRI, ultrasound, mammography | Candidate marks, triage flags, organ / lesion measurements |
| Surgery and planning | Pre-op CT/MRI plus intra-op video | 3D plan, tool tracking, overlay on the field |
| Pathology | Whole-slide images | Cell / tissue class, mitotic count, tumor / grade flags |
| Remote monitoring | Room or bedside camera | Fall, gait, activity, non-contact breathing rate |
| Dermatology | Clinical or dermoscopic photo | Lesion risk score; size / color change over time |
| Endoscopy / colonoscopy | Live GI video | Polyp or lesion highlight on a second screen |
| Ophthalmology | Fundus photo, OCT | DR grade, macular / glaucoma markers |
| Documentation | Scanned forms, labels, report text on a screen | OCR and a draft note the clinician edits |
Where the pixels actually come from
Diagnostic imaging is the part that shipped. A net trained on labeled studies flags a nodule, a bleed, a PE, a fracture, a mass. The radiologist still signs. That workflow is the radiology page, not this one — artificial intelligence in radiology.
Surgery. Pre-op volumes become a 3D plan. Intra-op cameras track tools or overlay anatomy. The surgeon still cuts. A robot cart with a camera is not autonomous surgery.
Pathology. A scanner turns the glass slide into a pyramid of pixels. The model counts mitoses, flags tumor, or grades. The pathologist still signs. Stain and scanner shift will move the score; you re-validate on your lab’s slides.
Remote monitoring. A camera watches a room or a bed and raises a fall or inactivity flag. Privacy is the product constraint, not a slide. Silhouette-only or on-device processing is how you keep a person from being a livestream.
Dermatology. A phone photo or a dermoscopic image gets a risk score. Lighting, angle, and skin-tone coverage in the training set decide whether that score is usable. A high score is a biopsy path, not a cancer diagnosis.
Endoscopy and colonoscopy. A model watches the live video and highlights a polyp on a second monitor. The endoscopist still withdraws and biopsies. This is one reason the old “medical imaging” URL belongs here: GI video is medical imaging with a different sensor.
Ophthalmology. Fundus photos and OCT are graded for diabetic retinopathy, macular degeneration, or glaucoma markers. Some systems are cleared for autonomous DR screening in a defined population. That is the exception, not the rule for the rest of this page — most imaging AI is assistive.
Documentation is the admin leftover: OCR on a requisition, a draft findings paragraph. Useful. Not a reason to keep a second “computer vision in healthcare” URL.
Will AI replace clinicians?
No. The models are narrow. A lung-nodule net does not read a trauma pan-scan, does not take a call from the surgeon, and does not own the signed report. Volume grew; so did the need for people who can drop the false flags and keep the real ones.
Most cleared imaging AI is assistive (FDA SaMD), not autonomous. A mark is a candidate. The clinician still signs. “Will AI replace radiologists” is answered on the radiology page. The same answer holds for pathologists, endoscopists, and dermatologists.
How a clinic actually starts
The “in medicine” and “in medical imaging” posts that used to live on this site were this explainer with a different noun. The real sequence is shorter.
- Name one problem. “Flag intracranial hemorrhage on non-contrast head CT and bump the worklist.” Not “bring computer vision to the hospital.”
- Get the pixels and the labels. Pull DICOM from PACS, or slides from the scanner. De-identify before anything leaves the premises. Labels come from reports, biopsies, or expert reads — not from a vendor deck.
- Re-validate on your machines. A paper AUC is not a site number. Vendor, protocol, and population shift the score.
- Land the flags in the viewer the clinician already uses. PACS / RIS / a web DICOM viewer. A separate portal is how tools die.
- Keep a person on the output. Worklist, viewer, signed report. Build vs fine-tune vs buy is a cost decision, not a different product.
What it does not do
- It does not replace the reader. The signed report is still a person. That is why most imaging computer vision is cleared as assistive, not autonomous.
- It fails on shift. A model trained on one vendor, protocol, stain, or hospital will miss or over-call on another. You re-validate on your scanners and your slides.
- It inherits the labels. Narrow demographics, one site, or noisy reports become biased flags. That is a dataset problem, not a “fairness module.”
- It is not a market forecast. Vendor CAGRs and “$X billion by 2032” are not a reason to buy, and they are not this page.
- It is not a general computer-vision lesson. Pixels, classification, detection, and segmentation as the spine live on what is computer vision.
If the question is diagnosis / CAD as a specialty, that is AI for medical diagnosis. If the question is the broader healthcare stack, that is what is artificial intelligence in healthcare. Radiology as its own job is artificial intelligence in radiology.
PYCAD builds the imaging side of this — custom pipelines and web DICOM viewers when the study has to live in a clinic app. Case studies.