DICOM & Medical Imaging Standards
If you build a healthcare ML system and you have not yet had a fight with DICOM, you have not actually shipped a healthcare ML system. Every CT, every MRI, every X-ray, every ultrasound in every hospital on earth lands in DICOM. Loving it is optional. Reading it correctly is not.
Learning Objectives
After this lesson, you will be able to:
- Explain what DICOM is, why every clinical imaging pipeline hits it, and what makes it different from PNG/JPEG
- Decode the four-level DICOM hierarchy (Patient, Study, Series, Instance) and why ML pipelines often break the Series boundary
- Read a DICOM file in Python with pydicom, including the pixel array, the rescale slope/intercept, and the window/level
- Anonymize a DICOM correctly for ML training (PHI removal per the DICOM PS3.15 Basic Profile, not just blanking PatientName)
- Handle the three pitfalls that bite every newcomer: byte order, photometric interpretation (MONOCHROME1 vs MONOCHROME2), and HU conversion
- Order CT/MRI slices correctly into a 3D volume for ML — the SliceLocation vs ImagePositionPatient trap
- Connect DICOM to HIPAA, PACS, and the C-STORE/C-FIND protocols you will see referenced in every hospital integration project
Prerequisites
Before You Start
#Why DICOM Will Show Up On Day Three Of Every Healthcare ML Project
If your project touches any imaging modality — radiology, cardiology, ophthalmology, pathology whole-slide imaging, dental, dermoscopy, even some ultrasound and endoscopy workflows — the data does not arrive as PNG. It arrives as DICOM, or it lives in a PACS that speaks DICOM, or it transits a vendor neutral archive that stores DICOM, or it comes off the modality as DICOM and gets converted to NIfTI by a research group that promptly loses half the metadata you actually needed.
#What DICOM Actually Is
DICOM stands for Digital Imaging and Communications in Medicine. It is maintained by the DICOM Standards Committee (NEMA, with WG-26 specifically for pathology and many other working groups). The current standard is PS3 and it spans about 22 parts and 6,000+ pages — yes, really.
DICOM is three things at once:
- A file format. A
.dcmfile is a sequence of (tag, VR, length, value) tuples. The tag identifies the field, the VR is the "Value Representation" (UI = unique identifier, DA = date, PN = person name, US = unsigned short, etc.), the length is in bytes, and the value is the data. - A network protocol. Modalities, PACS servers, and viewers talk to each other over DIMSE (DICOM Message Service Element) on top of TCP, traditionally on port 104 or 11112. The core operations are C-STORE (push an image), C-FIND (search), C-MOVE (pull an image), and C-ECHO (ping).
- A workflow standard. DICOM defines Worklists, Modality Performed Procedure Step, Structured Reports, Presentation States, RT (radiation therapy) objects, and many other "IODs" (Information Object Definitions).
For an ML engineer, you mostly care about (1) the file format and (3) the metadata. You will brush against (2) the network protocol when integrating with a hospital, and a member of your team will spend three weeks of their life debugging it.
#The Four-Level DICOM Hierarchy
Every DICOM object lives in a four-level hierarchy. Learn this hierarchy. Drill it. Tattoo it on your forearm. It is the single most important data model in healthcare imaging.
| Level | Unique ID | Real-world meaning |
|---|---|---|
| Patient | PatientID (0010,0020) | A person |
| Study | StudyInstanceUID | One imaging encounter, e.g. "Chest CT on 2025-11-04" |
| Series | SeriesInstanceUID | One acquisition or reconstruction within the study |
| Instance | SOPInstanceUID | One image file (one slice for CT, one frame for X-ray) |
Where ML pipelines break: beginners shuffle Instances. They take a chest CT, treat each slice as an independent training example, throw them all into a random 80/10/10 split, and unwittingly leak the same patient across train, val, and test. This is the single most common reason a published medical AI model collapses in real validation — including some that have made it through FDA submission and then mysteriously underperformed in deployment.
Rule: split on PatientID. Always. No exceptions.
#Reading a DICOM File in Python
pydicom is the canonical Python library. It is mature, well-maintained, and is what every healthcare ML team uses. There is also SimpleITK for scientific volume handling and GDCM under the hood for some compressed transfer syntaxes.
pythonreference · read-only
import pydicom
# Read the file. This parses the header but lazy-loads pixel data.
ds = pydicom.dcmread("scan.dcm")
# Access metadata by name (the "keyword" form)
print(ds.PatientID) # e.g. "ANON-00451"
print(ds.Modality) # "CT", "MR", "CR", "DX", "MG", "US", ...
print(ds.SeriesDescription) # "Axial Chest 1.25mm"
print(ds.SliceThickness) # 1.25
print(ds.PixelSpacing) # [0.7, 0.7] (mm per pixel, row, column)
# Access by tag (always works even when keyword unknown)
print(ds[0x0010, 0x0020].value) # PatientID
# Pixel data
import numpy as np
arr = ds.pixel_array # shape (rows, cols) for 2D, (frames, rows, cols) for multi-frame
print(arr.dtype, arr.shape, arr.min(), arr.max())That is the easy part. Now for the parts that bite.
#Pitfall 1: Rescale Slope and Intercept (Hounsfield Units)
CT scanners store pixels as a 12- or 16-bit integer that is not the Hounsfield Unit (HU). To recover HU you must apply the per-file rescale:
This is a universal CT bug. If your model expects HU and you feed it raw stored values, every CT pixel is off by ~1024. Loss converges to a local minimum that has nothing to do with anatomy. The model looks like it works on the research dataset (already converted) and fails on every real PACS export.
#Pitfall 2: Photometric Interpretation (MONOCHROME1 vs MONOCHROME2)
The
PhotometricInterpretation tag tells you whether the maximum pixel value is black or white.- MONOCHROME2. High pixel value = white. (Used in CT, MR, almost everywhere.)
- MONOCHROME1. High pixel value = black. Inverted. (Still occasionally used for X-rays and some legacy mammography.)
If you ignore this tag, half your X-rays will be color-inverted relative to the other half and your CNN will learn nothing.
pythonreference · read-only
if ds.PhotometricInterpretation == "MONOCHROME1":
arr = arr.max() - arr # invertThere is also
RGB, PALETTE COLOR, YBR_FULL_422, and others for color modalities — handle them explicitly or you will silently grab garbage.#Pitfall 3: Windowing
A chest CT covers ~4,000 HU of dynamic range (air at -1000, bone at +3000) but a screen can show ~256 grey levels. The radiologist applies a window that maps a clinically relevant range to displayable values.
- Lung window: WL = -600, WW = 1500. Shows lung parenchyma well.
- Soft tissue: WL = 40, WW = 400. Shows abdomen, mediastinum.
- Bone: WL = 400, WW = 1800. Shows fractures.
Every DICOM stores recommended windows in
WindowCenter (0028,1050) and WindowWidth (0028,1051). For ML you typically either train on raw HU values (preferred for CT, no information loss) or pick a fixed window for the task (lung window for nodule detection, bone window for fracture). Never train on whatever window happens to be in the header — that is the radiologist's preference, not a feature of the image.#Building a 3D Volume from a CT Series
A chest CT is typically 200-700 instances (slices). To build the 3D numpy volume you must sort them in physical Z order. Do not trust the file name. Do not trust the InstanceNumber. Use
ImagePositionPatient (0020,0032), which gives the (x, y, z) coordinates of the slice's top-left pixel in patient coordinates.pythonreference · read-only
slices = [pydicom.dcmread(p) for p in glob.glob("series/*.dcm")]
slices.sort(key=lambda d: float(d.ImagePositionPatient[2]))
volume = np.stack([s.pixel_array for s in slices]) # (Z, H, W)
hu_volume = volume * slices[0].RescaleSlope + slices[0].RescaleInterceptIf slices have varying RescaleSlope/Intercept (rare but it happens with multi-vendor reformats), apply per-slice. If the Z spacing is not uniform (common with breath-held acquisitions), resample to isotropic before any 3D CNN.
#Anonymization for ML Training
You must remove Protected Health Information (PHI) before data leaves the covered entity for ML purposes. The DICOM Standard PS3.15 defines the Basic Application Confidentiality Profile plus a set of optional profiles (Clean Pixel Data, Clean Recognizable Visual Features, Retain Longitudinal Temporal Information With Modified Dates, etc.).
Naïve approach (wrong, will get you in trouble): blanking PatientName and PatientID.
Correct approach:
- Apply the DICOM PS3.15 Basic Profile. Removes 100+ tags including the obvious ones (PatientName, PatientID, PatientBirthDate, PatientSex, PatientAddress, AccessionNumber, InstitutionName) and the less-obvious ones (ReferringPhysicianName, OperatorsName, RequestingPhysician, IssuerOfPatientID, OtherPatientIDsSequence).
- Re-map UIDs deterministically. Replace StudyInstanceUID, SeriesInstanceUID, SOPInstanceUID with new UIDs derived from a salted hash. Keep the mapping in a separate secured database — you may need it for re-linking labels later.
- Shift dates (do not blank them). A 6-month follow-up CT compared to a baseline is clinically meaningful. Shift every date for a patient by the same random offset of 0-365 days. Preserves intervals, destroys absolute dates.
- Scrub burned-in PHI in the pixels. Old ultrasound and CR studies sometimes have the patient name typed into the image pixels themselves. You need OCR-based detection (or a hand-coded crop) for these — DICOM tag scrubbing does not touch them.
- Strip private tags by default. Vendor private tags (group number is odd) can contain raw debug data including patient identifiers. Remove unless you have audited each one.
- Document everything in a de-identification log for your IRB.
The DICOM Standard PS3.15 Annex E is the authoritative list. The
pydicom-deid and presidio libraries automate most of it.#The 60-Line Reference Script
This is a defensible starting point. It reads a DICOM, applies HU conversion, fixes photometric interpretation, de-identifies, and writes the cleaned copy.
pythonreference · read-only
"""Minimal DICOM anonymizer + HU normalizer for ML training pipelines."""
import hashlib
import pydicom
from pydicom.uid import generate_uid
import numpy as np
# Tags removed under DICOM PS3.15 Basic Profile (subset — full list is ~100 tags)
PHI_TAGS_TO_REMOVE = [
"PatientName", "PatientBirthDate", "PatientSex", "PatientAddress",
"PatientTelephoneNumbers", "OtherPatientIDs", "OtherPatientNames",
"ReferringPhysicianName", "RequestingPhysician", "PerformingPhysicianName",
"OperatorsName", "InstitutionName", "InstitutionAddress",
"InstitutionalDepartmentName", "AccessionNumber", "StudyID",
"DeviceSerialNumber", "StationName",
]
SALT = b"replace-with-secret-from-your-secrets-manager"
def hash_id(raw_id: str) -> str:
return hashlib.sha256(SALT + raw_id.encode()).hexdigest()[:16]
def shift_date(date_str: str, offset_days: int) -> str:
from datetime import datetime, timedelta
if not date_str or len(date_str) < 8:
return ""
d = datetime.strptime(date_str[:8], "%Y%m%d") + timedelta(days=offset_days)
return d.strftime("%Y%m%d")
def to_hu(ds: pydicom.Dataset) -> np.ndarray:
arr = ds.pixel_array.astype(np.int16)
slope = float(getattr(ds, "RescaleSlope", 1.0))
inter = float(getattr(ds, "RescaleIntercept", 0.0))
hu = arr * slope + inter
# Honor photometric interpretation for X-rays
if ds.get("PhotometricInterpretation", "") == "MONOCHROME1":
hu = hu.max() - hu
return hu.astype(np.int16)
def anonymize(in_path: str, out_path: str, date_offset_days: int = 0) -> np.ndarray:
ds = pydicom.dcmread(in_path)
# 1. Compute HU for ML before we touch metadata
hu_array = to_hu(ds)
# 2. Replace patient ID with deterministic hash
original_pid = str(ds.get("PatientID", "UNKNOWN"))
ds.PatientID = "ANON-" + hash_id(original_pid)
# 3. Re-map UIDs (keep linkage by hashing the originals)
ds.StudyInstanceUID = generate_uid(entropy_srcs=[ds.StudyInstanceUID, SALT.hex()])
ds.SeriesInstanceUID = generate_uid(entropy_srcs=[ds.SeriesInstanceUID, SALT.hex()])
ds.SOPInstanceUID = generate_uid(entropy_srcs=[ds.SOPInstanceUID, SALT.hex()])
ds.file_meta.MediaStorageSOPInstanceUID = ds.SOPInstanceUID
# 4. Shift dates
for tag in ("StudyDate", "SeriesDate", "AcquisitionDate", "ContentDate"):
if tag in ds:
setattr(ds, tag, shift_date(getattr(ds, tag), date_offset_days))
# 5. Wipe known PHI tags
for tag in PHI_TAGS_TO_REMOVE:
if tag in ds:
delattr(ds, tag)
# 6. Remove all private (odd-group) tags by default
ds.remove_private_tags()
ds.save_as(out_path)
return hu_arrayThis is a starting point. A production pipeline adds: burned-in pixel scrubbing (OCR pass), private tag whitelisting per vendor, full PS3.15 tag coverage via
pydicom-deid, audit logging, and an integration test that round-trips known PHI to verify it has actually been removed.#PACS, C-STORE, C-FIND, and DICOMweb
You will eventually need to pull data from a hospital PACS. The classical protocol is DIMSE over TCP:
- C-ECHO. Application-level ping. Always test first.
- C-FIND. Query. You send a "query Identifier" with the tags you want to match (PatientID, StudyDate, Modality) and the level (Patient/Study/Series/Image). The PACS responds with matching entries.
- C-MOVE. Instruct the PACS to push images to a third Application Entity (your storage SCP).
- C-GET. Pull images directly. Less commonly enabled.
- C-STORE. Push an image to the PACS (or to your SCP).
Each end is identified by an AE Title (Application Entity Title), and connections are negotiated with "associations" that declare which "SOP classes" and "transfer syntaxes" each side supports. The Python library here is pynetdicom.
The modern alternative is DICOMweb — HTTP/JSON-flavored DICOM:
- QIDO-RS. Query (replaces C-FIND).
- WADO-RS. Retrieve (replaces C-MOVE/C-GET).
- STOW-RS. Store (replaces C-STORE).
If you have a choice, choose DICOMweb. Easier to firewall, easier to authenticate (OAuth2), and it returns JSON metadata. Vendors like Google Healthcare API, AWS HealthImaging, and Azure Health Data Services all expose DICOMweb endpoints.
#HIPAA, DICOM, and the Compliance Stack
HIPAA's Safe Harbor rule (45 CFR §164.514(b)(2)) lists 18 categories of identifiers that must be removed for data to be considered de-identified. DICOM PS3.15 Annex E maps these categories to specific tags. The cross-reference:
| HIPAA Identifier | DICOM Tag(s) |
|---|---|
| Names | PatientName, ReferringPhysicianName, OperatorsName |
| Geographic subdivisions | PatientAddress, InstitutionAddress |
| Dates (except year) | StudyDate, SeriesDate, PatientBirthDate, AcquisitionDateTime |
| Phone, fax, email | PatientTelephoneNumbers, PatientPhoneNumbers |
| Medical record numbers | PatientID, AccessionNumber |
| Device identifiers | DeviceSerialNumber, StationName |
| Biometric identifiers | Burned-in pixels (face on head CT, retinal scans) |
| Full-face photographs | Burned-in pixels |
Face removal on head CT/MR is a specific compliance hazard. A 3D rendering of a head CT identifies the patient about as well as a photograph. The standard mitigation is "defacing" — running a tool like pydeface or FreeSurfer's mri_deface before sharing. NIH dataset releases require it. Not doing it for a brain MRI dataset is a guaranteed IRB rejection.
You also need a signed Business Associate Agreement (BAA) with your cloud provider before any PHI hits their infrastructure (AWS, Google Cloud, Azure all offer BAAs but only for specific services — using a non-BAA service is the most common HIPAA violation in startups).
DICOM Standard PS3.15: Security and System Management Profiles
DICOM Standards Committee (2024)
Annex E (Attribute Confidentiality Profiles) is the canonical specification of which tags to remove for de-identification. The 'Basic Application Confidentiality Profile' is the minimum bar; the 'Clean Pixel Data' and 'Clean Recognizable Visual Features' profiles handle the burned-in-PHI and defacing cases.
#ML-Specific Considerations
3D volumes. Resample to isotropic spacing (1mm³ for most CT/MR tasks) before any 3D CNN. The native voxel sizes vary wildly across scanners — a single dataset may mix 0.5×0.5×0.625 mm and 0.7×0.7×5.0 mm volumes, and a 3D CNN trained on the mix learns mostly the spacing rather than anatomy.
Multi-frame DICOM. Per-frame metadata lives in the PerFrameFunctionalGroupsSequence (5200,9230) for Enhanced CT/MR objects. For ultrasound cine loops, the frames are in
pixel_array with shape (N, H, W) but the per-frame timestamps live in FrameTime, FrameTimeVector, or FrameAcquisitionDateTime.Window/level as augmentation. For X-ray and CT tasks, sampling random window/level during training is a strong augmentation that mimics radiologist-preference variability. Use it instead of brightness/contrast.
Image-level vs study-level labels. A radiology report describes the entire study (or sometimes the patient). Your training labels are study-level. Your model often makes instance-level predictions. The aggregation function (max, mean, top-K, attention-based MIL) is a core design choice and is the topic of most published medical-imaging ML papers in the last five years.
Class imbalance is brutal. Lung-cancer screening: ~0.5% positive. Retinal hemorrhage screening: ~2%. You need class-balanced sampling, focal loss or class weights, and threshold tuning at deployment. ROC AUC alone is misleading — always report sensitivity at a fixed specificity (e.g., sensitivity at 95% specificity).
#Try It Yourself
pythonplayground.py · Pyodide
#Quick Check
Quick Check1 / 5
You are training a chest CT model on 10,000 DICOMs from 800 patients. What is the correct split strategy?
#Key Takeaways
- DICOM is the format AND the protocol AND the workflow standard. Every clinical imaging deployment hits it, and the metadata is often more valuable than the pixels themselves
- The Patient/Study/Series/Instance hierarchy is sacred. Always split datasets on PatientID, never on Series or Instance
- The three universal pitfalls: rescale (HU conversion), photometric interpretation (MONOCHROME1 inversion), and windowing — get these right or your model silently trains on garbage
- Anonymization is legally required under HIPAA Safe Harbor. Use the DICOM PS3.15 Basic Profile, shift dates rather than blanking them, and scrub burned-in pixel PHI separately
- PACS speaks C-STORE/C-FIND/C-MOVE over DIMSE, or DICOMweb over HTTP. Choose DICOMweb for new integrations; QIDO/WADO/STOW is much easier to operate than classical DIMSE
- 3D volumes need isotropic resampling and IPP-based sorting. Never trust InstanceNumber or file names to give you the right Z order
You can now read a DICOM, convert it to HU, anonymize it, and reason about how it moves through a hospital network, and that closes out the ML Engineering track. Twenty-three lessons from the data pipeline to multi-model GPU serving, EU-AI-Act compliance, incident response, and PACS-grade clinical integration. The remaining tracks are about you, not the system: Track Claude Code shows you how to wield Claude Code itself as part of your engineering practice, and the Career track is the job-market layer — resume, interview patterns, and the production-disaster case studies that make sure you are the engineer who keeps these systems running.