Computational Pathology Pipeline

The pipeline must efficiently read diverse proprietary WSI formats. OpenSlide and Bio-Formats libraries provide a unified API to access pixel data from formats like Aperio SVS, Hamamatsu NDPI, and Philips iSyntax without loading the entire image into memory. Ingestion includes parsing OME-XML metadata for magnification, scanner type, and staining protocol. A robust ingestion layer handles corrupt files, incomplete uploads, and format-specific quirks through retry logic and dead-letter queues.

Before inference, every slide undergoes automated QC to prevent garbage-in-garbage-out failures. Focus quality assessment uses Laplacian variance to flag blurry regions. Artifact detection identifies tissue folds, air bubbles, and pen marks via semantic segmentation. Tissue detection separates foreground from glass background using Otsu thresholding on low-resolution thumbnails. Slides failing QC thresholds are routed for manual review or rescanning.

The pipeline tessellates tissue regions into fixed-size patches (e.g., 256×256 or 512×512 pixels) at a target magnification. Stain normalization using Macenko or Vahadane methods standardizes color appearance across labs. Patches are extracted only from tissue foreground to avoid wasted computation on glass. A tissue mask generated during QC guides extraction coordinates. The output is a structured dataset of tensors ready for batched inference.

Inference across 100,000+ patches per slide demands horizontal scaling. The pipeline distributes patch batches across GPU clusters using Ray or Kubernetes orchestration. Models—often Vision Transformers or Pathology Foundation Models like UNI or CONCH—run in mixed precision (FP16) for throughput. A feature store caches patch embeddings to avoid recomputation. Slide-level aggregation uses attention-based MIL pooling to produce a final diagnostic probability.

Post-inference, patch-level probabilities are rendered as a color-coded heatmap overlay on the WSI pyramid for visual interpretability. Red regions indicate high tumor probability; blue indicates normal tissue. The pipeline generates structured DICOM SR or PDF reports including slide-level classification, confidence scores, and quantitative metrics like TIL density or tumor burden percentage. Heatmaps are stored as pyramidal TIFFs for efficient viewing in digital slide archives.

Every processing step is instrumented with OpenTelemetry traces and metrics. The pipeline records provenance: which scanner produced the image, which model version ran inference, and which normalization parameters were applied. A metadata ledger (often PostgreSQL or a graph database) links each output to its input WSI and processing parameters. This audit trail is essential for FDA SaMD compliance and debugging model drift in production.