X Li¹*, G Sotolongo², Y Mowery³, J Hodgin⁴, A Janowczyk⁵, L Barisoni⁶, K Lafata⁷, (1) Duke University, Durham, NC, (2) Duke University Health System, Durham, NC, (3) Duke University Medical Center, Durham, NC, (4) University of Michigan, Ann Arbor, MI, (5) Case Western Reserve University, Cleveland, OH,(6) Duke University, Durham, NC, (7) Duke University, Durham, NC

• 
  X Li

MO-FG-BRB-4 (Monday, 7/11/2022) 1:45 PM - 3:45 PM [Eastern Time (GMT-4)]

Ballroom B

Purpose: Develop a deep learning pipeline to quantify the immune microenvironment on hematoxylin and eosin (H&E) whole slide digital pathology images and validate with immunohistochemistry (IHC) at single cell resolution.

Methods: We developed a novel end-to-end pipeline that integrates the following steps: tissue preparation, image acquisition and registration, deep learning classification, and topological graph analysis. First, formalin-fixed, paraffin-embedded specimens of inflamed kidney tissue were cut at 2 microns, stained with H&E, and digitized into whole slide images at 40X magnification. IHC was subsequently performed on the same section with antibodies against CD3/CD20 to tag T+B lymphocytes and the tissue was re-scanned at 40X. Second, to identify pixel locations on H&E corresponding to CD3/CD20 protein expression, the IHC images were fused to their paired H&E images using a multiscale registration algorithm. All nuclei were computationally segmented on H&E via the StarDist algorithm. Using the measured CD3/CD20 positive cells as ground-truth, nuclei were labeled as either lymphocytes or non-lymphocytes to generate a ground-truth mask (GTM). Third, the paired {H&E, GTM} images were used to train a Hover-Net deep learning model, which – given only an H&E image – simultaneously segments and classifies lymphocytes. Finally, using the predicted lymphocyte locations as nodes, graphs were calculated to characterize the topology of the lymphocytic microenvironment. The predicted topology on H&E was compared to the measured topology on IHC via the structural similarity index measure (SSIM).

Results: A total of 22,732 nuclei (12,618 lymphocytes) were used to train the Hover-Net model, which was ultimately tested on 4,772 nuclei from an independent tissue specimen. Model precision, recall, f1, and AUC was 0.69, 0.77, 0.73, and 0.74, respectively. SSIM between predicted topology on H&E and measured topology on IHC was 0.82.

Conclusion: Our data suggests that deep learning can mimic IHC to predict lymphocytic topology on H&E images.

Funding Support, Disclosures, and Conflict of Interest: DoD / CDMRP W81XWH2110248

Keywords

Tissue Characterization, Image Fusion, Feature Extraction

Taxonomy

IM- Optical : Microscope-based imaging

Contact Email