Skip to main content
NEJM AI homepage
Access provided by
AWP FRANCE SAS

Monthly & Annual Options

Subscribe Now
Access provided by
AWP FRANCE SAS

Monthly & Annual Options

Subscribe Now

Abstract

The search and retrieval of digital histopathology slides are important tasks that have yet to be solved. In this case study, we investigate the clinical readiness of four state-of-the-art histopathology slide search engines — Yottixel (“one yotta pixel”), SISH (self-supervised image search for histology), HSHR (High-Order Correlation-Guided Self-Supervised Hashing-Encoding Retrieval), and RetCCL (Retrieval with Clustering-Guided Contrastive Learning) — on both unseen datasets and several patient cases. We provide a qualitative and quantitative assessment of each model’s performance in providing retrieval results that are reliable and useful to pathologists. We found high levels of performance across all models using conventional metrics for tissue and subtyping search. Upon testing the models on real patient cases, we found that the results were still less than ideal for clinical use. On the basis of our findings, we propose a minimal set of requirements to further advance the development of accurate and reliable histopathology image search engines for successful clinical adoption. (Funded by The University of Texas Rising STARs [Science and Technology Acquisition and Retention] Program and The Cancer Prevention & Research Institute of Texas.)

Notes

A data sharing statement provided by the authors is available with the full text of this article.
Supported by The University of Texas System (Rising STARs ‘Science and Technology Acquisition and Retention’ Program Award to Dr. Luber) and The Cancer Prevention & Research Institute of Texas (First Time Faculty Award Grant #RR220015 to Dr. Luber).
The test slides along with the updated source code for all three methods used to generate the results can be found at https://github.com/jacobluber/PathologySearchComparison. Instructions to fully replicate the results are provided. The pathology slides generated for this study are available for download at https://zenodo.org/records/10835156. All other data used are publicly available at https://portal.gdc.cancer.gov/ and https://www.cancerimagingarchive.net/. The list of data included in the database can also be found in the GitHub repository.
Disclosure forms provided by the authors are available with the full text of this article.
We thank the following for their contributions: David Fernandez-Hazoury, M.D., Department of Pathology, Harbor-University of California, Los Angeles Medical Center; Kenechukwu Ojukwu, M.D., M.P.P., Bone/Soft Tissue Fellow, Department of Pathology and Laboratory Medicine, National Clinician Scholars Program Fellow; Alex Oliveira Kowaleski, M.D., University of California, Los Angeles Department of Pathology and Laboratory Medicine; Julie Y. Kim, D.O., M.S., Harbor-University of California, Los Angeles Medical Center; and Daniel P. Stefanko, M.D. Ph.D., University of California, Los Angeles Department of Pathology and Laboratory Medicine.

Supplementary Material

Supplementary Appendix (aics2300019_appendix.pdf)
Disclosure Forms (aics2300019_disclosures.pdf)
Data Sharing Statement (aics2300019_data-sharing.pdf)

Information & Authors

Information

Published In

History

Submitted: June 30, 2023
Revised: February 8, 2024
Accepted: February 28, 2024
Published online: April 25, 2024
Published in issue: April 25, 2024

Topics

Authors

Affiliations

Helen H. Shang, M.D., M.S. https://orcid.org/0000-0002-4684-6859
Department of Internal Medicine, Ronald Reagan University of California, Los Angeles Medical Center, Los Angeles
Department of Computer Science and Engineering, The University of Texas at Arlington, Arlington
Mohammad Sadegh Nasr, Ph.D. https://orcid.org/0000-0001-9675-5640
Department of Computer Science and Engineering, The University of Texas at Arlington, Arlington
Multi-Interprofessional Center for Health Informatics, The University of Texas at Arlington, Arlington
Jai Prakash Veerla, B.S. https://orcid.org/0009-0000-5023-0769
Department of Computer Science and Engineering, The University of Texas at Arlington, Arlington
Multi-Interprofessional Center for Health Informatics, The University of Texas at Arlington, Arlington
Jillur Rahman Saurav, B.S. https://orcid.org/0000-0001-7451-9962
Department of Computer Science and Engineering, The University of Texas at Arlington, Arlington
Multi-Interprofessional Center for Health Informatics, The University of Texas at Arlington, Arlington
Department of Computer Science and Engineering, The University of Texas at Arlington, Arlington
Multi-Interprofessional Center for Health Informatics, The University of Texas at Arlington, Arlington
Department of Computer Science and Engineering, The University of Texas at Arlington, Arlington
Multi-Interprofessional Center for Health Informatics, The University of Texas at Arlington, Arlington
Department of Computer Science and Engineering, The University of Texas at Arlington, Arlington
Chace Moleta, M.D., M.S. https://orcid.org/0009-0000-5867-548X
Department of Pathology & Laboratory Medicine, Ronald Reagan University of California, Los Angeles Medical Center, Los Angeles
Jitin Makker, M.B.B.S., M.D. https://orcid.org/0000-0001-6197-8641
Department of Pathology & Laboratory Medicine, Ronald Reagan University of California, Los Angeles Medical Center, Los Angeles
Department of Computer Science and Engineering, The University of Texas at Arlington, Arlington
Multi-Interprofessional Center for Health Informatics, The University of Texas at Arlington, Arlington
Department of Bioengineering, The University of Texas at Arlington, Arlington

Notes

Dr. Luber can be contacted at [email protected] or at the Department of Computer Science and Engineering, The University of Texas at Arlington, 655 West Mitchell St., Arlington, TX 76019-9800.
Drs. Shang and Nasr contributed equally to this article.

Metrics & Citations

Metrics

Altmetrics

Citations

Export citation

Select the format you want to export the citation of this publication.

Cited by

  1. Are Stronger Feature Representations All You Need for Histology Image Search?, NEJM AI, 1, 5, (2024)./doi/full/10.1056/AIe2400314
    Abstract
Loading...

View Options

View options

PDF

View PDF

Full Text

View Full Text

Media

Figures

Other

Tables

Share

Share

CONTENT LINK

Share

Your trusted guide to rigorous evaluation of clinical applications of AI.

Monthly & Annual Options

Subscribe Now