Researchers at Microsoft, Providence Health System, and the University of Washington have developed a new artificial intelligence model for diagnosing cancer based on analysis of more than a billion images of tissue samples from more than 30,000 patients.
The open-access model, known as Prov-GigaPath, is described in research published today by the journal Nature and is already being used in clinical applications.
“The rich data in pathology slides can, through AI tools like Prov-GigaPath, uncover novel relationships and insights that go beyond what the human eye can discern,” study co-author Carlo Bifulco, chief medical officer of Providence Genomics, said in a statement. “Recognizing the potential of this model to significantly advance cancer research and diagnostics, we felt strongly about making it widely available to benefit patients globally. It’s an honor to be part of this groundbreaking work.”
The effort to develop Prov-GigaPath used AI tools to identify patterns in 1.3 billion pathology image tiles obtained from 171,189 digital whole-slides provided by Providence. The researchers say this represented the largest pre-training effort to date with whole-slide modeling, drawing upon a database five to 10 times larger than other datasets such as The Cancer Genome Atlas.
Whole-slide imaging, which transforms a microscope slide of tumor tissue into a high-resolution digital image, has become a widely used tool for digital pathology. However, one standard gigapixel slide is thousands of times larger than typical natural images, posing a challenge for conventional computer vision programs.
Microsoft’s GigaPath platform employed a set of AI-based strategies to break up the large-scale images into more manageable 256-by-256-pixel tiles and look for patterns associated with a wide range of cancer subtypes.
Another step in the process involved fine-tuning the Prov-GigaPath model by associating the image data with real-world pathology reports. The researchers used OpenAI’s GPT-3.5 generative AI platform to “clean” the reports, removing information that was irrelevant to cancer diagnosis.
To assess Prov-GigaPath’s performance, the researchers established a digital pathology benchmark that included nine cancer subtyping tasks and 17 analytical tasks.
“Prov-GigaPath attains state-of-the-art performance on 25 out of 26 tasks, with significant improvement over the second-best model on 18 tasks,” two of the study’s authors at Microsoft, Hoifung Poon and Naoto Usuyama, said in a blog posting about the research.
Poon and Usuyama said the AI-assisted approach to digital pathology “opens new possibilities to advance patient care and accelerate clinical discovery” but added that much more work still remains.
“Most importantly, we have yet to explore the impact of GigaPath and whole-slide pretraining in many key precision health tasks such as modeling tumor microenvironment and predicting treatment response,” they wrote.
The open-access nature of Prov-GigaPath represents a significant decision by the research partners. Making the model freely available allows cancer researchers and diagnostic labs worldwide to benefit from the technology without licensing barriers.
The 1.3 billion image tiles analyzed during development represent an unprecedented scale for medical AI training. This massive dataset allows the model to learn subtle patterns that might be missed with smaller training sets.
The 171,189 digital whole-slides from Providence provide diverse examples of cancer presentations across different patient populations, tumor types, and disease stages. This diversity helps the model generalize to new cases it encounters in clinical practice.
The five to 10 times larger database compared to The Cancer Genome Atlas demonstrates the advantage of partnering with a major health system. Providence’s extensive pathology archives provided the raw material for this breakthrough.
Whole-slide imaging has transformed pathology from a microscope-based discipline to a digital field. Pathologists can now review cases remotely, consult with colleagues worldwide, and apply computational analysis impossible with physical slides.
The gigapixel size of whole-slide images creates computational challenges. A single slide can contain billions of pixels, far exceeding the capacity of standard image analysis techniques designed for photographs or medical scans.
The 256-by-256-pixel tile strategy breaks these massive images into pieces computers can process while maintaining the spatial relationships between tiles. The AI learns to analyze individual tiles and understand how they relate to neighboring regions.
The pattern recognition across cancer subtypes means the model learns features common to many cancers as well as characteristics specific to particular types. This broad training improves performance on both common and rare cancer varieties.
The fine-tuning process using real-world pathology reports teaches the model to connect visual patterns in tissue samples with diagnostic conclusions. This supervised learning phase grounds the model in actual clinical practice.
The use of GPT-3.5 to clean pathology reports demonstrates how different AI technologies can work together. The language model removes extraneous information, allowing the vision model to focus on diagnostically relevant patterns.
The digital pathology benchmark with nine cancer subtyping tasks and 17 analytical tasks provides rigorous testing across diverse diagnostic challenges. This comprehensive evaluation gives confidence in the model’s capabilities.
The 25 out of 26 tasks where Prov-GigaPath achieved state-of-the-art performance demonstrates its superiority over previous approaches. The 18 tasks with significant improvement over the second-best model show substantial advances rather than marginal gains.
