wenty years ago, a board of atmospheric experts published a report that revolutionized the meteorology industry with a single phrase: research to operations (R2O). This term was coined to describe the challenge of transitioning satellite data into operational use, or as it was described, bypassing the “Valley of Death” that swallowed up research before it could see the light of day.
The report asked an important question: What if the industry could build a bridge between research and operations?
Which it did. After defining R2O, the meteorological community saw immense uptick in bridge-building initiatives. The National Oceanic and Atmospheric Administration, for example, developed a new initiative focused on best practices and incentivization programs to reduce the research to operations burden and help research become a reality.
Today, the health care industry is facing an eerily similar chasm.
While completing a master’s degree from Tufts University School of Medicine in Boston, I explored the wealth of challenges posed by the R2O revolution and what learnings from it can be brought into the health care industry. Identifying the challenges was a simpler task: getting innovative research into health care workflows is well documented.
Take the use of artificial intelligence (AI) and machine learning (ML) in medicine. According to one report, fewer than 10% of machine learning models make it into production across all industries. The percentage is even lower in health care, given the additional barriers of security, accessibility, specialization, and regulation. Since 1997, when the FDA first approved an AI/ML-enabled care device, PubMed lists more than 26,000 publications on machine learning, AI, and health care. Yet as I write this there are now just 343 FDA-approved AI/ML enabled care devices (including software services).
The first clear step was to create a common term to discuss the problem. As renowned meteorologist William Hooke eloquently stated in a post for the American Meteorology Society’s “The Front Page” blog, “R2O matters. Simply put, it’s the key to realizing societal benefit from research and development.”
With a common goal in mind, meteorology looked at interoperability solutions: By developing connections between satellites and data systems, the industry was able to develop better real-time reporting. Interoperability will save lives in health care as well, and will help bridge the R2O chasm the industry faces, but bridging isn’t the place to stop.
The key discovery of my work landed here: Interoperability is not enough. Just because a clinician can access novel research does not mean she can use it in caring for her patients. Focusing on artificial intelligence, I identified three determinants to actualize the value and promise of AI: explainable, transparent and actionable. In a nutshell, models must be understood, trusted, and most importantly, useful.
As John D. Halamka, Suchi Saria, and Nigam Shah wrote recently in First Opinion, “to realize the full potential of artificial intelligence (AI) and machine learning (ML) for patients, researchers must foster greater confidence in the accuracy, fairness, and usefulness of clinical AI algorithms.”
Several programs are already at work solving for R2O in health care. A few examples include:
- Bridge2AI, a program funded by the National Institutes of Health’s Common Fund to “propel biomedical research forward”
- MedPerf, an open benchmarking platform for medical artificial intelligence
- Model Cards, a standard developed by my company, Google, to bring consistency to transparency and explainability data best practices
- HealthIT.gov’s Focus Areas, offering opportunities to engage in regulatory change
Moving forward means prioritizing and rewarding work that focuses on truly bringing research into operation — and thus into patients’ lives. R2O means incentivizing researchers to see their work beyond publication. It is developing interoperable standards and putting them to meaningful use. It is working to foster trust in technologies by breaking biases and building with an equitable lens. R2O is working across the care continuum to ensure that all users from all backgrounds can understand, trust, and use the technology.
https://www.statnews.com/2022/03/31/what-health-care-must-learn-from-meteorology-about-the-importance-of-r2o/
