Artificial Intelligence: Medical AI in A Real-World Setting is Very Different from a Lab – Google
Google deployed a deep learning system for diabetic retinopathy in Thai clinics.
When Google conducted field trials of its AI system for diabetic retinopathy in Thailand, the results were an eye-opener. The first study of its kind, it evaluated how nurses use an AI system to screen patients for the disease. While lab results were highly accurate, the system encountered a lot of difficulties in real-life testing. (MIT Technology Review)
Thailand’s initiative against diabetic retinopathy
Diabetic retinopathy happens when high blood sugar levels damage blood vessels in the retina and cause loss of vision even blindness. Thailand has targeted to annually screen 60% of its population for the disease.
Google’s AI can take an eye scan as input and identify the presence of diabetic retinopathy in fewer than 10 minutes with higher than 90% accuracy. The system looks within images for clues to the presence of the disease such as blocked or leaky blood vessels.
The question was, how does it perform in a real-life, clinical setting?
To find out, Emma Beede, a UX researcher at Google Health, and her team deployed Google’s deep-learning system at 11 eye clinics in Thailand.
How it worked
In a manual system, the nurses took the eye scans and sent them off to a specialist. The round trip to diagnosis could take as much as 10 weeks.
Google’s 10-minute diagnosis should have been revolutionary. Only it wasn’t.
Over the 11 months of the testing, the researchers found the system failed when presented with a poorer quality eye scan compared to what it was trained on. The reason accounted for nearly 20% of a failure to deliver a result.
The nurses had to repeat the imaging process and sometimes had to request the patient to return another day.
Again, erratic or slow internet connections meant it took very long to upload the scans to the cloud. That kept patients waiting at the clinic.
Based on these observations Google is now reworking its systems and procedures for the diabetic retinopathy algorithm.
“Deploying an AI system by considering a diverse set of perspectives in the design and development process is just one part of introducing new health technology that requires human interaction,” writes Beede. “It’s important to also study and incorporate real-life evaluations in the clinic, and engage meaningfully with clinicians and patients before the technology is widely deployed.”
Latest Alternative Investment News
Hedge Funds Up Amid Volatile Markets. Hedge funds were up nearly two percent in the month of June. This means that the industry faced its third monthly gain in a…
Silver Lake Group Close on AMC Deal Private equity firm, Silver Lake Group, is on the brink of making a deal with AMC Entertainment Holdings. The American movie theater giant…
SoFi becomes one more fintech looking to crossover into mainstream banking after it filed a de novo application with the Office of the Comptroller of the Currency (OCC) for a…
Sony (NYSE: SNE) announced Thursday its deal to buy a minority stake in Epic Games, the private company behind the battle-royale juggernaut of a game known as Fortnite. It is…