Butterfly Network’s Teleguidance feature on an iPhone, with little arrows that direct using AR. A doctor can be somewhere else.
Butterfly Network
For the most up-to-date news and information about the coronavirus pandemic, visit the WHO website.
My wife fell down the stairs over a month ago and we didn’t know if her foot was broken or not. We didn’t go to the doctor to find out. We’re not the only ones skipping in-person doctor visits. So when I saw Butterfly Network’s Zoom demo of their handheld scanner — and how a doctor could look for possible fractures from afar — I thought: When can our house get one of these?
Butterfly Network’s handheld ultrasound device, Butterfly IQ, has been available since late 2018. Connected to a phone or tablet, it can act like a little portable medical tool. But it’s designated for “doctors or qualified practitioners only.” I saw a demo last year at Apple’s WWDC and I’ve thought about it ever since. What grabbed me was the device’s promise of a future of remote patient assistance, using pop-up 3D instructions that could guide a patient to use the device properly, too.
Recent FDA changes allow medical imaging tools to be used remotely by doctors in emergency situations. This means Butterfly Network’s tech could be used at home now, using remote AR-assistance software that taps into Apple’s ARKit to render 3D directions as a person uses it.
I see the little hair trimmer-sized scanner point over someone’s body, while in another window, I see the pulsing echoes of the ultrasound. I also see arrows and markers, which can be placed there on a patient’s iPhone or iPad to remotely guide where to look, or to mark up areas of interest. Butterfly Networks’ VP of Product, Matt de Jonge, thinks of it like a “magic mirror” that helps someone orient the scanner and follow a doctor’s live guidance at the same time.
The Butterfly IQ’s capabilities include looking at pregnancies, possible broken bones, or even looking at lung conditions: The company is currently using the tech to identify possible COVID-19 symptoms in the lungs of patients being examined remotely.
The TeleGuidance feature works on more recent iPhones and iPads (iPhone XR and later, iPad Mini 5th gen and more recent Airs and Pros) that have the power to handle the AR and also room-scanning machine learning guidance needed to track the ultrasound device’s position in relation to a human body. The handheld scanner also works with some Android phones, but not with the AR guidance yet. Butterfly Network’s de Jonge sees more advanced phones with better computer vision and room scanning (the new iPad Pro’s Lidar, or improving software efforts) as helping more down the road.
Butterfly Network is careful to make clear that right now, the FDA permission is temporary for the duration of the coronavirus emergency. Per the company, “We are working closely with FDA to bring safe, effective, remote-guided ultrasound imaging to the medical and patient community. We are grateful for the opportunity to advance clinical care during this challenging time.”
The Expert View, where a doctor can send messages to guide the person using the ultrasound equipment and monitor results.
Butterfly Network
I’ve used AR headsets like the Microsoft Hololens that have explored remote assistance and guidance via AR and Google Maps uses floating arrows to do the same. This is the first time I’ve seen this for an essential home need like telemedicine.
The ultrasound hardware is, to be clear, expensive: $2000 plus subscription services, although the company says that doctors could prescribe them, or loan them. There’s a possibility of insurance coverage. That’s why I think it’ll be some time before I get one of these in my own home.
The idea of AR assistance in general, though, is something that could keep being useful in all sorts of ways. A technician guiding you through home repair, or technical support. A contractor trying to understand your home and offer advice. Doctors. This type of home assistance isn’t really here much yet, but it looks like a promising next step, maybe even after lockdown.
The information contained in this article is for educational and informational purposes only and is not intended as health or medical advice. Always consult a physician or other qualified health provider regarding any questions you may have about a medical condition or health objectives.
