Digital Health & Wearables: Tracking Challenges & Measuring Successes
Introduction
Metrics is a word that you would often hear if you have worked in any business setting. Measuring metrics and tracking progress against such metrics is a favorite and enduring management tool. However such enthusiasm for measuring minutiae of every process also usually leads to several ‘gaming the metrics’ initiatives, not only to project superior performance but also to provide false assurance to consumers, partners and sometimes even regulators. Charles Goodhart, a British economist, came up with the idea and it became known as his ‘law’ subsequently! Jono Hey, the brilliant Sketchplanations artist, sketches this beautifully…
Healthcare ecosystem is not new to such measurements and trackings and there are of course specific ways in which such healthcare metrics are gamed routinely. In 2018, researchers observed that ‘readmission rate 30 days after hospital discharge’ was one such example. When regulators tried to enforce this measure as a performance metric, hospitals simply delayed readmissions to day 31 after hospital discharge or even discouraged readmission altogether. Perversely, this resulted in a higher mortality rate (which incidentally also results in a lower readmission rate) and a lower standard of care, especially for patients with chronic diseases.
Wearables, wearables everywhere…
Why are we discussing Goodhart’s Law in an article about digital health and wearables? After reviewing the research and commentaries about the progress we have made so far about using wearables in healthcare, I think we may have a risk of falling into what I would call ‘the Goodhart trap’.
First let’s take a look at the current status of wearables in healthcare.
There is a lot of attention being given to wearables and sensors especially considering the ‘virtualization’ of patient care and medical research through the pandemic years. In 2021 alone, about 100 clinical trials were published that had a wearable component included. A look at registrations in clinicaltrials.gov shows a similar trend-
The wearable devices are themselves becoming more and more sophisticated, with several additional features being added at a faster pace. Several such wearable devices also received regulatory clearance/approval-
The chart above shows an important aspect of the wearables use- based on the intended use and claim being made about the wearable, regulators treat them differently. Regulators around the world classify medical devices including wearables as Class I (low risk) to Class III (Class IIa and IIb in the EMA). Most of the current wearables in the market are consumer-grade and fall under the Class I device definition; however there are some others who have sought and received a higher class approval, thus making them fit-for-medical usage. Examples include Omron’s HeartGuide (Class II) and Zoll Medical’s LifeVest (Class III).
Using wearables has been projected to bring in several benefits to patients (faster results, point-of-care usage, instant feedback), clinicians (titrating to individualized treatment, longitudinal view of health data) and health systems (potential cost savings). 320 million consumer health and wellness wearables are predicted to ship worldwide next year, with that number projected to surge to 440 million wearables by 2024, according to Deloitte Global’s 2022 Technology, Media, and Telecommunications Predictions.
A lot is still to be done
However, through 2021 and before, researchers, clinicians and other healthcare system players have also pointed out several checkboxes that would need to be ticked before we start to see wearable usage bring in these benefits, and this needs to happen at each step of the wearable data journey. American Society of Clinical Oncology Educational Book summarizes these issues well-
- Data collection: Consent requirements, local vs cloud storage, internet connectivity, tools to integrate data
- Data storage: Central storage requirements and burden, data transfer
- Data management: Access control and security, System updates
- Data Interpretation: Geographical differences in commonly assigned tags such as time and location, data gaps, continuous vs summary statistics
Data ownership and regulations is also an ever-evolving field for wearables. In the US, for wearables in the consumer-grade category, HIPAA coverage is not mandated; this despite likely containing health‐related information, data can be shared in a deidentified, aggregate manner. However EU GDPR does not differentiate between medical devices and consumer wearables within a medical context and hence requires consent for reuse.
Several user-level questions complicate the issues further- patients, healthcare providers and support staff all would need orientation and detailed training for how to use and not misinterpret results from the use of wearable. Access and connectivity may further deepen the existing healthcare inequities.
Lastly, perhaps the most important challenge is integrating wearable data with the rest of the user/patient records. In a 2020 Deloitte survey of U.S. physicians, providers said that in order for them to start using a new technology, that technology must increase efficiency and be integrated into their electronic health record (EHR) system. Yet only 10% of the physicians surveyed said they had integrated data from wearables into their EHRs.
The current state of such integration is non-existent or patchy at best. In absence of such integration, any interpretation and clinical guidance based on individual data points coming out of wearable can be useless, overburden the already stretched healthcare systems with false alarms or even lead to further harm to patients.
The way forward is getting clearer
Smuck et al presented a fantastic overview of factors that can lead to successful implementation of wearables -
They recommend starting with a focus on a specific disease area and problem and lead us right upto payment/reimbursement methodology. Some key highlights of their suggestions-
- Identifying clinician, patient and system specific problems that were limiting their ability to effectively manage patients with a particular health issue
- Not forgetting that the wearable is only one part of a larger digital health program
- Incorporating wearable into specialized integrated digital care delivery systems, involving health coaching, automated appointment reminders, disease-focused digital education, and automated medication dosing with human assistance when needed; further integration of wearable data into existing health data records
- Shifting the burden of support and technological knowledge from clinicians to the tech support team
- Inclusion of health support teams to tackle health-focused problems such as lifestyle management, medication compliance, nutrition, and health education
Overcoming Goodhart’s law
And so we return back to Goodhart’ law; if we continue to develop wearables in a fragmented manner, not considering the myriad other connects this data will have with the rest of user data, we will make it into a target that will be useless.
Becoming aware of these current challenges for wearable implementation and avoiding over-reliance on a single metric, esp in a complex, multi-connected disease process can be the best way to avoid the Goodhart trap.
As David Manheim writes eloquently in his blog, measurement finesses complexity, which is frequently irreducible. So faulty intuition, untrusted partners, and complex systems can be understood via intuitive, trustworthy, simple metrics. He warns us-
Measurement sometimes becomes a substitute for good judgement, a way to cover-your-rear, and an excuse for doing fun math and coding instead of dealing with messy and hard to understand human interactions!
For us to benefit from the advantages that wearable technology offers, we have to be diligent in making sure that we use this technology and the metrics it provide within a well-integrated healthcare system trying to answer a focussed challenge.
