Things & Thinks XVIII
Before we begin, wishes to all readers for a happy 2021! This year will continue to be an important year for healthcare in general and more specifically for digital health. I will continue to bring these news through Things & Thinks, with my usual curation of news and then trying to piece together the disparate elements.
If you missed it, I wrote two summaries at the end of last year, one for my 2020 writing and reading and another one about all the prior 2020 Things & Thinks issues; I will be happy to hear your thoughts.
In this eighteenth issue of the newsletter, I bring a case study about ‘algorithmic’ decision making in healthcare and its actual rollout in the field, followed by a fantastic guide that will help any digitization project. I follow these two with the usual dose of digital health news, ending with Chart of the Fortnight and Tweet of the Fortnight.
Case study-How not to do algorithms
Almost at the same time I was reading about the USDS playbook, I came across the story about how Stanford Med messed up their COVID-19 vaccination program using an ill-defined algorithm. Here’s the background info and what seems to have gone wrong (sources 1, 2, and 3)-
- We all know by now that because of limited supplies of #COVID19 vaccine, every government is rationing the rollout and each one is making recommendations about the order in which people should get vaccinated.
- At the Stanford Medical Center, only 7 resident physicians out of 1300 were shown to be prioritized for the first 5000 doses; when they saw who else had made the list, including administrators and doctors seeing patients remotely from home.
- So what was the rationale? What has been termed by media as ‘algorithm’ is not actually a sophisticated machine-learning algorithm, but a rule-driven formula that takes up ‘risk factors’, assigns ‘weightages’ and provides a ‘score’. The Stanford formula, leaked by protesting residents, appeared to consider three risk factors-“employee-based variables,” which have to do with age; “job-based variables”; and guidelines from the California Department of Public Health. For each category, staff received a certain number of points, with a total possible score of 3.48. Presumably, the higher the score, the higher the person’s priority in line.
- As with every problem blamed on technology rather than human bias, it was apparent immediately that the calculations assumed a lot of things with negligence of ground reality. The employee variable score automatically gives high scores to below-25/above-65 age group, deprioritizing the middle group (to which most front-line residents belong.) Residents also lost points because of their ‘location’, as most residents rotate between departments.
- After on-site protests, social media backlash and reviews, hospital authorities apologized and claimed to update their ‘algorithm’.
Whitney R. Robinson (@WhitneyEpi) summarized the situation really well-
Applying algorithms (like one that prioritizes vaccination) to a new context can give really biased results if you don’t understand the mechanisms by which the algorithm worked in your original population
Digitization Resource-United States Digital Services Playbook
I came across the United States Digital Services Playbook while browsing and was really impressed by both the USDS value sets as well as the content of the playbook.
According to their website, USDS is a group of technologists from diverse backgrounds working across the federal government to transform critical services for the people. These specialists join for tours of civic service to create a steady influx of fresh perspectives. Their values themselves look like something everyone in the innovation business should adopt-
- Hire and empower great people
- Find the truth. Tell the truth.
- Optimize for results, not optics.
- Go where the work is.
- Create momentum.
- Design with users, not for them.
With these values backing their work, they have released a great resource for digitization projects for US government agencies through ‘Digital Services Playbook’. Once again, this set of 13 plays can work for any start-up, large organization, government agency who want to work on digitization projects.
Of the 13 listed plays, I felt that the following were more important and urgent, especially in low resource settings like India-
- PLAY 1 Understand what people need: From design, prototyping and deployment, each step must include ‘real people’
- PLAY 2 Address the whole experience, from start to finish: Think about the whole journey and remove siloes- PLAY 2 Address the whole experience, from start to finish-we need to understand the different ways people will interact with our services, including the actions they take online, through a mobile application, on a phone, or in person. Every encounter — whether it’s online or offline — should move the user closer towards their goal.
- PLAY 8 Choose a modern technology stack: this one is particularly pragmatic-digital services teams should consider using open source, cloud-based, and commodity solutions across the technology stack, because of their widespread adoption and support by successful consumer and enterprise technology companies in the private sector.
- PLAY 12 Use data to drive decisions: At every stage of a project, we should measure how well our service is working for our users. This includes measuring how well a system performs and how people are interacting with it in real-time.
- PLAY 13 Default to open: By building services more openly and publishing open data, we simplify the public’s access to government services and information, allow the public to contribute easily, and enable reuse by entrepreneurs, nonprofits, other agencies, and the public. Once again, written here for government agencies, this play should apply more universally.
Overall, I think this is a great resource for everyone working on innovation projects.
Digital Healthcare news-Cybersecurity, BigTech, Digital health in India
One of the most neglected healthcare topic continues to be cybersecurity. Throughout December, there were reports about attacks across various levels in healthcare- European Medicines Agency in Netherlands was hacked, most probably hacking into BioNTech’s covid vaccine regulatory submission; CyberMDX, a cybersecurity services firm, reported cybersecurity vulnerability in GE’s medical imaging devices, that might lead to exploitation of sensitive patient data and/or allowing the attackers to impact availability of the system. Since Nov 2020, Check Point reported a 45% increase in cyberattacks targeting healthcare organizations globally.
On the big tech in healthcare front, Apple received new 510(K) for an updated version of its ECG feature, allowing the tool to add a category called A-Fib with high heart rate. Google launched Google Health Studies app, similar to Apple’s Research app, supposed to help streamline study recruitment for consumers and shows them how survey and sensor data are being employed for health research. According to Google, the app will use unique secure aggregation tool called federated analytics, which includes querying subsets of phones and then taking those different queries and combining them together; By the time the aggregated statistics are actually brought together and surfaced to the researcher or surfaced for anyone to see, there is almost no possibility of being able to go back and see individualized data. The individualized data is never leaving the device. Finally, towards the end of 2020, Philips, earlier well-known consumer electronics conglomerate and now a healthcare focused company, bought BioTelemetry for $2.8B. BioTelemetry is a cardiac diagnostics and monitoring firm.
India’s vaccination program continued to raise more questions than it answered. Two vaccines are now approved, although under unclear conditions not present in the current drug/vaccine regulations and even caught the vaccine manufacturers into doubts. One of the vaccines hasn’t yet completed its Phase III trial. Its vaccine digitization program also has raised several questions including fake apps/websites. India also released final version of Health Data Management Plan under its National Digital Health Mission.
Tidbits
- Healthcare Chart of the Fortnight: As COVID vaccinations have started around the world, it is interesting to see how various countries are rolling out their vaccination programs. The below chart shows vaccination doses per 100 people…
- Healthcare Tweet of the Fortnight: This tweet about new year resolutions, healthy eating and nature of health habits was a bit too close to reality for many!
