EHR safety, a false sense of security, and epistemological problems in the Medical Informatics community

Neil Versel is an author of many articles about healthcare information technology.  He recently published "Patient safety in the balance: Questions mount about EHRs and a wide range of patient safety concerns" (http://www.healthcareitnews.com/news/patient-safety-balance) in Healthcare IT News, August 27, 2014, also appearing in the Sept. 2014 print edition.

His article contained a sidebar on my views "A 'false sense of security'" (http://www.healthcareitnews.com/news/false-sense-security).

I thought my feedback back to him would be of interest, centering on a published comment of the typical hyper-enthusiast variety made by a prominent member of my Medical Informatics professional community.  The view exemplifies a widely-held but erroneous (and decidedly non-scientific) viewpoint in my professional community - and many others, including government -  regarding drawing conclusions in the face of significant evidentiary gaps and impediments regarding harms.

Here are my emails to Mr. Versel.  They speak for themselves, especially the short second supplementary email seen at the bottom of this blog post

-- SS

-------------------------------------------------------------

---

EMAIL #1:

From: Silverstein,Scot
Sent: Friday, September 05, 2014 10:05 AM
To: Neil Versel
Subject: Re:" A 'false sense of security'"

Neil,

Thanks for citing me in your article at http://www.healthcareitnews.com/news/patient-safety-balance, http://www.healthcareitnews.com/news/false-sense-security etc.

Want to call your attention to one of my largest disappointments and frustrations in health IT that evolved over the past decade or two, namely, cheerleading for the technology in the face of commonly understood Western science and epistemology ("the theory of knowledge, especially with regard to its methods, validity, and scope. Epistemology is the investigation of what distinguishes justified belief from opinion").

My concerns are best exemplified by the comment posted at the article site by a prominent member of my Medical Informatics professional community:

"The question is not whether EHRs are safe or not; sometimes they are, and sometimes they are not. The real question is whether they are safer than paper records, and the weight of scientific evidence is that they are."

The actual truth, considering the explicit statements with explicit reasons given by IOM, FDA, ECRI, and others as to why what we know about electronic harms is incomplete ("tip of the iceberg", see below), is that "while the potential for electronic systems to be safer than paper exists, the scientific evidence is inadequate to make any statement about comparative safety on the ground."

The data inadequacy is made far worse by the known and stated systematic impediments to discovery and diffusion of the harms, and a lack of robust studies on ACTUAL harms due to paper-based record keeping, especially and critically regarding optimal and well-staffed paper systems.  (Try a Medline search on that topic!)

The hyper-enthusiasts who favor the theoretical over reality on the ground and who push for speedy national rollout without proper safeguards, surveillance and regulation, are, in fact, killing people. [I.e, whose injuries and deaths at the hands of bad health IT were otherwise preventable had appropriate industry safeguards been in place - ed.]

There is actually nothing to argue, nothing to debate on this issue.

Some material backing up these assertions is below, mostly already covered in my blog-based 11 points.  [As at http://hcrenewal.blogspot.com/2014/04/fda-on-health-it-risk-reckless-or.html - ed.]

Cheers,

Scot

------------------------------

Examples: 

1)  IOM report on HIT safety, 2012 (as I report at http://hcrenewal.blogspot.com/2012/03/doctors-and-ehrs-reframing-modernists-v.html , midway down):

.. While some studies suggest improvements in patient safety can be made, others have found no effect. Instances of health IT–associated harm have been reported. However, little published evidence could be found quantifying the magnitude of the risk.
Several reasons health IT–related safety data are lacking include the absence of measures and a central repository (or linkages among decentralized repositories) to collect, analyze, and act on information related to safety of this technology. Another impediment to gathering safety data is contractual barriers (e.g., nondisclosure, confidentiality clauses) that can prevent users from sharing information about health IT–related adverse events. These barriers limit users’ abilities to share knowledge of risk-prone user interfaces, for instance through screenshots and descriptions of potentially unsafe processes. In addition, some vendors include language in their sales contracts and escape responsibility for errors or defects in their software (i.e., “hold harmless clauses”). The committee believes these types of contractual restrictions limit transparency, which significantly contributes to the gaps in knowledge of health IT–related patient safety risks. These barriers to generating evidence pose unacceptable risks to safety.[IOM (Institute of Medicine). 2012. Health IT and Patient Safety: Building Safer Systems for Better Care (PDF). Washington, DC: The National Academies Press, pg. S-2.]

Also in the IOM report:

… “For example, the number of patients who receive the correct medication in hospitals increases when these hospitals implement well-planned, robust computerized prescribing mechanisms and use barcoding systems. But even in these instances, the ability to generalize the results across the health care system may be limited. For other products— including electronic health records, which are being employed with more and more frequency— some studies find improvements in patient safety, while other studies find no effect.
More worrisome, some case reports suggest that poorly designed health IT can create new hazards in the already complex delivery of care. Although the magnitude of the risk associated with health IT is not known, some examples illustrate the concerns. Dosing errors, failure to detect life-threatening illnesses, and delaying treatment due to poor human–computer interactions or loss of data have led to serious injury and death.”

2)  FDA Internal memo on HIT risk, 2010 (as i report at http://hcrenewal.blogspot.com/2010/08/smoking-gun-internal-fda-memorandum-of.html):

... In summary, the results of this data review suggest significant clinical implications and public safety issues surrounding Health Information Technology. The most commonly reported H-IT safety issues included wrong patient/wrong data, medication administration issues, clinical data loss/miscalculation, and unforeseen software design issues; all of which have varying impact on the patient’s clinical care and outcome, which included 6 death and 43 injuries. The absence of mandatory reporting enforcement of H-IT safety issues limits the number of relevant MDRs and impedes a more comprehensive understanding of the actual problems and implications.

 This is especially true considering the FDA's own noted limitations of their information sources:

Limitations of the MAUDE search and final subset of MDRs include the following:

1. Not all H-IT safety issue MDRs can be captured due to limitations of reporting practices including

... (a) Vast number of H-IT systems that interface with multiple medical devices currently assigned to multiple procodes making it difficult to identify specific procodes for H-IT safety issues;
... (b) Procode assignments are also affected by the ability of the reporter/contractor to correctly identify the event as a H-IT safety issue;
... (c) Correct identification by the reporter of the suspect device brand name is challenged by difficulties discerning the actual H-IT system versus the device it supports.
2. Due to incomplete information in the MDRs, it is difficult to unduplicate similar reports, potentially resulting in a higher number of reports than actual events.
3. Reported death and injury events may only be associated with the reported device but not necessarily attributed to the device.
Memo: H-IT Safety Issues
4 Correct identification by the reporter of the manufacturer name is convoluted by the inability to discern the manufacturer of the actual H-IT system versus the device it supports.
5 The volume of MDR reporting to MAUDE may be impacted by a lack of understanding the reportability of H-IT safety issues and enforcement of such reporting.

3)  Jeff Shuren MD JD, head of FDA CDRH (as I report at http://hcrenewal.blogspot.com/2010/02/fda-on-health-it-adverse-consequences.html):

The Office of the National Coordinator for Health IT held a meeting of the HIT Policy Committee, Adoption/Certification Workgroup on February 25, 2010. The topic was "HIT safety." The agenda, presenters and presentations are available at this link.

At this meeting FDA testimony was given by Jeffrey Shuren, Director of FDA’s Center for Devices and Radiological Health. Dr. Shuren noted several categories of health IT-induced adverse consequences known by FDA. This information was striking:

He wrote:

... In the past two years, we have received 260 reports of HIT-related malfunctions with the potential for patient harm – including 44 reported injuries and 6 reported deaths. Because these reports are purely voluntary, they may represent only the tip of the iceberg in terms of the HIT-related problems that exist.

Even within this limited sample, several serious safety concerns have come to light. The reported adverse events have largely fallen into four major categories: (1) errors of commission, such as accessing the wrong patient’s record or overwriting one patient’s information with another’s; (2) errors of omission or transmission, such as the loss or corruption of vital patient data; (3) errors in data analysis, including medication dosing errors of several orders of magnitude; and (4) incompatibility between multi-vendor software applications and systems, which can lead to any of the above.

4)  ECRI Institute Deep Dive study of health IT safety (as I report at http://hcrenewal.blogspot.com/2013/02/peering-underneath-icebergs-water-level.html):

Participating facilities submitted health IT related events during the nine-week period starting April 16, 2012, and ending June 19, 2012. ECRI Institute PSO pulled additional health IT events that were submitted by facilities during the same nine-week period as part of their routine process of submitting event reports to ECRI Institute PSO’s reporting program. The PSO Deep Dive analysis consisted of 171 health IT-related events submitted by 36 healthcare facilities, primarily hospitals

(There were 8 injuries and 3 possible deaths associated with these 171 IT-related events over just 9 weeks, suggesting a damn serious problem way ahead of paper errors.)

5)  Statement by Medical Director of ECRI, same link as above:

Karen Zimmer, MD, medical director of the institute, says the reports of so many types of errors and harm got the staff's attention in part because the program captured so many serious errors within just a nine-week project last spring.  The volume of errors in the voluntary reports was she says, "an awareness raiser."

"If we're seeing this much under a voluntary reporting program, we know this is just the tip of the iceberg; we know these events are very much underreported."

6)  ECRI's continuing concerns based on continuous data submitted by their PSO member hospitals that health IT systems are "one of the top ten (and in fact in 2014, risk #1) technology risks in healthcare" (as I report at http://hcrenewal.blogspot.com/2014/04/in-ecri-institutes-new-2014-top-10.html)

This is a short list, but you get the idea about making any statements about comparative risks of paper v. electronic.

-----------------------------------------------------------------------------------------
 

---

EMAIL #2: 

 From: Silverstein,Scot 
Sent: Friday, September 05, 2014 10:08 AM
To: Neil Versel
Subject: RE: Re:" A 'false sense of security'"

Not to mention that the "real" question isNOT "whether they [electronic systems] are safer than paper records."

The real question is:  is it ethical, based on the current clinical reality and incomplete state of knowledge, to push this technology on patients?

I also could add to that question:  "...especially without informed consent?"

Scot

-----------------------------------------------------------------------------------------

-- SS