Frank E. Block, Jr.: Will Surveillance Alarm Systems for Opioid-Induced Respiratory Depression Make Things Better or Worse?

AAMI and other organizations have been urging hospitals to address the problem of deaths from opioid-induced respiratory depression by installing surveillance alarm systems for all patients who are receiving opioids. But would such systems actually work? Ideally, one would perform a suitable study to investigate the issue. The objection is that such a study would be expensive and would take a long time.

But let’s use the literature to see if we can determine what would happen with such a system. AH Taenzer et al. reported the experience with their surveillance alarm system at Dartmouth. The system decreased rescue events by 2.2 per 1,000 patient days. Thus, the surveillance system detected one rescue event every 455 patient days. Dartmouth also reported that there were four alarms per patient per day. But there was only one critical event every 455 days. So in 455 days, there was one true positive alarm signal and 1,819 false positive alarm signals, for a false alarm rate of 99.95%! And even if there were one true positive alarm signal every 30 days (instead of every 455 days), with four total alarm signals per day, the rate of false alarms would still be over 99%!

And the problem is that of alarm fatigue, as was addressed in some detail at the 2011 AAMI Medical Device Alarm Summit. Indeed, groups within AAMI are working to decrease the huge number of false alarms that we have today. As Maria Cvach notes in her paper, “Monitor Alarm Fatigue: An Integrative Review”:

“When the alarm is viewed as a ‘nuisance,’ the caregiver may disable, silence, or ignore the warning that is intended to make the environment safer. Rather than creating a safer environment, a large number of nuisance alarms have an opposite effect, resulting in desensitization.”

In fact, one human factors study shows that people ignore the alarms in direct proportion to the perceived percentage of false alarms. In her paper, titled “Fewer but Better Auditory Alarms Will Improve Patient Safety,” Judy Edworthy writes that:

“If an alarm system is perceived to be 90% reliable, then people will respond slightly more than 90% of the time. If a system is perceived to be 10% reliable, then they will respond only 10% of the time. Of course, the 10% of the time that they respond to the system is probably not the 10% of the time that the system is signalling correctly, so effectively the alarm system is rendered almost useless when false alarm rates are high. The practical consequence of this is that alarms which are installed on a ‘better safe than sorry’ basis are likely to make responses to them less—rather than more—reliable.”

 But the hue and cry from AAMI is that we have to “do something”! Yes, but we have to do something that will work! In 2012, The Joint Commission issued a Sentinel Event Alert on “safe use of opioids,” with several excellent steps to increase the safety of opioid use. We should push for widespread implementation of these suggestions in order to decrease the incidence of opioid-induce respiratory depression.

Do the people at AAMI who are working on the respiratory depression issue talk to the people at AAMI who are working on the alarm fatigue issue? If they were talking, they would not be recommending adding a surveillance system with more than 99% false alarms to the myriad false alarms that we have already. The respiratory depression deaths would be replaced by, or more likely augmented by, the alarm fatigue deaths.

Frank E. Block, Jr., MD, is a research professor in the Departments of Physics and Astronomy and a faculty fellow of the Vanderbilt Institute for Integrative Biosystems Research and Education at Vanderbilt University, and research professor of anesthesiology at Vanderbilt University Medical Center.

12 thoughts on “Frank E. Block, Jr.: Will Surveillance Alarm Systems for Opioid-Induced Respiratory Depression Make Things Better or Worse?

  1. I have an article coming out (some day) in the Journal of Clinical Engineering related to alarm fatigue. My perspective is the issue of system complexity. Additional alarms add to the complexity of the alarm fatigue scenario. Although complexity issues have rational trends, the outcomes become multiplied exponentially. Having too many choices of ‘the right thing to do’ is detrimental to patient health.

  2. As a mother who lost a a beautiful 18-yearold daughter to RD in 2010, there are issues with the way some are viewing the practicality of continuous monitoring of CO2. There are many hospitals that have continuous monitoring of CO2 that have had awesome results. St. Francis Health in Indianapolis has had monitoring for six years with zero, yes zero, adverse events. Since high levels of CO2 are one of the first indications that a patient is in distress, yes, it is where the money should be spent. I believe my daughter would still be alive today if she would have been monitored. Let me think: Yep, “one life.”

    • For those of you who are following this thoughtful conversation, note that we have a new AAMI podcast on the topic. I personally just finished listening to it and found the discussion accessible and scientifically rigorous, as well as compelling in addressing every objection. My favorite point was made in the form of a question: if your family member were in the hospital and at risk of respiratory depression, what would you want? This podcast, like all AAMI podcasts, is free. No log in or registration required. Please feel free to share and distribute as you see fit.
      (it’s Episode 7)

      • Almost all of us want maximum resources deployed when it is our loved one that is involved. But is this system thinking? And actually what I would want (I think) is 24 hour one-on-one direct observation, not reliance on a monitor.

  3. The Taenzer paper reports that there was a decrease of 2.2 rescue events per 1000 patient days with the new system. In other words, 2.2 additional rescue events were picked up that were not picked up with the old system of regular nursing checks, SpO2 spot checks by respiratory therapy, in-room (non-networked) monitors, or whatever else may have in use at the time. Many of those same events were probably picked up in the same way even after the new system was installed, and thus would have been picked up with or without the new system.

    Also, the paper reports that the number of ICU transfers was decreased by 2.7 per 1000 patient days. I am assuming that the 2.7 ICU transfers included some or all of the 2.2 rescue events, but for the sake of argument, let’s say that there were 4.9 total events per 1000 patient days picked up by the new system. There are no other improvements in patient OUTCOME reported in the paper.

    Now as I understand, no alarms on any devices were turned off when the new system was installed. Therefore ALL the alarms on the new system were ADDITIONAL new alarms, as they did not replace any old alarms. Thus, the “cost” of picking up the 4.9 additional events was approximately 3,995 additional alarm signals per 1,000 patient days! Besides the 4.9 additional events, the Taenzer study did not find any other changes in patient outcome. Therefore the other 3,995 alarm signals, even if they changed patient CARE (and were thus “actionable” alarms), did not change patient OUTCOME. Thus, there were 3,995 UNNECESSARY alarms, and those 3,995 alarms are indeed false alarms.

    I’m still waiting for the evidence that a surveillance alarm system will make things better and not worse. If anyone has real and accurate data from a properly-designed controlled trial about the percentage of alarms that changed patient outcome vs. the percentage of alarms that did not change patient outcome, then let’s please see those data in the peer-reviewed literature as soon as possible!

  4. On a less emotional and maternal level, the important thing we need to learn about continuous monitoring, is that somebody must be monitoring the monitors. Unless you have a ” closed loop system,” alarm fatigue can be a dangerous problem for a patient in danger. Thank you Jim Welch for speaking up in response to this post. The reason AAMI has devoted so much time and so many resources to the idea of continuous electronic monitoring is because patients are dying without monitors. And, yes, I am not a doctor, but I hope the idea of ” if we can just save one life ” is not a fad. It is the culture of care we are all working toward in 2015.
    Lenore Alexander

  5. Hello Dr. Block:

    It has been a long time.

    I believe you are correct, if we introduce technology and products which have historically been designed and used in acute care environments into the non-acute segments of healthcare, then this alarm fatigue issue will follow. However, there are middleware solutions which can organize and attenuate the alarms, as well as products which have been designed specifically for the non-acute settings. With these non-acute specific products, these systems alarms/alert systems have also been designed for these non acute clinical settings. EarlySense for example has been shown to produce just two-three alarms per eight-hour nursing shift. While in the acute systems business at Siemens, we were actively searching for ways to expand market coverage, and we did it with telemetry. In retrospect, this may be a part of the alarm fatigue problem of today. Products which were designed for almost one on one caregiver to patient care, such as in the OR, PACU, or ICUs, just do not have a good fit in the non-acute side of care. These products have shown to produce hundreds of alarms in an eight-hour shift, thus enhancing the problem of alarm fatigue.

    On the other hand, opioid use, is prescribed everywhere, even in long-term care environments. Today, patients who are being treated in a long-term care facility for Orthopedic Rehab, are given opioids with absolutely no monitoring, other than a four to eight-hour vitals check. These are catastrophic events waiting to happen. Some form of continuous monitoring is better for these patients than none at all.

    One size does not fit all. I believe that the proper technology and alarm/alert systems for the environment is one key to this issue.

    Tim O’Malley
    EarlySense, Inc.

  6. Measuring the performance of monitoring systems in a clinical environment is a difficult task, and we appreciate the attempt to understand the impact of monitoring using the data we published about the Dartmouth-Hitchcock surveillance system. However, the false alarm rate calculation presented here is flawed as it assumes that the only true positive alarms associated with the surveillance system were those that led to rescue events. The intended purpose of surveillance monitoring is to enable early detection of deterioration, not just to signal rescue events (or decrease ICU transfers or the need for opioid reversals, although these are all possible outcomes). When implemented as an integral part of the care process, surveillance system alarms can provide clinicians with sufficient time to implement any number of clinical interventions that actually prevent rescue events (or ICU transfers or opioid reversals, etc.). Such alarms would certainly be considered true positives. Unfortunately, interventions that prevent adverse events from occurring rarely get documented, especially to the degree that would be necessary to accurately calculate system specificity or sensitivity. Nevertheless, estimating a false alarm rate based solely on the occurrence of a subset of adverse events presents a false picture of system performance and its impact on clinicians and patients.

  7. The sentence about the “if we can save just one life ” perspective is one of the most offensive things I have seen in my many years involved with patient safety. William Hyman, do you have children? Did you want to reconsider this response to the article above?

    • I do not wish to reconsider. It is reality. Here is an analogy I have used before: I live in a high-rise building with many older people. There is a risk of heart attack among the population. A life might be saved if there was an ambulance parked in front of the building at all times. But this is true for every other building, each of which would need its own full-time ambulance. This is not affordable or practical.

  8. The Taenzer publication focused on the reduction of ICU transfers to higher cost-of-care areas and not actionable alarm events. The study used escalation of care as a proxy for outcomes. During the 11-month study, there were many reports from nurses where early warning alarms (for example, dropping oxygen saturations) alerted the nurse to respond, implement treatment at bedside, and avoid potential escalation of care. The sensitivity, specificity, and positive predictive values for alarm were not captured in this study. Dartmouth reported in the APSF Newsletter Spring-Summer 2012 (Post Operative Monitoring – The Dartmouth Experience) that the reduction in ICU transfers resulted in a $1.4M annual savings for the study unit. The 4.4 alarms per patient per day, the reduction in ICU transfers, and the resulting economic savings is compelling, at least for one institution, to justify continuous monitoring for patients receiving opiods.

    The Dartmouth study is observational and therefore subject to scientific debate for generalization across all institutions. However, the cost of such studies are not expensive since hospitals already capture patient transfer data.

    Many of the members of the AAMI committee on respiratory depression are also on AAMI committees for alarm reduction as well as standards. The exchange of information between these groups is ever increasing due to the excellent leadership provided by the AAMI staff.

    We (Sotera Wireless) are fully engaged in this topic and related patient safety communities. Our experience has been, and continues to be, that if nuisance alarms are reduced to a minimal level, nursing workflows are optimized, and critical thinking training implemented, then patients, caregivers, and hospitals realize a safer environment. Our alarm rates are consistently below eight alarms per patient per day (all vital signs). When we survey nurses, “What percentage of the alarms require clinical action at bedside to reverse an abnormal vital sign?” we get estimates 75% and above. We understand this is not a scientifically valid protocol, but it does represent the perceived value by nursing end users. The most frequent alarm is low O2 saturation (alarm setting of 85% with a persistence of more than 30 seconds). The most common root causes are dislodgment of the O2 cannula, and undiagnosed OSA. We are in the process of initiating several formal studies.

  9. Thank you for giving as a rational and important discussion of the issue and the proposed “solution.” The idea that alarms can decrease vigilance and responsiveness rather than increase them is not new, and needs to be considered when advocating any alarm-based solution.

    Also missing is any comparative risk assessment. Is the opioid group the most at risk group in the hospital? Given limited resources, is this the most effective place to spend money (if it even was a good idea? Or is this just chasing another fad risk with a “if-we-can-save-just-one-life” perspective? That’s a perspective which ignores limitation of resources and the need to prioritize risk.

    I was disappointed when I learned that AAMI had jumped on this issue without adequate study.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s