We like to pretend we know how to measure risks and benefits and then compare them. While risk is perhaps a little more developed in terms of the now-classic combination of severity and frequency (likelihood), benefit has no corresponding rubric. But even if it did, we still don’t know how to determine how much risk is worth how much benefit.
Things get even more complicated when comparing two different options with different risks and benefits. The Food and Drug Administration (FDA) addressed such a comparison in a guidance document last September. This guidance replaces the draft that was released more than four years ago.
We might like to think that “me too” devices have the same risks and benefits as their 510(k) predicates, or that newer versions (that are still substantially equivalent) have lower risk and the same or greater benefit. However, the guidance document raises the question of different parts of the risk/benefit spectrum in which a device has either higher risks and higher benefits or lower risks and lower benefits. Note that higher risks and the same or lower benefits is not an attractive argument for a new device and it is one that is not likely to be made. Lower or the same risk and higher or the same benefits is attractive and not controversial.
A comparison that involves higher risk and higher benefit is an interesting and more challenging place to be. We certainly hope that new devices will bring greater benefits, and such claims are often made. But since we don’t know how to compare risks and benefits for a single device, we also don’t know how to make such a comparison between devices. High risk/high benefit devices might also enter the legal realm of “unavoidably unsafe,” although this argument is not seen prior to litigation.
A situation that is rarely addressed is when there are both lower risks and lower benefits. This might be the case in a simplified device that does less than another device, but does it with less risk. Such lower risks could arise from its technical capabilities or from ease of use, although from the human factors perspective ease of use is also a technical issue. In a different scenario, there could be a device that offers a treatment modality that might not be quite as good as an alternative but that has fewer side effects. It would be an interesting marketing challenge, and one which I don’t recall ever seeing, to promote a technology as “not as good but safer.”
These considerations could also involve the cost of the technology, with lower cost being a benefit. While cost is not addressed in the FDA guidance, we all know that we operate in an environment of fiscal constraints, although the significance of restraint in device purchasing in terms of the total institutional budget is not often discussed.
I recently saw this cost issue addressed in terms of the capability of various monitors. It was argued that basic, lower cost monitors might provide all the capability some patients are likely to need, while higher cost, more advanced monitors might be needed by other patients. Furthermore, some patients might do fine with no monitoring. If cost was not a factor, you might buy the high-capability monitors and use them on all patients, putting aside for now the questions of complexity and user skills. Unfortunately, this would be an expensive proposition and the money might be better spent elsewhere.
A mix of monitors is a tempting solution but requires possibly critical decision making on how many are needed and which patients get which monitor. This can be further challenging if we are forced to look back and justify why a patient who had an adverse event was on the low-end monitor when the high-end monitor might have saved them.
These are difficult questions. Long ago, at an AAMI session that I was moderating, a plaintiff lawyer was asked for the “one worst thing” in device selection and management. He answered that it was not knowing why you did what you did, and correspondingly, not being able to explain it.
We live every day with personal and professional choices that involve risks and benefits. Sometimes we deal with these rationally, sometimes irrationally, and sometimes with no thought at all. At least professionally, we must aspire to shift the balance toward rationality.
William Hyman, ScD, is an adjunct professor of biomedical engineering at The Cooper Union and a consultant in New York. He is also a professor emeritus of biomedical engineering at Texas A&M University.