William Hyman: The Little-Discussed Device Learning Curve

A reasonable expectation in the adoption of new medical technology is that the direct users will require training and experience in achieving a reasonable level of comfort and efficacy. A corollary of this expectation is that during the initial period, there will be at least a greater challenge in using the equipment and a possible higher complication rate. Some of these complications associated with the learning period may be benign, but others may result in adverse patient events.

The service function has similar learning issues. It will take time for those providing service to become adept at what they do. In the interim, service may take longer, and there may be a risk of equipment damage or less-than-adequate status when the device is returned to use if there are particularly challenging steps. Adequate servicer qualifications are also relevant to maintenance requirements from The Joint Commission and Centers for Medicare & Medicaid Services.

The idea that the quality of at least initial performance is time dependent is sometimes reflected in the idea of the “learning curve,” which suggests that something is being plotted vs. time. This might be the number of complications, in which case the resultant curve is preferably declining with time. Or if complication-free results were plotted, the curve would preferably rise with time. In fact, it would be best if this curve rose rapidly with time. It is a curiosity in this regard that the expression a “steep learning curve” usually means that the procedure or device is difficult to learn or become proficient at. It is not clear what is being plotted against what to represent challenging learning as a “steep” curve, and it may be that this verbal expression does not have a graphic or mathematical equivalent.

As clinical users of medical technology move from novice to proficient, the patient experience presumably moves from higher to lower risk, i.e., early patients would be expected to be at greater risk of complications as compared to later patients who are out on the flatter part of a proficiency vs time curve. A separate issue is the level at which individual users plateau. Despite the general expectation of improvement over time, there is little research that determines the number of patients or procedures in a particular situation required to obtain a more-or-less steady-state proficiency. This raises a related question of informed consent: Do patients need to be told explicitly that they are in the early group being treated with a new technology or one that is new to their provider? This is quite different from the informed consent required for a clearly identified experiment (i.e., clinical trial). This question is related to the recent move by the U.S. Food and Drug Administration to allow some Class III devices to move from premarket data collection to postmarket data collection. Premarket data collection involves, of necessity, a registered clinical trial with Institutional Review Board (IRB) approval and associated informed consent that the patient is participating in a study. Will postmarket data collection require the same?

On a personal basis, do you want to be the first person treated with a technology that is newly in the hands of your provider? The fifth? The 10th? Do you expect to be told that you are in the pioneering group? Is being a pioneer something that appeals to you? Of course, your answer might depend on the nature of the medical situation and the availability of alternate treatments or providers.

My family once faced this exact question. A family member was given a fairly significant surgical treatment recommendation. We asked for a referral for a second opinion, which we received at a major academic medical center. The recommendation there was for a laser-based treatment that would be far less traumatic. The patient accepted this second approach, and the original doctor was told the patient would not be back because of the availability elsewhere of the laser treatment. The provider said (and I am not making this up), “I’m getting that kind of laser next week.” This was not convincing to us, and we went with the more experienced user, although this was admittedly not based on any data other than our expectation that experience matters. Our decision left someone else to be the first patient of the new laser owner and therefore the first data point on that doctor’s learning curve. This is perhaps an ethical dilemma: Can I be committed to the introduction of better medical technology and the need for learning, yet not be willing to be the one learned on?

This dilemma can be partially addressed by mandatory and transparent training regimens in which users, as appropriate to the technology, first have non-patient training, then watch a number of live patient events, and finally are watched and mentored for a second series in which they take the lead. They might even read the instruction manual, a practice which is not always followed. Only then could they be on their own. This can be a time-consuming process and is generally not mandatory, although hospital credentialing may set such standards as may state government in limited instances. The manufacturer’s role in assuring skill level is even less well established except in the relatively few instances in which the FDA has mandated training as a condition of sale. A manufacturer may also have information about individual users results that are not otherwise available. Should a manufacturer continue selling a product to a user when the manufacturer knows that the user’s complication rate is above average?

Given the uncertainty of adequate training and experience, an appropriate patient question might be “How were you trained to do this?” along with “How many have you done?” An equally important question: “How have your results been and how does this compare to other reported results?” In the case of some devices such as implants, this might be followed with the question, “Why do you use brand X instead of any of the others?”

William Hyman, ScD, is professor emeritus of biomedical engineering at Texas A&M University. He now lives in New York where he is adjunct professor of biomedical engineering at The Cooper Union.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s