Usability testing of instructions for use (IFU) and labels is a requirement for medical devices and pharmaceuticals. You will not receive approval from the FDA or other governing bodies without objective evidence of usability. This article discusses what you need to know about usability testing requirements.

Why is the FDA focusing on the readability and usability of IFUs and labels?

You and I, and most of the people we know, live in an educated bubble. And for many, it is a bubble of advanced education and significant professional expertise. We deal with the highest level of professionals in our respective fields. Outside of our bubble, 45 million people in this country are functionally illiterate and read below a fifth grade level. And according to the National Assessment of Adult Literacy, approximately 50% of Americans read so poorly that they cannot read a prescription drug label.¹

However, educated laypeople and healthcare professionals also use IFUs and labels. Does that mean that readability and usability are unnecessary concerns for these groups? The answer, of course, is that well-written and usable instructions are critical for all users.

For example, the formative testing of the IFU for a device for OR anesthesiologists identified some needed changes. The reading level of the document was not a problem, but the results showed that edits needed to be made to improve the description of the device, increase the font size, improve images, clarify procedure instructions, and add instructions on the use of a stimulation clip.

Scientists involved in the formative testing of an in vitro diagnostic device had to remove a strip of foil covering a well that contained reagents. Every one of them asked what they were supposed to do with the strip: “Do I throw this away? Do I need it later to cover the well again?” These are perfectly reasonable questions and obviously, they needed to be addressed in the IFU.

The ability to read something quickly and immediately understand it has very little to do with IQ or educational level, as can be seen in the two previous examples. The more the reader has to think about what is being said, the more an opportunity for error is introduced into the process. I’ve never yet had a user say, “I really hate those instructions because they are much too easy to understand.”

Readability Level Is an Important Requirement

First, you should analyze the readability level of your document. Readability refers to how easy a document is to read and understand. It is typically shown as the U. S. grade level in school that would have to be completed in order to read and understand the text. For example, a grade level of 9.7 means the reader must have completed the ninth grade, seventh month of school. The FDA recommends a maximum of eighth-grade reading level for lay users, and preferably sixth to eighth grade.²

There are a number of formulas for calculating readability, most of which are available for free and will do all the work for you. Microsoft Word contains the Flesch-Kincaid grade level analysis and the Flesch Reading Ease calculation. You want to aim for a low grade-level score and a high Reading Ease score. Simply open your document in Word, click Tools in the menu bar and (depending on your version of Word and what operating system you’re using) look for Spelling and Grammar and then check the Options box and look for  “Show readability statistics.”

There are also a number of usability calculators available for free online. Online-Utility.org automatically calculates four different indexes.

What Kind of Usability Testing Must Be Conducted?^3,4

Usability testing is an iterative process. Although it may seem to be an additional expense, it more than pays for itself. There is measurable proof that it reduces development time and costs, and increases user satisfaction.⁵

Formative testing is done during development. The point of formative testing is to identify problems if they exist and gain insights into how to fix them. Formative testing can be done with a working draft of the document. The formative report should contain a full description of the test process and any questions users must answer about the instructions or tasks to be performed, the selection process for users, summary and discussion of users’ answers, problems identified, and revisions made to the relevant documents, as well as examples of the original and revised documents.

Summative testing is done at the end of the development process; it is not the place to discover problems. Its purpose is to provide validation of your claim that the IFU and label are written at the appropriate grade level and are easy to read and understand. It is a required part of your submission. Testing is done to determine the risk associated with any inability to read or understand the text. Testing must be done with a final mock-up of the IFU or label as the user will see it. The final report should include all of the items mentioned in the formative test process as well as a risk analysis.

Remember to include the results of the testing in your technical files.

How Do You Select Participants?

Who are the users of the device? How many different types of people will interact with it? As an example, think about a self-injection device that is intended to be used at home. You would have three different user groups:

• People who are injecting themselves
• Caregivers
• Healthcare professionals such as nurses or LVNs (Licensed Vocational Nurse)

The groups could be divided into users who have experience giving shots and those who are beginners. You would have to establish a working definition of experienced vs. beginners, but you gain different insights from each group. Experienced users conceptually understand how the device is supposed to work but may presume it works exactly like other devices they’ve used and, consequently, use it incorrectly. Beginners provide insights into the learning curve of using an unfamiliar device. Both types of insights help formulate the contents of written instructions.

Identify the user characteristics you want (or don’t want). In addition to possible major user groups as discussed, consider educational level, experience, age, gender, and any other characteristic that might have an impact on a user’s ability to read and understand instructions. Where do you find appropriate users? Once you’ve identified the characteristics you want (or don’t want), there are agencies that will recruit them for you for a fee. Consider self-help groups, patient support groups, community centers, and parent and child groups as sources of participants. Participants are reimbursed for their time and possibly travel. Remember the Physician Payment Sunshine Act proscriptions about payments to physicians and consider using a third-party to do the testing and payments. In fact, if you don’t have in-house human factors experts, it’s a reasonable goal to use outside consultants to do all of the usability testing and reporting.

How Many People Are Needed for Each Type of Testing?

Formative testing can be done with five to seven people from each user group. For the example of the self-injection device above, you might have a total of six people in each major category (self-injectors, caregivers, healthcare professionals) and then subdivide each group into three experienced users and three beginners.

Summative testing requires at least 15 people in each user category. For the injection device example, I suggest 16 people in each user category so that when you subdivide each category into experienced and beginning users, you have an equal number in each group. It just makes the subsequent analyses of results easier.

A Must-Read Guidance

U.S. Department of Health and Human Service, Food and Drug Administration, Center for Devices and Radiological Health (CDRH). (April 2001). Guidance on Medical Device Labeling; Final Guidance for Industry and FDA Reviewers. Retrieved from www.fda.gov/downloads/medicaldevices/deviceregulationandguidance/guidancedocuments/ucm070801.pdf

References

1. National Center for Education Statistics, The Health Literacy of America’s Adults: Results from the 2003 National Assessment of Adult Literacy. Retrieved from https://nces.ed.gov/pubsearch/pubsinfo.asp?pubid=2006483
2. U.S. Department of Health and Human Service, Food and Drug Administration, Center for Devices and Radiological Health. (April 2001). Guidance on Medical Device Labeling; Final Guidance for Industry and FDA Reviewers. Retrieved from www.fda.gov/downloads/medicaldevices/deviceregulationandguidance/guidancedocuments/ucm070801.pdf
3. ANSI/AAMI HE75:2009/(R) 2013. Human factors engineering – Design of medical devices.
4. ANSI/AAMI/IEC 62366:2007/(R) 2013. Medical devices – Application of usability engineering to medical devices.
5. Bias, R.G. & Mayhew, D.J. (2005) Cost-Justifying Usability: An Update for the Internet Age. Second Edition.