Dr. Christopher Joseph Devine, President, Devine Guidance International
Devine Guidance

The FDA Loves Stats

By Dr. Christopher Joseph Devine
Dr. Christopher Joseph Devine, President, Devine Guidance International

Providing sample size rationale is a must.

Dr. D hopes all of you dads out there had a fabulous Father’s Day celebration with your families and friends. Dr. D’s day began with the writing this week’s article for MedTech Intelligence. This week the doctor will be brief, as Dr. D writes about a not-too-often cited violation of §820.250 (Statistical Techniques). Considering all of the tools available to help device establishments implement a robust approach to sampling, including one of the doctor’s favorite books “Juran’s Handbook,” it is difficult for Dr. D to comprehend how a device establishment can fail to provide sample size rationale. It really makes no difference either as an inspection tool or a salient component of validations: Providing adequate sample size rationale is understood as a fundamental quality requirement. Rest assured that our friends from FDA will review validation activities during one of their friendly visits for a cup of coffee and an inspection. If the investigator believes that an establishment’s approach to sample size selection is questionable or missing sufficient rationale, then a Form 483 observation will be forthcoming. All the Chief Jailable Officer (CJO) will be able to do is head off to the local speakeasy, after the inspection, and quaff (look-it-up) mass quantities of alcohol. Enjoy!

Warning Letter – May 16, 2016

Typically, when an offending establishment racks up nine inspectional observations (multiple subparts), bad things are going to happen. If you are the CJO of an offending establishment that has the honor of receiving more Form 483 observations than fingers, you will not need a Tarot card reading to see bad things are in your establishment’s future (Note: thumbs do not count as fingers). However, if you were born with a condition known as polydactyly (look-it-up), than you would have a couple of extra digits (former Major League pitcher Antonio Alfonseca was blessed with this interesting quirk of genetics), and the magic number would be 10 Form 483’s.  Seriously, really bad things start to happen when the Form 483 observations reach double digits, and the doctor isn’t talking about fingers, people.

Warning Letter Excerpt

Observation Six (6) “Sampling plans are not written and based on valid statistical rationale, as required by 21 CFR 820.250(b). Specifically, the following process validations did not have sampling plans based on valid statistical rationale.”

21 CFR, Part 820.250(b) – Statistical Techniques

(a) Where appropriate, each manufacturer shall establish and maintain procedures for identifying valid statistical techniques required for establishing, controlling, and verifying the acceptability of process capability and product characteristics.

(b) Sampling plans, when used, shall be written and based on a valid statistical rationale. Each manufacturer shall establish and maintain procedures to ensure that sampling methods are adequate for their intended use and to ensure that when changes occur the sampling plans are reviewed. These activities shall be documented.
 
Compliance for Dummies

Broken record time my dear readers: If your establishment does not have an SOP for the selection and application of sampling plans, there is no time like the present to script one. Please keep in mind, most device establishments think of sampling plans in terms of inspection activities (typically C=0); however, validation activities are important too. Can you say reliability and confidence level? You see, when a sample size of n=3 is stated and no rationale is provided, this not going to work for FDA. Can you identify the two reasons why this would be an issue for an investigator? If your answer is: (a) the sample size is just plan-old ridiculous and (b) valid statistical rationale is a QSR requirement, you would be correct.

Dr. D strongly recommends the application of ANSI/ASQ Z1.4 (Sampling Procedures and Tables for Inspection by Attributes) and ANSI/ASQ Z1.9 (Sampling Procedures and Tables for Inspection by Variables for Percent Nonconforming) for day-to-day inspection activities within a manufacturing facility. There is no reason to reinvent the proverbial wheel. However, when performing testing in support of product development testing activities (verification or validation) or pursuing process validation testing, it is time to break out “Juran’s Handbook” and become acquainted with k-values and the impact of sample size selection on reliability and confidence intervals. For example, if your engineering group is attempting to achieve a 90% reliability / 90% confidence level using variables data and the sample size identified in n=3, then Houston, we have a problem. The sample size should be stated as n=22, when employing a Non-Parametric Binomial Reliability Test. When in doubt, Dr. D recommends visiting http://reliabilityanalyticstoolkit.appspot.com/sample_size.

Additionally, the sample size rationale depicted in test protocols should always state the reliability/confidence levels and the associated sample size, which is premised on attribute or variable data being collected. The rationale must clearly state the logic behind the sample size(s) being employed. Dr. D would like share a fairly not-well-kept secret about FDA: Our friends from the agency really do like device establishments to spell it out.

Takeaways

For this week’s guidance, the doctor will share two takeaways with the readers. One: If you do not own a copy of “Juran’s Handbook”, Dr. D strongly suggests acquiring a copy. It will be money well spent. Two: When scripting protocols, clearly spell out the sample size rationale including:

  • Reliability and confidence levels
  • Actual sample size employed
  • Number of failures permitted (hopefully zero)
  • Type of data being collected (variable versus attribute)

In closing, thank you again for joining Dr. D, and the doctor hopes you find value in the guidance provided. Until the next installment of DG, cheers from Dr. D., and best wishes for continued professional success.

References

  1. Code of Federal Regulation. (April 2015). Title 21 Part 820: Quality system regulation. Washington, D.C.: U.S. Government Printing Office.
  2. Devine, C. (2011). Devine guidance for complying with the FDA’s quality system   regulation – 21 CFR, Part 820. Charleston, SC: Amazon.
  3. Devine, C. (2013). Devine guidance for managing key attributes of a FDA-compliant quality management system – 21 CFR, Part 820 Compliance. Charleston, SC: Amazon.
  4. FDA. (May 2016). Inspections, Compliance, Enforcement, and Criminal Investigations. Qiagen Sciences, LLC. Accessed June 17, 2016. Retrieved from http://www.fda.gov/ICECI/EnforcementActions/WarningLetters/2016/ucm501292.htm

About The Author

Dr. Christopher Joseph Devine, President, Devine Guidance International