the development of test methods to
create the operability region for the
method by first running a screening design to test the effects of various candidate test
method variables. There are typically a large number of
variables tested in the screening design to reduce the risk of
missing any important variables. The variables found to have
the largest effects (both positive and negative) are studied in
a subsequent optimization experiment, the output of which
is operating window for the method which serves the same
function as the Design Space for a product or process. As a
result we refer to this as the “Test Method Design Space”.
In a recent test method development project 11 variables
were studied in 24 runs using a Plackett-Burman screening
design. The four variables with the largest effects were evaluated further in a 28-run optimization experiment producing the
design space for the method. The next step in the development
was to assess the effects of raw material variations
Test Method Repeatability and Reproducibility is an important assessment once the method has been developed
initially. Thus is done with using a Gage Repeatability and
Reproducibility study referred to as a Gage R&R Study.
In the study 5-10 samples are evaluated by 2-4 analysts using 2-4 repeat tests sometimes involving 2-4 test instruments.
Output from such a study produces quantitative measures
of repeatability, reproducibility and measurement resolution.
These statistics are then used to evaluate the value of the
method to be used for product release and process improvement. The variance estimates obtained are also often used to
design sampling plans to monitor the performance of the process going forward.
Test Method Ruggedness. Sometimes we find that as a test
method is used the observed variation in the test results be-
comes too large. What do I do now you ask? One possibility is
to evaluate the measurement process/procedure for rugged-
ness. Measurement method is “rugged,” if it is immune to mod-
est (and inevitable) departures from the conditions specified in
the method (Youden 1961). Ruggedness (sometimes called ro-
bustness) tests study the effects of small variations in the how
the method is used. There are other sources of variation in a
measurement method in addition to instruments and analysts
which are typically the subject of Gage R&R studies. Such vari-
ables include raw material sources and method variables such
as time and temperature. Ruggedness can be evaluated using
Quality data also reduces the risk of poor process perfor-
mance and defective pharmaceuticals reaching patients.
Measurement is a process that is developed, controlled and
improved just like a manufacturing process. Indeed, quality data
are the product of measurement processes (Snee 2005). Quality
by Design (QbD), introduced by the FDA in 2005, is focused on
the development, control and improvement of processes. Data
are central to QbD and in turn QbD concepts, methods and
tools can be used to develop, control and improve measurement processes (Borman, etal 2007: Schweitzer, etal 2010. As a
result QbD and test methods have a complementary relationship; each can be used to improve the other.
This article discusses the concepts, methods and tools of
QbD that have been successfully used to design, control and improve measurement systems. The specific approaches are summarized in Table 1 and discussed in the following paragraphs.
The concepts and methods involved will be introduced and
illustrated with pharma and biotech case studies and examples.
Design of Experiments, an effective QbD tool is used in
■ By Ronald D. Snee, PhD, Snee Associates, LLC
Using good data leads to good science and improved products
Methods and Tools Analysis Purpose
QbD in Test Method
QbD Approach Speed Up Method Development
and Reduce Risk
Design of Experiments using
Screening and Optimization Experiments
Method Development including
Creation of Test Design Space
Improve Measurement Quality
Method Robustness Studies Create Methods Robust to Small
Variations in How the Method is Used
Blind Control Sample Continued Verification of Method
Repeatability and Reproducibility
Process Variation Studies Assess Process Variation to Determine
the Relative Contributions of the
manufacturing process, sampling
Procedure and Test Method to the
Observed Process Variation
Development & Validation