Perspectives in Clinical Research

: 2013  |  Volume : 4  |  Issue : 4  |  Page : 221--226

A data-driven approach to quality risk management

Demissie Alemayehu, Jose Alvir, Marcia Levenstein, David Nickerson 
 Specialty Care Business Unit, Clinical Affairs, Statistics (DA, JA, ML) and Worldwide Research & Development, Clinical Quality Management (DN), Pfizer, Inc., New York, USA

Correspondence Address:
Jose Alvir
Pfizer, Inc., 235 East 42nd Street, NYO-219-07-01, New York, NY 10017


Aim: An effective clinical trial strategy to ensure patient safety as well as trial quality and efficiency involves an integrated approach, including prospective identification of risk factors, mitigation of the risks through proper study design and execution, and assessment of quality metrics in real-time. Such an integrated quality management plan may also be enhanced by using data-driven techniques to identify risk factors that are most relevant in predicting quality issues associated with a trial. In this paper, we illustrate such an approach using data collected from actual clinical trials. Materials and Methods: Several statistical methods were employed, including the Wilcoxon rank-sum test and logistic regression, to identify the presence of association between risk factors and the occurrence of quality issues, applied to data on quality of clinical trials sponsored by Pfizer. Results: Only a subset of the risk factors had a significant association with quality issues, and included: Whether study used Placebo, whether an agent was a biologic, unusual packaging label, complex dosing, and over 25 planned procedures. Conclusion: Proper implementation of the strategy can help to optimize resource utilization without compromising trial integrity and patient safety.

How to cite this article:
Alemayehu D, Alvir J, Levenstein M, Nickerson D. A data-driven approach to quality risk management.Perspect Clin Res 2013;4:221-226

How to cite this URL:
Alemayehu D, Alvir J, Levenstein M, Nickerson D. A data-driven approach to quality risk management. Perspect Clin Res [serial online] 2013 [cited 2022 Aug 17 ];4:221-226
Available from:

Full Text


The current paradigm in drug development entails the conduct of complex and large trials, recruiting patients globally, and often relying on clinical research organization alliance partners to manage the trial execution. With the increasing size and complexity of trials, there is a corresponding need to be vigilant about patient safety, data quality, and trial integrity. Most trials employ resource-intensive approaches to oversee quality, with frequent on-site visits, to identify issues in a reactive manner, and may often fail to effectively address critical risks to quality. The prevailing practice of frequent site visits and extensive source document verification has also contributed to the sky-rocketing cost of clinical research. Accordingly, there is a nascent focus on new approaches to quality risk management by the pharmaceutical industry and other stakeholders in the clinical trial enterprise. [1],[2],[3],[4] One approach that is gaining momentum is a holistic strategy to quality management that incorporates risk management principles, borrowing ideas from the manufacturing sector as described in International Conference on Harmonisation Q8, Q9 and Q10. [5],[6],[7] Central to the approach is the concept of quality-by-design, which in the context of clinical trials translates into building quality into the trial design (e.g., protocol) and processes to execute the trial rather than managing the quality of the trial retrospectively or in a reactive manner. [8]

Pfizer has launched a pilot project in partnership with the US Food and Drug Administration that is designed to test one model for prospectively designing quality into clinical trials and systematically managing quality during study conduct. [9] The approach, known as the integrated quality management plan (IQMP), is built based on the following core principles:

Quality is built-in at the time of protocol development and systematically managed during study conduct through a process of continuous improvement;Critical to quality factors, and related metrics and associated performance expectations are identified a priori and actual performance is measured and actively managed throughout the duration of study conduct;Risks to quality are prospectively identified, prioritized, and mitigated.In this paper, we discuss a quantitative approach to complement the IQMP efforts using statistical models to identify risk factors that require closer scrutiny both before and during study conduct. Quantitative and data-driven approaches help minimize some of the pitfalls associated with actions taken in a subjective manner. In particular, such an approach, if executed meticulously, tends to provide results that are reproducible and often generalizable. However, the generalizability of the findings is dependent on the quality and magnitude of the data. This would often involve gathering numerical data, in a cost-effective and systematic fashion, from a fairly large number of studies that are representative of future trials.

The rest of the paper is organized as follows. In Section 2, we describe the data and analytical approaches and discuss the results in Section 3. In the last Section, we highlight the implications of the data-driven strategy with regard to optimal resource utilization, and suggest success factors that are critical for an IQMP.

 Materials and Methods

Data source and description

Data were obtained from seventy-three select ongoing late-stage clinical trials from across a variety of business units and therapeutic areas over several years at Pfizer.

Two separate questionnaires were completed for each study by the respective study teams. The first was a forward-looking assessment seeking to identify the level of risk perceived to be associated with risk factors that are related to eight different risk categories (i.e., asset characteristics, subjects, protocol, locations, site operations, vendors/outsourcing, monitoring, and drug supply). [Table 1] lists the prospectively identified risk factors that were collected for each trial. The second was a backward-looking assessment that identified the issues that actually occurred during study conduct based on a standard set of common issues critical to quality requirements. [Table 2] lists the issues that were assessed in the course of the trial conduct, and used to define the dependent variable for subsequent statistical analysis.{Table 1}{Table 2}

Statistical methods

To identify relevant risk factors that require closer scrutiny in future quality management initiatives, several statistical methods were employed. Quality issues were defined both as binary (i.e., presence or absence of a quality indicator) as well as counts (i.e., number of issues satisfying quality criteria). In the following, we present the results of the analyses performed using the latter.

In the univariate analysis, a Wilcoxon rank-sum test was used to identify the presence of association between the risk factors and the occurrence of quality issues. The results of the preliminary analyses were then used to reduce the number of risk factors for inclusion in multiple regression models. Due to the skewed nature of the data, it was necessary to use the non-parametric Wilcoxon rank-sum test for the univariate analyses, and a log transformation for the regression models.


Of the 73 protocols in the database, there were 72 studies that had at least one issue with a mitigation plan. Ten (13.7%) studies had at least one issue without a mitigation plan in place. For a preliminary analysis, a binary outcome was defined using the presence or absence of a specified number of issues with or without a mitigation plan. However, a binary definition tended to involve some degree of subjectivity and arbitrariness. As a result, actual counts of issues observed in a study were used in the definition of the dependent variable and reported in subsequent analyses.

[Table 3] gives a partial list of the risk factors and associated P values. Based on the univariate analyses, nine risk factors had a significant (P < 0.05) or marginally significant (P < 0.10) association with the number of quality issues. Incidentally, these factors included some intuitive ones that are known to lead to quality problems. The significant factors were: Unusual packaging/labeling, dosing complexity; a biologic compound, size of planned procedures over the course of the trial, whether investigator discretion was permitted in measurement decisions; whether the drug was self-administered; use of placebo; and number of exclusion criteria. Two of the risk factors, namely the co-sponsorship of the development program and the number of vendors used to manage the study were marginally significant.{Table 3}

The median numbers of issues for study drugs that require unusual labeling or involve complex dosing were 18, compared to only 10 when the opposite was the case. Similarly, a biologic compound tended to result in greater median number of issues than a non-biologic study drug (13 vs. 9, respectively). The results for the other significant and marginally significant risk factors trended in the same direction.

The above risk factors that were identified in the univariate analyses were then included in a multiple regression analysis. The regression analysis further identified five risk factors with significant predictive values when taken jointly: Whether study used Placebo, whether an agent was a biologic, unusual packaging label, complex dosing, and over 25 planned procedures.

It should be noted that the multiple regression approach was limited by the size of the data that was available for analysis. In addition, there were a few instances where the relevant data were missing. Despite those limitations, the approach has the potential to guide risk mitigation activities by identifying those risk factors that require increased attention.


In this paper, we proposed the use of a data-driven approach to enhance an integrated quality management strategy. While the results presented in the paper are intended to illustrate the approach, with robust and more reliable data, the approach can serve to identify risk factors that may need to be mitigated more closely. A meticulous application of the approach has the potential to maximize resource use in risk mitigation activities.

The advantages of quantitative and data-driven approaches rest largely on the ability to make decisions based on objective, rather than subjective, criteria. This in turn requires numerical data collected from a fairly large number of studies, to ensure result validity and generalizability. To the extent possible, the data collection method should be simple and cost-effective.

In any quality risk management exercise, success in ensuring patient safety and trial integrity is a function of several variables. Most notably, fancy models or complex quality management plans cannot be a substitute for strict adherence to Good Clinical Practice. In addition, it is important to collaborate and share experiences with other internal and external stakeholders, including regulatory bodies. For optimal impact, it is also essential to establish the necessary infrastructure, including processes, tools, and systems to make the quality management plan and findings of quantitative exercises scalable implementable.

A key feature of any continuous improvement project is the need to revisit current thinking and update operating models, informed by accumulating data. Accordingly, the quantitative analyses proposed in this paper should periodically be updated and refined using new data and until a reasonably steady state is achieved.


1Journot V, Pignon JP, Gaultier C, Daurat V, Bouxin-Métro A, Giraudeau B, et al. Validation of a risk-assessment scale and a risk-adapted monitoring plan for academic clinical research studies - The Pre-Optimon study. Contemp Clin Trials 2011;32:16-24.
2Davis JR, Nolan VP, Woodcock J, Estabrook RW. Assuring Data Quality and Validity in Clinical Trials for Regulatory Decision Making; Workshop Report; Roundtable on Research and Development of Drugs, Biologics, and Medical Devices, Institute of Medicine. Washington D.C.: National Academy Press; 1999. Available from: [Accessed 2013 Jan 28].
3Clinical Trials Transformation Initiative, 2011. Workshop on Quality Risk Management: Making Clinical Trials Fit for Purpose. Available from: [Accessed 2013 Jan 28].
4European Medicines Agency, 2011. Reflection paper on risk based quality management in clinical trials EMA/INS/GCP/394194/2011. Available from: Accessed 2013 Jan 28.
5International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use, 2009. Harmonised Tripartite Guideline: Pharmaceutical Development Q8(R2) Current Step 4 version. Available from: [Accessed 2013 Jan 28].
6International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use, 2005. ICH Harmonised Tripartite Guideline: Quality Risk Management Q9. Available from: [Accessed 2013 Jan 28].
7International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use, 2011. ICH Harmonized Tripartite Guidance: Pharmaceutical Quality Systems Q10. Available from: [Accessed 2013 Jan 28].
8Yu LX. Pharmaceutical quality by design: Product and process development, understanding, and control. Pharm Res 2008;25:781-91. Available from: [Accessed 2013 Jan 28].
9Sprenger K, Nickerson D, Meeker-O′Connell A, Morrison BW. Quality by Design in Clinical Trials: A Collaborative Pilot with FDA. Therapeutic Innovation & Regulatory Science. Available from: [Accessed 2013 Jan 28].