Home  |  About us  |  Editorial board  |  Ahead of print  | Current issue  |  Archives  |  Submit article  |  Instructions |  Search  |   Subscribe  |  Advertise  |  Contacts  |  Login 
  Users Online: 549Home Print this page Email this page Small font sizeDefault font sizeIncrease font size  

 
ORIGINAL ARTICLE
Ahead of print publication  

Assessment of risk of bias in randomized controlled trials published in Indian journals pertaining to pharmacology


 Department of Pharmacology and Therapeutics, Seth G S Medical College and KEM Hospital, Mumbai, Maharashtra, India

Date of Submission29-Jan-2021
Date of Decision19-Apr-2021
Date of Acceptance28-Jun-2021
Date of Web Publication01-Dec-2021

Correspondence Address:
Saurabh R Patil,
Department of Pharmacology and Therapeutics, Seth G S Medical College and KEM Hospital, Parel, Mumbai - 400 012, Maharashtra
India
Login to access the Email id

Source of Support: None, Conflict of Interest: None

DOI: 10.4103/picr.PICR_19_21

   Abstract 

Context: Number of trials in India shows an increasing trend. As these trials will shape clinical practice, their quality is of utmost importance. Among many tools to assess the quality of randomized control trials (RCTs), risk of bias (RoB) is most robust.
Aims: To understand the quality of trials being carried out in India in terms of RoB.
Settings and Design: We aimed to assess the RoB in a set of RCTs published in Indian pharmacology of randomized trials from journals pertaining to pharmacology.
Subjects and Methods: We used published journal articles as source of information for randomized clinical trials and evaluated them using Cochrane RoB tool 2.0.
Statistical Analysis Used: Descriptive statistics were used.
Results: 158 trials published in seven journals were evaluated in six different domains. Overall evaluation for 97% (153) trials was “high risk,” while 3% (5) were in “some concerns” category, with no trials categorized as “low risk. 74% articles showed a high risk of bias in the domain of 'selection of reported results. Nearly half articles scored “low risk” in domains of “missing data” and “deviations in assignment to intervention.” The study results showed a slowly increasing trend of average RoB over the last 10 years.
Conclusions: The study shows concerning rise in RoB in various domains RCTs published in Pharmacology journals in India.

Keywords: Quality, randomized control trials, reporting, risk of bias



How to cite this URL:
Patil SR, Bhide SS. Assessment of risk of bias in randomized controlled trials published in Indian journals pertaining to pharmacology. Perspect Clin Res [Epub ahead of print] [cited 2022 Jan 16]. Available from: https://www.picronline.org/preprintarticle.asp?id=331671


   Introduction Top


Risk of bias (RoB) is tool to assess the quality of research. Internal validity of a study is the extent to which the observed results represent the truth in the population we are studying. RoB is concerned with internal validity as it evaluates risk of methodological errors based on available information.[1] Data generated from clinical trials can be offset from the truth due to multiple factors.[2],[3] RoB is one of the factors, and higher RoB may result in overestimates of the effects.[4],[5],[6] For this reason, studies included in a synthesis of evidence undergo prior RoB assessment.[7]

Number of trials in India shows an increasing trend.[8] These trials will shape clinical practice, and therefore, as a pilot study, we have assessed RoB in a set of randomized trials published in India.


   Subjects And Methods Top


The study was exempted from review by the institutional ethics committee (EC/OA-24/2019) as data from public domain were used.

Journals published in India were included, containing the word “Pharmacology” in the title and those with an online portal. Journals with no search utility on portal were excluded as were the journals whose portals returned no results for search query.

We have used articles publishing trial results as a source of information. On the selected journals' online portals, articles were searched using multiple combinations of keywords “randomized,” “control,” “controlled,” “trial,” “RCT,” and “study” on April 12, 2019. Articles that published the results of randomized controlled trial (RCT) and whose full texts were freely available were included. Animal studies, observational studies, and surgical or AYUSH interventions were excluded. Duplicates were removed. Full texts were retrieved and each article was assigned a unique identifier. Cochrane RoB tool version 2.0 was used for the purpose of evaluation.[3]

To accelerate, streamline and reduce errors in evaluation process, an electronic case record form was created. Answers to one question served as decision points to decide which question was presented next. This process exactly mirrored the algorithms outlined for trial evaluation from the RoB tool documentation. Google Forms were used for this purpose.

Functional evaluation and validation of the forms were done using ten sample evaluation comparing results to the pen-and-paper method [Annexure 1]. It was also validated for inter-evaluator variability. Two independent investigators evaluated each article separately. The name of journal, authors, and other identifying information were censored from the full text of the articles. The investigators performed evaluations on a different set of ten articles as a training exercise before carrying out evaluations of articles included in study. All six domains were evaluated in each study [Annexure 1].

Evaluation result for each domain was recorded as “low risk,” “some concerns,” or “high risk.” Thus, each evaluated article generated six values. These values were replaced with values of “1,” “2,” and “3,” respectively, for analysis.

Descriptive statistics were used to analyze the data.


   Results Top


Seven journals satisfied the inclusion and exclusion criteria. The average impact factor of these journals was 0.746 (range 0.167–1.15). From these journals, a total of 381 search results were obtained. Duplicates were removed, resulting in 240 unique articles. After application of inclusion and exclusion criteria, 158 articles were eligible for evaluation [Figure 1] and [Figure 2]. As per the RoB 2.0 tool, 97% (153) articles fell into “high-risk” category. Other 3% (5) articles were in “some concerns” category. None of the articles were in “low-risk” category [Figure 3]. Domain-wise distribution showed that most of the articles had high RoB in domain of “selection of reported results;” while nearly half of the articles fared low risk in “missing data” and “deviations in assignment to intervention” domains [Figure 4]. Average RoB for a particular year was calculated by taking arithmetic mean of RoB values of the articles published in the same year. A year-wise comparison showed a slowly increasing trend over the past 10 years [Figure 5].
Figure 1: Selection of articles for analysis. IJBCP: International Journal of Basic and Clinical Pharmacology; IJP: Indian Journal of Pharmacology; IJPCS: International Journal of Pharmacology and Clinical Sciences; IJPP: Indian Journal of Pharmacy and Pharmacology; JPP: Journal of Pharmacology and Pharmacotherapeutics; NJPPP: National Journal of Physiology, Pharmacy and Pharmacology

Click here to view
Figure 2: Journal-wise distribution. Impact factors are given in brackets. IJBCP: International Journal of Basic and Clinical Pharmacology (0.786); IJP: Indian Journal of Pharmacology (1.040); IJPCS: International Journal of Pharmacology and Clinical Sciences (0.564); IJPP: Indian Journal of Pharmacy and Pharmacology (1.15); JPP: Journal of Pharmacology and Pharmacotherapeutics (0.774); NJPPP: National Journal of Physiology, Pharmacy and Pharmacology (0.167)

Click here to view
Figure 3: Overall Risk of Bias as per RoB tool 2.0 guideline

Click here to view
Figure 4: Domain-wise distribution of risk of bias

Click here to view
Figure 5: Year-wise trend of risk of bias

Click here to view



   Discussion Top


It was observed that a high proportion of trials had many of their domains rated as high risk. This is attributable partly be to flawed design and/or execution of the trial and partly due to lack of critical information in published article. Algorithms in RoB 2.0 tool tend to mark the domain as high risk if the information is missing. This is expected as the tool is used for weeding out the trials from being included in the evidence synthesis process. This simply highlights importance of the good practice of making trial-related information available for evidence synthesis processes.

Domain-wise discussion

There is a large repertoire of tools available to assess quality of trials by evaluating RoB.[9] Cochrane RoB tool is validated. It is used more widely for assessment of RoB as compared to other tools.[3],[10]

Assessment of individual trials is objective by algorithms' design and also due to blinding. The tool used for the purpose is validated and time-tested. Detailed domain-wise analysis has imparted more granularity to our analysis.

The study has some limitations. Only Indian journals pertaining to field of Pharmacology were included for analysis. We have only included open access journals due to limitations in funding in accessing paid journals. This has led to selection of journals with lower impact factors which reflect a poor quality of publication. Such selection may have affected our results. We also did not use other sources for information in this study. Vale has talked about how using only published article as source of information may is to be done cautiously.[11] The evaluation was done only for the primary outcome.


   Conclusions Top


Almost all of evaluated RCTs (97%) were of high RoB. Most high risk domains were those of “Selection of reported results” and “Deviations arising due to assignment to intervention.” The year-wise distribution of RoB shows an increasing trend.

Acknowledgment

We would like to acknowledge the Head of the Department Dr. Sandhya Kamat for continual support.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.

 
   References Top

1.
Katikireddi SV, Egan M, Petticrew M. How do systematic reviews incorporate risk of bias assessments into the synthesis of evidence? A methodological study. J Epidemiol Community Health 2015;69:189-95.  Back to cited text no. 1
    
2.
Sackett DL. Bias in analytic research. J Chronic Dis 1979;32:51-63.  Back to cited text no. 2
    
3.
Sterne JA, Savović J, Page MJ, Elbers RG, Blencowe NS, Boutron I, et al. RoB 2: A revised tool for assessing risk of bias in randomised trials. BMJ 2019;366:l4898.  Back to cited text no. 3
    
4.
Schulz KF, Chalmers I, Hayes RJ, Altman DG. Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA 1995;273:408-12.  Back to cited text no. 4
    
5.
Wood L, Egger M, Gluud LL, Schulz KF, Jüni P, Altman DG, et al. Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: Meta-epidemiological study. BMJ 2008;336:601-5.  Back to cited text no. 5
    
6.
Balk EM, Bonis PA, Moskowitz H, Schmid CH, Ioannidis JP, Wang C, et al. Correlation of quality measures with estimates of treatment effect in meta-analyses of randomized controlled trials. JAMA 2002;287:2973-82.  Back to cited text no. 6
    
7.
Presentation of Assessments of Risk of Bias; 2019. Available from: https://handbook-5-1.cochrane.org/chapter_8/8_6_presentation_of_assessments_of_risk_of_bias.htm. [Last accessed on 2019 Apr 02].  Back to cited text no. 7
    
8.
Bhide SS, Tadvi F, Maurya M, Bhojne S, Chandrakar P. Assessment of clinical trials registered at clinical trial registry of India over past decade: An audit. Int J Clin Trials 2016;3:238-43.  Back to cited text no. 8
    
9.
Page MJ, McKenzie JE, Higgins JP. Tools for assessing risk of reporting biases in studies and syntheses of studies: A systematic review. BMJ Open 2018;8:e019703.  Back to cited text no. 9
    
10.
Jørgensen L, Paludan-Müller AS, Laursen DR, Savović J, Boutron I, Sterne JA, et al. Evaluation of the Cochrane tool for assessing risk of bias in randomized clinical trials: Overview of published comments and analysis of user practice in Cochrane and non-Cochrane reviews. Syst Rev 2016;5:80.  Back to cited text no. 10
    
11.
Vale CL, Tierney JF, Burdett S. Can trial quality be reliably assessed from published reports of cancer trials: Evaluation of risk of bias assessments in systematic reviews. BMJ 2013;346:f1798.  Back to cited text no. 11
    


    Figures

  [Figure 1], [Figure 2], [Figure 3], [Figure 4], [Figure 5]



 

 
Top
 
 
  Search
 
     Search Pubmed for
 
    -  Patil SR
    -  Bhide SS
    Access Statistics
    Email Alert *
    Add to My List *
* Registration required (free)  

 
  In this article
    Abstract
   Introduction
   Subjects And Methods
   Results
   Discussion
   Conclusions
    References
    Article Figures

 Article Access Statistics
    Viewed163    
    PDF Downloaded11    

Recommend this journal