|
 |
| ORIGINAL ARTICLE |
|
| Year : 2023 | Volume
: 14
| Issue : 1 | Page : 16-19 |
|
Assessment of risk of bias in randomized controlled trials published in Indian journals pertaining to pharmacology
Saurabh R Patil, Shruti Shripad Bhide
Department of Pharmacology and Therapeutics, Seth G S Medical College and KEM Hospital, Mumbai, Maharashtra, India
| Date of Submission | 29-Jan-2021 |
| Date of Decision | 19-Apr-2021 |
| Date of Acceptance | 28-Jun-2021 |
| Date of Web Publication | 01-Dec-2021 |
Correspondence Address:
Dr. Saurabh R Patil
Department of Pharmacology and Therapeutics, Seth G S Medical College and KEM Hospital, Parel, Mumbai - 400 012, Maharashtra
India
 Source of Support: None, Conflict of Interest: None  | Check |
DOI: 10.4103/picr.PICR_19_21
 Abstract | | |
Context: Number of trials in India shows an increasing trend. As these trials will shape clinical practice, their quality is of utmost importance. Among many tools to assess the quality of randomized control trials (RCTs), risk of bias (RoB) is most robust.
Aims: To understand the quality of trials being carried out in India in terms of RoB.
Settings and Design: We aimed to assess the RoB in a set of RCTs published in Indian pharmacology of randomized trials from journals pertaining to pharmacology.
Subjects and Methods: We used published journal articles as source of information for randomized clinical trials and evaluated them using Cochrane RoB tool 2.0.
Statistical Analysis Used: Descriptive statistics were used.
Results: 158 trials published in seven journals were evaluated in six different domains. Overall evaluation for 97% (153) trials was “high risk,” while 3% (5) were in “some concerns” category, with no trials categorized as “low risk. 74% articles showed a high risk of bias in the domain of 'selection of reported results. Nearly half articles scored “low risk” in domains of “missing data” and “deviations in assignment to intervention.” The study results showed a slowly increasing trend of average RoB over the last 10 years.
Conclusions: The study shows concerning rise in RoB in various domains RCTs published in Pharmacology journals in India. Keywords: Quality, randomized control trials, reporting, risk of bias
How to cite this article:
Patil SR, Bhide SS. Assessment of risk of bias in randomized controlled trials published in Indian journals pertaining to pharmacology. Perspect Clin Res 2023;14:16-9 |
How to cite this URL:
Patil SR, Bhide SS. Assessment of risk of bias in randomized controlled trials published in Indian journals pertaining to pharmacology. Perspect Clin Res [serial online] 2023 [cited 2023 Mar 30];14:16-9. Available from: http://www.picronline.org/text.asp?2023/14/1/16/331671 |
| Introduction | |  |
Risk of bias (RoB) is tool to assess the quality of research. Internal validity of a study is the extent to which the observed results represent the truth in the population we are studying. RoB is concerned with internal validity as it evaluates risk of methodological errors based on available information.[1] Data generated from clinical trials can be offset from the truth due to multiple factors.[2],[3] RoB is one of the factors, and higher RoB may result in overestimates of the effects.[4],[5],[6] For this reason, studies included in a synthesis of evidence undergo prior RoB assessment.[7]
Number of trials in India shows an increasing trend.[8] These trials will shape clinical practice, and therefore, as a pilot study, we have assessed RoB in a set of randomized trials published in India.
| Subjects And Methods | | |
The study was exempted from review by the institutional ethics committee (EC/OA-24/2019) as data from public domain were used.
Journals published in India were included, containing the word “Pharmacology” in the title and those with an online portal. Journals with no search utility on portal were excluded as were the journals whose portals returned no results for search query.
We have used articles publishing trial results as a source of information. On the selected journals' online portals, articles were searched using multiple combinations of keywords “randomized,” “control,” “controlled,” “trial,” “RCT,” and “study” on April 12, 2019. Articles that published the results of randomized controlled trial (RCT) and whose full texts were freely available were included. Animal studies, observational studies, and surgical or AYUSH interventions were excluded. Duplicates were removed. Full texts were retrieved and each article was assigned a unique identifier. Cochrane RoB tool version 2.0 was used for the purpose of evaluation.[3]
To accelerate, streamline and reduce errors in evaluation process, an electronic case record form was created. Answers to one question served as decision points to decide which question was presented next. This process exactly mirrored the algorithms outlined for trial evaluation from the RoB tool documentation. Google Forms were used for this purpose.
Functional evaluation and validation of the forms were done using ten sample evaluation comparing results to the pen-and-paper method [Annexure 1]. It was also validated for inter-evaluator variability. Two independent investigators evaluated each article separately. The name of journal, authors, and other identifying information were censored from the full text of the articles. The investigators performed evaluations on a different set of ten articles as a training exercise before carrying out evaluations of articles included in study. All six domains were evaluated in each study [Annexure 1].
Evaluation result for each domain was recorded as “low risk,” “some concerns,” or “high risk.” Thus, each evaluated article generated six values. These values were replaced with values of “1,” “2,” and “3,” respectively, for analysis.
Descriptive statistics were used to analyze the data.
| Results | | |
Seven journals satisfied the inclusion and exclusion criteria. The average impact factor of these journals was 0.746 (range 0.167–1.15). From these journals, a total of 381 search results were obtained. Duplicates were removed, resulting in 240 unique articles. After application of inclusion and exclusion criteria, 158 articles were eligible for evaluation [Figure 1] and [Figure 2]. As per the RoB 2.0 tool, 97% (153) articles fell into “high-risk” category. Other 3% (5) articles were in “some concerns” category. None of the articles were in “low-risk” category [Figure 3]. Domain-wise distribution showed that most of the articles had high RoB in domain of “selection of reported results;” while nearly half of the articles fared low risk in “missing data” and “deviations in assignment to intervention” domains [Figure 4]. Average RoB for a particular year was calculated by taking arithmetic mean of RoB values of the articles published in the same year. A year-wise comparison showed a slowly increasing trend over the past 10 years [Figure 5]. | Figure 1: Selection of articles for analysis. IJBCP: International Journal of Basic and Clinical Pharmacology; IJP: Indian Journal of Pharmacology; IJPCS: International Journal of Pharmacology and Clinical Sciences; IJPP: Indian Journal of Pharmacy and Pharmacology; JPP: Journal of Pharmacology and Pharmacotherapeutics; NJPPP: National Journal of Physiology, Pharmacy and Pharmacology
Click here to view |
 | Figure 2: Journal-wise distribution. Impact factors are given in brackets. IJBCP: International Journal of Basic and Clinical Pharmacology (0.786); IJP: Indian Journal of Pharmacology (1.040); IJPCS: International Journal of Pharmacology and Clinical Sciences (0.564); IJPP: Indian Journal of Pharmacy and Pharmacology (1.15); JPP: Journal of Pharmacology and Pharmacotherapeutics (0.774); NJPPP: National Journal of Physiology, Pharmacy and Pharmacology (0.167)
Click here to view |
| Discussion | | |
It was observed that a high proportion of trials had many of their domains rated as high risk. This is attributable partly be to flawed design and/or execution of the trial and partly due to lack of critical information in published article. Algorithms in RoB 2.0 tool tend to mark the domain as high risk if the information is missing. This is expected as the tool is used for weeding out the trials from being included in the evidence synthesis process. This simply highlights importance of the good practice of making trial-related information available for evidence synthesis processes.
Domain-wise discussion
There is a large repertoire of tools available to assess quality of trials by evaluating RoB.[9] Cochrane RoB tool is validated. It is used more widely for assessment of RoB as compared to other tools.[3],[10]
Assessment of individual trials is objective by algorithms' design and also due to blinding. The tool used for the purpose is validated and time-tested. Detailed domain-wise analysis has imparted more granularity to our analysis.
The study has some limitations. Only Indian journals pertaining to field of Pharmacology were included for analysis. We have only included open access journals due to limitations in funding in accessing paid journals. This has led to selection of journals with lower impact factors which reflect a poor quality of publication. Such selection may have affected our results. We also did not use other sources for information in this study. Vale has talked about how using only published article as source of information may is to be done cautiously.[11] The evaluation was done only for the primary outcome.
| Conclusions | | |
Almost all of evaluated RCTs (97%) were of high RoB. Most high risk domains were those of “Selection of reported results” and “Deviations arising due to assignment to intervention.” The year-wise distribution of RoB shows an increasing trend.
Acknowledgment
We would like to acknowledge the Head of the Department Dr. Sandhya Kamat for continual support.
Financial support and sponsorship
Nil.
Conflicts of interest
There are no conflicts of interest.
| References | | |
| 1. | Katikireddi SV, Egan M, Petticrew M. How do systematic reviews incorporate risk of bias assessments into the synthesis of evidence? A methodological study. J Epidemiol Community Health 2015;69:189-95.  |
| 2. | Sackett DL. Bias in analytic research. J Chronic Dis 1979;32:51-63. |
| 3. | Sterne JA, Savović J, Page MJ, Elbers RG, Blencowe NS, Boutron I, et al. RoB 2: A revised tool for assessing risk of bias in randomised trials. BMJ 2019;366:l4898. |
| 4. | Schulz KF, Chalmers I, Hayes RJ, Altman DG. Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA 1995;273:408-12. |
| 5. | Wood L, Egger M, Gluud LL, Schulz KF, Jüni P, Altman DG, et al. Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: Meta-epidemiological study. BMJ 2008;336:601-5. |
| 6. | Balk EM, Bonis PA, Moskowitz H, Schmid CH, Ioannidis JP, Wang C, et al. Correlation of quality measures with estimates of treatment effect in meta-analyses of randomized controlled trials. JAMA 2002;287:2973-82. |
| 7. | |
| 8. | Bhide SS, Tadvi F, Maurya M, Bhojne S, Chandrakar P. Assessment of clinical trials registered at clinical trial registry of India over past decade: An audit. Int J Clin Trials 2016;3:238-43. |
| 9. | Page MJ, McKenzie JE, Higgins JP. Tools for assessing risk of reporting biases in studies and syntheses of studies: A systematic review. BMJ Open 2018;8:e019703. |
| 10. | Jørgensen L, Paludan-Müller AS, Laursen DR, Savović J, Boutron I, Sterne JA, et al. Evaluation of the Cochrane tool for assessing risk of bias in randomized clinical trials: Overview of published comments and analysis of user practice in Cochrane and non-Cochrane reviews. Syst Rev 2016;5:80. |
| 11. | Vale CL, Tierney JF, Burdett S. Can trial quality be reliably assessed from published reports of cancer trials: Evaluation of risk of bias assessments in systematic reviews. BMJ 2013;346:f1798. |
[Figure 1], [Figure 2], [Figure 3], [Figure 4], [Figure 5]
|
