|Ahead of print publication
A retrospective cross-sectional descriptive study to critically appraise the quality of reporting of health economic evaluations conducted in the Indian setting
Sandeep Kumar Gupta, Ravi Kant Tiwari, Raj Kumar Goel
Department of Pharmacology, Heritage Institute of Medical Sciences, Varanasi, Uttar Pradesh, India
|Date of Submission||23-Aug-2019|
|Date of Decision||06-Jan-2020|
|Date of Acceptance||29-Jan-2020|
|Date of Web Publication||08-Jan-2021|
Sandeep Kumar Gupta,
Heritage Institute of Medical Sciences, Varanasi - 221 311, Uttar Pradesh
Source of Support: None, Conflict of Interest: None
| Abstract|| |
Background: The reporting quality of economic research could benefit from enhanced quality assurance procedures. At present, there are small numbers of health economic researches being conducted with Indian context or setting. There is not much clarity about the reporting quality of health economic researches being conducted with Indian context or setting.
Objective: The primary objective is to of this study was to appraise the quality of reporting of health economic evaluations conducted in the Indian setting and published between January 2014 and December 2018.
Materials and Methods: This was a retrospective, cross-sectional, descriptive analysis. The MEDLINE in PubMed, Google Scholar, and Science Direct were systematically searched to search for economic evaluations. The consolidated health economic evaluation reporting standards statement checklist was utilized to assess the quality of reporting of the included studies. For grading the quality of the included health economic assessments, the Quality of Health Evaluation Studies (QHES) instrument was used.
Results: Thirty studies fulfilled the inclusion criteria and were included in the study. The mean QHES score was 80.26 (standard deviation = 8.06). Twenty-five (83.33%, 95% confidence interval [CI]: 0.66–0.92) of the article mentioned perspective of the study. Twenty-nine (96.66%, 95% CI: 0.83–0.99) of the article described the effects of uncertainty for all input parameters. Twenty (66.66%, 95% CI: 0.48–0.80) of the article reported all funding sources.
Conclusions: Overall, the quality of reporting of the included health economic studies was good, which reemphasizes their usefulness in supporting the decision-making procedure about better medicine. The finding of this study will be a small step toward ensuring robust and high-quality health economics data in India.
Keywords: Consolidated health economic evaluation reporting standards statement, health economics, quality of health evaluation studies scale, quality of reporting, quality-adjusted life year, sensitivity analysis
|How to cite this URL:|
Gupta SK, Tiwari RK, Goel RK. A retrospective cross-sectional descriptive study to critically appraise the quality of reporting of health economic evaluations conducted in the Indian setting. Perspect Clin Res [Epub ahead of print] [cited 2022 May 17]. Available from: https://www.picronline.org/preprintarticle.asp?id=306572
| Introduction|| |
The use of health-economic becomes unavoidable in developing countries such as India, where the aim of public health-care systems is to augment productivity in resource allocation to drug therapies. Health economics is also required for repayment purposes for the health insurance industry, which is growing swiftly in developing countries. The induction of health economics research in health-care policy decision-making will help more efficient resource allotment.,, In fact, only a few years after the prelude of the health economics, many countries started to apply health economic analyses in the execution of their health care program. However, it has been observed that even when health economic studies are available, they are not methodically or steadily applied in decision-making. Furthermore, there is a dearth of policies that boost the use of health economic evaluations in medicine selection. Moreover, health economic studies are relatively expensive and time-consuming to conduct; therefore, policymakers should fund such studies to make informed policy decisions. Nevertheless, the good quality of health economic research data is imperative for health-care policy decision-making. The quality evaluation of health economic research data can help correct flaws and further enhance the productivity and quality of an economic evaluation. However, quality evaluation of health economic research is an arduous task. However, the promulgation of the consolidated health economic evaluation reporting standards (CHEERS) statement in 2013 has made this task relatively easier. This statement gave commendations to augment the quality of reporting of health economic research. By following CHEERS statement, authors can reduce many reporting flaws found in health economic researches. Unlike CHEERS instruments, the quality of health evaluation studies (QHES) scale offers numerical aggregates that can even be scrutinized statistically.
There is proof that the reporting quality of economic research could benefit from enhanced quality assurance procedures. At present, there are small numbers of health economic researches being conducted with Indian context or setting. However, there is not much clarity about the reporting quality of health economic researches being conducted with Indian context or setting., Very few studies have evaluated the reporting quality of the economics researches being conducted in either India, Asia-Pacific region or South Asian countries.,,, Hence, the aim of this study was to evaluate the quality of reporting of economic evaluations conducted in Indian setting using CHEERS statement and QHES instrument.
The primary objective was to appraise the quality of reporting of health economic evaluations conducted in the Indian setting and published between January 2014 and December 2018.
| Materials and Methods|| |
A retrospective, cross-sectional, descriptive analysis is to assess the quality of reporting of the economic evaluation. This research was based exclusively on information available in public domain.
The MEDLINE in PubMed was systematically searched to search for economic evaluations. A methodical search of Google Scholar and Science Direct was also conducted in the same period to identify economic evaluations.
Literature search strategy
Proper combinations of various search terms were used for systematic search. These search items encompassed “pharmacoeconomic,” “drug economic,” “health economics,” “medical economics,” “cost-effectiveness analysis,” “cost measures,” “cost-minimization analysis,” “cost analysis,” “cost-utility analysis,” “healthcare cost,” “cost-benefit analysis,” “cost,” “India,” and “drug cost.” The present study included economic evaluations published from January 2014 to December 2018.
The inclusion criteria were: (1) full economic evaluation, (2) model-based or clinical trial based economic assessment, (3) comparative study assessing the costs and health outcomes between 2 or more interventions, (4) original research articles, (5) studies conducted in Indian setting or context, and (6) studies published between from January 2014 to December 2018.
Exclusion criteria were: (1) reviews or short communication or editorials or commentaries or study protocol, (2) multiple-country comparisons, (3) not an economic analysis of medical-related interventions, (4) studies without a comparator group, (5) focused only on either cost or efficacy of interventions, (6) cost-of-illness study, (7) only abstract or conference proceedings, and (8) veterinary studies.
The CHEERS statement checklist was utilized to assess the quality of reporting of the included studies. This instrument comprises a 24-item checklist substantiating the existence of explicit items in the economic evaluations. Because CHEERS checklist contains directives relating to all the subsections of health economic studies, it will help in increasing transparency and comprehensive reporting of studies. It can help in subverting faulty decision making due to poor reporting of health economic studies.
For grading the quality of the included health economic assessments, the QHES instrument was used. The QHES tool comprises of 16 benchmarks in the arrangement of “yes or no” questions. Every benchmark has a point allotted in the range of 1–9, which are utilized to create an overall score ranging from 0 to 100. QHES scores <50 will be considered as an index of poor quality., Although there is no uniform inference of the QHES score, the score between 75 and 90 will be considered as an indicator of good quality, and anything above 90 will be considered as an indicator of excellent quality. The numerical score obtainable with the QHES might empower users to come to the conclusion about the comparative quality of diverse studies and to simplify the decision-making procedure. It can confirm that higher-quality studies play a greater part in the decision-making procedure in India.
Descriptive statistical analysis was utilized to delineate the attributes of the studies. The lower and upper limits of the 95% confidence interval (CI) for the proportions were calculated. The SPSS statistical software package was used for data analysis SPSS (Statistical Package for the Social Sciences), version 16; SPSS, IBM Corporation, Chicago, Illinois, USA.
| Results|| |
Two hundred and sixty-nine records were identified through PubMed database searching. Additional records identified through Google Scholar and Science Direct were 103. Hence, a total of 372 articles were identified through literature search. Fifty-eight duplicate articles were removed after initial screening. Title and abstract of the remaining 314 articles were further screened and 21 articles were excluded. Two hundred and ninety-three full-text articles were further assessed for eligibility and 263 of these articles did not fulfill the study inclusion criteria. Thirty studies fulfilled the inclusion criteria and were included in the study [Figure 1]. [Table 1] presents a summary of the included health economic assessments and their demographic data. A summary of the descriptive and reporting characteristics of the included health economic studies are provided in [Table 2]. Only four studies were published in journals with impact factors >5.0. Eleven (36.66%) and 7 (23.33%) studies were published in 2018 and 2017, respectively. In 17 (56.66%) of the studies, country of the first author was India but 11 (36.66%) of the studies were published in the USA-based journal. Out of the included 30 studies, 28 (93.33%) studies were cost-effectiveness studies [Table 1]. Twenty-six (86.66%) studies were model-based. The decision-analytic model/decision tree model/combination of decision tree and Markov model/Markov model were the most utilized model in 15 (50%) of the studies. The time horizon was not mentioned in only 2 (6.66%) of the studies [Table 2]. Lifetime time horizon was the most commonly used time horizon in 8 (26.66%) of the studies. Perspective was not mentioned in 5 (16.66%) of the studies. Health-care system/provider perspective was the most commonly utilized perspective in 9 (30%) of the studies. The evaluation of uncertainty was not mentioned in only 1 (3.33%) of the studies. Probabilistic sensitivity analysis was most commonly used for the evaluation of uncertainty in 8 (26.66%) of the studies. The discount rate was not mentioned in only 6 (20%) of the studies. Three percent was the most commonly employed discount rate in 23 (76.66%) of the study. The quality-adjusted life-year (QALY) was the most commonly utilized measure of health outcome in 13 (43.33%) of the studies.
|Figure 1: Flow diagram of citations through the retrieval and the screening process|
Click here to view
|Table 1: Summary of the included health economic assessments and their demographic data|
Click here to view
|Table 2: A summary of the descriptive and reporting characteristics of the included health economic studies|
Click here to view
The mean QHES score was 80.26 (standard deviation = 8.06). The article that had primary authors from countries other than India had a higher mean QHES score compared to article with primary authors from India, but the difference was not statistically significant (81 vs. 79.94; P = 0.7300). The grading of the quality of the included assessments with the QHES instrument is shown in [Table 3]. Fourteen (46.66%) studies had QHES score ≥70 but <80. Twelve (40%) had QHES score ≥80 but <90. Three (10%) studies had QHES score ≥90. One (3.33%) study had QHES score ≥50 but <60 [Table 3].
|Table 3: Summary statistics of quality of health evaluation studies scores|
Click here to view
The results of quality assessment with the CHEERS statement checklist are shown in [Table 4]. Of the 30 studies included in the study, 29 studies (96.96%, 95% CI: 0.83–0.99) appropriately identified the study as an economic evaluation or used more specific terms in the title. Twenty-four studies (80%, 95% CI: 0.62–0.90) provided structured abstract with series of headings. Twenty-six (86.66%, 95% CI: 0.70–0.94) of the studies had given a description of the background and objectives with an appropriate explanation of the importance of the question. Twenty (66.66%, 95% CI: 0.48–0.80) of the article had described the eligible population and subgroups. Thirty (100%, 95% CI: 0.88–1) of the article had provided a clear description of the location, setting, or other relevant aspects of the system in which decisions need to be made. Twenty-five (83.33%, 95% CI: 0.66–0.92) of the article mentioned perspective of the study. Thirty (100%, 95% CI: 0.88–1) of the article describe the comparators and mention why they were chosen. Twenty-eight (93.33%, 95% CI: 0.78–0.98) of the article mention time horizon over which costs and consequences were evaluated. Twenty-four (80%, 95% CI: 0.62–0.90) of the article mentioned the discount rate (s) used for costs and outcomes. Thirty (100%, 95% CI: 0.88–1) of the article described what outcomes were used as the measure of benefit. Twenty-five (83.33%, 95% CI: 0.66–0.92) of the article reported the date of the price, method of price adjustment and currency and methods used for the currency conversion. Twenty-seven (90%, 95% CI: 0.74–0.96) of the article described the model structure being used for the analysis and explain why it is appropriate for use in the study. Twenty-seven (90%, 95% CI: 0.74–0.96) of the article listed the model assumptions. Twenty-nine (96.66%, 95% CI: 0.83–0.99) of the article described the effects of uncertainty for all input parameters. Twenty (66.66%, 95% CI: 0.48–0.80) of the article reported all funding sources. Twenty-six (86.66%, 95% CI: 0.70–0.94) of the article had disclosed a conflict of interest of the study contributors.
|Table 4: CHEERS checklist-Items to include when reporting economic evaluations of health interventions|
Click here to view
| Discussion|| |
Countries such as India have inadequate resources to manage the high load of communicable and noncommunicable diseases. Due to this, lawmakers are searching for ways to bridge the gap between the resources available and actual healthcare needs. Health economics can be one of the solutions to increase resource efficiency in health care. However, in most developing countries, the health economics has had little impact on medicine selection. Despite these hurdles, it is time to increase the use of health economic analyses in developing countries through improved training, support, and law-making. In this study, the mean QHES score was found to be 80.26 which was an indicator of good quality. A good quality was found with the QHES instrument because mean score of more than 70 was obtained with all the study except one [Table 3]. In a study by Desai et al., in 2012 which assessed the quality of pharmacoeconomic studies in India, the mean QHES score was 86. In a systematic review of the quality of pharmacoeconomic studies of China by Jiang et al., in 2014, the mean QHES score was 80 ± 10. It has been stated that the internal validity of economic studies cannot be judged by QHES. In addition, it has been further stated that there is better acceptability for the QHES among health-care policy decision-makers than among health economists. A pilot testing of the QHES in numerous setting particularly in the context of developing countries such as India, would lead to enhanced acceptance of the QHES. Another limitation of QHES is that instead of using a constant scale for each criterion, QHES uses yes/no replies.
The findings of QHES instrument were corroborated with the detailed quality check of the included studies with the CHEERS checklist. In this study, 96.66% of the articles suitably denoted the title and recognized the study as an economic analysis. In a report by Stawowczyk and Kawalec, in 2018, all the studies had adequately described the titles by identifying the study as an economic analysis and by mentioning the compared interventions. In this study, the maximally used economic assessment method was cost-effectiveness analysis. Similarly, in a report by Mehta and Nerurkar 2018, Desai et al., in 2012, cost-effectiveness analysis was the maximally used assessment method. Mehta and Nerurkar, in 2017 have further stated that this could be due to informal availability of figures on effectiveness in terms of outcomes and straightforward estimation methods.
The perspective of a pharmacoeconomic study is essential as it governs the types of costs to be measured. Expressing the perspective of the economic study is also vital for the reader to infer and apply the study conclusions. In this study, perspective was mentioned in 83.33% of the included studies [Table 4]. Health-care system/provider perspective was most commonly used followed by the societal perspective [Table 2]. In a study by Desai et al., in 2012, 50% of the studies reported the perspective. In a report by Stawowczyk and Kawalec, in 2018, the study perspective was described in majority of the included studies and public payer perspective was most commonly employed. In an assessment of economic evaluations of Korea by Yim EY et al., in 2012, majority of the studies mentioned the perspectives and 72% of them were evaluated from a societal perspective.
In this study, the time horizon was stated in 93.33% of studies, and the most commonly employed time horizon was lifetime [Table 2] and [Table 4]. The lifetime time horizon was mostly used as per a report of Stawowczyk and Kawalec, in 2018 and they have further stated that a time horizon encompassing lifetime is better for chronic ailments. In a study by Catalá-López et al., in 2016, the time horizon was mentioned in 97.8% of the studies and more than a 1-year horizon was employed in 78% of the studies.
In this study, the most frequently used modeling techniques used was decision-analytic model or decision tree model or combination of decision tree and Markov model [Table 2]. Decision analytical models epitomize an arrangement of chance events and decisions over time and are suitable for acute incidents of illness, but Markov models characterize recurring health states and are valuable in delineating chronic illness. Decision analysis is advantageous specifically in conditions where there is ambiguity about the balance of probable benefits and hazards, and costs, accompanying various health strategies. In this study, QALY was the most frequently employed outcome. In a report by Stawowczyk and Kawalec, in 2018, the QALY was the most frequently employed outcome in 88% of the studies. Majority of the guidelines endorse QALY as an outcome. In the present study, the most commonly used discount rate was 3% [Table 2]. Discount rate choice for cost and benefits depends on the projected comparative discrepancies in budgets and productivity over time. This estimation is very vague. Consequently, the exact choice for the discount rate of costs and benefits is uncertain. Commonly, the discount rate is taken at either 3% or 5% per annum.
A precondition of economic study is to execute a sensitivity analysis to estimate the uncertainty in the economic interpretations. In this study, 96.66% analysis mentioned sensitivity analysis [Table 4] and probabilistic sensitivity analysis was most commonly used for the evaluation of uncertainty [Table 2]. In a study by Nguyen et al., in 2017, sensitivity analysis was discussed in 80% reports. Sensitivity analysis can evaluate the discrepancy in the effectiveness, discount rate, costs, etc. The most commonly scrutinized form of uncertainty is that associated with the modeling procedures. Different measures of sensitivity analysis such as one-way or multiway analysis can be utilized as per the situations. In certain conditions, probabilistic analysis should be utilized for sensitivity analysis. In this study, 66.66% of the study mentioned about source of funding [Table 4]. In a study by Jiang et al., in 2014, 85% of the studies revealed their sponsor. Listing of the sponsor ensures transparency in the research conduct.
This study had many limitations. The studies included in this research were very diverse and had varied settings, varied patient populations, etc. There is always the chance of publication bias because of the inclusion of only published studies. Furthermore, because this study is based on literature searches in PubMed, Google Scholar, and Science Direct databases only, this analysis may not be considered exhaustive. Moreover, it should be noted that the CHEERS checklist is used to scrutinize only the quality of reporting rather than the quality of conduct of a health economic study. The CHEERS statement was designed based on earlier reporting specifications and with the help of a Delphi forum comprising of 47 members from diverse backgrounds. The creators of CHEERS themselves conceded that the constitution of the forum may have prejudiced the emphasis of the checklist, and subsequently, it might be inadequate in its usage for system dynamic models and its usage in both public health and in the context of developing countries such as India. Moreover, the evaluation procedure is not entirely independent of researcher's opinions or theoretical understanding. In this study, the interpretation of data was inevitably subjective. The assessment by multiple independent researchers would have been ideal to reduce bias and this is one of the limitations of the study.
| Conclusions|| |
Overall, the quality of reporting of the included health economic studies was good, but there is a scope for improvement. The findings of this study confirmed that the number of health economic studies in indexed journal has increased in the past 2 years. Journals can improve the quality of reporting of health economic studies by demanding adherence to CHEERS guideline from the authors and using QHES score as an indicator of good quality. There should be collaboration between researchers, regulatory bodies, journal editors, and policy-makers to raise the standard of health economic studies conducted in India or Indian context. Moreover, health economics should be taught in more detail in pharmacology undergraduate curriculum, and regulation should be in place to encourage healthy economic principles in choice of drug.
Financial support and sponsorship
Conflicts of interest
There are no conflicts of interest.
| References|| |
Mori A, Gavaza P, Robberstad B. Role of pharmacoeconomics in developing countries. Farmeconomia 2013;14:3-5.
Elsisi GH, Kaló Z, Eldessouki R, Elmahdawy MD, Saad A, Ragab S, et al
. Recommendations for reporting pharmacoeconomic evaluations in Egypt. Value Health Reg Issues 2013;2:319-27.
Ma H, Jian W, Xu T, He Y, Rizzo JA, Fang H. Quality of pharmacoeconomic research in China: A systematic review. Medicine (Baltimore) 2016;95:e5114.
Ahmad A, Patel I, Parimilakrishnan S, Mohanta GP, Chung H, Chang J. The role of pharmacoeconomics in current Indian healthcare system. J Res Pharm Pract 2013;2:3-9.
] [Full text]
Tran BX, Nong VM, Maher RM, Nguyen PK, Luu HN. A systematic review of scope and quality of health economic evaluation studies in Vietnam. PLoS One 2014;9:e103825.
Alefan Q, Rascati K. Pharmacoeconomic studies in world health organization eastern Mediterranean countries: Reporting completeness. Int J Technol Assess Health Care 2017;33:215-21.
McGhan WF, Al M, Doshi JA, Kamae I, Marx SE, Rindress D. The ISPOR good practices for quality improvement of cost-effectiveness research task force report. Value Health 2009;12:1086-99.
Husereau D, Drummond M, Petrou S, Carswell C, Moher D, Greenberg D, et al
. Consolidated health economic evaluation reporting standards (CHEERS) statement. BMJ 2013;346:f1049.
Catalá-López F, Ridao M, Alonso-Arroyo A, García-Altés A, Cameron C, González-Bermejo D, et al
. The quality of reporting methods and results of cost-effectiveness analyses in Spain: A methodological systematic review. Syst Rev 2016;5:6.
Pandey P, Pandey RD, Shah V. Evaluation of quality of pharmacoeconomic studies in Asia-pacific region and identification of influencing variables. Value Health Reg Issues 2018;15:70-5.
Mishra D, Nair SR. Systematic literature review to evaluate and characterize the health economics and outcomes research studies in India. Perspect Clin Res 2015;6:20-33.
] [Full text]
Desai PR, Chandwani HS, Rascati KL. Assessing the quality of pharmacoeconomic studies in India: A systematic review. Pharmacoeconomics 2012;30:749-62.
Mehta M, Nerurkar R. Evaluation and characterization of health economics and outcomes research in SAARC nations. Ther Innov Regul Sci 2018;52:348-53.
Ofman JJ, Sullivan SD, Neumann PJ, Chiou CF, Henning JM, Wade SW, et al
. Examining the value and quality of health economic analyses: Implications of utilizing the QHES. J Manag Care Pharm 2003;9:53-61.
Spiegel BM, Targownik LE, Kanwal F, Derosa V, Dulai GS, Gralnek IM, et al
. The quality of published health economic analyses in digestive diseases: A systematic review and quantitative appraisal. Gastroenterology 2004;127:403-11.
Jiang S, Ma X, Desai P, Yang L, Rascati K. A systematic review on the extent and quality of pharmacoeconomic publications for China. Value Health Reg Issues 2014;3:79-86.
Motheral B. Assessing the value of the quality of health economic studies (QHES). J Manag Care Pharm 2003;9:86-7.
Stawowczyk E, Kawalec P. A systematic review of the cost-effectiveness of biologics for ulcerative colitis. Pharmacoeconomics 2018;36:419-34.
Liu CC, Lui J, Oddone Paolucci E, Rudmik L. Systematic review of the quality of economic evaluations in the otolaryngology literature. Otolaryngol Head Neck Surg 2015;152:106-15.
Yim EY, Lim SH, Oh MJ, Park HK, Gong JR, Park SE, et al
. Assessment of pharmacoeconomic evaluations submitted for reimbursement in Korea. Value Health 2012;15:S104-10.
Milne RJ. Pharmacoeconomic models in disease management. Dis Manage Health Outcomes 1998;4:119-34.
Ademi Z, Kim H, Zomer E, Reid CM, Hollingsworth B, Liew D. Overview of pharmacoeconomic modelling methods. Br J Clin Pharmacol 2013;75:944-50.
Dang A, Likhar N, Alok U. Importance of economic evaluation in health care: An Indian perspective. Value Health Reg Issues 2016;9:78-83.
Cleemput I, van Wilder P, Huybrechts M, Vrijens F. Belgian methodological guidelines for pharmacoeconomic evaluations: Toward standardization of drug reimbursement requests. Value Health 2009;12:441-9.
Nguyen HN, Ly KN, Vo QT. Assessing the quality of health economic evaluation research by cheers instrument: A critical literature review in Laos, Cambodia, and Myanmar. J App Pharm Sci 2017;06:222-8.
Tsokeva Zh, Sokolova K, Yovtchev Y. Handling the results of pharmacoeconomic evaluations. Trakia J Sci 2006;4:1-6.
Tanna DG, Baio G. Methodological issues for the economic evaluation of health interventions: A concise state of the art. Epidemiology Biostatistics and Public Health 2013;10:E8917-1.
Al Aqeel SA, Al-Sultan M. The use of pharmacoeconomic evidence to support formulary decision making in Saudi Arabia: Methodological recommendations. Saudi Pharm J 2012;20:187-94.
Tai BB, Bae YH, Le QA. A systematic review of health economic evaluation studies using the patient's perspective. Value Health 2016;19:903-8.
Sanghera S, Frew E, Roberts T. Adapting the CHEERS statement for reporting cost-benefit analysis. Pharmacoeconomics 2015;33:533-4.
Kaptchuk TJ. Effect of interpretive bias on research evidence. BMJ 2003;326:1453-5.
[Table 1], [Table 2], [Table 3], [Table 4]