Next Article in Journal
Changes in Microbial Community Structures under Reclaimed Water Replenishment Conditions
Next Article in Special Issue
Effectiveness of Maintenance Immunosuppression Therapies in a Matched-Pair Analysis Cohort of 16 Years of Renal Transplant in the Brazilian National Health System
Previous Article in Journal
Facilitating Eudaimonic Well-Being in Mental Health Care Organizations: The Role of Servant Leadership and Workplace Civility Climate
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Review

Economic Evaluations Informed Exclusively by Real World Data: A Systematic Review

1
Teaching, Research & Innovation Unit, Parc Sanitari Sant Joan de Déu–Institut de Recerca Sant Joan de Déu, 08830 Barcelona, Spain
2
Primary Care Prevention and Health Promotion Network (redIAPP), 08007 Barcelona, Spain
3
CIBER of Epidemiology and Public Health (CIBERESP), 28029 Madrid, Spain
4
Faculty of Health Sciences, Universidad Icesi, 760030 Cali, Colombia
5
Institut Universitari d’Investigació en Atenció Primària Jordi Gol (IDIAPJGol), 08007 Barcelona, Spain
6
Universitat Autònoma de Barcelona, 08193 Bellaterra (Cerdanyola del Vallès), Spain
7
Health Science School, Universitat de Girona, 17071 Girona, Spain
*
Author to whom correspondence should be addressed.
Int. J. Environ. Res. Public Health 2020, 17(4), 1171; https://0-doi-org.brum.beds.ac.uk/10.3390/ijerph17041171
Submission received: 9 January 2020 / Revised: 3 February 2020 / Accepted: 8 February 2020 / Published: 12 February 2020

Abstract

:
Economic evaluations using Real World Data (RWD) has been increasing in the very recent years, however, this source of information has several advantages and limitations. The aim of this review was to assess the quality of full economic evaluations (EE) developed using RWD. A systematic review was carried out through articles from the following databases: PubMed, Embase, Web of Science and Centre for Reviews and Dissemination. Included were studies that employed RWD for both costs and effectiveness. Methodological quality of the studies was assessed using the Consolidated Health Economic Evaluation Reporting Standards (CHEERS) checklist. Of the 14,011 studies identified, 93 were included. Roughly half of the studies were carried out in a hospital setting. The most frequently assessed illnesses were neoplasms while the most evaluated interventions were pharmacological. The main source of costs and effects of RWD were information systems. The most frequent clinical outcome was survival. Some 47% of studies met at least 80% of CHEERS criteria. Studies were conducted with samples of 100–1000 patients or more, were randomized, and those that reported bias controls were those that fulfilled most CHEERS criteria. In conclusion, fewer than half the studies met 80% of the CHEERS checklist criteria.

1. Introduction

Given limited health resources and growth of health technologies, it is vital to use these resources efficiently. Economic evaluation (EE) studies are a useful tool to facilitate decision-making regarding which health technologies to use and/or finance. EE studies can be performed directly, based on individual patient data during randomized controlled trials (RCT “piggyback”), or by employing data from pragmatic studies [1]. EE analyses can also be carried out using decision models. Studies that employ models can obtain data and parameters from RCTs, pragmatic studies, expert opinion or a combination of these sources. Other sources of data for EE are observational studies using Real World Data [RWD]. This information typology can be used to measure both effects and costs. As such, RWD can be used for decision models [2,3], pragmatic studies and retrospective observational studies. This review focuses on complete EE studies that use RWD but are not based on decision models. Although there are a large number of definitions of RWD, for the purposes of this review we refer to RWD as “Data used for decision-making that are not collected in conventional RCTs” [4,5]. Real World Evidence is generated from RWD to inform a conclusion or a judgment [6].
The main strengths of RWD are the large populations, which let us develop studies in low-prevalence diseases (unfeasible in the context of an RCT) and extensive time periods that allow examination of long-term effects. Consequently, and due to the use of software for management, storage and analysis of large data sets collected during daily clinical practice, use of RWD has been constantly increasing in recent years. The use of RWD is widely employed by Health Technology Agencies with several aims [7]. It is used for epidemiologic purposes to assess clinical effectiveness and safety of medications and other health technologies [8,9], for assessing adherence to and persistence with pharmacological treatments [10], to assess the direct and indirect costs of a certain treatment and/or procedures [11] and it is also considered a valid source for market access and reimbursement [2]. RWD also has some limitations: information considered as RWD is not always all the information we would like to achieve the research goals; some confounding factors might not be taken into account; RWD is susceptible to misclassification or omission as these data are collected during routine clinical practice with no research purpose as distinct from the use of the case report form (CRF) in clinical trials [12]. Once these limitations are addressed, the potential of using RWD to create evidence to guide improvements in health care systems will be notorious.
Developing economic evaluation using RWD will help in decision-making, optimizing resources and gaining external validity. Consequently, the main research goal of this paper is to review the characteristics and assess the quality of full economic evaluations developed using RWD worldwide. Our aim is to provide information to decision-makers on the methodological quality of the evaluated studies and identify the strengths and weaknesses with a view to improving such studies.

2. Materials and Methods

This was a systematic review of published studies. A protocol was drafted to conduct a systematic review following the recommendations of Preferred Reporting Items for Systematic Reviews and Meta-Analyses for systematic review protocols (PRISMA-P) (http://www.prisma-statement.org/Extensions/Protocols.aspx). [13].
Protocol and registration. The search protocol was registered at Figshare (https://figshare.com/articles/Systematic_Review_Protocol/7265576) in October 2018. The registration code is 7265576.
Eligibility criteria. Defined in PICO format in accordance with the aims of the present study (Population: any, Intervention: any, Comparator: any, Outcome: Full economic evaluation). Studies were considered complete economic evaluations if they compared two or more alternatives and assessed both costs and outcomes (effectiveness) and thus could be considered as cost-effectiveness analysis, cost-utility analysis and cost-benefit analysis [1]. There were no restrictions regarding date or publication status. Only studies in English or Spanish were included.
Sources of information. A search was performed of the main clinical databases: PubMed, Embase, Web of Science (WOS) and the Centre for Reviews and Dissemination (CRD). This search was carried out between the 3rd of June and the 21st of August 2018.
Search strategy. The search strategy was adapted to each of the databases. An example of the PubMed Search is provided in Supplementary Material 1.
Inclusion criteria. Complete economic evaluations that used RWD (data not collected in conventional RCT [5]) for both costs and effectiveness.
Exclusion criteria. Decision-Analytic Models, cost-minimization analysis, cost-analysis, reviews, meta-analysis, comments, protocols, use of ad-hoc questionnaires in costs or effects, poster or presentations at congress or workshops, grey literature, abstract only, not health-related, book or letter.
Study selection. Selection was conducted by independent pairs according to established inclusion and exclusion criteria. In cases of disagreement, a third reviewer assessed the article. Six researchers participated in the process.
All article titles and abstracts identified in the search were initially reviewed using the Rayyan QCRI platform (https://rayyan.qcri.org) [14]. This site allows identification of duplicated articles and establishes the degree of agreement between reviewer pairs.
Finally, those articles considered potentially eligible were fully reviewed by the reviewer pairs.
Information extraction process. Information extraction was performed independently in duplicate. The 6 researchers participated in the process. Study information was recorded in an Excel template with the following information: title, year of publication, country and study setting, year(s) of inclusion of the study population, sample size, illness of the population (Using the International Classification of Diseases, ICD-10 [15]), population-medication (for those studies including a population receiving a medication or group of medications, ATC [16] was/were recorded) population demographic data, intervention, control, study methodology, source of RWD, costs, outcomes, time horizon, perspective, risk of bias and methods for controlling it, and source of funding.
Evaluation of methodological quality of included studies. Assessment of the methodological quality of studies was conducted using the Consolidated Health Economic Evaluation Reporting Standards (CHEERS) checklist [17], ISPOR Task Force Report. This list contains 24 items for evaluation. Each item is assigned values from 0 to 2; where 0, represents non-compliance with the item evaluated, 1 represents compliance and 2 means not applicable. Not applicable is recorded in those items that refer to economic evaluation models (examples 15 and 16), as articles using economic decision models were excluded for the purposes of the study.
Study quality was also assessed by the independent pairs and differences were resolved through consensus. When consensus was not reached, a third reviewer was consulted. Selection of studies to be evaluated by each researcher was randomized.
Studies using the RWD checklist were not assessed [6] as some of the questions were included in the CHEERS list and, additionally, gave rise to ambiguities in interpretation by the different reviewers.

3. Results

Of the 14,011 studies identified, 593 were potentially eligible following review of title and abstract. Figure 1 shows article selection and reasons for exclusion, mainly those studies using economic evaluation models, studies not using RWD or those that did not perform complete economic evaluations. A total of 93 articles were finally included [18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99,100,101,102,103,104,105,106,107,108,109,110].
Table 1 shows the characteristics of the included studies. Most were published between 2011 and 2018. The country with the most studies included was the USA (21.5%), while Europe provided 22 (notably France, Spain, Germany and United Kingdom). Sixteen studies did not report the country where it was carried out. Approximately 50% of studies performed the economic evaluation in a hospital setting and 29% did not specify the setting. Public funding was the type most frequently reported.
Regarding study population, the diseases most frequently assessed were neoplasms (14: chapter II), especially lung cancer and colorectal cancer (each representing 17% of the total of neoplasm studies), followed by breast cancer and hepatocellular cancer. Some 58% of studies did not report a medication or have it as an inclusion criterion for the study population. In studies where it was included, the most frequent therapeutic group (23%) was antineoplastic and immunomodulatory agents. Pharmacological interventions were the most commonly evaluated (41%) followed by surgical (21%). Three out of four controls used were usual care or pharmacological treatment.
Table 2 shows methodological information on RWD. The main source of RWD costs and outcomes was information systems, such as the use of administrative databases. Most frequent sample size was between 100 and 1000. Some 83% of studies were not randomized. The most frequently mentioned potential bias was confounding selection and the most common forms of control described were sensitivity analysis and propensity score.
Table 3 shows methodological aspects related to the economic evaluation. All studies employed direct costs while 15% combined direct and indirect. Mean survival measured as Life-years, life-years gained, life-years or mortality was the most frequently used clinical outcome in 46% of studies, while only 17% can be considered cost-utility evaluations. Around 50% of studies used a time horizon between 1 and 5 years. A total of 41% of studies carried out the evaluation from the payer’s perspective.
Supplementary Material 2 and 3 show general characteristics, aspects of RWD and information on the economic evaluation in each study.
The methodological quality of the economic evaluation for each included study is shown in Figure 2. It can be seen that the study items most frequently fulfilled with respect to the CHEERS checklist were the title, study population/subgroup, comparators, selection of clinical results and discussion. The least frequently fulfilled were setting and location, discount rate and characterizing heterogeneity. Of the items evaluated, two were not applicable in 100% of the reviewed studies as they are items on decision models (items 15 and 16) and item 12 did not apply to economic evaluation studies that did not use QALY as a measure of effectiveness. Forty-four studies (47.31%) met at least 80% of the CHEERS criteria (17 or more items, not including items 12, 15 and 16) and 5 (5%) met at least 50% of checklist items.
The CHEERS checklist focuses on the methodological aspects of EE rather than RWD. However, we examined whether there was any relationship between studies that met CHEERS criteria and RWD aspects, such as sample size, study randomization or whether the authors applied bias control methodology or not. This information is shown in Table 4, where it can be seen that most studies with a sample size lower than 1000 met at least 80% of CHEERS criteria, while 60% of those with 1000 or more reached the same threshold. Randomized studies and those that reported bias control met the highest percentage of CHEERS criteria (80% or more).
Please see Supplementary Material 4 for details on study quality assessments.

4. Discussion

An assessment was conducted of the methodological quality of the economic evaluations in the 93 studies using RWD. Pharmacological interventions were the most frequently evaluated. The most commonly used clinical outcome was survival. Approximately half of the studies used a time horizon of 1–5 years and carried out the evaluation from the funding entity’s perspective. Roughly half of the studies met 80% of CHEERS criteria. The most frequently fulfilled items were discussion and measurement of clinical effectiveness and the least were setting and location and characterizing heterogeneity.
Some studies did not specify in which country or setting the EE was performed, which is an important aspect for decision-makers or researchers who would like to use the study as a basis. As the decision to apply the results of an EE study is used for inclusion and or reimbursement, it will depend on other aspects of the context in which it is carried out.
A systematic review of the use of RWD for EE in Germany was identified [111] and some comparisons with this are presented below. However, some inclusion criteria differ from those of the present study, such as studies that use routine data as a source of information for costs or effects or both. Our review includes studies that use RWD in costs and effects. Moreover, Gansen [111] only included studies based on data from Germany and the authors put no restrictions on the publication language while, in contrast, we placed no restrictions on specific country databases but did restrict the publication languages.
Gansen [111] found that the principal illnesses assessed were cardiovascular diseases followed by type II diabetes mellitus whereas neoplasms were the main focus of evaluation in our review. This may be due to the fact that Gansen [111] only reviewed studies performed in Germany. Regarding the higher frequency of evaluations related to neoplasm interventions, it may be because some, such as lung cancer, are among the most frequently diagnosed and cause the most mortality [112,113]. Approximately 50% of the EE studies on neoplasm interventions were conducted in Canada (89% of studies in this country were on this illness group) and the United States (40% of total illnesses).
QALYs were used as a measure of effectiveness in only 17% of studies, despite being the health outcome recommended by EE methodological guidelines [114,115], especially in illnesses that compromise quality of life, such as cancer, musculoskeletal diseases or mental disorders. Furthermore, QALYS are a useful measure of effectiveness as they allow comparison between interventions in distinct illnesses. Nevertheless, use of QALYs in the EE reviewed was higher than in the Gansen [111] studies, which identified only two studies using this outcome (5.7%). It is possible that one of the reasons why QALYs were not included in EE using RWD is because quality of life is not systematically recorded in health systems or that the utilities cannot be estimated with registered clinical outcomes.
To ensure quality in an EE study based on RWD in analysis of cost and outcomes variables, it is important to have a good source of information for use of RWD. Quality will depend on the degree of data reliability and how to use it to respond to specific questions [5]. A crucial aspect is sample size, which in the reviewed studies varied between less than 100 and more than 5000. Sample size affects results, so it is vital to determine what range of sample size ensures that the economic evaluation has sufficient validity.
Study randomization is another key issue in use of RWD where the potential risk of bias is the most significant limitation that can arise in this type of data [5]. In this review, there was a predominance of non-randomized studies as sources of RWD were mainly information systems, where randomization is not possible in retrospective studies. However, some studies reported using methods to control bias or the risk of bias. Nevertheless, despite the application of statistical approaches to adjust for selection bias in observational studies, they do not have the methodological rigor of RCTs [5] that favor internal validity but are easily generalized to a more heterogeneous population [4,116]. More critical are those studies that did not report biases or methods to control them which, in this review, totaled approximately 20 studies.
The relationship between quality of RWD and EE methodological quality is reflected in the results of this review where studies with sample sizes of 1000 or more, randomized and with bias control, met more CHEERS criteria.
Only one study was performed in two countries [75] which evaluated the effectiveness and costs of an intervention. This may be due to the complexity of harmonizing data from differing health and registration systems.
Temporal horizon is a fundamental aspect in EE. However, we observed that some studies did not report it, or if they did, its use was not justified. Very few studies employed the society perspective, which is the most extensive [1] and, depending on the EE, the most appropriate, as there are costs borne by families that could represent 30% of total costs [115]. To approach the study from the society perspective, information on costs is required which is not normally recorded as, in most cases, the sources of information were administrative databases and/or medical records. In common with the Gansen review, most studies employed the payer’s perspective. Furthermore, some studies did not specify the study’s perspective.
Regarding the results of the methodological quality assessment, study strengths (items marked “yes” for 80% or more of the studies reviewed) were: proper placement of the title; defined aims, study population and comparators; description of health-outcome selection; measurement of effectiveness; and an adequate discussion section. Areas for improvement were also identified (items marked “no” for 50% or more of studies reviewed). For instance, half of the studies did not specify relevant aspects of decision-making procedures. EE setting and location can greatly affect results. Approximately half of the studies reported a discount rate and very few characterized heterogeneity, that is, they did not explain whether differences in costs, effects or cost-effectiveness were due to variations between patient subgroups or other observed variability. Heterogeneity is of great relevance in studies that use RWD as the patients included do not usually meet strict inclusion criteria or, at least, these are not defined prior to their participation. Consequently, discussion of possible differences due to distinct subject characteristics is crucial.
Approximately half of the studies fulfilled 80% of checklist items. This is in line with results reported by Gansen [111], who also identified characterization of heterogeneity as a weakness; although, in contrast with this review, the German review identified other weaknesses, such as not reporting the discount rate (item no. 9), currency, price date and conversion, and characterization of uncertainty.
As reflected in this review and the one conducted in Germany, use of RWD in EE is increasing [111]. RWD are an essential source for coverage, funding and health-technology reimbursement decisions [5]. However, as mentioned previously, these studies have some methodological limitations, such as confounding selection, identified by the authors of the studies in this review as the main potential risk. In another review [2], the main biases were confounding and missing data. A further risk in this type of information, frequent but not detailed, is registration quality. Registration accuracy and the heterogeneity of each variable considered as health service use should be reflected.
These RWD limitations represent an important methodological challenge as the benefits of RWD with respect to RCTs. These designs should be complementary given that RCTs continue to be the standard for demonstrating the clinical efficacy of interventions. However, it is necessary to determine their effectiveness through long-term observational studies, pragmatic studies and use of administrative databases [5].
Assessment by independent pairs of researchers and methodological rigor were applied throughout the review. In cases of discrepancies, consensus was sought and if agreement was not reached, a third researcher made the decision. We hope that the results obtained contribute information on the methodological quality of the evaluated studies and are useful to decision-makers in terms of recognizing good quality studies on health technology. Nevertheless, this review has some limitations.
One limitation identified was ambiguity in interpreting the checklist used to evaluate EE methodological quality, especially on some items such as CHEERS item 18 which refers to study parameters. Conducting the review using independent pairs facilitated identification of the ambiguities and increased the likelihood of the results being transparent and comparable over time [117]. In addition, the researchers came from different disciplines (pharmacy, medicine and economics) which allowed a comprehensive view and was a strength of the study.
Another limitation was that only studies in English or Spanish were included, a restriction which led to the omission of published studies that met other inclusion criteria.
The methodological quality of the performance of the RWD studies was not assessed, although the one reported by the authors of the reviewed studies was considered valid. There are various checklists available to evaluate the quality of RWD studies. One is that proposed by the ISPOR Task Force [6], which consists of 7 items related to Hypothesis Evaluating Treatment Effectiveness (HETE). As noted in the systematic review protocol registered at Figshare, the intention was to use this checklist with the studies included. However, although we largely adhered to the protocol, RWD quality was not assessed with this checklist as the items seemed ambiguous and, as mentioned above, some items appeared on the CHEERS list, so it was decided to use the CHEERS one only. The possible reason for the non-tracking of the ISPOR Task Force checklist is its publication date, as most articles were published prior to this date.
Another checklist is that proposed by Kreif, which consists of 5 questions dealing with some statistical aspects [118]. Nevertheless, we did not use it as, although it was complementary, it did not substitute the EE checklist and had some limitations such as not covering some aspects of statistical analysis. Moreover, this checklist did not allow a description of which specific statistical method should be prescribed to approach selection bias in analysis of cost-effectiveness [2].
For future studies, it would be beneficial to have a validated, adapted tool available to assess the methodological quality of EE studies using RWD. Health Technology Assessment agencies in Europe could collaborate with RWD use policies and recommendations on practical aspects of RWD collection and analysis [7]. These policies could act as a basis for the use of RWD in EE. This standardization is essential because, as the results of this review showed, the use of RWD to create Real World Evidence is increasing. Despite its limitations, the use of this evidence is becoming more and more necessary. In the future, its limitations should be considered and compensated. This evidence will be crucial in enabling improvements in the quality, safety and value of health care.

5. Conclusions

This review demonstrates that the use of RWD in carrying out EE with individual patient data is an increasingly common practice. A total of 93 studies were identified that met these conditions and their methodological quality was assessed using the CHEERS checklist. It was observed that fewer than half of the studies fulfilled 80% of checklist criteria. It shows the low quality of the EE included in this review. More attention should be paid to the reporting of methodologies and results in EE.
Meeting CHEERS checklist criteria is associated with RWD methodological aspects; studies with samples sizes of 1000 or more, randomized studies and those that reported some method of controlling bias were those that had the greatest likelihood of fulfilling 80% of CHEERS criteria.
Some important methodological differences were noted in RWD use and in the presentation of results, which represents a considerable challenge regarding standardization of methodology in EE using RWD. Meeting this challenge would facilitate decision-making among policy makers.
Use of the CHEERS checklist showed that there are important aspects of RWD that are not considered and that the use of more than one checklist is neither practical nor efficient and, as such, it would be valuable to have an EE checklist that includes RWD available.

Supplementary Materials

The following are available online at https://0-www-mdpi-com.brum.beds.ac.uk/1660-4601/17/4/1171/s1, Supp-Material-1-Search- strategy; Suppl-Material-2-Charact; Suppl-Material-3-RWD-EE; Suppl-Material-4-CHEERS.

Author Contributions

Conceptualization, I.A.-L., M.R.-V., E.P.-R. and C.G.-C.; Methodology, E.P.-R., M.R.-V., C.G.-C., and I.A.-L.; Software, E.P.-R. and I.A.-L.; Data Curation, All Authors, Formal Analysis, All authors.; Writing—Original Draft Preparation, E.P.-R. and I.A.-L.; Writing—Review & Editing, All authors; Visualization, E.P-R.; Supervision, I.A.-L. and M.R.-V. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding. The APC was funded by Primary Care Prevention and Health Promotion Network (redIAPP).

Acknowledgments

We thank the CIBERESP (CIBER in Epidemiology and Public Health, CB16/02/00322) and its Cohort Real World Data Subprogram, and the European Regional Development Fund for its support in the development of this study. We are grateful to Stephen Kelly for his contribution to editing the English of the article. We are grateful to Edward Burn for his contribution in the protocol definition and title and abstract search. MRV has a “Miguel Servet” research contract from the Instituto de Salud Carlos III (ISCIII) of the Ministry of Economy and Competitiveness (Spain) (CP19/00029).

Conflicts of Interest

The authors declare that they have no conflict of interest related to this systematic review.

References

  1. Drummond, M.F.; McGuire, A. Economic Evaluation in Health Care: Merging Theory with Practice; Oxford University Press: Oxford, UK, 2005. [Google Scholar]
  2. Bowrin, K.; Briere, J.B.; Levy, P. Cost-effectiveness analyses using real-world data: An overview of the literature. J. Med. Econ. 2019, 22, 545–553. [Google Scholar] [CrossRef] [PubMed]
  3. Terkola, R.; Antonanzas, F.; Postma, M. Economic evaluation of personalized medicine: A call for real-world data. Eur. J. Health Econ. 2017, 18, 1065–1067. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  4. Makady, A.; de Boer, A.; Hillege, H.; Klungel, O.; Goettsch, W. What Is Real-World Data? A Review of Definitions Based on Literature and Stakeholder Interviews. Value Health 2017, 20, 858–865. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  5. Garrison, L.P.; Neumann, P.J.; Erickson, P.; Marshall, D.; Mullins, C.D. Using real-world data for coverage and payment decisions: The ISPOR real-world data task force report. Value Health 2007, 10, 326–335. [Google Scholar] [CrossRef] [Green Version]
  6. Berger, M.L.; Sox, H.; Willke, R.J.; Brixner, D.L.; Eichler, H.G.; Goettsch, W.; Madigan, D.; Makady, A.; Schneeweiss, S.; Tarricone, R.; et al. Good practices for real-world data studies of treatment and/or comparative effectiveness: Recommendations from the joint ISPOR-ISPE Special Task Force on real-world evidence in health care decision making. Pharmacoepidemiol. Drug Saf. 2017, 26, 1033–1039. [Google Scholar] [CrossRef]
  7. Makady, A.; Ham, R.T.; de Boer, A.; Hillege, O.; Klungel, O.; Goettsch, W. GetReal Workpackage 1 Policies for Use of Real-World Data in Health Technology Assessment (HTA): A Comparative Study of Six HTA Agencies. Value Health 2017, 20, 520–532. [Google Scholar] [CrossRef] [Green Version]
  8. Peters, B.J.M.; Janssen, V.E.M.T.; Schramel, F.M.; van de Garde, E.M.W. Quantitative and qualitative assessment of real world data comparative effectiveness research of systemic therapies in lung oncology: A systematic review. Cancer Epidemiol. 2016, 44, 5–15. [Google Scholar] [CrossRef]
  9. Cha, M.-J.; Choi, E.-K.; Han, K.-D.; Lee, S.R.; Lim, W.H.; Oh, S.; Lip, G.Y.H. Effectiveness and Safety of Non-Vitamin K Antagonist Oral Anticoagulants in Asian Patients with Atrial Fibrillation. Stroke 2017, 48, 3040–3048. [Google Scholar] [CrossRef]
  10. Lemstra, M.; Nwankwo, C.; Bird, Y.; Moraros, J. Primary nonadherence to chronic disease medications: A meta-analysis. Patient Prefer. Adherence 2018, 12, 721–731. [Google Scholar] [CrossRef] [Green Version]
  11. Aznar-Lou, I.; Fernández, A.; Gil-Girbau, M.; Sabés-Figuera, R.; Fajó-Pascual, M.; Peñarrubia-Maria, M.T.; Serrano-Blanco, A.; Moreno-Peral, P.; Sánchez-Niubó, A.; March-Pujol, M. Impact of initial medication non-adherence on use of healthcare services and sick leave: A longitudinal study in a large primary care cohort in Spain. Br. J. Gen. Pract. 2017, 67, e614–e622. [Google Scholar] [CrossRef] [Green Version]
  12. Klonoff, D.C. The Expanding Role of Real-World Evidence Trials in Health Care Decision Making. J. Diabetes Sci. Technol. 2019, 14, 147–149. [Google Scholar] [CrossRef]
  13. Shamseer, L.; Moher, D.; Clarke, M.; Ghersi, D.; Liberati, A.; Petticrew, M.; Shekelle, P.; Stewart, L.A.; PRISMA-P Group. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: Elaboration and explanation. BMJ 2015, 350, g7647. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  14. Ouzzani, M.; Hammady, H.; Fedorowicz, Z.; Elmagarmid, A. Rayyan-a web and mobile app for systematic reviews. Syst. Rev. 2016, 5, 210. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  15. Organización Panamericana de la Salud. Clasificación Estadística Internacional de Enfermedades y Problemas Relacionados con la Salud, 10th ed.; Organización Panamericana de la Salud: Washington, DC, USA, 1995.
  16. WHO Collaboration Centre for Drug Statistics Methodology. ATC/DDD Index 2020 [Internet]; Norwegian Institute of Public Health: Oslo, Norway, 2019. Available online: https://www.whocc.no/atc_ddd_index/ (accessed on 1 July 2019).
  17. Husereau, D.; Drummond, M.; Petrou, S.; Carswell, C.; Moher, D.; Greenberg, D.; Augustovski, F.; Briggs, A.H.; Mauskopf, J.; Loder, E.; et al. Consolidated Health Economic Evaluation Reporting Standards (CHEERS)—Explanation and elaboration: A report of the ISPOR Health Economic Evaluation Publication Guidelines Good Reporting Practices Task Force. Value Health 2013, 16, 231–250. [Google Scholar] [CrossRef] [Green Version]
  18. Tsai, P.-C.; Liu, T.-W.; Huang, C.-F.; Yeh, M.L.; Day, C.Y.; Chuang, W.L.; Huang, J.F.; Yu, M.L. A real world cost effectiveness analysis of interferon-based therapy for HCV naive super-responders. J. Chin. Med. Assoc. 2018, 81, 670–675. [Google Scholar] [CrossRef]
  19. Tsai, P.-C.; Liu, T.-W.; Hsieh, M.-H.; Yeh, M.L.; Liang, P.C.; Lin, Y.H.; Huang, C.I.; Huang, C.F.; Hsieh, M.Y.; Hou, N.J.; et al. A real-world impact of cost-effectiveness of pegylated interferon/ribavarin regimens on treatment-naive chronic hepatitis C patients in Taiwan. Kaohsiung J. Med. Sci. 2017, 33, 44–49. [Google Scholar] [CrossRef] [Green Version]
  20. Gimenez-Manzorro, A.; Garcia-Gonzalez, X.; Rodriguez-Gonzalez, C.G.; Ochoa-Palominos, A.; Sanjurjo-Sáez, M.; Clemente-Ricote, G. Coste-efectividad y seguridad de telaprevir y boceprevir para el tratamiento de la hepatitis C crónica en la práctica clínica. Gastroenterol. Hepatol. 2015, 38, 575–582. [Google Scholar] [CrossRef]
  21. Deshmukh, A.A.; Zhao, H.; Das, P.; Chiao, E.Y.; You, Y.N.; Franzini, L.; Lairson, D.R.; Swartz, M.D.; Giordiano, S.H.; Cantor, S.B. Clinical and Economic Evaluation of Treatment Strategies for T1N0 Anal Canal Cancer. Am. J. Clin. Oncol. 2018, 41, 626–631. [Google Scholar] [CrossRef]
  22. Thein, H.-H.; Qiao, Y.; Zaheen, A.; Jembere, N.; Sapisochin, G.; Chan, K.K.W.; Yoshida, E.M.; Earle, C.C. Cost-effectiveness analysis of treatment with non-curative or palliative intent for hepatocellular carcinoma in the real-world setting. PLoS ONE 2017, 12, e0185198. [Google Scholar] [CrossRef]
  23. Parikh, N.D.; Marshall, V.D.; Singal, A.G. Survival and cost-effectiveness of sorafenib therapy in advanced hepatocellular carcinoma: An analysis of the SEER-Medicare database. Hepatology 2017, 65, 122–133. [Google Scholar] [CrossRef]
  24. Gilden, D.M.; Kubisiak, J.M.; Pohl, G.M.; Ball, D.E.; Gilden, D.E.; John, W.J.; Wetmore, S.; Winfree, K.B. Treatment patterns and cost-effectiveness of first line treatment of advanced non-squamous non-small cell lung cancer in Medicare patients. J. Med. Econ. 2017, 20, 151–161. [Google Scholar] [CrossRef] [PubMed]
  25. Thein, H.-H.; Isaranuwatchai, W.; Qiao, Y.; Wong, K.; Sapisochin, G.; Chan, K.K.W.; Yoshida, E.M.; Earle, C.C. Cost-effectiveness analysis of potentially curative and combination treatments for hepatocellular carcinoma with person-level data in a Canadian setting. Cancer Med. 2017, 6, 2017–2033. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  26. Chen, Y.; Lairson, D.R.; Chan, W.; Huo, J.; Du, X.L. Cost-Effectiveness of Novel Agents in Medicare Patients with Multiple Myeloma: Findings from a U.S. Payer’s Perspective. J. Manag. Care Spec. Pharm. 2017, 23, 831–843. [Google Scholar] [CrossRef] [PubMed]
  27. Woldemichael, A.; Onukwugha, E.; Seal, B.; Hanna, N.; Mullins, C.D. Sequential Therapies and the Cost-Effectiveness of Treating Metastatic Colon Cancer Patients. J. Manag. Care Spec. Pharm. 2016, 22, 628–639. [Google Scholar] [CrossRef]
  28. Lin, C.-Y.; Fang, H.-Y.; Feng, C.-L.; Li, C.C.; Chien, C.R. Cost-effectiveness of neoadjuvant concurrent chemoradiotherapy versus esophagectomy for locally advanced esophageal squamous cell carcinoma: A population-based matched case-control study. Thorac. Cancer 2016, 7, 288–295. [Google Scholar] [CrossRef]
  29. Hsia, T.-C.; Tu, C.-Y.; Fang, H.-Y.; Liang, J.A.; Li, C.C.; Chien, C.R. Cost and effectiveness of image-guided radiotherapy for non-operated localized lung cancer: A population-based propensity score-matched analysis. J. Thorac. Dis. 2015, 7, 1643–1649. [Google Scholar]
  30. Lairson, D.R.; Parikh, R.C.; Cormier, J.N. Cost-Effectiveness of Chemotherapy for Breast Cancer and Age Effect in Older Women. Value Health 2015, 18, 1070–1078. [Google Scholar] [CrossRef] [Green Version]
  31. Guglieri-Lopez, B.; Perez-Pitarch, A.; Porta-Oltra, B. Effectiveness, toxicity, and economic evaluation of vinflunine for the treatment of patients with transitional cell carcinoma in the Spanish outpatient setting. Anticancer Drugs 2015, 26, 860–865. [Google Scholar] [CrossRef]
  32. Wong, M.C.S.; Ching, J.Y.L.; Chan, V.C.W.; Sung, J.J.Y. The comparative cost-effectiveness of colorectal cancer screening using faecal immunochemical test vs. colonoscopy. Sci. Rep. 2015, 5, 13568. [Google Scholar] [CrossRef]
  33. Poonawalla, I.B.; Lairson, D.R.; Chan, W. Cost-Effectiveness of Neoadjuvant Chemotherapy versus Primary Surgery in Elderly Patients with Advanced Ovarian Cancer. Value Health 2015, 18, 387–395. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  34. Meenan, R.T.; Anderson, M.L.; Chubak, J. An economic evaluation of colorectal cancer screening in primary care practice. Am. J. Prev. Med. 2015, 48, 714–721. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  35. Lairson, D.R.; Parikh, R.C.; Cormier, J.N.; Du, X.L. Cost-utility analysis of platinum-based chemotherapy versus taxane and other regimens for ovarian cancer. Value Health 2014, 17, 34–42. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  36. Chien, C.-R.; Hsia, T.-C.; Chen, C.-Y. Cost-effectiveness of chemotherapy combined with thoracic radiotherapy versus chemotherapy alone for limited stage small cell lung cancer: A population-based propensity-score matched analysis. Thorac. Cancer 2014, 5, 530–536. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  37. Shaya, F.T.; Breunig, I.M.; Seal, B. Comparative and cost effectiveness of treatment modalities for hepatocellular carcinoma in SEER-Medicare. Pharmacoeconomics 2014, 32, 63–74. [Google Scholar] [CrossRef]
  38. Khor, S.; Beca, J.; Krahn, M. Real world costs and cost-effectiveness of Rituximab for diffuse large B-cell lymphoma patients: A population-based analysis. BMC Cancer 2014, 14, 586. [Google Scholar] [CrossRef] [Green Version]
  39. Byrd, J.K.; Smith, K.J.; de Almeida, J.R. Transoral Robotic Surgery and the Unknown Primary: A Cost-Effectiveness Analysis. Otolaryngol. Head Neck Surg. 2014, 150, 976–982. [Google Scholar] [CrossRef] [Green Version]
  40. Cho, E.; Kang, M.H.; Choi, K.S. Cost-effectiveness of Korea’s National Cervical Cancer Screening Program. Asian Pac. J. Cancer Prev. 2013, 14, 4329–4334. [Google Scholar] [CrossRef] [Green Version]
  41. Shah, M.; Winfree, K.B.; Peterson, P. Cost effectiveness of first-line pemetrexed plus platinum compared with other regimens in the treatment of patients with nonsquamous non-small cell lung cancer in the US outpatient setting. Lung Cancer 2013, 82, 121–127. [Google Scholar] [CrossRef] [Green Version]
  42. Corso, A.; Mangiacavalli, S.; Cocito, F. Long term evaluation of the impact of autologous peripheral blood stem cell transplantation in multiple myeloma: A cost-effectiveness analysis. PLoS ONE 2013, 8, e75047. [Google Scholar] [CrossRef]
  43. Griffiths, R.I.; Gleeson, M.L.; Mikhael, J.; Danese, M.D. Impact on Medical Cost, Cumulative Survival, and Cost-Effectiveness of Adding Rituximab to First-Line Chemotherapy for Follicular Lymphoma in Elderly Patients: An Observational Cohort Study Based on SEER-Medicare. J. Cancer Epidemiol. 2012, 2012, 978391. [Google Scholar] [CrossRef] [Green Version]
  44. Griffiths, R.I.; Gleeson, M.L.; Mikhael, J. Comparative effectiveness and cost of adding rituximab to first-line chemotherapy for elderly patients diagnosed with diffuse large B-cell lymphoma. Cancer 2012, 118, 6079–6088. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  45. Cromwell, I.; van der Hoek, K.; Malfair Taylor, S.C. Erlotinib or best supportive care for third-line treatment of advanced non-small-cell lung cancer: A real-world cost-effectiveness analysis. Lung Cancer 2012, 76, 472–477. [Google Scholar] [CrossRef] [PubMed]
  46. Nelson, M.A.; Shetty, S.; Kulakodlu, M. A comparison of mortality and costs associated with FOLFOX versus FOLFIRI in stage IV colorectal cancer. J. Med. Econ. 2011, 14, 179–186. [Google Scholar] [CrossRef] [PubMed]
  47. Cromwell, I.; van der Hoek, K.; Melosky, B.; Peacock, S. Erlotinib or docetaxel for second-line treatment of non-small cell lung cancer: A real-world cost-effectiveness analysis. J. Thorac. Oncol. 2011, 6, 2097–2103. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  48. Lin, Y.-F.; Lairson, D.R.; Chan, W. The costs and cost-effectiveness of allogeneic peripheral blood stem cell transplantation versus bone marrow transplantation in pediatric patients with acute leukemia. Biol. Blood Marrow Transplant. 2010, 16, 1272–1281. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  49. Shih, Y.-C.T.; Pan, I.-W.; Tsai, Y.-W. Information technology facilitates cost-effectiveness analysis in developing countries: An observational study of breast cancer chemotherapy in Taiwan. Pharmacoeconomics 2009, 27, 947–961. [Google Scholar] [CrossRef]
  50. Aletti, G.D.; Podratz, K.C.; Moriarty, J.P. Aggressive and complex surgery for advanced ovarian cancer: An economic analysis. Gynecol. Oncol. 2009, 112, 16–21. [Google Scholar] [CrossRef]
  51. Vu, T.; Ellard, S.; Speers, C.H. Survival outcome and cost-effectiveness with docetaxel and paclitaxel in patients with metastatic breast cancer: A population-based evaluation. Ann. Oncol. 2008, 19, 461–464. [Google Scholar] [CrossRef]
  52. Borie, F.; Daures, J.-P.; Millat, B.; Tretarre, B. Cost and effectiveness of follow-up examinations in patients with colorectal cancer resected for cure in a French population-based study. J. Gastrointest. Surg. 2004, 8, 552–558. [Google Scholar] [CrossRef]
  53. Greanya, E.D.; Taylor, S.C.M.; BscPharm, F.H. Temozolomide for malignant gliomas in British Columbia: A population-based cost-effectiveness analysis. J. Oncol. Pharm. Pract. 2004, 10, 201–209. [Google Scholar] [CrossRef]
  54. Nakama, H.; Zhang, B.; Fattah, A.S. A cost-effective analysis of the optimum number of stool specimens collected for immunochemical occult blood screening for colorectal cancer. Eur. J. Cancer 2000, 36, 647–650. [Google Scholar] [CrossRef]
  55. Leivo, T.; Salminen, T.; Sintonen, H. Incremental cost-effectiveness of double-reading mammograms. Breast Cancer Res. Treat. 1999, 54, 261–267. [Google Scholar] [CrossRef]
  56. Daliri, A.A.K.; Haghparast, H.; Mamikhani, J. Cost-effectiveness of prophylaxis against on-demand treatment in boys with severe hemophilia A in Iran. Int. J. Technol. Assess. Health Care 2009, 25, 584–587. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  57. Jeong, Y.J.; Kim, H.; Baik, S.J. Analysis and comparison of the cost-effectiveness of statins according to the baseline low-density lipoprotein cholesterol level in Korea. J. Clin. Pharm. Ther. 2017, 42, 292–300. [Google Scholar] [CrossRef] [PubMed]
  58. Lajara, R.; Davidson, J.A.; Nikkel, C.C.; Morris, T.L. Clinical and cost-effectiveness of insulin delivery with V-GO((R)) disposable insulin delivery device versus multiple daily injections in patients with type 2 diabetes inadequately controlled on basal insulin. Endocr. Pract. 2016, 22, 726–735. [Google Scholar] [CrossRef] [PubMed]
  59. Smith, D.H.; O’Keeffe-Rosetti, M.; Owen-Smith, A.A. Improving Adherence to Cardiovascular Therapies: An Economic Evaluation of a Randomized Pragmatic Trial. Value Health 2016, 19, 176–184. [Google Scholar] [CrossRef]
  60. Hsieh, H.-M.; Gu, S.-M.; Shin, S.-J. Cost-Effectiveness of a Diabetes Pay-For-Performance Program in Diabetes Patients with Multiple Chronic Conditions. PLoS ONE 2015, 10, e0133163. [Google Scholar] [CrossRef]
  61. Hsieh, H.-M.; Tsai, S.-L.; Shin, S.-J. Cost-effectiveness of diabetes pay-for-performance incentive designs. Med. Care 2015, 53, 106–115. [Google Scholar] [CrossRef]
  62. Tim-Aroon, T.; Harmon, H.M.; Nock, M.L. Stopping Parenteral Nutrition for 3 Hours Reduces False Positives in Newborn Screening. J. Pediatr. 2015, 167, 312–316. [Google Scholar] [CrossRef]
  63. Dodd, J.M.; Ahmed, S.; Karnon, J. The cost-effectiveness of providing antenatal lifestyle advice for women who are overweight or obese: The LIMIT randomised trial. BMC Obes. 2015, 2, 14. [Google Scholar] [CrossRef] [Green Version]
  64. DeKoven, M.; Lee, W.C.; Bouchard, J. Real-world cost-effectiveness: Lower cost of treating patients to glycemic goal with liraglutide versus exenatide. Adv. Ther. 2014, 31, 202–216. [Google Scholar] [CrossRef] [Green Version]
  65. Kolu, P.; Raitanen, J.; Rissanen, P.; Luoto, R. Cost-effectiveness of lifestyle counselling as primary prevention of gestational diabetes mellitus: Findings from a cluster-randomised trial. PLoS ONE 2013, 8, e56392. [Google Scholar] [CrossRef]
  66. Tarraga-Lopez, P.J.; Celada-Rodriguez, A.; Cerdan-Oliver, M. A pharmacoeconomic evaluation of statins in the treatment of hypercholesterolaemia in the primary care setting in Spain. Pharmacoeconomics 2005, 23, 275–287. [Google Scholar] [CrossRef]
  67. Kamagata, E.; Yamada, K. Improvements in Quality-Adjusted Life Years and Cost-Utility After Pharmacotherapy for Premenstrual Dysphoric Disorder: A Retrospective Study. Clin. Drug Investig. 2018, 38, 49–55. [Google Scholar] [CrossRef] [PubMed]
  68. Gisev, N.; Shanahan, M.; Weatherburn, D.J. A cost-effectiveness analysis of opioid substitution therapy upon prison release in reducing mortality among people with a history of opioid dependence. Addiction 2015, 110, 1975–1984. [Google Scholar] [CrossRef] [PubMed]
  69. Cocchi, A.; Mapelli, V.; Meneghelli, A.; Preti, A. Cost-effectiveness of treating first-episode psychosis: Five-year follow-up results from an Italian early intervention programme. Early Interv. Psychiatry 2011, 5, 203–211. [Google Scholar] [PubMed]
  70. Ypinga, J.H.L.; de Vries, N.M.; Boonen, L.H.H.M. Effectiveness and costs of specialised physiotherapy given via ParkinsonNet: A retrospective analysis of medical claims data. Lancet Neurol. 2018, 17, 153–161. [Google Scholar] [CrossRef]
  71. Holland, M.; Noeller, J.; Buatti, J. The cost-effectiveness of surgery for trigeminal neuralgia in surgically naive patients: A retrospective study. Clin. Neurol. Neurosurg. 2015, 137, 34–37. [Google Scholar] [CrossRef] [PubMed]
  72. Diebold, G.; Ducray, F.; Henaine, A.-M. Management of glioblastoma: Comparison of clinical practices and cost-effectiveness in two cohorts of patients (2008 versus 2004) diagnosed in a French university hospital. J. Clin. Pharm. Ther. 2014, 39, 642–648. [Google Scholar] [CrossRef] [PubMed]
  73. Bose, S.; Ang, M.; Mehta, J.S. Cost-effectiveness of Descemet’s stripping endothelial keratoplasty versus penetrating keratoplasty. Ophthalmology 2013, 120, 464–470. [Google Scholar] [CrossRef]
  74. Michel, M.; Becquemin, J.-P.; Marzelle, J. Editor’s Choice—A Study of the Cost-effectiveness of Fenestrated/branched EVAR Compared with Open Surgery for Patients with Complex Aortic Aneurysms at 2 Years. Eur. J. Vasc. Endovasc. Surg. 2018, 56, 15–21. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  75. Aissaoui, N.; Morshuis, M.; Maoulida, H. Management of end-stage heart failure patients with or without ventricular assist device: An observational comparison of clinical and economic outcomes. Eur. J. Cardiothorac. Surg. 2018, 53, 170–177. [Google Scholar] [CrossRef] [PubMed]
  76. Ariyaratne, T.V.; Ademi, Z.; Huq, M. The Real-World Cost-Effectiveness of Coronary Artery Bypass Surgery Versus Stenting in High-Risk Patients: Propensity Score-Matched Analysis of a Single-Centre Experience. Appl. Health Econ. Health Policy 2018, 16, 661–674. [Google Scholar] [CrossRef]
  77. Sogaard, R.; Lindholt, J.S. Cost-effectiveness of population-based vascular disease screening and intervention in men from the Viborg Vascular (VIVA) trial. Br. J. Surg. 2018, 105, 1283–1293. [Google Scholar] [CrossRef] [PubMed]
  78. Sloth, A.D.; Schmidt, M.R.; Munk, K. Cost-effectiveness of remote ischaemic conditioning as an adjunct to primary percutaneous coronary intervention in patients with ST-elevation myocardial infarction. Eur. Hear. J. Acute Cardiovasc. Care 2017, 6, 244–253. [Google Scholar] [CrossRef] [PubMed]
  79. Scholz, J.; Portela, L.D.; Abe, T.M.O. Cost-effectiveness analysis of smoking-cessation treatment using electronic medical records in a cardiovascular hospital. Clin. Trials Regul. Sci. Cardiol. 2016, 14, 1–3. [Google Scholar] [CrossRef]
  80. Baumler, M.; Stargardt, T.; Schreyogg, J.; Busse, R. Cost effectiveness of drug-eluting stents in acute myocardial infarction patients in Germany: Results from administrative data using a propensity score-matching approach. Appl. Health Econ. Health Policy 2012, 10, 235–248. [Google Scholar] [CrossRef]
  81. Schafer, P.E.; Sacrinty, M.T.; Cohen, D.J. Cost-effectiveness of drug-eluting stents versus bare metal stents in clinical practice. Circ. Cardiovasc. Qual. Outcomes 2011, 4, 408–415. [Google Scholar] [CrossRef] [Green Version]
  82. Huang, Y.; Zhang, R.; Culler, S.D.; Kutner, N.G. Costs and effectiveness of cardiac rehabilitation for dialysis patients following coronary bypass. Kidney Int. 2008, 74, 1079–1084. [Google Scholar] [CrossRef] [Green Version]
  83. Kim, L.G.; PScott, R.A.; Ashton, H.A.; Thompson, S.G. A sustained mortality benefit from screening for abdominal aortic aneurysm. Ann. Intern. Med. 2007, 146, 699–706. [Google Scholar] [CrossRef]
  84. van Mastrigt, G.A.P.G.; Heijmans, J.; Severens, J.L. Short-stay intensive care after coronary artery bypass surgery: Randomized clinical trial on safety and cost-effectiveness. Crit. Care Med. 2006, 34, 65–75. [Google Scholar] [CrossRef]
  85. Barnett, P.G.; Chen, S.; Boden, W.E. Cost-effectiveness of a conservative, ischemia-guided management strategy after non-Q-wave myocardial infarction: Results of a randomized trial. Circulation 2002, 105, 680–684. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  86. Reed, S.O.; Mullins, C.D.; Magder, L.S. Cost effectiveness of abciximab during routine medical practice. Pharmacoeconomics 2000, 18, 265–274. [Google Scholar] [CrossRef] [PubMed]
  87. Ronaldson, S.J.; Raghunath, A.; Torgerson, D.J.; Van Staa, T. Cost-effectiveness of antibiotics for COPD management: Observational analysis using CPRD data. ERJ Open Res. 2017, 3, 00085-2016. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  88. Price, D.; Small, I.; Haughney, J. Clinical and cost effectiveness of switching asthma patients from fluticasone-salmeterol to extra-fine particle beclometasone-formoterol: A retrospective matched observational study of real-world patients. Prim. Care Respir. J. 2013, 22, 439–448. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  89. Fleming, N.S.; Ogola, G.; Ballard, D.J. Implementing a standardized order set for community-acquired pneumonia: Impact on mortality and cost. Jt Comm. J. Qual. Patient Saf. 2009, 35, 414–421. [Google Scholar] [CrossRef]
  90. Weiss, K.; Buxton, M.; Andersson, F.L.; Lamm, C.-J.; Liljas, B.; Sullivan, S.D. Cost-effectiveness of early intervention with once-daily budesonide in children with mild persistent asthma: Results from the START study. Pediatr. Allergy Immunol. 2006, 17, s21–s27. [Google Scholar] [CrossRef]
  91. Rystedt, J.M.L.; Tingstedt, B.; Montgomery, F.; Montgomery, A.K. Routine intraoperative cholangiography during cholecystectomy is a cost-effective approach when analysing the cost of iatrogenic bile duct injuries. HPB 2017, 19, 881–888. [Google Scholar] [CrossRef] [Green Version]
  92. Kolker, J.L.; Damiano, P.C.; Flach, S.D. The cost-effectiveness of large amalgam and crown restorations over a 10-year period. J. Public Health Dent. 2006, 66, 57–63. [Google Scholar] [CrossRef]
  93. Owesen, C.; Aas, E.; Aroen, A. Surgical reconstruction is a cost-efficient treatment option for isolated PCL injuries. Knee Surg. Sports Traumatol. Arthrosc. 2018, 26, 1053–1058. [Google Scholar] [CrossRef] [Green Version]
  94. Raman, T.; Nayar, S.K.; Liu, S. Cost-Effectiveness of Primary and Revision Surgery for Adult Spinal Deformity. Spine 2018, 43, 791–797. [Google Scholar] [CrossRef]
  95. Gum, J.L.; Hostin, R.; Robinson, C. Impact of cost valuation on cost-effectiveness in adult spine deformity surgery. Spine J. 2017, 17, 96–101. [Google Scholar] [CrossRef] [PubMed]
  96. Carreon, L.Y.; Glassman, S.D.; Ghogawala, Z. Modeled cost-effectiveness of transforaminal lumbar interbody fusion compared with posterolateral fusion for spondylolisthesis using N(2)QOD data. J. Neurosurg. Spine 2016, 24, 916–921. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  97. Alvin, M.D.; Lubelski, D.; Abdullah, K.G. Cost-Utility Analysis of Instrumented Fusion Versus Decompression Alone for Grade I L4-L5 Spondylolisthesis at 1-Year Follow-up: A Pilot Study. Clin. Spine Surg. 2016, 29, E80–E86. [Google Scholar] [CrossRef] [PubMed]
  98. Joensuu, J.T.; Aaltonen, K.J.; Aronen, P. Cost-effectiveness of biologic compared with conventional synthetic disease-modifying anti-rheumatic drugs in patients with rheumatoid arthritis: A Register study. Rheumatology 2016, 55, 1803–1811. [Google Scholar] [CrossRef] [Green Version]
  99. Heinrich, S.; Rapp, K.; Stuhldreher, N. Cost-effectiveness of a multifactorial fall prevention program in nursing homes. Osteoporos. Int. 2013, 24, 1215–1223. [Google Scholar] [CrossRef]
  100. Lubowitz, J.H.; Appleby, D. Cost-effectiveness analysis of the most common orthopaedic surgery procedures: Knee arthroscopy and knee anterior cruciate ligament reconstruction. Arthroscopy 2011, 27, 1317–1322. [Google Scholar] [CrossRef]
  101. Barra, L.; Pope, J.E.; Payne, M. Real-world anti-tumor necrosis factor treatment in rheumatoid arthritis, psoriatic arthritis, and ankylosing spondylitis: Cost-effectiveness based on number needed to treat to improve health assessment questionnaire. J. Rheumatol. 2009, 36, 1421–1428. [Google Scholar] [CrossRef] [Green Version]
  102. Huddleston, J.M.; Long, K.H.; Naessens, J.M. Medical and surgical comanagement after elective hip and knee arthroplasty: A randomized, controlled trial. Ann. Intern. Med. 2004, 141, 28–38. [Google Scholar] [CrossRef]
  103. Hsieh, H.-M.; Lin, M.-Y.; Chiu, Y.-W. Economic evaluation of a pre-ESRD pay-for-performance programme in advanced chronic kidney disease patients. Nephrol. Dial. Transplant. 2017, 32, 1184–1194. [Google Scholar] [CrossRef]
  104. Taylor, J.S.; Marten, C.A.; Munsell, M.F. The DISINFECT Initiative: Decreasing the Incidence of Surgical INFECTions in Gynecologic Oncology. Ann. Surg. Oncol. 2017, 24, 362–368. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  105. Pan, M.; Huang, L.-Y.; Zhen, L.; Li, D.-Z. A cost-effectiveness analysis comparing two different strategies in advanced maternal age: Combined first-trimester screening and maternal blood cell-free DNA testing. Taiwan J. Obstet. Gynecol. 2018, 57, 536–540. [Google Scholar] [CrossRef] [PubMed]
  106. Haggstrom, J.; Sampaio, F.; Eurenius, E. Is the Salut Programme an effective and cost-effective universal health promotion intervention for parents and their children? A register-based retrospective observational study. BMJ Open 2017, 7, e016732. [Google Scholar] [CrossRef] [PubMed]
  107. Tracy, S.K.; Hartz, D.L.; Tracy, M.B. Caseload midwifery care versus standard maternity care for women of any risk: M@NGO, a randomised controlled trial. Lancet 2013, 382, 1723–1732. [Google Scholar] [CrossRef]
  108. Dodson, T.B.; Pfeffle, R.C. Cost-effectiveness analysis of open reduction/nonrigid fixation and open reduction/rigid fixation to treat mandibular fractures. Oral Surg. Oral Med. Oral Pathol. Oral Radiol. 1995, 80, 5–11. [Google Scholar] [CrossRef]
  109. Lopez-Picazo, J.J.; Ruiz, J.C.; Sanchez, J.F.; Ariza, A.; Aguilera, B. A randomized trial of the effectiveness and efficiency of interventions to reduce potential drug interactions in primary care. Am. J. Med. Qual. 2011, 26, 145–153. [Google Scholar] [CrossRef]
  110. Sevick, L.K.; Santana, M.-J.; Ghali, W.A.; Clement, F. Prospective economic evaluation of an electronic discharge communication tool: Analysis of a randomised controlled trial. BMJ Open 2017, 7, e019139. [Google Scholar] [CrossRef]
  111. Gansen, F.M. Health economic evaluations based on routine data in Germany: A systematic review. BMC Health Serv. Res. 2018, 18, 268. [Google Scholar] [CrossRef]
  112. Cheng, T.-Y.D.; Cramb, S.M.; Baade, P.D.; Youlden, D.R.; Nwogu, C.; Reid, M.E. The International Epidemiology of Lung Cancer: Latest Trends, Disparities, and Tumor Characteristics. J. Thorac. Oncol. 2016, 11, 1653–1671. [Google Scholar] [CrossRef] [Green Version]
  113. Ferlay, J.; Soerjomataram, I.; Dikshit, R.; Eser, S.; Mathers, C.; Rebelo, M.; Parkin, D.M.; Forman, D.; Bray, F. Cancer incidence and mortality worldwide: Sources, methods and major patterns in GLOBOCAN 2012. Int. J. Cancer 2015, 136, E359–E386. [Google Scholar] [CrossRef]
  114. Puig-Junoy, J.; Oliva-Moreno, J.; Trapero-Bertrán, M.; Abellán-Perpiñán, J.M.; Brosa Riestra, M. Guía y Recomendaciones para la Realización y Presentación de Evaluaciones Económicas y Análisis de Impacto Presupuestario de Medicamentos en el Ámbito del CatSalut; Servei Català de la Salut: Barcelona, Spain, 2014; Available online: https://catsalut.gencat.cat/web/.content/minisite/catsalut/proveidors_professionals/medicaments_farmacia/farmaeconomica/caeip/gaeip_publica_castellano_octubre2014_catsalut.pdf (accessed on 9 January 2020).
  115. Ortega Eslava, A.; Marín Gil, R.; Fraga Fuentes, M.D.; López-Briz, E.; Puigventós Latorre, F. Guía de Evaluación Económica e Impacto Presupuestario en los Informes de Evaluación de Medicamentos; Guía práctica; Sociedad Española de Farmacia Hospitalaria: Madrid, Spain, 2019; Available online: https://gruposdetrabajo.sefh.es/genesis/genesis/Documents/GUIA_EE_IP_GENESIS-SEFH_19_01_2017.pdf (accessed on 9 January 2020).
  116. Mulder, R.; Singh, A.B.; Hamilton, A.; Das, P.; Outhred, T.; Morris, G.; Bassett, D.; Baune, B.T.; Berk, M.; Boyce, P.; et al. The limitations of using randomised controlled trials as a basis for developing treatment guidelines. Evid. Based Ment. Health 2018, 21, 4–6. [Google Scholar] [CrossRef] [PubMed]
  117. Watts, R.D.; Li, I.W. Use of Checklists in Reviews of Health Economic Evaluations, 2010 to 2018. Value Health 2019, 22, 377–382. [Google Scholar] [CrossRef] [PubMed]
  118. Kreif, N.; Grieve, R.; Sadique, M.Z. Statistical methods for cost-effectiveness analyses that use observational data: A critical appraisal tool and review of current practice. Health Econ. 2013, 22, 486–500. [Google Scholar] [CrossRef] [PubMed]
Figure 1. Flow chart of paper selection.
Figure 1. Flow chart of paper selection.
Ijerph 17 01171 g001
Figure 2. Quality assessment according CHEERS check list. Items 15 (Choice of model) and 16 (assumptions) only apply to mathematical models and were not scored. NA: Not apply
Figure 2. Quality assessment according CHEERS check list. Items 15 (Choice of model) and 16 (assumptions) only apply to mathematical models and were not scored. NA: Not apply
Ijerph 17 01171 g002
Table 1. Characteristics of studies included.
Table 1. Characteristics of studies included.
Characteristicsn (%)
Year publication
1995–20106 (6.45)
2011–201559 (63.44)
2016–201828 (30.11)
Region/Country
Europe22 (23.66)
United States20 (21.51)
China11 (11.83)
Canada9 (9.68)
Australia4 (4.3)
Others6 (6.45)
Not reported16 (17.20)
Setting
Hospital45 (48.39)
Primary care15 (16.13)
Hospital and primary care3 (3.23)
Specialized care3 (3.23)
Not reported27 (29.03)
Source of funding
Public42 (45.16)
Private15 (16.13)
Public and private3 (3.23)
No funding4 (4.3)
Not reported29 (31.18)
Eligibility criteria based on disease (ICD10)
Chapter II: Neoplasms35 (37.63)
Chapter IV: Endocrine, nutritional and metabolic diseases10 (10.75)
Chapter IX: Diseases of the circulatory system14 (15.05)
Chapter XIII: Diseases of the musculoskeletal system and connective tissue10 (10.75)
Others24 (25.81)
No disease1 (1.08)
Eligibility criteria based on treatment
ATC L (antineoplastic and immunomodulating agents)22 (23.66)
Other ATC codes (A, B, C, J, N and R)14 (15.05)
Radiotherapy2 (2.15)
Cellular therapy1 (1.08)
Not reported/not applied54 (58.06)
Intervention
Pharmacological38 (40.86)
Surgical20 (21.51)
Screening program9 (9.68)
Prevention program (education)6 (6.45)
Management program7 (7.53)
Rehabilitation4 (4.3)
Radiotherapy3 (3.23)
Cellular therapy2 (2.15)
Others4 (4.3)
Control
Usual care38 (40.86)
Pharmacological29 (31.18)
Surgical17 (18.28)
Screening program3 (3.23)
Others6 (6.45)
ICD: International Statistical Classification of Diseases; ATC: Anatomical therapeutic clinical.
Table 2. Methodological information about Real World Data (RWD).
Table 2. Methodological information about Real World Data (RWD).
Characteristicsn (%)
RWD source
Information systems63 (67.74)
Clinical records12 (12.9)
Clinical records and information systems11 (11.83)
Not reported7 (7.53)
Sample size
<1009 (9.68)
Between 100 and 100038 (40.86)
Between 1000 and 500024 (25.81)
>500018 (19.35)
Not reported4 (4.3)
Study methodology
Not randomized77 (82.8)
Randomized16 (17.2)
Reported potential risk of bias *
Confounding selection39 (41.94)
Confounding factors14 (15.05)
Adjustment variables not specified14 (15.05)
Missing data10 (10.75)
Bias in measurement or registry10 (10.75)
Censoring data3 (3.23)
Time-related bias2 (2.15)
Not reported19 (20.43)
Reported methods to control *
Sensitivity analysis27 (29.03)
Propensity score25 (26.88)
Bootstrap20 (21.51)
Adjusting by confounding6 (6.45)
Imputation missing data5 (5.38)
Not reported22 (23.66)
* Some studies reported different potential risks of bias or methods to control.
Table 3. Methodologic information about economic evaluation.
Table 3. Methodologic information about economic evaluation.
Characteristicsn (%)
Costs/inputs
Direct costs79 (84.95)
Direct and indirect costs14 (15.05)
Clinical outcomes/outputs *
Life years (LYS, LYG, mortality)43 (46.24)
Quality-adjusted life years (QALY)16 (17.2)
Achievement of therapeutic objectives (LDL, Ha1C, viral response, etc.)9 (9.68)
Events avoided8 (8.6)
Cases diagnosed6 (6.45)
Others **21 (22.58)
Time horizon ***
<1 year18 (19.35)
Between 1 and 5 years42 (45.16)
>5 years14 (15.05)
Lifetime3 (3.23)
Not reported18 (19.35)
Perspective ****
Payer38 (40.86)
Health system20 (21.51)
Hospital8 (8.6)
Society4 (4.3)
Other2 (2.15)
Not reported23 (24.73)
* Some studies reported different clinical outcomes, ** Successful therapeutic or surgical programs, health questionnaires or quality of life, *** Two studies used 2 different time horizons, **** Two studies used different perspectives.
Table 4. Relation between CHEERS and RWD.
Table 4. Relation between CHEERS and RWD.
CharacteristicsMeet Less Than 80% of Criteria *
n (%)
Meet 80% or More Criteria *
n (%)
CHEERS51 (54.84)42 (45.16)
Sample size
<1006 (11.76)3 (7.14)
Between 100 and 100027 (52.94)11 (26.19)
Between 1000 and 50009 (17.65)15 (35.71)
>50008 (15.69)10 (23.81)
Not reported1 (1.96)3 (7.14)
Randomized
Yes7 (13.73)9 (21.43)
No44 (86.27)33 (78.57)
Reported control bias
Yes31 (60.78)41 (97.62)
No20 (39.22)1 (2.38)
* 17 or more criteria (of 21, excluding those that did not apply 15, 16) or that in the great majority did not apply (12).

Share and Cite

MDPI and ACS Style

Parody-Rúa, E.; Rubio-Valera, M.; Guevara-Cuellar, C.; Gómez-Lumbreras, A.; Casajuana-Closas, M.; Carbonell-Duacastella, C.; Aznar-Lou, I. Economic Evaluations Informed Exclusively by Real World Data: A Systematic Review. Int. J. Environ. Res. Public Health 2020, 17, 1171. https://0-doi-org.brum.beds.ac.uk/10.3390/ijerph17041171

AMA Style

Parody-Rúa E, Rubio-Valera M, Guevara-Cuellar C, Gómez-Lumbreras A, Casajuana-Closas M, Carbonell-Duacastella C, Aznar-Lou I. Economic Evaluations Informed Exclusively by Real World Data: A Systematic Review. International Journal of Environmental Research and Public Health. 2020; 17(4):1171. https://0-doi-org.brum.beds.ac.uk/10.3390/ijerph17041171

Chicago/Turabian Style

Parody-Rúa, Elizabeth, Maria Rubio-Valera, César Guevara-Cuellar, Ainhoa Gómez-Lumbreras, Marc Casajuana-Closas, Cristina Carbonell-Duacastella, and Ignacio Aznar-Lou. 2020. "Economic Evaluations Informed Exclusively by Real World Data: A Systematic Review" International Journal of Environmental Research and Public Health 17, no. 4: 1171. https://0-doi-org.brum.beds.ac.uk/10.3390/ijerph17041171

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop