Next Article in Journal
Serological Response and Relationship with Gender-Sensitive Variables among Healthcare Workers after SARS-CoV-2 Vaccination
Next Article in Special Issue
Novel Machine Learning Approach to Predict and Personalize Length of Stay for Patients Admitted with Syncope from the Emergency Department
Previous Article in Journal
Vav1 Selectively Down-Regulates Akt2 through miR-29b in Certain Breast Tumors with Triple Negative Phenotype
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
JPM
Volume 12
Issue 6
10.3390/jpm12060990
jpm-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Prediction of All-Cause Mortality Following Percutaneous Coronary Intervention in Bifurcation Lesions Using Machine Learning Algorithms

by
Jacopo Burrello
1,†,
Guglielmo Gallone
2,†,
Alessio Burrello
3,
Daniele Jahier Pagliari
4,
Eline H. Ploumen
5,
Mario Iannaccone
6,
Leonardo De Luca
7,
Paolo Zocca
5,
Giuseppe Patti
8,
Enrico Cerrato
9,
Wojciech Wojakowski
10,
Giuseppe Venuti
11,
Ovidio De Filippo
2,
Alessio Mattesini
12,
Nicola Ryan
13,
Gérard Helft
14,
Saverio Muscoli
15,
Jing Kan
16,
Imad Sheiban
17,
Radoslaw Parma
18,
Daniela Trabattoni
19,
Massimo Giammaria
20,
Alessandra Truffa
21,
Francesco Piroli
2,
Yoichi Imori
22,
Bernardo Cortese
23,
Pierluigi Omedè
2,
Federico Conrotto
2,
Shao-Liang Chen
16,*,
Javier Escaned
24,
Rosaly A. Buiten
5,
Clemens Von Birgelen
5,
Paolo Mulatero
1,
Gaetano Maria De Ferrari
2,*,
Silvia Monticone
1,‡ and
Fabrizio D’Ascenzo
2,‡,§add Show full author list remove Hide full author list
1
Division of Internal Medicine and Hypertension, Department of Medical Sciences, University of Turin, 10126 Turin, Italy
2
Division of Cardiology, Department of Medical Sciences, University of Turin, 10126 Turin, Italy
3
Department of Electrical, Electronic and Information Engineering, University of Bologna, 40126 Bologna, Italy
4
Department of Control and Computer Engineering, Polytechnic University of Turin, 10129 Turin, Italy
5
Cardiology Department, Medisch Spectrum Twente, Thoraxcentrum Twente, 7412 Enschede, The Netherlands
6
Cardiology Department, San Giovanni Bosco Hospital, 10154 Turin, Italy
7
Division of Cardiology, S. Giovanni Evangelista Hospital, Tivoli, 00019 Rome, Italy
8
Coronary Care Unit and Catheterization Laboratory, A.O.U. Maggiore della Carità, 28100 Novara, Italy
9
Department of Cardiology, San Luigi Gonzaga Hospital, 10043 Orbassano, Italy
10
Department of Cardiology, Medical University of Silesia, 40-752 Katowice, Poland
11
Division of Cardiology, A.O.U. “Policlinico-Vittorio Emanuele”, 95123 Catania, Italy
12
Structural Interventional Cardiology, Careggi University Hospital, 50134 Florence, Italy
13
Department of Cardiology, Aberdeen Royal Infirmary, Aberdeen AB25 2ZN, UK
14
Department of Cardiology, Pierre and Marie Curie University, 75005 Paris, France
15
Department of Medicine, Università degli Studi di Roma Tor Vergata, 00133 Rome, Italy
16
Division of Cardiology, Nanjing First Hospital, Nanjing Medical University, Nanjing 210029, China
17
Division of Cardiology, Pederzoli Hospital, 37019 Peschiera del Garda, Italy
18
Department of Cardiology, University Clinical Hospital, 02-091 Warsaw, Poland
19
Department of Cardiovascular Sciences, IRCCS Centro Cardiologico Monzino, 20138 Milan, Italy
20
Division of Cardiology, Ospedale Maria Vittoria, 10144 Turin, Italy
21
Division of Cardiology, ASL Cardinal Massaia Hospital, 14100 Asti, Italy
22
Department of Cardiovascular Medicine, Nippon Medical School, Sendagi, Bunkyo-ku, Tokyo 113-8602, Japan
23
Division of Cardiology, San Carlo Clinic, 20037 Milan, Italy
24
Division of Cardiology, Hospital San Carlos, Complutense University, 28040 Madrid, Spain
 
add Show full affiliation list
 
remove Hide full affiliation list
*
Authors to whom correspondence should be addressed.
†
These authors contributed equally to this work.
‡
These authors contributed equally to this work.
§
A list of study collaborators can be found as Supplementary Materials.
J. Pers. Med. 2022, 12(6), 990; https://0-doi-org.brum.beds.ac.uk/10.3390/jpm12060990
Submission received: 13 May 2022 / Revised: 9 June 2022 / Accepted: 14 June 2022 / Published: 17 June 2022
(This article belongs to the Special Issue Cardiovascular Disease Risk Stratification in the Era of Machine Learning and Personalized Medicine)
Browse Figures
Review Reports Versions Notes

Abstract

:
Stratifying prognosis following coronary bifurcation percutaneous coronary intervention (PCI) is an unmet clinical need that may be fulfilled through the adoption of machine learning (ML) algorithms to refine outcome predictions. We sought to develop an ML-based risk stratification model built on clinical, anatomical, and procedural features to predict all-cause mortality following contemporary bifurcation PCI. Multiple ML models to predict all-cause mortality were tested on a cohort of 2393 patients (training, n = 1795; internal validation, n = 598) undergoing bifurcation PCI with contemporary stents from the real-world RAIN registry. Twenty-five commonly available patient-/lesion-related features were selected to train ML models. The best model was validated in an external cohort of 1701 patients undergoing bifurcation PCI from the DUTCH PEERS and BIO-RESORT trial cohorts. At ROC curves, the AUC for the prediction of 2-year mortality was 0.79 (0.74–0.83) in the overall population, 0.74 (0.62–0.85) at internal validation and 0.71 (0.62–0.79) at external validation. Performance at risk ranking analysis, k-center cross-validation, and continual learning confirmed the generalizability of the models, also available as an online interface. The RAIN-ML prediction model represents the first tool combining clinical, anatomical, and procedural features to predict all-cause mortality among patients undergoing contemporary bifurcation PCI with reliable performance.

Graphical Abstract

1. Introduction

The evolution of both stent technology and implantation techniques has translated into improved clinical outcomes following percutaneous coronary intervention (PCI) in complex anatomical and procedural settings, such as coronary bifurcation lesions [1,2,3,4,5].
However, real-life contemporary registries [6,7] still report a considerable risk of adverse events in this high-risk subset, warranting precise prognostication.
Available risk scores to predict adverse events associated with PCI are based on study populations with a small proportion of bifurcation lesions [8,9,10,11]. The external performance of such models in this lesion setting remains modest [12]. The absence of dedicated algorithms to predict long-term outcomes of PCI in coronary bifurcations clashes with the abundant evidence demonstrating a significantly poorer short- and long-term prognosis of these lesions compared to the overall PCI population, most likely as a result of multifaceted differences in procedural technique, hemorheology and vessel healing that makes PCI in bifurcations less forgiving than in other anatomical subsets [5,13,14,15,16]. While the risk stratification of ischemic endpoints is pivotal to informing patient management and therapeutic choices, mortality prediction remains an important goal to improve physician-patient communication, orientate follow-up and clinical decision making, and allow comparative effectiveness research in order to guide procedural strategy and further technical advances [17,18]. However, to date, there is no available predictive tool to predict long-term mortality following bifurcation PCI.
In the clinical research field of risk prediction following PCI, available studies favored either clinical [8] or anatomy/procedure [11,19] focused approaches to assess residual risk rather than exploiting the multidimensional nature of risk, which may be better determined by integrating these factors, especially in the bifurcation setting. Moreover, traditional prognostic risk assessment is constructed upon a limited selection of variables based on a priori assumptions, potentially omitting routinely assessed, powerful outcome predictors. This potential limit of classical inferential statistics could be overcome by machine learning (ML) that adopts a radically different approach, focusing on algorithmic representations of data and their classification in order to establish and quantify the relationships among variables [20]. Indeed, ML has emerged as a powerful approach to circumvent the limitations of current methods by applying computational algorithms to large datasets with numerous multiparametric variables, capturing high-dimensional, non-linear relationships among clinical features to make data-driven outcome predictions [21]. The effectiveness of this strategy has been shown across several cardiovascular applications, where ML was superior to validated traditional risk stratification tools, including the prediction of adverse events among patients with coronary artery disease or heart failure undergoing cardiac resynchronization therapy [22,23,24]. Thus, we developed an ML-based risk stratification model integrating clinical, anatomical, and procedural features to predict mortality following bifurcation PCI, utilizing a large international cohort of patients undergoing coronary bifurcation PCI with contemporary stents [5]. The model was then validated in a large population derived from two randomized trials of contemporary stents.

2. Materials and Methods

2.1. Study Population

The present study includes 4094 patients with a coronary bifurcation lesion treated with contemporary, very thin drug-eluting stents. The ML-based model was developed and internally validated using the RAIN (very thin stents for patients with left main or bifurcation in real life, NCT03544294) registry population (defined as the discovery cohort in the present study). RAIN is a multicenter retrospective registry including consecutive patients undergoing unprotected left main or coronary bifurcation PCI from 2014 to 2017 at 23 institutions worldwide [5]. Patients undergoing ostial/mid-shaft left main PCI or patients with incomplete clinical, angiographic, procedural, and outcome data were excluded from this study. Thus, 2393 patients undergoing bifurcation PCI with very thin strut stents comprised the final discovery cohort. The discovery cohort was randomly divided into a training cohort (n = 1795) and an internal validation cohort (n = 598).
The external validation cohort for the ML-based model was obtained from the merged DUTCH PEERS (Durable polymer-based stent Challenge of Promus ElemEnt versus Resolute integrity: TWENTE II) trial and BIO-RESORT trial patient cohorts [2,25,26]. More specifically, 1701 patients with a coronary bifurcation lesion treated with very thin stents (n = 465 from DUTCH-PEERS, n = 1236 from BIO-RESORT) with follow-up truncated at 2-years comprised the external validation cohort.
Detailed descriptions of the study cohorts are provided as Supplementary Materials. Cardiovascular risk factors, clinical presentation, angiographic features, use of intravascular imaging, bifurcation technique details, characteristics of the treated lesion, and implanted stents were collected in a dedicated database. The primary endpoint of the study was all-cause mortality at two years, while all-cause death at 30 days and at one year were evaluated as secondary endpoints. The study complies with the Declaration of Helsinki, all of the patients provided informed consent for inclusion in the registries, and local institutional review board approval was obtained by each center.

2.2. Model Development

An overview of model development is provided in Figure 1A. The model was trained and validated according to TRIPOD guidelines. The discovery cohort was randomly split into a training (n = 1795) and an internal validation (n = 598) dataset. The Fisher score was used for feature selection in the training cohort, and the variables with a coefficient >0.75 were retained (Figure S1); the selected variables were used to develop predictive models. A grid search including 5 different ML classifiers (linear discriminant analysis [LDA], random forest [RF] regressor, support vector machine [SVM] with a linear or Gaussian kernel, and isolation forest) and 3 algorithms for data imbalance correction (synthetic minority over-sampling technique [SMOTE], SMOTE and nearest neighbors, and random oversampling methods) was performed on the training cohort, generating 13 models for mortality prediction after bifurcation PCI. Data imbalance correction algorithms were applied to avoid the accuracy paradox (a falsely high accuracy due to over-prediction of the most represented class); oversampling techniques impute simulated patient data starting from real patients from the discovery cohort in the virtual space created by patient parameters to balance the number of patients with death occurrence and patients without events during model training. LDA applies a linear combination approach; the predicted endpoint is derived from the following equation: “Endpoint (all-cause mortality) = LDAcoeff1*Variable1 + LDAcoeff2*Variable2 + … + LDAcoeffn*Variablen > tested thresholds”. The coefficients are generated by the algorithm to maximize the separation between groups (Death vs. No events), increasing precision estimates by variance reduction; variables represent patients’ features, selected as described above. The RF algorithm generates a pre-defined set of classification trees (“n” classification trees) with a fixed maximum number of splits for each tree. The predicted endpoint results from the outcome of each classification tree of the forest; if at least “(n/2) + 1” of “n” trees of the RF predicts death as an outcome, then this endpoint is assigned to the patient. Linear SVM builds a classification model to assign patients to their outcome given a linear boundary, while Gaussian SVM allows the patients to be divided using a non-linear boundary. The model equations are: “SVMcoeff0 + SVMcoeff1*Variable1 + SVMcoeff2*variable2 + …. + SVMcoeffn*Variablen”, and “SVMcoeff0 + SVMcoeff1*f(Variable1) + SVMcoeff2*f(variable2) + …. + SVMcoeffn*f(Variablen)”, respectively, where “f” is an exponential function coefficient. Isolation forest is a particular type of RF that uses unsupervised learning to discriminate anomalies (in this case, patients with death occurrence) from normal data (patients without events).
A random forest regressor algorithm with random oversampling correction yielded the highest accuracy for the prediction of death occurrence, and it is referenced throughout the manuscript as the RAIN-ML prediction model. A 10-fold cross-validation was used to select and tune the hyper-parameters (number of classification trees and number of splits) of the RAIN-ML model in the training cohort; the hyper-parameters reaching the highest accuracy in outcome prediction were selected. Thereafter, its performance was assessed by K-center cross-validation, risk stratification analysis, continual learning, and both internal and external validation. Overfitting bias was defined as the difference between the accuracy obtained during training and the accuracy during the internal or external validation. The model was developed to predict 2-years all-cause mortality; its performance was then assessed also at different time points (30-day and 1-year). A detailed description of the model development is provided in the Supplementary Materials (Extended Methods section).
A user-friendly online interface was designed to facilitate the application of the RAIN-ML prediction model in clinical practice (available at https://rain.hpc4ai.it; accessed on 12 May 2022).

2.3. Statistical Analysis

Categorical variables were reported as count and percentage and analyzed by chi-square test. Continuous variables were reported as median and interquartile ranges and analyzed by Mann–Whitney U-test. The analysis of the receiver operating characteristic (ROC) curves was performed to calculate the area under the curve (AUC) and to derive the best cut-off by evaluation of the Youden Index (J = sensitivity + specificity − 1). A two-tailed p-value of less than 0.05 was considered significant. Analyses were performed by IBM SPSS Statistics 26 (IBM, New York, NY, USA), Python 3.5 (library, scikit-learn), and GraphPad PRISM 8.0 (La Jolla, California, CA, USA).

3. Results

3.1. Characteristics of the Discovery Cohort

The discovery cohort (n = 2393) was used to develop and internally validate the RAIN-ML prediction model. The baseline characteristics of the patients undergoing bifurcation PCI (median age 69 [interquartile range, IQR: 61–77] years, male sex 76.0%) from the discovery cohort are reported in Table S1, as stratified by death occurrence. The discovery cohort was randomized into a training and an internal validation dataset to develop and test the predictive models (Table 1). There were no differences between the training and internal validation cohorts (Table S2). After a median follow-up of 274 (IQR 52–434) days, 137 (5.7%) patients died (103 and 34 from the training and internal validation cohort, respectively; 30-day, 1-year, and 2-year mortalities were 1.2%, 3.7%, and 5.2%, respectively).

3.2. Development and Internal Validation of the RAIN-ML Prediction Model

Several patient and lesion-related parameters differed significantly between patients in whom the all-cause death endpoint was or was not reached at 2-year follow-up (Table S1). Features associated with all-cause mortality were selected by Fisher score (see Methods and Extended Methods sections) in the training cohort. Of the 38 patient and lesion-related parameters, 13 were excluded leading to a final set of 25 input variables (13 related to patient history and clinical presentation, seven to coronary anatomy, and five to the PCI procedure; Figure S1). Chronic kidney disease (CKD, defined as a glomerular filtration rate < 60 mL/min/1.73 m2) was the best predictor of all-cause mortality, followed by the indication for PCI, first lesion vessel, diabetes, diffuse coronary disease, left ventricular ejection fraction (EF, %), kind of bifurcation, and age (Figure 1B).
Among the 13 trained models, a random forest regressor algorithm with random oversampling correction yielded the highest accuracy in predicting 2-years all-cause mortality and was selected as the RAIN-ML prediction model (Table S3). A representative classification tree of the random forest RAIN-ML model is shown in Figure 1C. After tuning (Table S4), the RAIN-ML model displayed an accuracy of 81.1% at training (AUC 0.791; 95% CI 0.742–0.840), and 79.8% at internal validation (AUC 0.768; 95% CI 0.669–0.868), with an overfitting effect of 1.3% (Figure 1D). The sensitivity and specificity were 82.5/81.0% during training and 67.6/80.5% during internal validation, with 85 of 103 and 23 of 34 patients experiencing the correctly classified endpoint.
The performance of the RAIN-ML model in all-cause mortality prediction was assessed at different time points: the model was developed to predict 2-year all-cause mortality as the primary endpoint, and then its performance was also assessed at 30-days follow-up, 1-year follow-up, and including all events; Figure 2). After 1 year, the AUC was 0.777 (95% CI 0.721–0.834) during training and 0.718 (95% CI 0.586–0.850) during internal validation (Figure 2B). After 2 years, the AUC was 0.799 (95% CI 0.745–0.852) during training and 0.736 (95% CI 0.624–0.847) during internal validation (Figure 2C).
To further confirm the generalizability of the RAIN-ML prediction model, we applied a K-center cross-validation approach to the discovery cohort (n = 2393). The analysis confirmed an acceptable performance in each of the 23 participating institutions, with a mean accuracy, sensitivity, and specificity of 75.3%, 60.6%, and 76.2%.

3.3. External Validation of the RAIN-ML Model

The patients from the external validation cohort were younger (65 [IQR 57–72] years), with a lower prevalence of cardiovascular risk factors, prior myocardial infarction, and coronary revascularizations compared to the discovery cohort (Table 1 and Table S5). At external validation, 1312 of 1701 patients were correctly classified according to death occurrence (Figure 1D), resulting in an accuracy of 77.1% (overfitting effect 4%). ROC curve analysis confirmed a good performance at all the evaluated times of follow-up. After 2 years, 39 patients died (2.3%), and the AUC was 0.706 (95% CI 0.619–0.794; Figure 2C). The predictive performance in the overall population (mixed discovery and external validation cohort; n = 4094) was similar, with an AUC of 0.769 (95% CI 0.728–0.810), 0.726 (95% CI 0.633–0.819), 0.758 (95% CI 0.707–0.810), and 0.786 (95% CI 0.743–0.830), respectively, considering all events, or a follow up of 30 days, 1 or 2 years (Figure 2).

3.4. Risk Stratification Analysis

In the mixed discovery and external validation cohorts, increasing coefficients of the RAIN-ML prediction model were directly correlated with the proportion of subjects with death occurrence (Figure 3A–D). Patient stratification according to the RAIN-ML model and the occurrence of death at follow-up are reported in Table S6. The lowest risk patients with an ML model coefficient of 0.10–0.19 displayed an all-cause mortality risk of 0.3%, 0.9%, and 2.3%, after 30 days, 1-year, and 2-year follow-up, respectively. On the other hand, the highest risk patients with an ML model coefficient of 0.90–1.00 displayed an all-cause mortality risk of 8.1%, 23.7%, and 72.2%, respectively, after 30 days, 1-year, and 2-year follow-up. Using cut-offs derived by ROC curve analysis to optimize sensitivity and specificity, we then stratified patients according to the predicted risk of all-cause mortality. For the RAIN-ML prediction model, a coefficient of less than 0.21 identified a low-risk subgroup of patients with a risk of 1.4% of all-cause mortality (35 of 2444 subjects), patients with a coefficient ranging between 0.21 and 0.70 showed an intermediate risk of 4.5% (52 of 1166 subjects), while a coefficient higher than 0.70 identified a risk of 18.4% (high-risk group; 89 of 484 subjects; Figure 3E). As compared to low risk, being categorized as intermediate risk and high risk was associated with increased (3.2-fold and 13.1-fold, respectively, both p < 0.001) mortality. The risk ranking approach (after the exclusion of patients at intermediate risk) led to a sensitivity/specificity of 71.8/85.9% with an overall accuracy of 85.3% when low-risk patients were compared to those classified as high risk.
The predictive performances of the RAIN-ML model are summarized in Table S7.

4. Discussion

A reliable and clinically relevant patient risk stratification is a prerequisite for adequate treatment selection, informed consent, and improved care, all key elements of modern personalized medicine. Following this guiding principle, we developed and validated a prediction model based on supervised machine learning algorithms to identify long-term all-cause mortality in patients undergoing PCI on coronary bifurcations.
Our model was first developed using the largest available bifurcation PCI registry reflective of contemporary practice, encompassing a wide range of very thin second-generation drug-eluting stents and procedural techniques, applied to a variety of clinical scenarios among all-comers at 23 institutions worldwide. Subsequently, our findings were externally validated using a large bifurcation PCI population derived from two randomized trials of second-generation drug-eluting stents [2,25,26].
The RAIN-ML model, correctly classifying 3245 of 4094 patients, showed a good discriminative capability for all-cause mortality prediction, confirmed at both internal and external validation, and by K-center cross-validation, with an accuracy of 81.1% during training and ranging between 77.1% and 79.8% during validation. According to the RAIN-ML prediction model, following bifurcation PCI, 6% of the patients displayed a 1-year mortality risk > 20%, while about 60% of them carried a 1-year mortality risk below 2%. The adoption of the RAIN-ML model in contemporary practice may allow for the performance of reliable and clinically relevant risk stratification to inform, personalize, and improve care. An accurate risk stratification would allow, on the one hand, targeted strategies in patients with the highest risk of death through a comprehensive evaluation and a tailored approach and, on the other hand, less intensive follow-up for those at low risk.
Despite several differences in the characteristics of the study cohorts, external validation yielded a good performance, suggesting good generalizability and robustness of the RAIN-ML model applicability. In particular, the cohorts of the BIO-RESORT and DUTCH-PEERS trials were composed of patients with a lower burden of cardiovascular risk factors and previous cardiovascular events, translating into lower death rates at follow-up as compared to the real-world clinical setting of the RAIN cohort. Moreover, in the external validation cohorts, “left ventricular ejection fraction”, a powerful feature of the RAIN-ML model, was coded dichotomically, and the variable “diffuse coronary disease” was unavailable. The model performed well despite these missing data and potentially limiting factors that might have affected model discrimination. Importantly, the model performed well at internal validation, suggesting its applicability in the real-world setting and its potential usefulness in daily clinical practice.

4.1. Rationale of the Study and Related Work

In this study, we focused on all-cause mortality as the primary endpoint to offer a comprehensive evaluation of the biological risk of this patient subset, which displays unique features as compared to the overall PCI population. Specifically, patients undergoing bifurcation PCI have higher short- and long-term all-cause mortality as compared to patients with non-bifurcation PCI [13,27], pointing to a peculiar association and possibly causal link of bifurcation lesions with mortality. If, on the one hand, the presence of bifurcation lesions is a potential proxy for a more severe atherosclerotic burden, on the other hand, it may also represent a subset biologically more prone to adverse events due to the peculiar rheological characteristics [14,15]. Moreover, specific procedural aspects are associated with long-term mortality in bifurcation PCI, highlighting the importance, beyond the lesion’s natural history, of PCI-related factors in determining the prognosis of this population [5,16]. However, presently there is no predictive tool reflective of current clinical practice available to predict long-term mortality following bifurcation PCI. For these reasons, we focused on a multifaceted approach to residual risk, based on a comprehensive evaluation of patient-, anatomy-, and procedure-related factors integrated by a machine learning approach able to handle multidimensional information and produce data-driven outcome prediction. A key advantage of this approach is that investigators do not generally need to specify which potential predictor variables to consider and in which combinations. A multidimensional approach may be highly relevant in this setting, which is characterized by high anatomical and procedural complexity. Indeed, previous scores to predict adverse events focusing on either patient-related or anatomy/procedure-related factors performed only modestly in patients with bifurcation PCI [28]. Specifically, in a previous analysis of the RAIN registry, the PCI complexity definition proposed by Giustino et al. [11] using validated and guideline-endorsed criteria (reflecting anatomy/procedure-related factors) was unable to discriminate post-procedural mortality (AUC 0.49), and the PARIS risk score [8] (reflecting patient-related factors) displayed only a modest discrimination capacity (AUC 0.65). More importantly, these tools showed potential for an accurate event prediction when combined, thus suggesting that a comprehensive evaluation of clinical, anatomical, and procedural features may better reflect residual risk [28].
Nevertheless, the aim of the RAIN-ML prediction model was not to guide bifurcation PCI based on pre-procedural risk but rather to provide the patient and the treating physician with information that also integrates procedural outcome predictors that may significantly modify the patient’s prognostic trajectory and, consequently, its clinical management and follow-up.

4.2. Perspectives

When compared to traditional scores, ML-based models do not always improve performance in terms of accuracy [29]. However, several advantages may become apparent in the long term [30,31,32,33,34,35,36,37]. Specifically, compared to the static nature of traditional scores, the performance of the RAIN-ML prediction model is dynamic, thanks to its evolutive learning feature allowing the model to improve its classification algorithm by learning strategies at the increased enrollment time and number of recruited patients.
An example of continual learning applied to the discovery cohort of the RAIN-ML model is presented in Figure 4: the model was trained at each time point on an increasing number of patients. From 3 months to 33 months of enrollment, the accuracy increased from 67.9% to 78.7% at validation. We thus will plan through a dedicated anonymized system integrated with the freely available online interface (https://rain.hpc4ai.it; accessed on 12 May 2022) to prospectively endorse RAIN-ML training to constantly improve outcome prediction. Moreover, future works should assess whether the integration of more granular features, such as the characteristics of plaque vulnerability evaluated by intravascular imaging, might have potentially improved outcome prediction. Similarly, features of non-cardiovascular comorbidities, which may be relevant to the mortality endpoint in this complex patient population, have not been evaluated in the present work: future studies may integrate these features with the RAIN-ML model to possibly improve its discriminative capability. Finally, prospective validation of the RAIN-ML model in an external real-world population remains desirable.

4.3. Limitations

This study has some limitations to be acknowledged. First, the RAIN registry was retrospective. However, the good generalizability demonstrated in the external validation cohort constituted by bifurcation PCI patients from two randomized trials is reassuring. Second, the model performance in the cohorts and at different time points ranged from moderate to good. As discussed, the integration of more granular technical and anatomical features along with data on non-cardiovascular comorbidities might have potentially improved outcome prediction. However, this would have limited the model adoption as these features are not yet routinely assessed in everyday clinical practice.
Third, our ML model requires several input variables that might discourage its use. However, all these variables are generally readily available, and the user-friendly online interface makes risk estimation at the different evaluated time-points an easy and quick procedure.

5. Conclusions

The RAIN-ML prediction model represents the first developed tool combining clinical, anatomical, and procedural features through a machine learning approach to predict all-cause mortality among patients undergoing contemporary coronary bifurcation PCI, with robust performance and generalizability for mortality prediction across different clinical scenarios and at different time points. The adoption of the RAIN-ML model has the potential to improve doctor-to-patient communication, patient management, and clinical research.

Supplementary Materials

6 Supplementary tables; 1 Supplementary Figure; The following supporting information can be downloaded at: https://0-www-mdpi-com.brum.beds.ac.uk/article/10.3390/jpm12060990/s1. Study list collaborators; Extended methods; Supplementary Table S1. Characteristics of patients from the discovery cohort; Supplementary Table S2. Discovery cohort: training vs. internal validation dataset; Supplementary Table S3. Grid Search: Model and Data Imbalance Correction; Supplementary Table S4. Tuning of the RAIN-ML model; Supplementary Table S5. Characteristics of discovery cohort vs. external validation cohort; Supplementary Table S6. Patient distribution and risk of all-cause mortality; Supplementary Table S7. Predictive performance of the RAIN-ML model; Supplementary Figure S1. Feature selection.

Author Contributions

S.M. (Silvia Monticone), F.D., P.M., G.M.D.F., J.B. and G.G. took part in the conception and design of the current analysis. A.B., D.J.P., J.B. and G.G. performed the analysis. J.B., S.M. (Silvia Monticone), G.G. and F.D. drafted the manuscript. All the other authors participate to patient recruitment and manuscript revision. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

The study was conducted in accordance with the Declaration of Helsinki and approved by the Institutional Review Board (ethic committee approval code: 00253/2021).

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

The data that support the findings of this study are available on reasonable request from the corresponding author.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Bangalore, S.; Kumar, S.; Fusaro, M.; Amoroso, N.; Attubato, M.; Feit, F.; Bhatt, D.L.; Slater, J. Short-and long-term outcomes with drug-eluting and bare-metal coronary stents: A mixed-treatment comparison analysis of 117 762 patient-years of follow-up from randomized trials. Circulation 2012, 125, 2873–2891. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  2. von Birgelen, C.; Kok, M.M.; van der Heijden, L.C.; Danse, P.W.; E Schotborgh, C.; Scholte, M.; Gin, R.M.T.J.; Somi, S.; van Houwelingen, K.G.; Stoel, M.G.; et al. Very thin strut biodegradable polymer everolimus-eluting and sirolimus-eluting stents versus durable polymer zotarolimus-eluting stents in allcomers with coronary artery disease (BIO-RESORT): A three-arm, randomised, non-inferiority trial. Lancet 2016, 388, 2607–2617. [Google Scholar] [CrossRef]
  3. Biolè, C.; Huczek, Z.; Nuñez-Gil, I.; Boccuzzi, G.; Autelli, M.; Montefusco, A.; Trabattoni, D.; Ryan, N.; Venuti, G.; Imori, Y.; et al. Daily risk of adverse outcomes in patients undergoing complex lesions revascularization: A subgroup analysis from the RAIN-CARDIOGROUP VII study (veRy thin stents for patients with left mAIn or bifurcatioN in real life). Int. J. Cardiol. 2019, 290, 64–69. [Google Scholar] [CrossRef] [PubMed]
  4. D’Ascenzo, F.; Omedè, P.; De Filippo, O.; Cerrato, E.; Autelli, M.; Trabattoni, D.; Ryan, N.; Venuti, G.; Muscoli, S.; Montabone, A.; et al. Impact of final kissing balloon and of Imaging on patients treated on unprotected left main coronary artery with thin-strut stents (from the RAIN-CARDIOGROUP VII study). Am. J. Cardiol. 2019, 123, 1610–1619. [Google Scholar] [CrossRef]
  5. Gaido, L.; D’Ascenzo, F.; Imori, Y.; Wojakowski, W.; Saglietto, A.; Figini, F.; Mattesini, A.; Trabattoni, D.; Rognoni, A.; Tomassini, F.; et al. Impact of kissing balloon in patients treated with ultrathin stents for left main lesions and bifurcations: An analysis from the RAIN-CARDIOGROUP VII study. Circ. Cardiovasc. Interv. 2020, 13, e008325. [Google Scholar] [CrossRef]
  6. D’Ascenzo, F.; Chieffo, A.; Cerrato, E.; Ugo, F.; Pavani, M.; Kawamoto, H.; di Summa, R.; Varbella, F.; Boccuzzi, G.; Omedè, P.; et al. Incidence and Management of Restenosis After Treatment of Unprotected Left Main Disease with Second-Generation Drug-Eluting Stents (from Failure in Left Main Study with 2nd Generation Stents–Cardiogroup III Study). Am. J. Cardiol. 2017, 119, 978–982. [Google Scholar] [CrossRef]
  7. Naganuma, T.; Chieffo, A.; Meliga, E.; Capodanno, D.; Park, S.-J.; Onuma, Y.; Valgimigli, M.; Jegere, S.; Makkar, R.R.; Palacios, I.F.; et al. Long-term clinical outcomes after percutaneous coronary intervention for ostial/mid-shaft lesions versus distal bifurcation lesions in unprotected left main coronary artery: The DELTA Registry (Drug-Eluting Stent for Left Main Coronary Artery Disease). JACC Cardiovasc. Interv. 2013, 6, 1242–1249. [Google Scholar] [CrossRef] [Green Version]
  8. Baber, U.; Mehran, R.; Giustino, G.; Cohen, D.J.; Henry, T.D.; Sartori, S.; Ariti, C.; Litherland, C.; Dangas, G.; Gibson, C.M.; et al. Coronary Thrombosis and Major Bleeding after PCI with Drug-Eluting Stents Risk Scores from Paris. J. Am. Coll. Cardiol. 2016, 67, 2224–2234. [Google Scholar] [CrossRef]
  9. Mehran, R.; Baber, U.; Steg, P.G.; Ariti, C.; Weisz, G.; Witzenbichler, B.; Henry, T.D.; Kini, A.S.; Stuckey, T.; Cohen, D.J.; et al. Cessation of dual antiplatelet treatment and cardiac events after percutaneous coronary intervention (PARIS): 2 years results from a prospective observational study. Lancet 2013, 382, 1714–1722. [Google Scholar] [CrossRef]
  10. Moretti, C.; D’Ascenzo, F.; Omedè, P.; Sciuto, F.; Presutti, D.G.; Di Cuia, M.; Colaci, C.; Giusto, F.; Ballocca, F.; Cerrato, E.; et al. Thirty-day readmission rates after PCI in a metropolitan center in Europe: Incidence and impact on prognosis. J. Cardiovasc. Med. 2015, 16, 238–245. [Google Scholar] [CrossRef]
  11. Giustino, G.; Chieffo, A.; Palmerini, T.; Valgimigli, M.; Feres, F.; Abizaid, A.; Costa, R.A.; Hong, M.-K.; Kim, B.-K.; Jang, Y.; et al. Efficacy and Safety of Dual Antiplatelet Therapy After Complex PCI. J. Am. Coll. Cardiol. 2016, 68, 1851–1864. [Google Scholar] [CrossRef] [PubMed]
  12. Raposeiras-Roubín, S.; Caneiro Queija, B.; D’Ascenzo, F.; Kinnaird, T.; Ariza-Solé, A.; Manzano-Fernández, S.; Templin, C.; Velicki, L.; Xanthopoulou, I.; Cerrato, E.; et al. Usefulness of the PARIS Score to Evaluate the Ischemic-hemorrhagic Net Benefit with Ticagrelor and Prasugrel After an Acute Coronary Syndrome. Rev. Esp. Cardiol. 2019, 72, 215–223. [Google Scholar] [CrossRef] [PubMed]
  13. Burzotta, F.; Annone, U.; Paraggio, L.; D’Ascenzo, F.; Biondi-Zoccai, G.; Aurigemma, C.; Romagnoli, E.; Verdirosi, D.; Trani, C.; Crea, F. Clinical outcome after percutaneous coronary intervention with drug-eluting stent in bifurcation and nonbifurcation lesions: A meta-analysis of 23 981 patients. Coron. Artery Dis. 2020, 31, 438–445. [Google Scholar] [CrossRef] [PubMed]
  14. Van Der Giessen, A.G.; Wentzel, J.J.; Meijboom, W.B.; Mollet, N.R.; Van Der Steen, A.F.W.; Van De Vosse, F.N.; De Feyter, P.J.; Gijsen, F.J.H. Plaque and shear stress distribution in human coronary bifurcations: A multislice computed tomography study. EuroIntervention 2009, 4, 654–661. [Google Scholar] [CrossRef]
  15. Lassen, J.F.; Burzotta, F.; Banning, A.P.; Lefèvre, T.; Darremont, O.; Hildick-Smith, D.; Chieffo, A.; Pan, M.; Holm, N.; Louvard, Y.; et al. Percutaneous coronary intervention for the left main stem and other bifurcation lesions: 12th consensus document from the European Bifurcation Club. EuroIntervention 2018, 13, 1540–1553. [Google Scholar] [CrossRef] [Green Version]
  16. Behan, M.W.; Holm, N.R.; De Belder, A.J.; Cockburn, J.; Erglis, A.; Curzen, N.P.; Niemelä, M.; Oldroyd, K.G.; Kervinen, K.; Kumsars, I.; et al. Coronary bifurcation lesions treated with simple or complex stenting: 5-year survival from patient-level pooled analysis of the Nordic Bifurcation Study and the British Bifurcation Coronary Study. Eur. Heart J. 2016, 37, 1923–1928. [Google Scholar] [CrossRef]
  17. Ambati, L.S.; El-Gayar, O.; Nawar, N. Design principles for multiple sclerosis mobile self-management applications: A patient-centric perspective. In Proceedings of the 27th Annual Americas Conference on Information Systems, AMCIS 2021, Online, 9–13 August 2021. [Google Scholar]
  18. Ambati, L.S.; El-Gayar, O.F.; Nawar, N. Influence of The Digital Divide and Socio-Economic Factors on Prevalence of Diabetes. Issues Inf. Syst. 2020, 21, 103–113. [Google Scholar]
  19. Costa, F.; Van Klaveren, D.; Feres, F.; James, S.; Räber, L.; Pilgrim, T.; Hong, M.-K.; Kim, H.-S.; Colombo, A.; Steg, P.G.; et al. Dual Antiplatelet Therapy Duration Based on Ischemic and Bleeding Risks After Coronary Stenting. J. Am. Coll. Cardiol. 2019, 73, 741–754. [Google Scholar] [CrossRef]
  20. Gallone, G.; Bruno, F.; D’ascenzo, F.; De Ferrari, G. What will we ask to artificial intelligence for cardiovascular medicine in the next decade? Minerva Cardiol. Angiol. 2022, 70, 92–101. [Google Scholar] [CrossRef]
  21. Dey, D.; Slomka, P.J.; Leeson, P.; Comaniciu, D.; Shrestha, S.; Sengupta, P.P.; Marwick, T.H. Artificial Intelligence in Cardiovascular Imaging: JACC State-of-the-Art Review. J. Am. Coll. Cardiol. 2019, 73, 1317–1335. [Google Scholar] [CrossRef]
  22. Motwani, M.; Dey, D.; Berman, D.S.; Germano, G.; Achenbach, S.; Al-Mallah, M.; Andreini, D.; Budoff, M.J.; Cademartiri, F.; Callister, T.Q.; et al. Machine learning for prediction of all-cause mortality in patients with suspected coronary artery disease: A 5-year multicentre prospective registry analysis. Eur. Heart J. 2016, 38, 500–507. [Google Scholar] [CrossRef] [PubMed]
  23. Tokodi, M.; Schwertner, W.R.; Kovács, A.; Tősér, Z.; Staub, L.; Sárkány, A.; Lakatos, B.K.; Behon, A.; Boros, A.M.; Perge, P.; et al. Machine learning-based mortality prediction of patients undergoing cardiac resynchronization therapy: The SEMMELWEIS-CRT score. Eur. Heart J. 2020, 41, 1747–1756. [Google Scholar] [CrossRef] [PubMed]
  24. D’Ascenzo, F.; Filippo, O.; De Gallone, G.; Mittone, G.; Deriu, M.A.; Iannaccone, M.; Ariza-Solé, A.; Liebetrau, C.; Manzano-Fernández, S.; Quadri, G.; et al. Machine learning-based prediction of adverse events following an acute coronary syndrome (PRAISE): A modelling study of pooled datasets. Lancet 2021, 397, 199–207. [Google Scholar] [CrossRef]
  25. Von Birgelen, C.; Sen, H.; Lam, M.K.; Danse, P.W.; Jessurun, G.A.J.; Hautvast, R.W.M.; van Houwelingen, G.K.; Schramm, A.R.; Gin, R.M.T.J.; Louwerenburg, J.W.; et al. Third-generation zotarolimus-eluting and everolimus-eluting stents in all-comer patients requiring a percutaneous coronary intervention (DUTCH PEERS): A randomised, single-blind, multicentre, non-inferiority trial. Lancet 2014, 383, 413–423. [Google Scholar] [CrossRef]
  26. Zocca, P.; Kok, M.M.; Tandjung, K.; Danse, P.W.; Jessurun, G.A.; Hautvast, R.W.; van Houwelingen, K.G.; Stoel, M.G.; Schramm, A.R.; Gin, R.M.T.J.; et al. 5-Year Outcome Following Randomized Treatment of All-Comers with Zotarolimus-Eluting Resolute Integrity and Everolimus-Eluting PROMUS Element Coronary Stents: Final Report of the DUTCH PEERS (TWENTE II) Trial. JACC Cardiovasc. Interv. 2018, 11, 462–469. [Google Scholar] [CrossRef]
  27. D’Ascenzo, F.; De Filippo, O.; Elia, E.; Doronzo, M.; Montefusco, A.; Pennone, M.; Angelini, F.; Franchin, L.; Bruno, F.; Salizzoni, S.; et al. Percutaneous vs. Surgical Revascularization for Patients with Unprotected Left Main Stenosis: A Meta-Analysis of 5 Years Follow-Up RCTs. Eur. Heart J. Qual. Care Clin. Outcomes 2020, 7, 476–485. [Google Scholar] [CrossRef]
  28. Gallone, G.; D’Ascenzo, F.; Conrotto, F.; Costa, F.; Capodanno, D.; Muscoli, S.; Chieffo, A.; Yoichi, I.; Pennacchi, M.; Quadri, G.; et al. Accuracy of the PARIS score and PCI complexity to predict ischemic events in patients treated with very thin stents in unprotected left main or coronary bifurcations. Catheter. Cardiovasc. Interv. 2020, 97, E227–E236. [Google Scholar] [CrossRef]
  29. Christodoulou, E.; Ma, J.; Collins, G.S.; Steyerberg, E.W.; Verbakel, J.Y.; Van Calster, B. A systematic review shows no performance benefit of machine learning over logistic regression for clinical prediction models. J. Clin. Epidemiol. 2019, 110, 12–22. [Google Scholar] [CrossRef]
  30. Gupta, V.; Mittal, M.; Mittal, V.; Gupta, A. ECG signal analysis using CWT, spectrogram and autoregressive technique. Iran J. Comput. Sci. 2021, 4, 265–280. [Google Scholar] [CrossRef]
  31. Parameswari, C.; Siva Ranjani, S. Prediction of atherosclerosis pathology in retinal fundal images with machine learning approaches. J. Ambient Intell. Humaniz. Comput. 2020, 12, 6701–6711. [Google Scholar] [CrossRef]
  32. Talaat, F.M.; Gamel, S.A. RL based hyper-parameters optimization algorithm (ROA) for convolutional neural network. J. Ambient Intell. Humaniz. Comput. 2022, 1–11. [Google Scholar] [CrossRef]
  33. Elavarasan, D.; Vincent, P.M.D.R. A reinforced random forest model for enhanced crop yield prediction by integrating agrarian parameters. J. Ambient Intell. Humaniz. Comput. 2021, 12, 10009–10022. [Google Scholar] [CrossRef]
  34. Jayavadivel, R.; Prabaharan, P. Investigation on automated surveillance monitoring for human identification and recognition using face and iris biometric. J. Ambient Intell. Humaniz. Comput. 2021, 12, 10197–10208. [Google Scholar] [CrossRef]
  35. Usharani, R.; Shanthini, A. Neuropathic complications: Type II diabetes mellitus and other risky parameters using machine learning algorithms. J. Ambient Intell. Humaniz. Comput. 2021, 12, 0123456789. [Google Scholar] [CrossRef]
  36. Saravanan, S.; Karthigaivel, R.; Magudeeswaran, V. A brain tumor image segmentation technique in image processing using ICA-LDA algorithm with ARHE model. J. Ambient Intell. Humaniz. Comput. 2021, 12, 4727–4735. [Google Scholar] [CrossRef]
  37. González, I.; Fontecha, J.; Bravo, J. Relationship between stride interval variability and aging: Use of linear and non-linear estimators for gait variability assessment in assisted living environments. J. Ambient Intell. Humaniz. Comput. 2019, 10, 2095–2109. [Google Scholar] [CrossRef]
Jpm 12 00990 g001 550
Figure 1. RAIN-ML model. The RAIN-ML prediction model was built in the discovery cohort (n = 2393). (A) The discovery cohort was randomized in a training and in an internal validation cohort. The model was developed in the training cohort (n = 1795): 5 machine learning models (linear discriminant analysis [LDA], random forest regressor [RF], support vector machine [SVM] with different kernels, and isolation forest) and 3 algorithms for dataset imbalance correction (SMOTE, Synthetic Minority Oversampling Technique, SMOTE & nearest neighbours, and random oversampling) have been evaluated; the best model was an RF with random oversampling algorithm (reported in bold). The model was then tested in the internal and external validation cohorts (n = 598, n = 1701, respectively) and by K-center cross-validation, risk stratification analysis, and continual learning. (B) Radar chart reporting the 8 normalized best predictors associated with patient outcome. (C) Representative classification tree from the RAIN-ML RF model. (D) Confusion matrix, real and predicted diagnosis (Death vs. No event), accuracy, sensitivity, and specificity for the RAIN-ML model at training, internal validation, and external validation. CKD, Chronic Kidney Disease; PCI, Percutaneous Coronary Intervention; EF, Ejection Fraction; ACS, Acute Coronary Syndrome.
Figure 1. RAIN-ML model. The RAIN-ML prediction model was built in the discovery cohort (n = 2393). (A) The discovery cohort was randomized in a training and in an internal validation cohort. The model was developed in the training cohort (n = 1795): 5 machine learning models (linear discriminant analysis [LDA], random forest regressor [RF], support vector machine [SVM] with different kernels, and isolation forest) and 3 algorithms for dataset imbalance correction (SMOTE, Synthetic Minority Oversampling Technique, SMOTE & nearest neighbours, and random oversampling) have been evaluated; the best model was an RF with random oversampling algorithm (reported in bold). The model was then tested in the internal and external validation cohorts (n = 598, n = 1701, respectively) and by K-center cross-validation, risk stratification analysis, and continual learning. (B) Radar chart reporting the 8 normalized best predictors associated with patient outcome. (C) Representative classification tree from the RAIN-ML RF model. (D) Confusion matrix, real and predicted diagnosis (Death vs. No event), accuracy, sensitivity, and specificity for the RAIN-ML model at training, internal validation, and external validation. CKD, Chronic Kidney Disease; PCI, Percutaneous Coronary Intervention; EF, Ejection Fraction; ACS, Acute Coronary Syndrome.
Jpm 12 00990 g001
Jpm 12 00990 g002 550
Figure 2. Predictive performance. Receiver operating characteristics curve to assess the area under the curve and its 95% confidence interval (lower and upper limits) for the RAIN-ML prediction model at training (n = 1795), internal (n = 598), external validation (n = 1701), and in the mixed cohort (n = 4094). (A) Performance at 30-day follow-up; (B) Performance at 1-year follow-up; (C) Performance at 2-year follow-up; (D) Performance considering all the events at follow-up.
Figure 2. Predictive performance. Receiver operating characteristics curve to assess the area under the curve and its 95% confidence interval (lower and upper limits) for the RAIN-ML prediction model at training (n = 1795), internal (n = 598), external validation (n = 1701), and in the mixed cohort (n = 4094). (A) Performance at 30-day follow-up; (B) Performance at 1-year follow-up; (C) Performance at 2-year follow-up; (D) Performance considering all the events at follow-up.
Jpm 12 00990 g002
Jpm 12 00990 g003 550
Figure 3. Stratification of all-cause mortality risk according to the RAIN-ML model. Patient distribution and risk stratification analysis in the mixed discovery and external validation cohort (n = 4094). (A–D) Histograms showing the proportion of patients (y-axis, %) stratified according to their outcome (No Event, grey vs. Death, black); on the x-axis are reported the ML coefficients (for the RAIN-ML prediction model). Patients were stratified considering death occurrence at different follow-ups (30 days, 1 year, 2 years, and all events). (E) The table shows confusion matrix reporting risk stratification analysis, sensitivity, and specificity, for the RAIN-ML prediction model * Sensitivity and specificity were derived on a mixed cohort composed of the low- and high-risk groups, after exclusion of patients at intermediate risk.
Figure 3. Stratification of all-cause mortality risk according to the RAIN-ML model. Patient distribution and risk stratification analysis in the mixed discovery and external validation cohort (n = 4094). (A–D) Histograms showing the proportion of patients (y-axis, %) stratified according to their outcome (No Event, grey vs. Death, black); on the x-axis are reported the ML coefficients (for the RAIN-ML prediction model). Patients were stratified considering death occurrence at different follow-ups (30 days, 1 year, 2 years, and all events). (E) The table shows confusion matrix reporting risk stratification analysis, sensitivity, and specificity, for the RAIN-ML prediction model * Sensitivity and specificity were derived on a mixed cohort composed of the low- and high-risk groups, after exclusion of patients at intermediate risk.
Jpm 12 00990 g003
Jpm 12 00990 g004 550
Figure 4. Continual Learning of RAIN-ML prediction model. Graphs showing diagnostic performance (left y-axes) with a continual learning strategy in the discovery cohort (n = 2393). Squares indicates the number of patients (right y-axes) over the enrolment time (x-axes). (A) Learning simulation for RAIN-ML prediction model at the increase in the enrollment time; 70% of the discovery cohort is used for training (patients enrolled first), 30% for validation (last enrolled patients). Mean and standard deviation are shown after 10 repetitions of the analysis. Accuracy at training: from 86.1% to 79.9%. Accuracy at validation: from 67.9% to 78.7%. (B) Accuracy, sensitivity, and specificity at validation.
Figure 4. Continual Learning of RAIN-ML prediction model. Graphs showing diagnostic performance (left y-axes) with a continual learning strategy in the discovery cohort (n = 2393). Squares indicates the number of patients (right y-axes) over the enrolment time (x-axes). (A) Learning simulation for RAIN-ML prediction model at the increase in the enrollment time; 70% of the discovery cohort is used for training (patients enrolled first), 30% for validation (last enrolled patients). Mean and standard deviation are shown after 10 repetitions of the analysis. Accuracy at training: from 86.1% to 79.9%. Accuracy at validation: from 67.9% to 78.7%. (B) Accuracy, sensitivity, and specificity at validation.
Jpm 12 00990 g004
Table
Table 1. Clinical, anatomical, procedural features and outcomes in the study cohorts.
Table 1. Clinical, anatomical, procedural features and outcomes in the study cohorts.
VariableTraining Cohort
(n = 1795)		Internal Validation Cohort
(n = 598)		External Validation Cohort
(n = 1701)
Patient parametersAge (years)70 [61; 77]69 [61; 78]65 [57; 72]
Sex (ref. male; n, %)1379 (76.9)440 (73.6)1329 (78.1)
Hypertension (ref. yes; n, %)1331 (74.2)460 (76.9)820 (48.2)
Hyperlipidemia (ref. yes; n, %)1074 (59.8)377 (63.0)N.A.
Diabetes (ref. yes; n, %)605 (33.7)200 (33.4)319 (18.8)
Smoker N.A.
No (n, %)872 (48.6)291 (48.6)
Previous (n, %)548 (30.5)181 (30.3)
Current (n, %)375 (20.9)126 (21.1)
CKD (ref. GFR < 60 mL/min; n, %)391 (21.8)147 (24.6)50 (2.9)
Previous PCI (ref. yes; n, %)605 (33.7)187 (31.3)301 (17.7)
Previous CABG (ref. yes; n, %)97 (5.4)29 (4.8)120 (7.1)
Previous MI (ref. yes; n, %)554 (30.9)183 (30.6)339 (19.9)
EF at echo (%)55 [55; 60]55 [55; 65]50 [50; 50] *
PCI indication
STEMI (n, %)305 (17.0)114 (19.1)383 (22.5)
NSTEMI (n, %)447 (24.9)133 (22.2)388 (22.8)
Unstable angina (n, %)257 (14.3)90 (15.1)296 (17.4)
Stable angina (n, %)440 (24.6)147 (24.5)634 (37.3)
Positive stress test (n, %)227 (12.6)82 (13.7)0 (0.0)
Planned angiography (n, %)119 (6.6)32 (5.4)0 (0.0)
ACS at presentation (ref. yes; n, %)1007 (56.1)337 (56.4)1067 (62.7)
STEMI at presentation (ref. yes; n, %)305 (17.0)114 (91.1)383 (22.5)
Kind of DAT (aspirin plus)
Clopidogrel (n, %)1170 (65.2)391 (65.4)1131 (66.4)
Ticagrelor (n, %)479 (26.7)162 (27.1)513 (30.2)
Prasugrel (n, %)146 (8.1)45 (7.5)57 (3.4)
Predilatation (ref. yes; n, %)First lesion vessel
LM (n, %)435 (24.2)160 (26.8)179 (10.5)
LAD (n, %)876 (48.9)296 (49.5)1044 (61.5)
Cx/MO (n, %)320 (17.8)98 (16.4)332 (19.5)
RCA (n, %)133 (7.4)33 (5.5)142 (8.3)
RI (n, %)31 (1.7)11 (1.8)4 (0.2)
Lesion localization
Ostial (n, %)64 (3.6)21 (3.5)63 (3.7)
Proximal (n, %)545 (30.4)187 (31.3)964 (56.7)
Middle (n, %)642 (35.7)209 (34.9)401 (23.6)
Distal (n, %)544 (30.3)181 (30.3)273 (16.0)
Type C lesion (ref. yes; n, %)685 (38.2)212 (35.5)N.A.
Severe calcification (ref. yes; n, %)261 (14.5)88 (14.7)397 (23.3)
Diffuse disease (ref. yes; n, %)700 (39.0)238 (39.8)N.A.
Kind of bifurcation
Distal LM (n, %)481 (26.8)174 (29.1)179 (10.5)
LAD/Diag (n, %)839 (46.7)284 (47.5)1045 (61.5)
Cx/MO (n, %)341 (19.0)109 (18.2)334 (19.6)
RCA/PL (n, %)134 (7.5)31 (5.2)143 (8.4)
Medina class
1,1,1 (n, %)605 (33.8)205 (34.2)378 (22.2)
1,1,0 (n, %)587 (32.7)198 (33.0)640 (37.6)
1,0,1 (n, %)174 (9.7)47 (7.9)85 (5.0)
0,1,1 (n, %)87 (4.8)32 (5.4)80 (4.7)
1,0,0 (n, %)158 (8.8)41 (6.9)119 (7.0)
0,1,0 (n, %)92 (5.1)47 (7.9)258 (15.2)
0,0,1 (n, %)92 (5.1)28 (4.7)141 (8.3)
Kind of strategy
Provisional (n, %)1474 (82.1)490 (81.9)1447 (85.1)
Two stents (n, %)321 (17.9)108 (18.1)254 (14.9)
Use of imaging
No (n, %)1186 (66.1)400 (66.9)1648 (96.9)
IVUS (n, %)586 (32.6)194 (32.4)36 (2.1)
OCT (n, %)23 (1.3)4 (0.7)17 (1.0)
Predilatation (ref. yes; n, %)1559 (86.9)531 (88.8)1375 (80.8)
Kind of balloon
Conventional (n, %)1521 (97.5)518 (97.5)1666 (97.9)
Angiosculpt (n, %)18 (1.2)4 (0.8)35 (2.1)
Scoring (n, %)20 (1.5)9 (1.7)0 (0.0)
Rotablator (ref. yes; n, %)106 (5.9)38 (6.4)25 (1.5)
Kind of stent on MB N.A.
Resolute Onyx (n, %)506 (28.6)186 (31.3)
Xience Alpine (n, %)454 (25.5)136 (22.9)
Synergy (n, %)374 (21.0)131 (22.1)
Ultimaster (n, %)165 (9.3)50 (8.4)
Biomatrix alpha (n, %)2 (0.1)2 (0.3)
Promus (n, %)276 (15.5)89 (15.0)
MB lesion diameter (mm)3.0 [2.8; 3.5]3.0 [2.8; 3.5]N.A.
MB lesion length (mm)22 [16; 28]23 [16; 28]15 [10; 22]
MB atmospheres (atm)12 [12; 16]14 [12; 16]N.A.
Stent on SB (ref. yes; n, %)321 (17.9)108 (18.1)442 (26.0)
SB lesion diameter (mm)2.3 [1.0; 2.8]2.5 [1.0; 2.8]N.A.
SB lesion length (mm)28 [20; 33]28 [20; 33]N.A.
SB atmospheres (atm)12 [10; 14]12 [12; 14]N.A.
POT (ref. yes; n, %)1384 (77.1)447 (74.7)N.A.
ATM Post (atm)20 [16; 22]20 [16; 20]N.A.
Final kissing balloon (ref. yes; n, %)746 (41.6)248 (41.5)366 (21.5)
OutcomeDeath (ref. yes; n, %)103 (5.7)34 (5.7)39 (2.3)
Median follow-up at the event (days)274 [61; 433]253 [23; 458]284 [65; 500]
Death within 30 days (ref. yes; n, %)20 (1.1)9 (1.5)6 (0.4)
Death within 1 year (ref. yes; n, %)68 (3.8)21 (3.5)22 (1.3)
Death within 2 years (ref. yes; n, %)94 (5.2)31 (5.2)39 (2.3)
Patient and lesion parameters in the discovery cohort (n = 2393) after randomization into training (n = 1795) and internal validation cohorts (n = 598), and in the external validation cohort (n = 1701). Variables are reported as median [interquartile range], or absolute number (percentage, %), as appropriated. CKD, Chronic Kidney Disease; PCI, Percutaneous Coronary Intervention; CABG, Coronary Artery Bypass Graft; MI, Myocardial Infarction; EF, Ejection Fraction; STEMI, ST-segment Elevated Myocardial Infarction; NSTEMI, Non-ST-segment Elevated Myocardial Infarction; ACS, Acute Coronary Syndrome; DAT, Double Antiaggregant Therapy; LM, Left Main; LAD, Left Anterior Descending; Cx/MO, Circumflex/Marginal; RCA, Right Coronary Artery; RI, Right Intermedius; Diag, Diagonal; PL, Posterior Left; IVUS, IntraVascular UltraSound; OCT, Optical Coherence Tomography; MB, Main Branch; SB, Side Branch; POT, Proximal Optimization Technique. *For the external validation cohort, EF was coded dichotomously and reported for each patient as a value of either “30%” or “50%”.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Burrello, J.; Gallone, G.; Burrello, A.; Jahier Pagliari, D.; Ploumen, E.H.; Iannaccone, M.; De Luca, L.; Zocca, P.; Patti, G.; Cerrato, E.; et al. Prediction of All-Cause Mortality Following Percutaneous Coronary Intervention in Bifurcation Lesions Using Machine Learning Algorithms. J. Pers. Med. 2022, 12, 990. https://0-doi-org.brum.beds.ac.uk/10.3390/jpm12060990

AMA Style

Burrello J, Gallone G, Burrello A, Jahier Pagliari D, Ploumen EH, Iannaccone M, De Luca L, Zocca P, Patti G, Cerrato E, et al. Prediction of All-Cause Mortality Following Percutaneous Coronary Intervention in Bifurcation Lesions Using Machine Learning Algorithms. Journal of Personalized Medicine. 2022; 12(6):990. https://0-doi-org.brum.beds.ac.uk/10.3390/jpm12060990

Chicago/Turabian Style

Burrello, Jacopo, Guglielmo Gallone, Alessio Burrello, Daniele Jahier Pagliari, Eline H. Ploumen, Mario Iannaccone, Leonardo De Luca, Paolo Zocca, Giuseppe Patti, Enrico Cerrato, and et al. 2022. "Prediction of All-Cause Mortality Following Percutaneous Coronary Intervention in Bifurcation Lesions Using Machine Learning Algorithms" Journal of Personalized Medicine 12, no. 6: 990. https://0-doi-org.brum.beds.ac.uk/10.3390/jpm12060990

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop