Special Issue on Ensemble Learning and Applications

Abstract

During the last decades, in the area of machine learning and data mining, the development of ensemble methods has gained a significant attention from the scientific community. Machine learning ensemble methods combine multiple learning algorithms to obtain better predictive performance than could be obtained from any of the constituent learning algorithms alone. Combining multiple learning models has been theoretically and experimentally shown to provide significantly better performance than their single base learners. In the literature, ensemble learning algorithms constitute a dominant and state-of-the-art approach for obtaining maximum performance, thus they have been applied in a variety of real-world problems ranging from face and emotion recognition through text classification and medical diagnosis to financial forecasting.

1. Introduction

This article is the editorial of the “Ensemble Learning and Their Applications”(https://0-www-mdpi-com.brum.beds.ac.uk/journal/algorithms/special_issues/Ensemble_Algorithms) Special Issue of the Algorithms journal. The main aim of this Special Issue is to present the recent advances related to all kinds of ensemble learning algorithms, frameworks, methodologies and investigate the impact of their application in a diversity of real-world problems. The response of the scientific community has been significant, as many original research papers have been submitted for consideration. In total, eight (8) papers were accepted, after going through a careful peer-review process based on quality and novelty criteria. All accepted papers possess significant elements of novelty, cover a diversity of application domains and introduce interesting ensemble-based approaches, which provide readers with a glimpse of the state-of-the-art research in the domain.

During the last decades, the development of ensemble learning methodologies and techniques has gained a significant attention from the scientific and industrial community [1,2,3]. The basic idea behind these methods is the combination of a set of diverse prediction models for obtaining a composite global model which produces reliable and accurate estimates or predictions. Theoretical and experimental evidence proved that ensemble models provide considerably better prediction performance than single models [4]. Along this line, a variety of ensemble learning algorithms and techniques have been proposed and found their application in various classification and regression real-word problems.

2. Ensemble Learning and Applications

The first paper is entitled “A Weighted Voting Ensemble Self-Labeled Algorithm for the Detection of Lung Abnormalities from X-Rays” and it is authored by Livieris et al. [5]. The authors presented a new ensemble-based semi-supervised learning algorithm for the classification of lung abnormalities from chest X-rays. The proposed algorithm exploits a new weighted voting scheme which assigns a vector of weights on each component learner of the ensemble based on its accuracy on each class. The proposed algorithm was extensively evaluated on three famous real-world benchmarks, namely the Pneumonia chest X-rays dataset from Guangzhou Women and Children’s Medical Center, the Tuberculosis dataset from Shenzhen Hospital and the cancer CT-medical images dataset. The presented numerical experiments showed the efficiency of the proposed ensemble methodology against simple voting strategy and other traditional semi-supervised methods.

The second paper is authored by Papageorgiou et al. [6] entitled “Exploring an Ensemble of Methods that Combines Fuzzy Cognitive Maps and Neural Networks in Solving the Time Series Prediction Problem of Gas Consumption in Greece”. This paper presents an innovative ensemble time-series forecasting model for the prediction of gas consumption demand in Greece. The model is based on an ensemble learning technique which exploits evolutionary Fuzzy Cognitive Maps (FCMs), Artificial Neural Networks (ANNs) and their hybrid structure, named FCM-ANN, for time-series prediction. The prediction performance of the proposed model was compared against that of the Long Short-Term Memory (LSTM) model on three time-series datasets concerning data from distribution points which compose the natural gas grid of a Greek region. The presented results illustrated empirical evidence that the proposed approach could be effectively utilized to forecast gas consumption demand.

The third paper “A Grey-Box Ensemble Model Exploiting Black-Box Accuracy and White-Box Intrinsic Interpretability” was written by Pintelas et al. [7]. In this interesting study, the authors proposed a new framework for the development of a Grey-Box machine learning model based on the semi-supervised philosophy. The advantages of the proposed model are that it is nearly as accurate as a Black-Box and it is also interpretable like a White-Box model. More specifically, in their proposed framework, a Black-Box model was utilized for enlarging a small initial labeled dataset, adding the model’s most confident predictions of a large unlabeled dataset. In the sequel, the augmented dataset was utilized for training a White-Box model which greatly enhances the interpretability and explainability of the final model (ensemble). For evaluating the flexibility as well as the efficiency of the proposed Grey-Box model, the authors used six benchmarks from three real-world application domains, i.e., finance, education, and medicine. Based on their detailed experimental analysis the authors stated that the proposed model reported comparable and sometimes better prediction accuracy compared to that of a Black-Box while being at the same time interpretable as a White-Box model.

The fourth paper was authored by Karlos et al. [8] entitled “A Soft-Voting Ensemble Based Co-Training Scheme Using Static Selection for Binary Classification Problems”. The authors presented an ensemble-based co-training scheme for binary classification problems. The proposed methodology is based on the imposition of an ensemble classifier as a base learner in the co-training framework. Its structure is determined by a static ensemble selection approach from a pool of candidate learners Their experimental results in a variery of classical benchmarks as well as the reported statistical analysis showed the efficacy and efficiency of their approach.

An interesting research entitled “GeoAI: A Model-Agnostic Meta-Ensemble Zero-Shot Learning Method for Hyperspectral Image Analysis and Classification was authored by Demertzis and Iliadis [9]. In this work, a new classification model was proposed, named MAME-ZsL (Model-Agnostic Meta-Ensemble Zero-shot Learning), which is based on zero-shot philosophy for geographic object-based scene classification. The attractive advantages of the proposed model are its training stability, its low computational cost, but mostly its remarkable generalization performance thought the reduction of potential overfitting. This is performed by the selection of features which do not cause the gradients to explode or diminish. Additionally, it is worth noticing that the superiority of MAME-ZsL model lies on the fact that the testing set contained instances whose classes were not contained in the training set. The effectiveness of the proposed architecture was presented against state-of-the-art fully supervised deep learning models on two datasets containing images from a reflective optics system imaging spectrometer.

Zvarevashe and Olugbara [10] presented a research paper entitled “Ensemble Learning of Hybrid Acoustic Features for Speech Emotion Recognition”. Signal processing and machine learning methods are widely utilized for recognizing human emotions based on extracted features from video files, facial images or speech signals. The authors studied the problem that many classification models were not able to efficiently recognize fear emotion with the same level of accuracy as other emotions. To address this problem, they proposed an elegant methodology for improving the precision of fear and other emotions recognition from speech signals, based on an interesting feature extraction technique. In more detail, their framework extracts highly discriminating speech emotion feature representations from multiple sources which are subsequently agglutinated to form a new set of hybrid acoustic features. The authors conducted a series of experiments on two public databases using a variety of state-of-the-art ensemble classifiers. The presented analysis which reported the efficiency of their approach, provided evidence that the utilization of the new features increased the generalization ability of all ensemble classifiers.

The seventh paper entitled “Ensemble Deep Learning for Multilabel Binary Classification of User-Generated Content is authored by Haralabopoulos et al. [11]. The authors presented an multilabel ensemble model for emotion classification which exploits a new weighted voting strategy based on differential evolution. Additionally, the proposed model used deep learning learners which comprised of convolutional and pooling layers as well as (LSTM) layers which are dedicated for such classification problems. To present the efficiency of their model, they conducted a performance evaluation, on two large and widely used datasets, against state-of-the-art single models and ensemble models which were comprised with the same base learners. The reported numerical experiments showed that the proposed model presented improved classification performance, outperforming state-of-the-art compared models.

Finally, the eighty paper ”Ensemble Deep Learning Models for Forecasting Cryptocurrency Time-Series” was authored by Livieris et al. [12]. The main contribution of this research is the combination of three of the most widely employed ensemble strategies: ensemble-averaging, bagging and stacking with advanced deep learning methodologies for forecasting the cryptocurrency hourly prices of Bitcoin, Etherium and Ripple. More analytically, the ensemble models utilized state-of-the-art deep learning models as component learners, which were comprised by combinations of LSTM, Bi-directional LSTM and convolutional layers. The authors conducted an exhaustive experimentation in which the performance of all ensemble deep learning models was compared on both regression and classification problems. The models were evaluated on forecasting of the cryptocurrency price on the next hour (regression) and also on the prediction of next price directional movement (classification) with respect to the current price. Furthermore, the reliability of all ensemble model as well as the efficiency of their predictions was studied by examining for autocorrelation of the errors. The detailed numerical analysis indicated that ensemble learning strategies and deep learning techniques can be efficiently beneficial to each other, and develop accurate and reliable cryptocurrency forecasting models.

We would like to thank the Editor-in-Chief and the editorial office of the Algorithms journal for their support and for trusting us with the privilege to edit a special issue in this high-quality journal.

3. Conclusions and Future Approaches

The motivation behind this Special Issue was to make a minor and timely contribution to the existing literature. It is hoped that the novel approaches presented in this Special Issue will be found interesting, constructive and appreciated by the international scientific community. It is also expected that they will inspire further research on innovative ensemble strategies and applications in various multidisciplinary domains. Future approaches may involve exploiting ensemble learning for improving prediction accuracy, machine learning explainability and enhancing model’s reliability.