Modeling Energy Demand—A Systematic Literature Review

Abstract

In this article, a systematic literature review of 419 articles on energy demand modeling, published between 2015 and 2020, is presented. This provides researchers with an exhaustive overview of the examined literature and classification of techniques for energy demand modeling. Unlike in existing literature reviews, in this comprehensive study all of the following aspects of energy demand models are analyzed: techniques, prediction accuracy, inputs, energy carrier, sector, temporal horizon, and spatial granularity. Readers benefit from easy access to a broad literature base and find decision support when choosing suitable data-model combinations for their projects. Results have been compiled in comprehensive figures and tables, providing a structured summary of the literature, and containing direct references to the analyzed articles. Drawbacks of techniques are discussed as well as countermeasures. The results show that among the articles, machine learning (ML) techniques are used the most, are mainly applied to short-term electricity forecasting on a regional level and rely on historic load as their main data source. Engineering-based models are less dependent on historic load data and cover appliance consumption on long temporal horizons. Metaheuristic and uncertainty techniques are often used in hybrid models. Statistical techniques are frequently used for energy demand modeling as well and often serve as benchmarks for other techniques. Among the articles, the accuracy measured by mean average percentage error (MAPE) proved to be on similar levels for all techniques. This review eases the reader into the subject matter by presenting the emphases that have been made in the current literature, suggesting future research directions, and providing the basis for quantitative testing of hypotheses regarding applicability and dominance of specific methods for sub-categories of demand modeling.

1. Introduction

The transformation of our energy system towards a more reliable, eco-friendly, and cost-effective one is a central goal of today’s energy policy. An integral part of the planning processes across different infrastructures are energy system models. As the scope of such models is expanding across multiple infrastructures and energy carriers [1] they become increasingly detailed and complex [2]. Hence, well-founded information on future energy demand with the high temporal and spatial resolution is one of the most crucial inputs for such models, having a direct impact on associated decision-making processes [3] affecting real-time grid operation as well as long-term infrastructure extension planning. Accordingly, there is a strong need for reliable models predicting and simulating energy demand (in this article, all methods for the mathematical representation of energy demand or consumption are summarized under the term “energy demand modeling”. Therefore, the terms energy consumption and energy demand are to be understood synonymously). Finally, energy demand modeling is the essential basis for all quantifications of demand flexibility.

There is an entire field of research revolving around the question of how energy demand can be modeled using a variety of approaches on different scales ranging from a global level down to a single appliance [4,5]. In the year 2009, there were 60 English articles indexed on “Web of Science”, which had “energy demand” (the query also included “energy consumption” as a synonym for “energy demand”) and “model” in their title. In 2020 this number had increased to 641. Energy demand models have a wide range of applications. As shown by Bhattacharyya and Timilsina [6], they can range from short-term energy consumption forecasting in energy grids and markets over a simulation of heat and electricity loads in buildings and industrial processes to econometric long-term projections of national energy demand. In this article both, future-oriented forecasting, as well as operational simulation of energy demand in technical systems, is addressed by the term modeling.

Several reviews have been published capturing the variety of approaches and describing the developments in energy demand modeling literature. 28 recent reviews have been analyzed for this article. An overview of their characteristics can be found in

Table A1. Characteristics of existing literature reviews. Overview of existing literature reviews by content. In each line, black squares (■) indicate topics covered in the given review. Only seven literature reviews used a systematic approach. Most reviews cover more than one sector or energy carrier and are concerned with analyzing model inputs (demand drivers). Few reviews show the number of articles reviewed. Only the present article covers all aspects.

Systematic	Energy Carriers				Sectors				Building Focused	Demand Drivers	Reviewed Articles	References
	Electricity	Thermal	Natural gas	Primary energy	Residential	Commercial	Industries	All sectors
■	■							■		■	41	[9]
■	■	■			■					■	n/a	[11]
■	■	■			■	■	■		■	■	63	[10]
■	■	■	■	■				■			483	[4]
■	■	■			■	■	■				130	[12]
■			■		■	■				■	39	[13]
■	■	■	■	■				■		■	116	[14]
	■	■						■	■	■	n/a	[205]
	■				■	■	■		■		n/a	[206]
	■		■				■				n/a	[207]
	■				■	■			■	■	31	[208]
	■							■		■	n/a	[5]
	■							■			n/a	[209]
	■				■	■	■			■	50	[23]
	■				■					■	n/a	[210]
	■					■			■	■	n/a	[211]
	■						■			■	n/a	[212]
	■										n/a	[213]
		■						■	■		n/a	[214]
	■	■						■	■		n/a	[215]
	■	■			■	■			■	■	n/a	[216]
	■	■			■				■		n/a	[217]
	■	■		■	■	■	■		■	■	n/a	[128]
	■	■			■				■	■	n/a	[218]
	■	■			■	■			■		n/a	[219]
	■	■			■	■	■		■		n/a	[220]
			■					■		■	17	[221]
	■							■			n/a	[222]
				■				■		■	n/a	[223,224]
■	■	■	■	■	■	■	■	■	■	■	419	This review

in the Appendix A. Seven out of these 28 stood out in terms of their systematic procedure ensuring transparency, replicability, and reduced bias following the conduct of a systematic literature review as described in [5,7,8]. These seven studies will be briefly presented in the following.

Kuster et al. [9] present a review on electric load forecasting techniques. 41 papers are reviewed regarding applied techniques, input data, pre-processing routines, geographic extend, temporal resolution, and horizon. While this review covers a variety of criteria, the number of reviewed articles could be extended across other energy carriers and sectors. In [10], 63 articles are reviewed which focus on energy consumption in buildings mainly applying ML techniques. The authors analyze the reviewed articles regarding techniques, types of feature, pre-processing, temporal granularity, data size, type of building, type of energy end-use, and performance measures. In [11], an analysis of the viability of various model inputs for residential energy consumption is given, focusing on socio-demographic, psychological, and contextual factors. In [4], Debnath and Mourshed present a review on forecasting techniques for supply and demand in energy planning models across all energy carriers. The authors present 483 models from articles published between 1985 and 2017. They discuss geographical extend, time frames, and performance measures, as well as specific criteria for techniques, such as the number of neurons in layers for artificial neural networks (ANN). While this review provides a wide-ranging analysis, data-related aspects are not included and a distinction between sectors is missing. Riva et al. [12] provide an analysis of 130 peer-reviewed studies on long-term rural energy planning, covering the electricity, oil, and heating sector on the demand and supply side. The reviewed studies are classified according to spatial coverage, planning horizon, energy carrier, mathematical models, and energy use. Šebalj et al. [13] review 39 articles on predicting natural gas consumption in the residential and commercial sector, published between 2003 and 2017. Articles are categorized regarding technique, input variables, spatial scope, and temporal horizon. Wei et al. [14] compiled a literature study on conventional and artificial intelligence-based models in energy consumption forecasting. 116 publications have been described with respect to purpose, temporal horizons, data properties, applied areas, pre-processing, and forecasting techniques. Additionally, forecasting accuracy is evaluated considering the MAPE.

Table 1. Overview of recent systematic literature by content. In each line, black squares (■) indicate topics covered in the given review. Most reviews cover several sectors or energy carriers and analyze model inputs and spatio-temporal features. Few reviews analyze model accuracies and only the present article covers all the aspects.

Techniques	Energy Carriers				Sectors				Spatio-Temporal Features			Input Data	Accuracy	Articles	Reference
	Electricity	Thermal	Natural gas	Primary energy	Residential	Commercial	Industries	All sectors together	Temp. horizon	Temp. resolution	Spatial resolution
■	■							■	■	■	■	■		41	[9]
■	■	■			■							■		n/a	[11]
■	■	■			■	■	■			■	■	■	■	63	[10]
■	■	■	■	■				■	■		■		■	483	[4]
■	■	■			■	■	■		■		■			130	[12]
■			■		■	■			■		■	■		39	[13]
■	■	■	■	■				■	■		■	■	■	116	[14]
■	■	■	■	■	■	■	■	■	■	■	■	■	■	419	This article

shows which aspects have been covered by recent systematic reviews. It reveals that none of the existing reviews provides comprehensive coverage regarding all of the aspects analyzed in the article at hand.

2. Methodology

The literature review follows a systematic procedure as recommended in [4,7,9]. The step-by-step procedure is shown in Figure 1.

This review provides a comprehensive description and well-structured presentation of the content of recent international literature on energy demand modeling. Therefore, a systematic and replicable analysis of a high number of articles was conducted regarding the utilized techniques as well as associated input data, accuracy, and spatio-temporal resolution across different energy carriers and sectors. This comprehensive and concise literature classification serves as a decision-base for fellow researchers for the selection of appropriate data-model combinations for their projects. Direct and easy access to articles corresponding to a particular set of criteria is provided through structured tables in the Appendix A. Moreover, the advantages and drawbacks of common techniques as well as countermeasures against disadvantages are presented. This review constitutes an exploratory study examining and categorizing a broad and up-to-date literature base regarding an unprecedented number of properties using descriptive statistical methods. Challenges and future research directions are suggested and the compiled material provides a basis for future hypothesis-based quantitative testing.

The article is organized as follows: In Section 2, the systematic review process is described. In Section 3, a description and classification of techniques are given. In Section 4 the results of the literature analysis are presented, starting with sectors and energy carriers and followed by results on modeling techniques, input data, temporal and spatial characteristics as well as accuracy. In Section 5 most significant results are discussed and in Section 6 future research directions are suggested. The paper concludes with Section 7.

To aim for recent and relevant literature, the search was limited to articles published between 2015 and 2020 in journals related to energy, engineering, modeling, and simulation or computer science in English. The literature base for this review is the result of a replicable query to Web of Science Core Collection, a database for international journal publications and conference proceedings [15], on 1 May 2021. A search string was derived from a keyword matrix containing keywords from the thematic groups “energy”, “demand” and “modeling”. The search string and keyword matrix can be found in the Appendix A of this review (see

Table A2. Keyword compilation. The table contains the keywords used for the literature research arranged by thematic groups. Keywords in the bottom cell are explicitly excluded, which leads to a more precise search result. The * is a truncation operator to retrieve words with variant zero to many characters.

Energy	Demand	Modeling
Electric *	Demand	Forecast *
Natural gas	Consumption	Estimat *
Heat	Load	Predict *
	Requirement	Project *
	Intensity	Simulation
		Disaggregation
		Planning
		Model *
		Bottom-up
		Top-down
Excluded keywords: storage, carbon, emission, price, optimization, vehicle, climate
Search string applied to the Web of Science Core Collection on 1 May 2021: (TI = ((electric* OR “natural gas” OR heat) AND (demand OR consumption OR load OR requirement OR intensity) AND (forecast* OR estimat* OR predict* OR project* OR simulation OR disaggregation OR planning OR model* OR “bottom-up” OR “top-down”)) NOT TS = (storage OR carbon OR emission OR price OR optimization OR vehicle OR climate))

). The search yielded 695 articles, which were then further scrutinized based on their title and abstract resulting in an exclusion of 276 articles due to non-matching topics or closed access despite institutional logins at the publishers’ websites. The final literature collection contains 419 articles.

Articles are analyzed according to the properties listed in

Table 2. Assessment criteria. Overview of analysis criteria defining the collected data during step three of the review procedure. Each item represents a property characterizing the techniques applied in the respective articles. A short description and possible values are given. For mutually exclusive criteria only one value is possible, while for non-exclusive properties multiple values can be given and counted multiple times.

Analysis Criteria	Description	Possible Values	Mutually Exclusive
Technique	Modeling technique applied	Artificial neural network, support vector machine, regression, autoregressive methods, etc.	No
Category of techniques	General category of applied technique	Statistical, machine learning, metaheuristic, stochastic/fuzzy/grey, and engineering-based techniques	No
Technique combination	A single technique or a combination of techniques was applied	Stand-alone or hybrid approach	Yes
Model inputs	Inputs for energy demand models serving as explanatory variables and predictors	Data describing historic load, calendar information, weather, economy, demographics, environment, prices, behavior, and information about the technical system	No
Energy carrier	Forecasted/modeled type of energy	Electricity, natural gas, energy for heating and cooling	No
Sector	Economic sector or consumer group which is modeled	Industrial, commercial, residential, all sectors	No
Technical system	Applications or technical systems, which are modeled	Power grid, gas grid, district heating, building, production	No
Spatial resolution	Spatial level of detail of models	Country, regions (e.g., district), households/buildings, appliances	Yes
Temporal resolution	Scale of time steps that are described by the models	Sub-hourly, hourly, daily, above daily	Yes
Temporal horizon	Timespan that is covered by the models	Short-term (up to one day), medium-term (several weeks or months), long-term (one year and above)	Yes
Accuracy	Performance evaluation of presented models	Numeric values for MAPE	No

. Given the variety of entries for all the criteria, they have been grouped in the column “possible values” in Table 2. The spatial resolution is defined by the smallest energy-consuming entity, which is modeled in the respective articles. For the temporal horizon, various categorizations exist in the literature [4,16]. The chosen definition is inspired by Wei et al. [14]. The MAPE is defined as the average absolute discrepancy between the predicted value and the actual value, expressed as a percentage of the actual value [17]. It is a unitless performance measure and not dependent on the magnitude of the system, which makes it appropriate for comparing the performance of techniques applied in different contexts [18]. Therefore, it is a widely used accuracy measure in energy demand modeling [5]. For the techniques, a variety of classifications can be found in the literature. The following section provides a clear definition of categories of techniques used for energy demand modeling.

The 419 articles represent the total population of units whose properties are analyzed and described. Hence, descriptive statistical techniques are used in order to illustrate the frequency and contingency of the properties of the articles, using bar plots and box plots. The data collected from the articles are categorical in all cases except for the MAPE value, which is of numerical continuous type. For the MAPE value, a histogram was plotted in order to illustrate its (non-symmetric) distribution.

After classification and analysis, the results are presented using plots and structured tables for the direct accessibility of articles. Subsequently, highlights are discussed regarding particular advantages and drawbacks of techniques.

3. Classification of Techniques

A variety of classifications for techniques for energy demand modeling exists in the literature. Debnath and Mourshed [4] distinguish between statistical, computational intelligence (CI), and mathematical programming as well as stand-alone and hybrid techniques. Within the category of statistical techniques, they define regression, time series analysis (TSA), and autoregressive conditional heteroscedasticity (ARCH) techniques. Within the category of CI, they mention ML, uncertainty, and metaheuristic techniques as well as expert-based methods. Hong and Fang [5] suggest the two general categories of statistical and artificial intelligence techniques, with the former comprising multiple linear regression and TSA techniques and the latter including ANN, fuzzy regression, support vector machines (SVMs), and gradient boosting machines. Wei et al. [14] distinguish between conventional techniques, including TSA, regression, and grey models, and artificial intelligence techniques, such as ANN and SVM. Kuster et al. [9] discuss the categories of TSA, regression models, ANN, SVM, and bottom-up techniques.

This article is based on the classification by Debnath and Mourshed [4], however, extended by engineering-based techniques, which have been mentioned by other authors and referred to as bottom-up techniques [6,9,19,20,21,22]. Expert-based systems and mathematical programming mentioned in [4] are not considered since none of the analyzed articles followed either approach. Furthermore, the category of uncertainty techniques, which consists of fuzzy logic and grey models according to [4], is complemented by stochastic models, which have been encountered several times during data collection. The category name “uncertainty technique” might evoke some ambiguity amongst readers, since uncertainty is a natural property of any forecasting attempt. Therefore, the category is renamed “stochastic/fuzzy/grey systems theory”. Based on the classifications of [4,9] the following five categories are defined: statistical, ML, metaheuristic, stochastic/fuzzy/grey, and engineering-based techniques.

3.1. Statistical Techniques

According to [4,5], this category consists of regression and TSA techniques. As shown in [6], techniques from this category have been used in econometrics to explore the interrelationship between energy demand and economic development.

Regression techniques are used to solve an underlying regression problem [23], which consists of finding an approximation of a functional relation between numerical input and output variables. To find a solution for the approximation, different methods are employed, oftentimes minimizing the sum of the squares of errors [24]. For linear relations, this can be done by the ordinary least squares method. For non-linear relations, methods of steepest descent are used [24] or kernel functions [25]. Typical examples for statistical regression as found among the reviewed articles are linear, nonlinear, logistic, quantile, and ridge regression. Outside of statistical techniques, non-parametric regression can be found where ML techniques, such as ANN, kernel regression, or regression trees are employed to derive the functional form and regression parameters from the data [26,27].

TSA techniques derive their predictions from a historic time series, i.e., historic energy consumption data. In their core, many TSA approaches represent regression models since the predicted value is estimated based on one or more previous values [9]. This category includes univariate time series models such as autoregressive moving average (ARMA) models. Popular other techniques are autoregressive integrated moving average (ARIMA) models for non-stationary time series, seasonal autoregressive integrated moving average (SARIMA) models for seasonality, and ARMA models with exogenous variables (ARMAX) [5,28]. A typical multivariate TSA method is vector auto-regression as used in [29,30]. As suggested in [4], the category of TSA also includes exponential smoothing models and ARCH techniques.

3.2. Machine Learning Techniques

Techniques from this category find broad application in energy demand modeling and prediction and can be divided into supervised and unsupervised learning approaches.

Supervised learning approaches use labeled training datasets to derive a function describing a relation between inputs and outputs based on examples of input-output pairs [31]. They can be applied to numerical variables in the case of regression problems and categorical variables in the case of classification problems [32]. Within this sub-category, common techniques are ANN and instance-based algorithms, such as k-nearest neighbor and kernel machines [33]. A common example for the latter is SVM, which can convert nonlinear problems in low-dimensional space to linear problems in high-dimensional space [34]. Furthermore, in this category, there are decision trees, Bayesian algorithms, and ensemble learning approaches, such as gradient boosting machines [5].

Unsupervised learning approaches are often applied to clustering problems. These algorithms deduce structures in an unlabeled input dataset, e.g., through finding similarities [35].

3.3. Metaheuristic Techniques

Metaheuristic techniques oftentimes are used to solve optimization problems and can be incorporated into other techniques to improve performance [23]. The category includes evolutionary algorithms, which mimic mechanisms that are inspired by biological processes, such as reproduction, mutation, recombination, and selection [4]. It includes genetic algorithms, particle swarm optimization, bee colony optimization, firefly algorithms, and more [36]. In combination with ML approaches, they can be employed for parameter optimization in SVM [37] or weight optimization in ANN [22,38], resulting in approaches such as the firefly algorithm neural network [23]. Genetic algorithms are also used for feature selection in ML approaches [39,40].

3.4. Stochastic, Fuzzy and Grey Systems Theory Techniques

The techniques in this category are used to model different types of uncertainty. Zimmerman describes two basic forms of uncertainty: the traditional logic of probability describes randomness regarding the occurrence of an event, whereas fuzziness, describes the ambiguity of an event, i.e., to what extent an event occurs [41]. As Hájek et al. [42] point out, probability and fuzzy logic represent different sorts of uncertainty. Probability theory is used to describe stochastic processes [43] in which future states of a system are described by their past states plus a random change.

In energy demand modeling, stochastic processes such as Markov chains are used for forecasting and simulation of load profiles [20,44,45,46]. Fuzzy logic is employed in the form of fuzzy time series [47], fuzzy regression models [5], fuzzy clustering [48], and adaptive neuro-fuzzy interference systems (ANFIS) [49]. Tien states that fuzzy uncertainty can be analyzed with the grey system theory [50] and Debnath and Mourshed classify them as uncertainty methods [4]. The grey theory was proposed by Deng in 1982 and was developed to estimate the behavior of an uncertain system given only a limited amount of data [51]. The fundamental grey model GM(1,1) relies on as few as four recent data points to forecast the future data point [50]. The model uses the least square method to obtain the parameters for the grey differential equations, which describe the change between time steps [51].

3.5. Engineering-Based Techniques

Engineering-based techniques follow a bottom-up approach and use a variety of external and internal parameters to describe an energy-consuming system in high detail [6,23]. Oftentimes, individual loads of end-use appliances are considered to obtain aggregate profiles [19]. Common examples are models on the level of individual dwellings [52] or industrial processes [53] as well as building simulations [54]. Engineering-based models have proven their worth in planning and design of technical systems, being able to simulate a system’s behavior under conditions, for which there has been little historic data recorded yet [21]. Furthermore, they are promising approaches regarding the inclusion of the effects of household composition and individual behavior of dwellers as well as testing demand-side management strategies [20]. While techniques from this category are widely deployed in practice, they have been less visible in scientific articles on energy demand modeling and rarely been included in literature reviews.

Geographic information systems (GIS) are used for referencing data to geographic shapes, which have a distinct location and orientation relative to a reference coordinate system. Most of the time they are used for visualization and mapping of energy consumption as well as for urban [55] and rural [56] planning of infrastructure and building simulation [54,57].

4. Results

A total number of 419 articles originating from 54 different countries was reviewed. The country with the highest output of articles during the last years is China (98) followed by the USA (40) and Turkey (31). The analysis of the publication dates shows a slight increase of articles over the last six years from 66 in 2015 to 77 in 2020.

4.1. Sectors and Energy Carriers

Figure 2 shows the energy carriers and economic sectors on which the articles were focused. It reveals that in most articles the consumer group is not limited to a single sector but rather comprises all sectors together. This is the case when the energy consumption of an entire (market-) region is modeled. However, targeted modeling of residential and commercial energy consumption is also common, while industrial consumers are analyzed rather rarely. A possible interpretation could be the lower availability of publicly available consumption data for the industrial sector compared to the other sectors. Moreover, industrial energy consumption tends to be less dependent on exogenous influencing factors (e.g., weather) and rather exhibits production-related temporal patterns that are difficult to predict without knowledge of internal processes, which can be company-specific and proprietary knowledge.

Figure 2 also shows that most articles focus on electricity demand. However, particularly within the residential and commercial sectors, various articles are modeling the consumption of thermal energy, i.e., demand for heating and cooling. This could be partially explained by the fact, that there is a significant number of articles focusing on energy in buildings in the residential and commercial sectors, where thermal energy accounts for the largest share of the energy consumed. A separate analysis of all technical systems showed that most articles focus on power grids, smart grids, and buildings (see Figure A3).

Table A3. Articles sorted by energy carrier and sector. A compilation of the results of the analysis regarding energy carriers and sectors, which allows for direct tracing of the respective references.

Energy Carrier	Sector	References
Electricity	All sectors	[5,16,27,29,30,33,34,39,40,47,48,51,52,58,59,60,69,70,71,72,73,81,82,88,89,90,94,95,97,100,103,105,106,108,109,110,111,113,117,118,129,135,136,138,139,140,141,142,147,148,149,157,161,162,163,165,168,169,170,172,176,177,178,179,181,182,184,185,186,189,190,191,202,203,225,226,227,228,229,230,231,232,233,234,235,236,237,238,239,240,241,242,243,244,245,246,247,248,249,250,251,252,253,254,255,256,257,258,259,260,261,262,263,264,265,266,267,268,269,270,271,272,273,274,275,276,277,278,279,280,281,282,283,284,285,286,287,288,289,290,291,292,293,294,295,296,297,298,299,300,301,302,303,304,305,306,307,308,309,310,311,312,313,314,315,316,317,318,319,320,321]
	Residential	[19,20,21,44,52,56,62,64,66,68,75,79,84,86,92,93,96,98,122,126,130,143,158,160,164,166,174,322,323,324,325,326,327,328,329,330,331,332,333,334,335,336,337,338,339,340,341,342,343,344,345,346,347,348,349,350,351,352,353,354,355,356,357,358,359,360,361,362,363,364,365,366,367,368,369,370,371,372,373,374,375,376,377,378,379,380,381,382,383,384,385,386,387,388,389,390,391,392,393]
	Commercial	[17,21,61,63,75,79,84,91,102,120,131,132,137,145,150,151,152,156,173,233,325,333,336,340,346,357,362,363,364,371,378,394,395,396,397,398,399,400,401,402,403,404,405,406,407,408,409,410,411,412,413]
	Industry	[45,49,51,53,79,84,155,180,226,233,249,336,351,362,371,408,414,415,416,417,418,419,420,421,422,423,424]
Gas	All sectors	[18,80,99,112,133,153,192,287,425,426,427,428,429,430,431,432,433,434,435,436]
	Residential	[104,116,437,438,439,440,441,442,443,444,445]
	Commercial	[83,104,116,137,438,441,443]
	Industry	[83,419,441,443,446]
Heating/cooling	All sectors	[37,87,146,175,183,447,448,449,450,451,452]
	Residential	[22,52,54,57,74,85,101,144,154,171,187,194,334,341,451,453,454,455,456,457,458,459,460,461,462,463,464]
	Commercial	[61,77,91,101,107,114,115,137,154,159,398,459,465,466,467,468,469,470,471,472,473]
	Industry	[53,419,474]

in Appendix A provides a structured reference list of the analyzed articles by energy carrier and sector. Fellow researchers can use this overview to find articles, which have similar objects of research with regard to the energy carriers and the sector.

4.2. Techniques and Input Data

A variety of techniques is employed in the field of energy demand modeling. Figure 3 shows the number of appearances (n) of each of the five major categories of techniques (see Section 3) across all articles as well as the proportion of energy carriers whose consumption was modeled.

It is revealed that ML and statistical techniques are employed in most of the reviewed articles. A possible reason could be that ML and statistical techniques, like TSA, can be applied to a variety of use cases with relatively little effort in terms of model configuration and data preparation. They mainly require historic load data as input, which can be complemented by a limited set of external parameters, such as calendar or weather information [51,58,59,60]. Conventional regression techniques, like multiple linear regression, are commonly used among the articles as well and perform as benchmarks for other approaches [61,62,63].

Stochastic, fuzzy and grey techniques can be implemented as stand-alone models [64], e.g., simulating load profiles by sampling from stochastic processes such as Gaussian processes or Markov chains [52,65,66]. However, the representation of uncertain outcomes through stochastic, fuzzy or grey expressions is often combined with other techniques such as statistical regression [67,68] or ANN [69,70], giving results in the form of membership functions, intervals or probabilistic density.

Metaheuristic techniques are often used as part of hybrid techniques. Typical application is the optimization of model parameters [34,71] or feature selection [39,49] in ML models. Another example is the application of genetic algorithms in order to create optimized models as the result of an evolutionary process in which model a configuration is refined over multiple generations [72,73].

In the case of engineering-based techniques, energy demand is derived from the specifications of the system and its technical details [53,74,75]. Hence, an accurate simulation with an engineering-based model can require a high amount of data and effort, which might reduce the widespread use of such approaches [76]. However, engineering-based techniques are used to a greater extend to model heating and cooling demands, especially in the context of building simulation.

GIS techniques are used to reference data to geo-spatial shapes and visualization in the form of geographical maps. This has been used in the context of building simulation, where geometry and orientation was considered [54,57,77,78] or planning of rural [56] or urban [79,80] grid infrastructure. In other cases regional loads and trends in spatial energy consumption were analyzed [21,81,82,83] or modelled using socioeconomic data [84,85,86].

Figure 4 provides more details on the ML techniques that are used in the articles. ANNs [87,88], by far, show the highest number of appearances, followed by instance-based algorithms such as SVMs [89,90]. Clustering algorithms such as the k-means algorithm are used frequently to split datasets into groups of maximum similarity. This can be applied to the formation of consumer groups but also to finding similar days in historic load datasets [91,92,93]. Ensemble learning algorithms combine multiple ML techniques that are individually trained in order to obtain an improved overall performance. This can be achieved by bootstrap aggregating (bagging) [94,95] and boosting [96,97]. Bayesian algorithms and decision trees use supervised learning algorithms and therefore are typically used for regression and classification problems, e.g., Bayesian networks [98,99] or classification and regression trees [100,101]. By combining multiple decision trees through bagging, where each tree is trained on a different sub-sample of the dataset, random forest ensembles are created [94,102,103].

Overall, in 210 out of 419 articles a combined approach was employed. Figure 5 shows the combinations among the five main categories. The self-arcs show the number of times the respective technique was used as a stand-alone approach or was combined with a technique from the same category, which is the case for 63 articles within the category of ML, 17 articles with statistical techniques, three articles with metaheuristic techniques and respectively four articles with stochastic/fuzzy/grey and engineering-based techniques.

The analysis shows that among the analyzed articles, engineering-based techniques have the highest proportion of stand-alone models and metaheuristic techniques have the highest proportion of hybrid models. ML techniques often form hybrid techniques with themselves as well as with statistical and stochastic/fuzzy/grey techniques.

A typical example for a combined approach is the employment of techniques for clustering or frequency analysis for upstream data preparation. These techniques refine input data, e.g., by signal decomposition through Fourier or wavelet transformation, before the data is fed into a downstream model implemented by TSA [95,104,105] or ML techniques [106,107,108,109,110,111,112], where each of the decomposed signals is predicted separately. Another example is the integration of fuzzy mathematics into ANNs resulting in ANFIS [59,113,114,115] or the incorporation of metaheuristic optimization algorithms into the training stage of an ANN [116,117] or SVM [34,37,71].

In other cases, an overall prediction will be given as a weighted average of the results of multiple models, which can stem from different categories. The calculation of the weights can be subject to various (metaheuristic) optimization techniques, allowing TSA, regression or ML techniques to be combined into one approach [59,113,118,119,120]. In the case of engineering-based techniques, there are examples of combining simulation results from stochastic processes with bottom-up models, which are predominantly used for predicting energy demand in households [52,121,122].

Another aspect of the reviewed articles concerns the respective datasets that serve as inputs for the models.

Table 3. Model inputs. Classification of possible data-sets used as model input along with examples of data-sets.

Model Inputs	Examples
Historic energy demand	Historic load, electricity, heating, cooling, or natural gas demand
Weather data	Outside temperature, atmospheric pressure, cooling and heating degree days, humidity, solar radiation, wind speed
Calendar data	Time of day, day of the week, month, holidays, bridge days, seasons, workday, working hours, operating time of appliance
Demographic or economic data	Economic indicators: gross domestic product (GDP), gross national income (GNI), level of production, income, import and export level of a region; demographic indicators: human development indices, population, number of dwellers/buildings/residences, age, sex, education, infant mortality
Technical system data	Appliance data: equipment installed, number of appliances, efficiency, material properties, air change ratio, flow rate, outlet/inlet temperatures, rated power of the equipment, impedance Building data: floor space, number of bedrooms, transmission factor, building type, age of the building, efficiency rating, geometry of the building, the status of refurbishment, window area, building material, indoor temperature, indoor humidity
Usage and behavioral data	Time-use survey data, building usage (main residency, rented, owned, etc.), occupancy/activity patterns, operation/usage time of a device
Energy prices	Electricity and gas prices, tariffs, payment methods

gives an overview of examples for different model inputs, reflecting the variety of input data that is used in the field.

Figure 6 gives an overview of the frequency of usage of different types of input data as well as an indication of whether they are used in combination with others (“Multiple demand drivers (DD)”) or as a “single DD”. It is revealed that historic energy demand is used in 85% of the articles, which highlights its significance as a data source. As historic demand can be forecasted by trend extrapolation and pattern recognition, e.g., in the case of ARMA models, it constitutes a stand-alone data source.

Weather data is used in 43% of the articles. Its frequent use could be explained by the fact that operational patterns of some end-user devices are correlated with weather phenomena, e.g., heating, cooling, and lighting systems. Calendar information is also widely used (in 29% of the articles) since energy consumption can exhibit daily, weekly or annual patterns. For example, in most cases, the metering data of a company will reflect working hours.

Usage and behavioral data can be used to describe the relationship between energy consumption and user behavior, i.e., a parameter indicating that a technical device is now in use. Regional demographic and economic data describe properties of a region, such as household income or economic output, which can serve as predictors for the energy consumption of this region. The relatively rare use of price data might seem surprising, but can be explained by the international literature base largely covering countries with regulated or only recently liberalised energy markets, such as China, where price signals have not been transmitted to end consumers in the past [123]. Furthermore, energy demand has shown to be price inelastic both in the short and long-term [124], and therefore energy prices are not be considered as dominant drivers of demand.

When taking a closer look at the usage of input data across the techniques (Figure 7), it becomes apparent that among the examined articles engineering-based techniques do not rely on historic energy demand data as much as the other techniques. Here, external parameters such as information on the technical system or weather data are essential, whereas historic energy demand is rather used for calibration and validation purposes.

Table 4. Techniques and input data used (1/5). Compiled results of the analysis on techniques and input data. Each cell provides a short assessment of the respective data-technique combination. The corresponding articles for each cell are documented in detail in the structured reference lists in Table A4, Table A5, Table A6 and Table A7 in Appendix A.

Technique	Advantages and Disadvantages of Techniques	Historic Energy Demand	Weather Data	Calendar Data	Demographic or Economic Data	Technical System Data	Usage or Behavioral Data	Energy Prices
ANN	+ Established for classification and regression problems (high performance), low effort, handle non-linear relations, big variety of pre-set models, no knowledge of technical system needed − Risk of over-fitting, difficult interpretation (black-box), feature engineering required	Contributions: 107 Impact: High; can be used as a single input Drawbacks: Outputs dependent on data quality and availability Outlook: Continuous intensive use	Contributions: 65 Impact: High; predictor for heating and cooling and lighting systems Drawbacks: Limited explanatory value for other applications than heating/cooling/ lighting Outlook: Continuous intensive use	Contributions: 44 Impact: High; Predictor for regular daily, weekly or annual patterns Drawbacks: Risk of overestimation of periodic routines, cannot account for special events without knowledge Outlook: Continuous intensive use	Contributions: 13 Impact: Low for short-term load prediction; high for long-term regional, sectoral, or national demand prediction Drawbacks: Low level of detail regarding individual consumer patterns, usually yearly or quarterly resolution Outlook: Continuous use in cases of long-term national or sectoral demand modeling	Contributions: 18 Impact: High; explanatory value regarding process internal and end-user devices Drawbacks: Difficult to collect, have to be measured using sensors, might be subject to data privacy or company secrets Outlook: Occasional use, could see intensification	Contributions: 6 Impact: Potentially high; explanatory value regarding individual consumer patterns Drawbacks: High effort to collect because the result of time-of- use surveys, subject to privacy or company secrets, difficult to predict, the output of simulations Outlook: Rare use, could see intensification	Contributions: 4 Impact: Low Drawbacks: Rarely considered as a predictor because demand has low price elasticity, price swings only in liberalized markets, difficult to obtain future values to use in predictions Outlook: Rare use
Instance-based	+ Established for classification problems, good performance with a high number of features, for kernel machines there are many pre-set kernel functions to choose from, transforms nonlinear relations into linear ones in the feature space, robust against overfitting − Medium/high effort, dependent on the right choice of kernel, memory-intensive and limited scalability for big datasets	Contributions: 35 Impact: High; usually complemented by additional features Drawbacks: Outputs dependent on data quality and availability Outlook: Continuous intensive use	Contributions: 19 Impact: High; predictor for heating and cooling and lighting systems Drawbacks: Limited explanatory value for other applications than heating/cooling/lighting Outlook: Continuous intensive use	Contributions: 15 Impact: High; Predictor for regular daily, weekly or annual patterns Drawbacks: Risk of overestimation of periodic routines, cannot account for special events without knowledge Outlook: Continuous intensive use	Contributions: 11 Impact: High for classification of regions or consumer groups; low for short-term load prediction Drawbacks: Low level of detail regarding individual consumer patterns, usually yearly or quarterly resolution Outlook: Continuous use in cases of regional or sectoral modeling	Contributions: 8 Impact: High; explanatory value regarding process internal and end-user devices Drawbacks: Difficult to collect, have to be measured using sensors, might be subject to data privacy or company secrets Outlook: Occasional use, could see intensification	Contributions: 0 Impact: Potentially high explanatory value for classification of typical time steps considering individual consumer patterns Drawbacks: High effort to collect because the result of time-of- use surveys, subject to privacy or company secrets, difficult to predict, the output of simulations Outlook: No use, could see intensification	Contributions: 2 Impact: Low Drawbacks: rarely considered as a predictor because demand has low price elasticity, price swings only in liberalized markets, difficult to obtain future values to use in predictions Outlook: Rare use
Clustering	+ Finding natural groupings in an unsupervised learning process, easy to implement, different algorithms in place based on geographic distance (K-means), graph distance (affinity propagation), density (DBSCAN), or a hierarchical approach − Assumptions on the number and shape of clusters can be necessary and lead to mistakes	Contributions: 34 Impact: High; used to find similar time steps or similar consumer groups Drawbacks: Outputs dependent on data quality and availability Outlook: Continuous intensive use	Contributions: 13 Impact: High; used for finding similar days, a predictor for heating and cooling and lighting systems Drawbacks: Limited explanatory value for other applications than heating/cooling/ lighting Outlook: Continuous intensive use	Contributions: 12 Impact: High; for regular daily, weekly or annual patterns Drawbacks: Risk of overestimation of periodic routines, cannot account for special events without knowledge Outlook: Continuous intensive use	Contributions: 9 Impact: High for classification of regions or consumer groups; low for short-term load prediction; Drawbacks: Low level of detail regarding individual consumer patterns, usually yearly or quarterly resolution Outlook: Continuous use in cases of regional or sectoral modeling	Contributions: 5 Impact: High; explanatory value regarding process internal and end-user devices Drawbacks: Difficult to collect, have to be measured using sensors, might be subject to data privacy or company secrets Outlook: Occasional use, could see intensification	Contributions: 2 Impact: Potentially high explanatory value for classification of typical time steps considering individual consumer patterns Drawbacks: High effort to collect because the result of time-of- use surveys, subject to privacy or company secrets, difficult to predict, the output of simulations Outlook: Rare use, could see intensification	Contributions: 1 Impact: Low Drawbacks: Rarely considered as a predictor because demand has low price elasticity, price swings only in liberalized markets, difficult to obtain future values to use in predictions Outlook: Rare use

,

Table 5. Techniques and input data used (2/5).

Technique	Advantages and Disadvantages of Techniques	Historic Energy Demand	Weather Data	Calendar Data	Demographic or Economic Data	Technical System Data	Usage or Behavioral Data	Energy Prices
Ensemble learning	+ Improved predictive performance by combi-ning the predictions of multiple models, in-creased robustness by reducing the variance of prediction errors − Requires additional knowledge to solve the bias-variance tradeoff	Contributions: 19 Impact: High; always used Drawbacks: Outputs dependent on data quality and availability Outlook: Continuous use	Contributions: 12 Impact: High; predictor for heating and cooling and lighting systems Drawbacks: Limited explanatory value for other applications than heating/cooling/ lighting Outlook: Continuous use	Contributions: 9 Impact: High; Predictor for regular daily, weekly or annual patterns Drawbacks: Risk of overestimation of periodic routines, cannot account for special events without knowledge Outlook: Continuous use	Contributions: 1 Impact: Low for short-term load prediction; high for long-term regional, sectoral, or national demand prediction, Drawbacks: Low level of detail regarding individual consumer patterns, usually yearly or quarterly resolution Outlook: Use in cases of regional or sectoral modeling	Contributions: 3 Impact: Potentially high; explanatory value regarding process internal and end-user devices Drawbacks: Difficult to collect, have to be measured using sensors, might be subject to data privacy or company secrets Outlook: Occasional use, could see intensification	Contributions: 1 Impact: Potentially high; explanatory value regarding individual consumer patterns Drawbacks: a high effort to collect because the result of time-of-use surveys, subject to privacy or company secrets, difficult to predict, the output of simulations Outlook: Rare use, could see intensification	Contributions: 1 Impact: Low Drawbacks: Rarely considered as a predictor because demand has low price elasticity, price swings only in liberalized markets, difficult to obtain future values to use in predictions Outlook: Rare use
Deep learning	+ Established for classification and regression problems, can learn complex patterns be using hidden layers creating intermediary representations of the data, reduced need for feature engineering using drop-out layers − Large amounts of training data required, specialized algorithms, computationally intensive to train, require additional expertise to tune	Contributions: 17 Impact: High; always used Drawbacks: Outputs dependent on data quality and availability Outlook: Continuous use	Contributions: 9 Impact: High; often used for short term load forecasting Drawbacks: Limited explanatory value for other applications than heating/ cooling/lighting Outlook: Continuous use	Contributions: 7 Impact: High; Predictor for regular daily, weekly or annual patterns Drawbacks: Risk of overestimation of periodic routines, cannot account for special events without knowledge Outlook: Continuous use	Contributions: 1 Impact: Low for short-term load prediction; high for long-term regional, sectoral, or national demand prediction Drawbacks: Low level of detail regarding individual consumer patterns, usually yearly or quarterly resolution Outlook: Use in cases of regional or sectoral modeling	Contributions: 1 Impact: Potentially high; explanatory value regarding process internal and end-user devices Drawbacks: Difficult to collect, have to be measured using sensors, might be subject to data privacy or company secrets Outlook: Rare use, could see intensification	Contributions: 1 Impact: Potentially high; explanatory value regarding individual consumer patterns Drawbacks: High effort to collect because the result of time-of-use surveys, subject to privacy or company secrets, difficult to predict, the output of simulations Outlook: Rare use, could see intensification	Contributions: 0 Impact: Low Drawbacks: Rarely considered as a predictor because demand has low price elasticity, price swings only in liberalized markets, difficult to obtain future values to use in predictions Outlook: No use
Bayesian algorithms	+ Established for classification, basic models (Naïve Bayes) have low implementation effort, good performance, good scalability, are able to handle conflicting/limited information and nonlinear relations − Design of advanced models (B. networks) requires expert knowledge or good data to learn from, computationally expensive, therefore simplifications are used, however, they assume conditional independence between input features, which is rarely true	Contributions: 9 Impact: High if used for forecasting Drawbacks: Outputs dependent on data quality and availability Outlook: Continuous use	Contributions: 8 Impact: High; used in almost all cases, often used for forecasting for heating and cooling demand Drawbacks: Limited explanatory value for other applications than heating/ cooling/lighting Outlook: Continuous use	Contributions: 5 Impact: High; used in short-term forecasts, Predictor for regular daily, weekly or annual patterns Drawbacks: Risk of overestimation of periodic routines, cannot account for special events without knowledge Outlook: Continuous use	Contributions: 2 Impact High for classification of regions or consumer groups; low for short-term load prediction; Drawbacks: Low level of detail regarding individual consumer patterns, usually yearly or quarterly resolution Outlook: Use in cases of regional or sectoral modeling	Contributions: 2 Impact: Low for forecasting, used in cases of algorithms for energy management and system control Drawbacks: Difficult to collect, have to be measured using sensors, might be subject to data privacy or company secrets Outlook: Sporadic use	Contributions: 1 Impact: Low for forecasting; potentially high for classification of typical time steps considering individual consumer patterns, used for simulations of demand in smart grids Drawbacks: High effort to collect because the result of time-of-use surveys, subject to privacy or company secrets, difficult to predict, the output of simulations Outlook: Rare use	Contributions: 1 Impact: Low, can be used for simulation of smart grids Drawbacks: Rarely considered as a predictor because demand has low price elasticity, price swings only in liberalized markets, difficult to obtain future values to use in predictions Outlook: Rare use

,

Table 6. Techniques and input data used (3/5). Continuation of Table 5.

Technique	Advantages and Disadvantages of Techniques	Historic Energy Demand	Weather Data	Calendar Data	Demographic or Economic Data	Technical System Data	Usage or Behavioral Data	Energy Prices
Decision trees	+ Established for classification, rarely used for regression, handles non-linear relationships by splitting data into homogenous sub-samples, low effort in data preparation, robust to outliers or missing values, good scalability − Risk of overfitting, very data sensitive, a small change in data can result in a major change of tree structure	Contributions: 7 Impact: High; always used Drawbacks: Outputs dependent on data quality and availability Outlook: Continuous use	Contributions: 5 Impact: High; used in almost all cases, often used for forecasting for heating and cooling demand Drawbacks: Limited explanatory value for other applications than heating/ cooling/lighting Outlook: Continuous use	Contributions: 5 Impact: High; used in short-term forecasts, a predictor for regular daily, week-ly or annual patterns Drawbacks: Risk of overestimation of periodic routines, cannot account for special events without knowledge Outlook: Continuous use	Contributions: 3 Impact: High for classification of regions or consumer groups; low for short-term load prediction; Drawbacks: Low level of detail regarding individual consumer patterns, usually yearly or quarterly resolution Outlook: Continuous use in cases of regional or sectoral modeling	Contributions: 3 Impact: High; explanatory value regarding process internal and end-user devices Drawbacks: Difficult to collect, have to be measured using sensors, might be subject to data privacy or company secrets Outlook: Occasional use, could see intensification	Contributions: 0 Impact: Potentially high; explanatory value for classification of typical time steps considering individual consumer patterns Drawbacks: High effort to collect because the result of time-of-use surveys, subject to privacy or company secrets, difficult to predict, the output of simulations, Outlook: Could see intensification	Contributions: 0 Impact: Low Drawbacks: Rarely considered as a predictor because demand has low price elasticity, price swings only in liberalized markets, difficult to obtain future values to use in predictions Outlook: Rare use
Regression	+ Low implementation effort, good performance, computationally inexpensive, white box character, measures against overfitting in place (regularization) − Risk of underfitting, sensitive to outliers, the underlying assumption that features are independent does not hold in reality	Contributions: 88 Impact: High; the dependent variable Drawbacks: Outputs dependent on data quality and availability Outlook: Continuous intensive use	Contributions: 50 Impact: High; one of the most used independent variables, especially for heating/ cooling/lighting Drawbacks: Limited explanatory value for other applications than heating/cooling/lighting Outlook: Continuous intensive use	Contributions: 31 Impact: High; predictor for regular daily, weekly or annual patterns Drawbacks: Risk of overestimation of periodic routines, cannot account for special events without knowledge Outlook: Continuous intensive use	Contributions: 31 Impact: High for long-term regional, sectoral or national demand prediction Drawbacks: Low level of detail regarding individual consumer properties, usually yearly or quarterly resolution Outlook: Continuous intensive use in cases of regional or sectoral modeling	Contributions: 13 Impact: High; explanatory value regarding process internal and end-user devices Drawbacks: Difficult to collect, have to be measured using sensors, might be subject to data privacy or company secrets Outlook: Occasional use, could see intensification	Contributions: 10 Impact: Potentially high; explanatory value regarding individual consumer patterns Drawbacks: High effort to collect because the result of time-of-use surveys, subject to privacy or company secrets, difficult to predict, the output of simulations Outlook: Occasional use, could see intensification	Contributions: 3 Impact: Low Drawbacks: Rarely considered as a predictor because demand has low price elasticity, price swings only in liberalized markets, difficult to obtain future values to use in predictions Outlook: Rare use
TSA/ARCH	+ Low implementation effort, data cleaning, and understanding is done at the same time, uncovers patterns in data like autocorrelation and seasonality, filters out noise, lots of pre-set models − Cannot predict unpreceded events, poor handling of outliers which can be propagated into future, not many techniques to deal with large numbers of variables and complex relationships, less suitable for long-term forecasting	Contributions: 78 Impact: High; always used, often used as a single input Drawbacks: Outputs dependent on data quality and availability Outlook: Continuous intensive use	Contributions: 20 Impact: Medium; used as an external variable, especially for heating/cooling/ lighting Drawbacks: Limited explanatory value for other applications than heating/cooling/lighting Outlook: Continuous use	Contributions: 19 Impact: Medium; used as an external variable, a predictor for regular temporal patterns Drawbacks: Risk of overestimation of periodic routines, cannot account for special events without knowledge Outlook: Continuous use	Contributions: 10 Impact: Low for short-term load prediction; medium for long-term regional, sectoral, or national demand prediction Drawbacks: Low level of detail regarding individual consumer patterns, usually yearly or quarterly resolution Outlook: Use in cases of regional or sectoral modeling	Contributions: 0 Impact: Low; use of many external variables is generally rare with TSA Drawbacks: Difficult to collect, have to be measured using sensors, might be subject to data privacy or company secrets Outlook: Rare use	Contributions: 1 Impact: Low; use of many external variables is generally rare with TSA Drawbacks: High effort to collect because the result of time-of-use surveys, subject to privacy or company secrets, difficult to predict, the output of simulations Outlook: Rare use	Contributions: 1 Impact: Low; since external variables are generally rarely used with TSA in general Drawbacks: Rarely considered as a predictor because of low price elasticity, price swings only in liberalized markets, difficult to obtain future values Outlook: Rare use

,

Table 7. Techniques and input data used (4/5). Continuation of Table 6.

Technique	Advantages and Disadvantages of Techniques	Historic Energy Demand	Weather Data	Calendar Data	Demographic or Economic Data	Technical System Data	Usage or Behavioral Data	Energy Prices
Stochastic	+ Capacity to handle uncertainty regarding the occurrence of events and produce variations of possible outcomes, the underlying assumptions about the randomness can be tested statistically, allowing to estimate not only the expected value but also the variations of the expected values Potentially high implementation effort can be computationally expensive, results can be difficult to communicate	Contributions: 33 Impact: High; used to define probability distribution on historic values Drawbacks: Outputs dependent on data quality and availability Outlook: Continuous intensive use	Contributions: 15 Impact: High; predictor for heating and cooling and lighting systems Drawbacks: Limited explanatory value for other applications than heating/cooling/lighting Outlook: Continuous intensive use	Contributions: 15 Impact: High; predictor for regular daily, weekly or annual patterns Drawbacks: Risk of overestimation of periodic routines, cannot account for special events without knowledge Outlook: Continuous intensive use	Contributions: 10 Impact: High for long-term regional, sectoral or national demand prediction Drawbacks: Low level of detail regarding individual consumer properties, usually yearly or quarterly resolution Outlook: Continuous intensive use in cases of regional or sectoral modeling	Contributions: 8 Impact: High; explanatory value regarding process internal and end-user devices Drawbacks: Difficult to collect, have to be measured using sensors, might be subject to data privacy or company secrets Outlook: Continuous use, could see intensification	Contributions: 8 Impact: High; explanatory value regarding individual consumer patterns Drawbacks: High effort to collect because the result of time-of-use surveys, subject to privacy or company secrets, difficult to predict, the output of simulations Outlook: Continuous use, could see intensification	Contributions: 2 Impact: Low; can be used for simulation of smart grids Drawbacks: Rarely considered as a predictor because demand has low price elasticity, price swings only in liberalized markets, difficult to obtain future values to use in predictions Outlook: Sporadic use
Fuzzy	+ Adopts vagueness in human reasoning modeling the degree of occurrence of an event, can display a range of possibilities for inputs by applying membership functions (fuzzyfication), can handle incomplete data, after the model’s ruleset is applied defuzzyfication can be done following different principles (e.g., weighted average), computationally inexpensive, − Results can be perceived as inaccurate, communication of results can be difficult, depends on expert knowledge to be set up, extensive validation and testing	Contributions: 32 Impact: High; exact usage also depends on the other part of the hybrid model (ML, TSA, etc.) Drawbacks: Outputs dependent on data quality and availability Outlook: Continuous intensive use	Contributions: 11 Impact: High; predictor for heating and cooling and lighting systems Drawbacks: Limited explanatory value for other applications than heating/cooling/lighting Outlook: Continuous intensive use	Contributions: 10 Impact: Medium; exact usage depends on an-other part of the hybrid model, usually no fuzziness about calendar information Drawbacks: Risk of overestimation of periodic routines, cannot account for special events without knowledge Outlook: Continuous use	Contributions: 7 Impact: High for long-term regional, sectoral or national demand prediction Drawbacks: Low level of detail regarding individual consumer properties, usually yearly or quarterly resolution Outlook: Continuous intensive use in cases of regional or sectoral modeling	Contributions: 3 Impact: Medium; exact usage depends on another part of the hybrid model, high explanatory value regarding process internal and end-user devices Drawbacks: Difficult to collect, have to be measured using sensors, might be subject to data privacy or company secrets Outlook: Occasional use, could see intensification	Contributions: 2 Impact: Potentially high; explanatory value regarding individual consumer patterns Drawbacks: High effort to collect because the result of time-of-use surveys, subject to privacy or company secrets, difficult to predict the output of simulations Outlook: Occasional use, could see intensification	Contributions: 1 Impact: Low; can be used for simulation of smart grids Drawbacks: Rarely considered as a predictor because demand has low price elasticity, price swings only in liberalized markets, difficult to obtain future values to use in predictions Outlook: Sporadic use
Metaheuristic	+ Provide alternative and promising algorithms to solve optimization problems and efficiently search the solution space, variety of metaheuristic algorithms in place which requires little implementation effort − No guarantee that the global maximum is attained, selection of algorithm can be difficult, depending on adequate parameter tuning	Contributions: 26 Impact: High; exact usage also depends on the other part of the hybrid model (ML, regression, etc.) Drawbacks: Outputs dependent on data quality and availability Outlook: Continuous intensive use	Contributions: 17 Impact: high, often used as a predictor for heating/cooling/lighting Drawbacks: Limited explanatory value for other applications than heating/cooling/lighting Outlook: Continuous intensive use	Contributions: 7 Impact: Medium; exact usage depends on an-other part of the hybrid model Drawbacks: Risk of overestimation of periodic routines, cannot account for special events without knowledge Outlook: Continuous use	Contributions: 7 Impact: High for long-term regional, sectoral or national demand prediction Drawbacks: Low level of detail regarding individual consumer properties, usually yearly or quarterly resolution Outlook: Continuous intensive use in cases of regional or sectoral modeling	Contributions: 2 Impact: Potentially high; explanatory value regarding process internal and end-user devices, exact usage depends on another part of the hybrid model Drawbacks: Difficult to collect, have to be measured using sensors, might be subject to data privacy or company secrets Outlook: Occasional use, could see intensification	Contributions: 1 Impact: Potentially high; explanatory value regarding individual consumer patterns Drawbacks: High effort to collect because the result of time-of-use surveys, subject to privacy or company secrets, difficult to predict, the output of simulations Outlook: Rare use, could see intensification	Contributions: 3 Impact: Low Drawbacks: Rarely considered as a predictor because demand has low price elasticity, price swings only in liberalized markets, difficult to obtain future values to use in predictions Outlook: Occasional use

and

Table 8. Techniques and input data used (5/5). Continuation of Table 7.

Technique	Advantages and Disadvantages of Techniques	Historic Energy Demand	Weather Data	Calendar Data	Demographic or Economic Data	Technical System Data	Usage or Behavioral Data	Energy Prices
Engineering-based	+ White box character, revealing detailed input-output relations, Able to simulate energy demand for explorative and normative scenarios including disruptions that have no historic record − Data and knowledge-intensive, prediction accuracy can be low due to simplifications regarding the system	Contributions: 21 Impact: High; historic demand used for validation of model outputs Drawbacks: Outputs dependent on data quality and availability Outlook: Continuous intensive use	Contributions: 22 Impact: high, often used as a predictor for heating/cooling/lighting Drawbacks: Limited explanatory value for other applications than heating/cooling/lighting Outlook: Continuous intensive use	Contributions: 8 Impact: Medium; not needed to describe physical input-output relations, a predictor for regular daily, weekly or annual patterns Drawbacks: Risk of overestimation of periodic routines, cannot account for special events without know-ledge Outlook: Continuous use	Contributions: 16 Impact: High for long-term regional, sectoral or national demand prediction Drawbacks: Low level of detail regarding individual consumer properties, usually yearly or quarterly resolution Outlook: Continuous intensive use in cases of regional or sectoral modeling	Contributions: 37 Impact: High; most important information to describe physical input-output relations, Drawbacks: High amount of data needed, difficult to collect, have to be measured using sensors, might be subject to data privacy or company secrets Outlook: Continuous intensive use	Contributions: 12 Impact: Potentially high; explanatory value regarding individual consumer patterns Drawbacks: High effort to collect because the result of time-of-use surveys, subject to privacy or company secrets, difficult to predict, the output of simulations Outlook: Continuous use, could see intensification	Contributions: 2 Impact: Low Drawbacks: Rarely considered as a predictor, not needed for physical input-output relations, demand has low price elasticity, price swings only in liberalized markets, difficult to obtain future values to use in predictions Outlook: Rare use

present the results of the analysis on the techniques and inputs used in the examined articles. They allow the reader to directly identify the compatibility of the data spectrum and methodology. The first two columns contain the techniques and associated advantages and disadvantages. The other columns refer to the input data types. In each cell a short assessment of the respective data-technique combination in the context of energy demand modeling is given in the following categories:

• “Contributions” refers to the number of relevant articles.
• “Impact” describes the importance (high, medium, low) of the data type for the respective technique considering different use-cases.
• “Drawbacks” refers to the weaknesses and limitations of the data-technique combination.

“Outlook” gives a qualitative assessment of the applicability in future research based on the number of contributions and the rating on impact and drawbacks, revealing possible research fields with high or limited potential for application.

All articles relevant for each cell, are documented in detail in the structured reference lists in

Table A4. Techniques and input data used per article (1/4) (thanks to the reviewers’ advice, Table 5, Table 6, Table 7, Table 8 and Table 9 have been designed in a clear and understandable structure, highlighting techniques and applications).

Method	Input	References
ANN	Historic energy demand	[16,21,40,45,48,49,51,58,59,60,68,69,70,72,73,80,82,87,88,91,94,96,97,99,100,101,106,108,110,112,113,114,115,116,117,118,130,131,132,135,136,137,138,145,146,147,148,153,154,159,161,162,164,165,166,167,168,172,173,174,176,184,185,191,229,235,237,244,248,249,252,255,260,261,263,265,267,270,277,278,283,286,288,293,294,325,340,342,346,348,349,352,354,356,360,363,366,373,399,402,405,407,427,428,442,448,453,454,456,470,475,476,477]
	Weather data	[21,22,40,68,69,70,80,82,87,91,96,97,101,106,112,114,115,118,130,132,136,138,145,146,147,148,151,153,154,159,164,165,172,173,174,176,183,191,229,237,244,255,261,263,267,275,325,340,348,349,356,363,402,405,407,427,432,442,448,454,456,470,476,477]
	Calendar data	[48,70,73,88,94,97,99,101,106,110,112,113,114,115,118,136,138,146,147,154,159,162,164,165,173,183,184,185,229,235,237,244,263,293,325,340,342,348,349,354,399,470,477]
	Demographic or economic data	[21,40,49,82,101,133,148,255,258,270,277,348,356]
	Technical system data	[22,68,80,82,87,91,96,101,144,159,269,354,366,402,416,453,457,474]
	Usage or behavioral data	[40,96,151,255,348,432]
	Energy prices	[49,96,148,427]
Instance based	Historic energy demand	[33,34,37,39,68,71,82,89,90,101,107,108,111,131,147,163,175,178,229,236,263,285,331,340,346,349,356,361,365,396,398,425,442,449,478]
	Weather data	[37,39,68,82,89,101,107,147,163,175,183,229,263,340,349,356,396,442,449]
	Calendar data	[39,101,111,147,183,229,236,263,331,340,349,396,425,449,478]
	Demographic or economic data	[34,82,89,101,107,178,258,285,331,356,478]
	Technical system data	[37,68,82,101,107,398,449,457]
	Usage or behavioral data	n/a
	Energy prices	[178,331]

,

Table A5. Techniques and input data used per article (2/4). Continuation of Table A4.

Method	Input	References
Clustering	Historic energy demand	[21,48,49,81,82,91,92,93,109,114,118,155,163,232,234,235,236,263,268,279,322,323,326,328,342,353,355,360,369,372,395,403,428,476]
	Weather data	[21,82,91,92,93,114,118,163,263,275,348,353,476]
	Calendar data	[48,92,93,109,114,118,235,236,263,326,342,348]
	Demographic or economic data	[21,49,81,82,93,232,234,348,395]
	Technical system data	[82,91,93,353,395]
	Usage or behavioral data	[348,395]
	Energy prices	[49]
Ensemble learning	Historic energy demand	[94,95,96,97,100,106,114,120,130,132,133,138,162,168,355,363,407,449,476]
	Weather data	[95,96,97,106,114,130,132,138,363,407,449,476]
	Calendar data	[94,95,97,106,114,120,138,162,449]
	Demographic or economic data	[133]
	Technical system data	[96,144,449]
	Usage or behavioral data	[96]
	Energy prices	[96]
Deep learning	Historic energy demand	[21,72,73,106,153,164,165,166,167,229,248,267,278,293,325,352,366]
	Weather data	[21,106,153,164,165,229,267,325,432]
	Calendar data	[73,106,164,165,229,293,325]
	Demographic or economic data	[21]
	Technical system data	[366]
	Usage or behavioral data	[432]
	Energy prices	n/a
Bayesian algorithms	Historic energy demand	[39,98,99,145,146,164,249,250,405]
	Weather data	[39,98,145,146,164,239,345,405]
	Calendar data	[39,98,146,164,239]
	Demographic or economic data	[99,345]
	Technical system data	[269,345]
	Usage or behavioral data	[98]
	Energy prices	[98]
Decision trees	Historic energy demand	[100,101,228,407,446,449,468]
	Weather data	[101,407,446,449,468]
	Calendar data	[101,228,446,449,468]
	Demographic or economic data	[101,446,468]
	Technical system data	[101,457,468]
	Usage or behavioral data	n/a
	Energy prices	n/a

,

Table A6. Techniques and input data used per article (3/4). Continuation of Table A5.

Method	Input	References
Regression	Historic energy demand	[17,27,30,61,62,63,75,84,85,86,91,96,118,131,133,134,138,140,143,146,148,149,150,152,154,156,157,158,159,160,168,181,189,192,202,230,231,234,239,240,242,243,255,262,263,268,270,273,277,280,282,285,289,292,327,329,331,337,338,340,344,356,357,363,364,367,371,395,397,398,400,401,404,404,407,438,439,440,442,446,447,465,468]
	Weather data	[27,61,62,63,75,84,91,96,118,138,140,141,142,146,148,149,150,154,157,159,160,183,239,242,255,289,292,329,338,340,363,367,394,397,404,407,438,439,440,442,446,447,449,465,468]
	Calendar data	[27,61,62,63,118,138,140,146,150,154,157,159,160,183,230,239,280,292,331,340,400,438,440,446,447,449,468]
	Demographic or economic data	[75,85,133,141,148,157,158,189,234,255,262,270,277,285,292,298,327,330,331,338,351,395,439,440,446,447,465,468]
	Technical system data	[61,63,75,85,96,144,152,158,159,327,329,330,337,338,344,358,394,395,398,400,449,458,468]
	Usage or behavioral data	[63,86,96,142,255,327,330,395,400,401]
	Energy prices	[63,96,148,280,331,439,458]
TSA/ ARCH	Historic energy demand	[17,29,30,39,47,48,59,60,79,81,95,104,105,109,113,120,131,134,136,143,155,156,162,167,178,179,180,182,202,227,231,240,241,243,245,246,247,249,250,251,253,256,259,261,263,264,265,271,272,273,276,281,283,288,290,294,296,297,324,333,339,342,343,346,356,363,367,369,371,373,403,407,414,418,430,443,479]
	Weather data	[39,95,136,143,180,202,243,253,259,261,263,264,290,339,343,356,363,367,407,430]
	Calendar data	[39,48,95,109,113,120,136,162,202,227,231,241,243,245,263,290,342,343,414]
	Demographic or economic data	[29,81,178,240,241,246,273,356,443]
	Technical system data	n/a
	Usage or behavioral data	[339]
	Energy prices	[178,241,246]
Stochastic	Historic energy demand	[20,27,29,30,33,34,40,44,45,64,66,68,79,93,98,133,139,167,176,184,202,203,228,238,243,246,257,276,294,324,414,418]
	Weather data	[27,40,52,68,93,98,139,176,202,203,238,243,334,347]
	Calendar data	[27,44,52,93,98,121,122,184,202,203,228,238,243,347,414]
	Demographic or economic data	[29,34,40,66,93,122,133,246,334,347]
	Technical system data	[20,52,66,68,93,121,334,347]
	Usage or behavioral data	[40,44,52,66,98,121,122,334]
	Energy prices	[98,246]
Fuzzy	Historic energy demand	[16,18,47,48,49,51,59,69,70,75,112,113,114,118,133,135,163,164,192,251,252,270,272,284,285,327,336,349,426,453,470]
	Weather data	[69,70,75,112,114,118,151,163,164,349,470]
	Calendar data	[48,70,112,113,114,118,164,284,349,470]
	Demographic or economic data	[49,75,133,270,285,327,336,475]
	Technical system data	[327,409,453]
	Usage or behavioral data	[151,327]
	Energy prices	[49]

and

Table A7. Techniques and input data used per article (4/4). Continuation of Table A4.

Method	Input	References
Meta- heuristic	Historic energy demand	[34,37,39,40,49,71,72,73,112,116,117,136,150,163,175,189,190,191,192,254,363,407,427,437,446,447]
	Weather data	[22,37,39,40,112,136,150,163,175,191,254,363,407,427,437,446,447]
	Calendar data	[39,73,112,136,150,446,447]
	Demographic or economic data	[34,40,49,189,190,446,447]
	Technical system data	[22,37]
	Usage or behavioral data	[40]
	Energy prices	[49,190,427]
Engineering-based	Historic energy demand	[53,75,78,85,129,186,187,287,291,326,329,341,353,359,368,370,371,374,408,431,469]
	Weather data	[52,53,54,74,75,77,78,187,194,329,332,334,347,353,368,408,450,466,467,469,480,481]
	Calendar data	[52,77,121,122,326,332,347,375]
	Demographic or economic data	[53,75,85,122,226,233,291,295,330,334,347,374,408,431,441,481]
	Technical system data	[19,52,54,74,75,77,85,121,186,187,188,194,274,295,329,330,332,334,341,347,353,374,375,406,409,415,417,429,450,466,469,471,474,480,481,482,483]
	Usage or behavioral data	[19,52,54,121,122,330,334,370,375,450,481,483]
	Energy prices	[295,332]

in Appendix A, which enables the reader to find examples of data-model combinations in recent literature.

4.3. Spatiotemporal Level of Detail

The spatial and temporal properties represent the level of detail applied within the articles and are common decision criteria for model selection since the level of detail of available input data substantially influences the resolution and scale of the output of a model. Figure 8 summarizes the core findings with regard to the applied techniques among the analyzed articles.

The analysis of the temporal horizon (Figure 8, left) reveals an overall higher number of occurrences for short- (≤one day) and long-term (≥one year) projections compared to medium-term. Among the articles, ML techniques are applied more frequently for short temporal horizons while engineering-based and uncertainty techniques are used more often for longer time frames. This is in line with a common view within the scientific literature, whereby engineering-based energy demand models are described as simulation approaches, which are suitable for modeling longer time spans in a realistic and reliable manner [76]. For the other approaches, there seems to be no particular tendency.

The analysis of the temporal resolution (Figure 8, middle) reveals an overall tendency towards hourly time steps or shorter. This could be because hourly time steps represent a reasonable compromise between the level of detail, availability, and data quantity for most projects since an hourly resolution allows to represent most human-driven impacts on consumption, such as daily routines, while the amount of data is still manageable. Additionally, a lot of weather and consumption data is tracked at least in hourly time steps. However, there are differences among the energy carriers: for example, there was no article that modeled natural gas consumption with shorter than hourly time steps. The difference in temporal resolution for electricity and gas most likely reflects system operation requirements and the respective metering infrastructure [125]. Figure 8 also shows that for time steps longer than one day, there is a decrease of ML approaches while statistical approaches are used more frequently.

The investigation on the spatial resolution (Figure 8, right) shows an overall tendency among the analyzed literature of modeling energy consumption on the level of single households and buildings or regional level, e.g., on a district-scale. Comparatively few approaches focus on the level of appliances or national consumption. Engineering-based techniques stand out in having a clear tendency towards the building and appliance level, reflecting the high level of detail which is characteristic of this technique. Scaling an engineering-based model to the national level requires high amounts of detailed technical system data and has only been done in a few cases [126]. In contrast, statistical techniques are used most frequently on large geographic scales, reflecting their traditional role in econometric analyses.

Table A8. Techniques and level of detail per article (1/2). Compiled results of the analysis on techniques, temporal horizon, and spatial resolution. The table follows a matrix structure: all articles referenced within a cell rely on the category of techniques specified.

Method	Temporal Horizon	Spatial Resolution	References
Machine learning	Short	Appliance	[45,107,163,164,166,236,265]
		Building/household	[40,59,87,91,92,98,112,117,120,130,137,154,162,168,173,175,248,279,323,331,342,346,348,355,360,365,366,369,381,387,392,396,398,399,402,403,407,413,419,448,462,477,478]
		Regional	[37,58,60,71,94,97,106,113,116,135,136,140,146,147,161,165,169,174,176,183,229,235,237,244,249,267,275,286,307,315,319,320,356,434,442,452,459,484]
		National	[16,73,118,155,294,311,317,318,428,432]
	Medium	Appliance	[352,373,474]
		Building/household	[101,102,115,132,143,325,340,349,378,388,395,410,421,445,446,454,456,470,476]
		Regional	[33,68,69,80,90,100,110,111,153,171,260,268,278,288,322,411,449]
		National	[95,191,250,261,263]
	Long	Appliance	n/a
		Building/household	[89,114,156,159,310,328,353,354,380]
		Regional	[21,34,48,70,82,96,103,108,131,141,170,234,270,302,305,309,362,384]
		National	[29,49,51,99,138,178,232,255,258,277,283,285,300,306,312,316,436]
Statistical	Short	Appliance	[231,265,344,418,423]
		Building/household	[59,61,62,63,91,120,154,158,162,168,331,335,335,339,342,346,367,369,379,387,393,394,398,403,407]
		Regional	[30,60,79,113,136,140,146,157,183,227,249,264,276,282,290,356,438,440,442]
		National	[118,142,155,182,242,243,292,294,308,318,330]
	Medium	Appliance	[373,401,420]
		Building/household	[17,75,102,143,180,324,337,338,340,343,390,395,446,473]
		Regional	[247,253,259,268,272,288,289,397,479]
		National	[95,245,246,250,251,261,263,444]
	Long	Appliance	[329]
		Building/household	[85,156,159,160,321,327,333,364,382,404,414]
		Regional	[47,48,96,104,105,131,141,149,150,177,234,241,256,270,271,280,281,296,297,301,305,357,391,397]
		National	[29,83,84,133,134,138,178,179,181,189,192,230,240,255,262,273,277,283,285,298,303,312,313,314,351,371,435,436,439,443]

and

Table A9. Techniques and level of detail per article (2/2). Continuation of Table A6.

Method	Temporal Horizon	Spatial Resolution	References
Stochastic/ Fuzzy/ Grey	Short	Appliance	[45,163,164,418]
		Building/household	[20,40,46,52,59,66,98,112,335,392,393]
		Regional	[30,79,113,135,176,276,299,434]
		National	[16,118,243,257,294,317]
	Medium	Appliance	n/a
		Building/household	[64,75,324,349,385,470]
		Regional	[33,68,69,272,411]
		National	[246,251]
	Long	Appliance	n/a
		Building/household	[114,321,327,347,382,414,463]
		Regional	[34,47,48,70,270,321,334,336,362]
		National	[18,29,49,51,121,133,192,284,285,435]
Meta- heuristic	Short	Appliance	[163]
		Building/household	[40,112,117,175,387,407,413,437]
		Regional	[37,71,116,136,320]
		National	[73]
	Medium	Appliance	n/a
		Building/household	[446]
		Regional	[254]
		National	[191]
	Long	Appliance	n/a
		Building/household	n/a
		Regional	[34,150]
		National	[49,189,190,192]
Engineering-based	Short	Appliance	[274,406,415,467,482]
		Building/household	[52,186,332,461]
		Regional	n/a
		National	[330]
	Medium	Appliance	[474,480]
		Building/household	[19,74,75,78,188,466]
		Regional	[129]
		National	[368]
	Long	Appliance	[194,329,375,483]
		Building/household	[54,77,85,341,347,353,463,481]
		Regional	[187,226,233,287,291,295,334,374,441]
		National	[126]

in Appendix A provide a structured list of references by techniques and levels of detail. The tables allow the reader to identify recent articles that share a similar combination of these properties, which enables fellow researchers to quickly find matching articles for their projects.

4.4. Prediction Accuracy

Prediction accuracy has a direct relationship with decision quality [113]. Therefore, the pursuit of performance enhancement and higher levels of accuracy is one of the driving factors of the development of new techniques and combinations among them. Given its importance, researchers might consider the forecasting accuracy as a criterion for model selection. According to Hong and Fan, the most used performance measure in the electric power industry is MAPE, due to its simplicity and transparency [5]. Lewis’ benchmark [127], which has been mentioned by several authors [18,128], suggests, that a MAPE value of 10 % or lower indicates high prediction accuracy.

In several literature reviews MAPE values of different techniques are compared [4,14,23]. Debnath and Mourshed suggest, that ML and hybrid approaches tend to perform more accurate compared to other techniques [4] and Wei et al. found that MAPE values of long-term projections tend to be better than for short-term projections [14]. However, other authors are reluctant to give clear recommendations, stating that the different choices of performance measures make it hard to categorize the methods from best to worst [4] and that the suitability of models finally depend on the dataset [23].

MAPE was the most frequently used accuracy measure among articles and was provided in 217 of 419 articles. Other measures like root mean square error (RMSE), normalized RMSE (nRMSE), mean absolute error (MAE), and coefficient of determination (R²) were encountered in several cases as well. Figure 9 shows the histogram of the MAPE values over-collected from all articles. The shape of the distribution as well as the values for skew (3.99) and kurtosis (23.07) give a strong indication that the values are not normally distributed. The same assumption can be derived from the histograms of the grouped MAPE values, e.g., by technique since they show similar characteristics.

Figure 10 shows that a direct comparison does not reveal a universal higher level of accuracy for any technique used among the articles. While engineering-based methods have a slightly higher median (green line), the means (red diamonds) are almost equal among all techniques. This could be due to the great variety of available methods within each category, allowing users to find techniques that are tailored to their use-cases. Furthermore, for every technique, particular measures and sub-routines have been developed to counter drawbacks and increase accuracy (see Section 4.5). An analysis of the MAPE values of techniques grouped by hybrid and stand-alone approaches yielded a similar result.

Figure 11 gives an indication, that different levels of spatial resolution could influence the accuracy of prediction. A higher level of detail seems to result in higher MAPE values, especially for articles in which loads of individual appliances were predicted showed. A possible interpretation could be, that aggregated loads on the level of countries or regions are easier to predict since they are smoother and more likely to show seasonal or trend-related patterns compared to loads of individual appliances. Disaggregated loads depend on the behavior of individual users having a higher degree of randomness, which naturally produces lower degrees of accuracy in their prediction. Similar analyses for articles grouped by the temporal horizon and the temporal resolution were conducted but did not show interpretable results (see Figure A1 and Figure A2 in the Appendix A).

4.5. Measures for Improvement of Accuracy

Techniques have become more flexible over the years to be adapted to the specific contexts and datasets in which they are used and sub-routines have been developed to counter drawbacks and improve predictive performance.

For ML techniques, the following measures to improve predictive performance have been found in the analyzed articles. To reduce overfitting, ensemble learning was employed to create independent predictions of multiple models and to use weighted averaged results [59,95,96,97,99,113,115,129,130,131,132,133,134]. Other measures against overfitting include the usage of incremental learning and dynamic neural networks, where the models are updated step by step during training phase [88,106,131,135] or restrictions on coefficients are implemented [136] as well as the introduction of dropout layers [137]. The adjustments of coefficients of the predicting variables in order to capture the essential properties of the training data and provide better generalization to yet unknown data points, is an important and widespread concept to avoid overfitting known as regularization. The idea is to use a regression technique to shrink or regularize the estimated coefficients, effectively discouraging the learning of a complex model and hence reducing risk of overfitting. The most common procedures are the least absolute shrinkage and selection operator (LASSO) [138,139,140,141,142] and the ridge regression [96,102,143,144]. Bayesian regularization was used in [145,146].

The curse of dimensionality [147] is a common problem, which occurs when dealing with high dimensional datasets. Elimination of insignificant features and appropriate selection is crucial for ML and regression techniques. This was achieved by using correlation and principal component analysis (PCA) in [61,107,148,149,150,151,152,153,154,155,156,157,158,159,160,161]. For ML approaches, feature selection was also done by genetic algorithms and decision trees in [39,40,100,111] and fuzzy based feature selection in [162,163,164]. To boost accuracy, deep learning techniques such as stacked auto encoders [21,165,166,167] or long short-term memory (LSTM) networks [153,168,169,170,171] are used. Another measure to increase accuracy for ANN is to vary the number of neurons and layers, as shown in [106,159,172,173].

Another popular measure is the pre-processing of data in order to eliminate outliers and noise, as well as isolate seasonal [113] or temperature related [174] patterns. For TSA and ML techniques this has been done by using wavelet or Fourier transformation in [39,58,90,108,111,120,175,176,177,178,179,180]. Particular attention was payed to the prediction of special events and holidays in [118,181,182,183].

Sometimes model performance suffers from the lack of data. In the case of ML approaches this can be compensated by the creation of virtual data through densification or latent information functions [184,185]. In engineering-based techniques insufficient data can be tackled by prioritization as well as the right choice of representative samples as done in [19,54,186,187,188].

ML approaches sometimes suffer from ending up in local shallow minima when optimizing parameters. The solutions can involve alternative (metaheuristic) optimization routines during training stage, such as the artificial bee colony algorithm for ANNs [116,189], the Cuckoo search [71] or the wolf pack algorithm [163] for SVMs.

4.6. Summary of Results

The following chapter provides a brief summary of advantages and disadvantages as well as countermeasures to cope with the drawbacks by category of techniques derived from the analyzed literature.

Table 9. Summary of advantages and disadvantages as well as countermeasures to compensate drawbacks.

Technique	Advantages	Disadvantages	Countermeasures
ML	High predictive performance; Relatively low implementation effort; Able to handle nonlinear relations; Many pre-set model configurations are available; Can be used without deeper knowledge of technical system	Black box character; Risk of overfitting; Course of dimensionality; Risk of getting stuck in shallow local minima	Regularization; Ensemble learning; Appropriate feature selection; Variation of input layers and neurons; Usage of metaheuristic optimization during the training stage
Statistical	Low implementation effort for basic models; White box character, revealing relations between independent and dependent variables; Especially TSA can be used with relatively low data requirements	Limitations when independent variables are correlated; Difficulties predicting extreme events and outliers; Slight risk of overfitting	Pre-processing of data, e.g., by transformation and decomposition; Variable selection using PCA; Coefficient adjustments using regularization
Stochastic/Fuzzy/Grey	Appropriately addresses uncertainty about inputs allowing to estimate expected outputs and output variations by using quantiles, intervals, or density functions as representations; Able to deal with incomplete/inaccurate data; Able to simulate energy demand based on stochastic processes, providing generated data as inputs for other models	Can be considered unsatisfying for decision-makers since model outputs are afflicted with probabilistic or fuzzy expressions; Long computing times for repeated simulation of stochastic processes	Variable elimination algorithms; Usage of evaluative labels on model outputs to make the uncertainty more understandable (e.g., uncertainty is high or low)
Metaheuristic	Provide alternative and promising methods to solve optimization problems and efficiently search the solution space to find global optima; can be applied to different types of problems; a high number of easy to implement algorithms in place; Can be incorporated into other models;	Requires additional knowledge and effort to implement in existing models; not unrestrictedly reliable in finding the optimal solution Can have low convergence rates and be time-consuming	Usage of existing and proven model combinations
Engineering-based	White-box character, revealing detailed input-output relations based on laws of physics; Able to simulate energy demand for explorative and normative scenarios including disruptions that have no historic record	Data and knowledge-intensive for a description of the technical system; prediction accuracy can be low due to simplifications regarding the system	Prioritization of datasets and choice of representative samples; Use of publicly available datasets for aggregated consumers

contains the most important elements.

ML techniques are used the most across the articles and showed an increase in usage over the last years. ML has the advantages of being able to handle nonlinear relations and achieve high levels of accuracy with quite low implementation effort [22]. Drawbacks lie in the black-box character [190], the tendency of overfitting and getting stuck at shallow local minima [116,191]. Countermeasures are the use of regularization procedures, the formation of model ensembles as well as feature selection and data pre-processing by decomposition. ML techniques dominate across all temporal and spatial levels, however with a slight tendency towards smaller time steps, horizons and scales.

Statistical modelling techniques have a long history in econometrics and are common in energy demand modelling, too. They are the second most used technique. Multiple linear regression is fast and simple to use and capable of explaining the relationship between independent and dependent variables. However, when independent variables are correlated, these models face difficulties [148]. One way of counteracting is by refining the variable selection process and reducing complexity with the help of PCA or coefficient shrinking with LASSO. TSA techniques are easy to use and efficient in modelling overall trends and seasonal patterns. Limits occur when it comes to forecasting extreme events or outliers. Countermeasures can be found in the transformation and decomposition of data [104].

Techniques for stochastic, fuzzy and grey systems modelling address different types of uncertainty. As discussed by Hong and Fan [5], having elements of uncertainty included in model outputs can be considered to be unsatisfying for decision makers in management positions who expect single point values. However, this branch of energy demand modelling can still be considered as a recent development, which will likely see an increase in popularity over the next years. Fuzzy and grey approaches are able to deal with incomplete or inaccurate data [69,192]. Stochastic simulations might run into long computing times, which can be countered by variable elimination algorithms [98].

Meta-heuristic approaches are mainly used as part of combined models, e.g., by introducing genetic feature selection algorithms [40,112] or by optimizing model parameters through an evolutionary process [72,73,189]. However, the integration of a metaheuristic algorithm into another technique requires additional effort and can have low convergence rates that has to be justified by improved results [193].

Engineering-based techniques derive energy demand from a bottom-up representation, which involves a detailed representation of input-output-relationships based on the laws of physics. This level of detail represents a fundamental difference to other techniques. Engineering-based approaches are commonly used in the context of building simulation [57,194]. However, the requirement of large amounts of parameters as well as the accurate representation of input-output relationships make the initial set-up of such models rather laborious. Once created, however, these models have the potential to predict different scenarios on a long temporal horizon, which makes them particularly relevant in the context of system planning. Furthermore, the forecasting based on historic data, as done by ML and TSA techniques, is unable to depict structural disruptions, such as the consequences of political interventions, technological breakthroughs or a pandemic.

5. Discussion

Compared to existing literature reviews on energy demand modelling presented in Section 1 ([4,9,10,11,12,13,14]) and Table A1, this study covers all sectors, energy carriers, categories of techniques, input data types, spatio-temporal characteristics, accuracy as well as advantages, disadvantages and typical countermeasures. A recent publication in this journal by Mosavi et al. [195] focuses on the application of ML models in the energy system comprising separate analyses of multiple studies including accuracy values on different scales. In comparison, the study at hand presents a more detailed analysis of the demand sector, including a wider range of techniques and a more extensive and structured literature base, ensuring comparability of the properties of different articles.

Most articles focus on electricity consumption, which is not surprising since it is the most valuable and expensive form of energy. Moreover, due to service level requirements of grid infrastructure and expansion of smart meters, vast amounts of detailed and high-quality data is available in this sector [5]. However, the authors expect an increasing number of smart metering devices also in the gas and heat grids, which will improve data availability in the future and facilitate integrated energy system planning and modelling [196,197].

Studies with a focus on buildings represent a significant overall share among analyzed articles and are particularly relevant in the context of modelling heating and cooling demand in the residential and commercial sector. Compared to relevant analyses on the building sector assessing efficiency of wood-based constructions [198], their energy consumption during use [199] or the potential of zero energy buildings [200,201], the review at hand covers all energy demand sectors up to a national scale.

The industry sector is underrepresented in the available articles. Considering the intense efforts regarding efficiency targets and demand side management potentials, there is a large interest in modelling the industry sector including the adoption of new technologies, as explained by Fleiter et al. [76]. The small number of articles can be explained by a lack of publicly available data as well as the reluctance to publish this kind of research; energy consumption of a company is often considered sensitive data since it implicitly contains information about production activity and efficiency [125]. The results of the review at hand indicate that the challenges pointed out by Fleiter et al. regarding data availability and transparency in the industrial sector persist [76].

Accuracy of prediction is one of the most important factors in decision making, not only to enable the right choice of models but also to allow stakeholders to understand the performance of the employed method. Unlike the findings of other authors ([4,14]), a tendency of higher accuracy for ML and hybrid techniques or for longer temporal horizons cannot be confirmed for the analyzed articles. It was shown that among the examined literature models with higher spatial detail have a tendency towards lower accuracy. This shows that comparing MAPE values of techniques irrespective of the individual context in which they are applied is of limited explanatory value. In order to robustly compare and discuss accuracy of different techniques they have to be applied to the same datasets, as done in forecasting competitions, such as GEFCom [27,139,202,203,204].

6. Challenges and Future Research Directions

In the context of the energy transition, the focus of planning and decision-making processes is expanding across infrastructures and sectors. Accordingly, the level of detail and complexity of energy systems and energy demand models is increasing. Looking at the results of the analysis, the research focus is on sector-unspecific electricity demand, i.e., without a focus on a particular consumer group. Given the importance of carbon-free energy carriers, a continuous high output of articles on electricity demand is expected but should be backed up by intensifying research on modeling heat and hydrogen demand. Compared to the residential and commercial sectors, the industrial sector has been less frequent in the focus. Given the significant decarbonization challenges this sector faces, intense research efforts are needed and should lead to an increasing number of publicly available studies.

ML-techniques, more precisely ANN, and statistical approaches are the predominant methods and historic energy demand, weather data, and calendar information are the most frequently applied inputs. Based on the analysis of data-technique combinations (Table 4, Table 5, Table 6, Table 7 and Table 8), continuous intensive use of data-driven ML-techniques can be expected, especially in ensembles and combined models integrating key strengths of other models such as Fuzzy expressions or metaheuristic optimization algorithms, using historic energy demand and publicly available input data. At the same time, given the high explanatory value of the technical system and appliance usage data, the authors see a high potential for an intensifying application of these inputs. This development is supported by the expansion of the sensor and metering sector, which will further increase the availability and quality of data, especially on the level of buildings and appliances. In addition, this level of detail is needed in order to accurately model demand flexibility options and their technical and economic constraints in the different sectors. This also enables a broader application of detailed engineering-based models, which are particularly suited for representing the input-output relations.

Future work should make use of the knowledge base provided by this literature review, inspiring hypothesis-driven analyses and quantitative testing, focusing on the applicability and dominance of specific data-method combinations for energy demand modeling.

7. Conclusions

A comprehensive and up-to-date systematic literature review about energy demand modeling regarding techniques, data, accuracy, energy carriers, sectors, and spatio-temporal level of detail was presented. 419 articles published between 2015 and 2020 were reviewed. References are structured by property and compiled in tables for easy access.

The analysis has shown that energy demand modeling is a research field with continuously high numbers of yearly publications. The analysis of the articles proved the current trend of increasing popularity for ML approaches. Statistical models such as regression and TSA are well established, whereas stochastic/fuzzy/grey and metaheuristic techniques often are used as part of combined approaches. Engineering-based techniques stand out as they provide a more detailed representation of the physical properties of the energy-consuming system resulting in higher external and internal data requirements. A research gap was identified regarding models for industrial energy demand.

The level of accuracy proves to be a difficult criterion for a ranking of techniques since attainable performance depends on the particular context. Among the articles, a higher level of detail, e.g., forecasting on the level of appliances, produced lower levels of accuracy compared to forecasts of aggregated loads on country levels.

The material presented here shows trends with regard to the prevailing combinations of methods and data and allows these trends to be tested on the basis of quantitative methods in the future. In particular, this review provides the basis for further analyses and quantitative testing of hypotheses regarding the applicability and dominance of specific methods for sub-categories of demand modeling.