Next Article in Journal
Maximizing Total Profit of Thermal Generation Units in Competitive Electric Market by Using a Proposed Particle Swarm Optimization
Next Article in Special Issue
The Effects of Knowledge Assets and Path Dependence in Innovations on Firm Value in the Korean Semiconductor Industry
Previous Article in Journal
Simultaneous Removal of Residual Sulfate and Heavy Metals from Spent Electrolyte of Lead-Acid Battery after Precipitation and Carbonation
Previous Article in Special Issue
Big Data Analytics in Government: Improving Decision Making for R&D Investment in Korean SMEs
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

The History and Evolution: A Big Data Analysis of the National Innovation Systems in South Korea

1
Data Analysis Division, South Korea Institute of Science and Technology Information, 66 Hoegi-ro, Dongdaemun-gu, Seoul 02456, Korea
2
Technology Commercialization Center, South Korea Institute of Science and Technology Information, 66 Hoegi-ro, Dongdaemun-gu, Seoul 02456, Korea
*
Author to whom correspondence should be addressed.
Sustainability 2020, 12(3), 1266; https://0-doi-org.brum.beds.ac.uk/10.3390/su12031266
Submission received: 17 January 2020 / Revised: 5 February 2020 / Accepted: 6 February 2020 / Published: 10 February 2020
(This article belongs to the Special Issue Big Data for Sustainable Anticipatory Computing)

Abstract

:
This study is a starting point to analyze South Korean national innovation systems (KNIS) using big data and provide insights for policy makers regarding how they implement the dynamic process of innovation systems. It examines KNIS that has developed over the past 14 years from 2003 to 2016 during the governments of Roh Moo-hyun, Lee Myung-bak, and Park Geun-hye. The aim of this study is to evaluate the KNIS in three ways. The first way is to analyze the NIS of the three governments based on data of 470,000 national research and development (R&D) projects, following which the second way is to compare innovative outcomes of the three governments. The last way is to figure out the characteristics of the KNIS in innovative performance. Our analysis reveals that the KNIS was developed and evolved from 2003 to 2008, maintained until 2012, and gradually declined, even though national R&D investment increased for 14 years. Empirical evidence highlights that policies implemented for more than a decade do not effectively link to economic outcomes, resulting in an imbalance between innovation input and innovation output. This study further argues that the use of NIS concept in South Korea seems to be skewed towards measuring national performance from a narrower perspective.

1. Introduction

This study is consistent with the views of Freeman (1982, 1987) [1,2] at the beginning of discussions about national innovation systems (NISs). Freeman (1987) [2] questioned how Japan, which depended on imitation and imports of foreign technology in the 1950s and 1960s, was able to achieve economic growth throughout the country with technological growth that surpassed that of the US and Europe in the 1980s. How could Japan grow more quickly than the US and Europe? Several scholars, such as Freeman (1987) [2], Lundvall (1992) [3], and Nelson (1993) [4], carried out comparative studies, particularly of Japan and its institutional differences pursued over many decades, while Freeman (1987) [2] developed the concept of a ‘system of innovation’.
Since the 2000s, South Korea has been making efforts to build the NIS to achieve technological innovation and thereby increase economic growth at the national level. The Roh Moo-hyun government (Roh government, 2003–2008) was established the South Korean national innovation system (KNIS), with the aim of achieving a gross national income (GNI) of USD 20,000. As the literature on NIS [1,3,4] outlined, the Roh government considered that NIS to be a core framework and foundation of the long-term economic development of a nation. The target was achieved in 2006, with a GNI of USD 20,823 [5]. Thereafter, the concept of the KNIS emerged as a tool for achieving national goals that were adjusted and reconstructed by the Lee Myung-bak government (Lee government, 2008–2013) and the Park Geun-hye government (Park government, 2013–2017).
Korea is currently facing an economic crisis. Gross domestic product (GDP) volatility, which shows the extent of economic growth, has shrunk by half compared with that of other major countries. GDP volatility is the standard deviation of the quarterly average growth rate. The result of dividing the volatility of GDP between 2010 and 2017 by the volatility between 2000 and 2007 shows the average for 35 OECD countries of 0.9, compared with that for South Korea of 0.5 [6]. However, in the same period, the government steadily increased research and development (R&D) investment from USD 3.45 billion in 2000 to USD 17 billion in 2017, expecting that more inputs might provide more opportunities for economic growth. For the period that the South Korean government adopted the concept of NIS, this study attempts to determine whether the investments of that time led to noticeable outcomes by analyzing the KNIS used from time of the Roh government, which first constructed NIS as a policy framework, to the recently completed term of the Park government.
This study is different from previous studies. First, it attempts to analyze the KNIS between 2003 and 2016, meaning when the KNIS emerged and how it was developed under three changes of government, rather than to provide simple characteristics of the KNIS at a specific point in time. NIS scholars have undertaken little longitudinal-based research to understand the evolution of NIS in a specific setting. Second, this study adopted a dataset of 470,000 national R&D projects of the National Science & Technology Information Service (NTIS) accumulated over 14 years, while previous studies have used survey results conducted by international organizations, such as the International Institute for Management Development (IMD) and the Organization for Economic Co-operation and Development (OECD). Along with background analysis of the KNIS, which includes content analysis of government policy documents, analysis of a large amount of data would support the findings of this study.
The rest of this paper is organized as follows. In Section 2, we examine the concept of NIS, including its theoretical background. In Section 3, we present the historical background of KNIS, from when it was established by the Roh government in 2003, to its most recent completion term in 2016 under the Park government. Section 4 describes the data and methods used in this paper to empirically analyze the development of KNIS. Section 5 presents the results of the analysis and examines the patterns of KNIS. Section 6 discusses policy issues associated with implementing the KNIS. Section 7 summarizes the results and limitations of this paper, and presents future research directions.

2. Background

2.1. Theoretical Considerations: Key Concepts

Since the NIS concept emerged in the mid-1980s, it has been used in designing and measuring innovative performance in many countries. NIS can be viewed as an institutional effort to recognize the state as an organic system and effectively promote innovation for the economic development of a country. Freeman (1987: 1) [2] who presented the concept of NIS in his book on Japan, argued that it is ‘the institutional network between the public and private sector organizations to carry out related activities and interactions to create, introduce, revise, and advance new technology.’ Freeman analyzed Japan’s rapid economic growth in the 1960s and 1980s as a result of policies in the Japanese system in which institutions, such as the Japanese Ministry of International Trade and Industry played a significant role in facilitating innovation and connecting it to national performance.
Although there is no single definition of NIS, Lundvall (1988, 1992) [3,7] defined NIS as: ‘…constituted by elements and relationships which interact in the production, diffusion and use of new, and economically useful knowledge … a national system encompasses elements and relationships, either located within or rooted inside the borders of a national state.’ In a similar vein, Metcalfe (1995) [8] defined NIS as a system of interconnected institutions that jointly and individually contribute to the development of new technologies and these institutions provided the framework within which government forms and implements policies.
These scholars adopted a broad perspective where NIS not only included institutions relating to exploring and searching processes, but also subsystems-training systems, labor systems, education systems and production and financial systems, all of which also have an impact on learning activities [9,10]. Based on this point, NIS is a system of all organizations and their relationships in a country that affects the creation, diffusion, and utilization of knowledge, which evolves through cultural factors, customs, and social influences [2,3,7].
Nelson instead viewed NIS as a national R&D system focusing on patent systems, government policies, and universities that affect technology innovation. In this regard, Nelson (1993) [4] conducted an empirical study which analyzed and compared the NIS of 15 countries and identified how innovators or organizations created different innovative outcomes. Although the NIS concept was differently defined by scholars, it can still be regarded as an analytical tool to explain the competitiveness of a nation [11]. Interactions and linkages are key factors showing that growth in interactions leads to improved innovative performance. Hence, innovation systems may grow through complementary interactions at three levels: first, firms and knowledge institutions; second, interactions among different markets; and third, interactions between market and nonmarket mechanisms [12,13,14].
In line with the above, the OECD (1997) [15] started to use NIS as an institutional framework to measure national innovation performance through the Science, Technology, and Industry (STI) Scoreboard of Indicators [16,17]. OECD (1997) [15] encouraged member nations to use the concept as it could provide better knowledge of differences among member nations, and it soon became a popular analytical tool for policy makers in many countries [3,18].

2.2. Analysis of the NIS

Since its appearance as a new conceptual framework, the NIS has drawn the attention of academics and policy makers for more than two decades. Scholars put effort into theory building, undertaking a major comparative study and case studies covering various nations. Policy makers have widely used the NIS concept as an analytical tool, as it further carries the potential to provide an effective mechanism that shapes the system, instead of viewing the system as a market with anonymous players [19,20,21].
First, in their case study, Arocena and Suzt (2000) [22] analyzed the innovation system of Latin America through innovation industrial surveys held between 1995 to 1998 in Argentina, Chile, Colombia, Mexico, Venezuela, and Uruguay. The study highlights that Latin America’s NIS is weak because dominant cultural patterns in Latin America underestimate the impact of knowledge and innovation. Next, Intarakumnerd et al. (2002) [23] conducted a case study to understand NIS in Thailand based on the R&D / Innovation Survey 2000 conducted by Thailand’s National Agency for Science and Technology Development Agency, Thailand was recognized as a failure case in establishing an NIS due to a mismatch between the economic structural change and the development level required for NIS. Although the case study in Thailand uses more diverse indicators than that of Latin America in explaining the NIS, these two macro-level case studies seem to have limits in being able to reveal ‘what is happening’ when implementing the NIS in different countries. Marxt and Brunner (2013) [24] conducted a case study of the Swiss innovation system. Through in-depth interviews and workshops with experts, they analyzed why Switzerland became a leader in innovation in Europe and presented nine suggestions for further development of the NIS. However, the study only shows how the system has emerged and developed in Switzerland by focusing on the analysis of current aspects of the NIS.
Unlike previous studies that adopted qualitative research methods and examined the NIS at a specific point in time, Chen et al. (2011) [25] analyzed the effect of NIS on R&D efficiency from 1998 to 2005 through a panel dataset of 24 countries. Intellectual property rights protection, knowledge stock, and human capital accumulation were used as useful indicators for measuring the NIS. Although it is meaningful that the NIS of 24 countries is compared and analyzed for a long period of time, NIS tends to be simply measured and understood through limited indicators such as patents, royalties or licensing fees, and journal publications, with no consideration for national characteristics of the countries. In this regard, Balzat and Hanusch (2004) [18] argued that lack or weak development of indicators could be further developed to explain specifics of the NIS (i.e., low income countries, highly industrialized countries, or newly industrialized countries).
Guan and Chen (2012) [26] criticized the NIS because research on NIS tended to remain at the theoretical level and further examined the different knowledge innovation processes of 22 OECD member countries at system level. The study of Guan and Chen (2012) [26] is of great significance because it compares and demonstrates the NIS by utilizing the eight indicators extracted from the publications of OECD and the World Competitiveness Report of the IMD, while the dataset of 22 countries is inconsistent in terms of the surveying time for each country. In addition, Chaminade et al. (2012) [27] conducted an empirical analysis of the NIS in Thailand. Indicators adopted by OECD STI Scoreboard of Indicators were used along with a dataset from the years 1999, 2001, and 2003. The scholars found a mismatch between the innovation policy and systemic problems in the Thai NIS through the advanced innovation survey conducted right after the major change of the Thai innovation system, and suggested the survey analysis should be reflected when establishing the Thai NIS.
In South Korea, the South Korea Institute of S&T Evaluation and Planning (KISTEP) developed the COmposite Science and Technology Innovation Index (COSTII) using the NIS framework [28]. Aiming at a comprehensive review of science and technology and measuring the KNIS, COSTII adopted the OECD STI Scoreboard of Indicators [29], which are useful not only to compare the KNIS with the rest of the OECD countries but also for analyzing the strengths and weakness of the KNIS. Since COSTII was introduced in 2006, it has been a popular tool among policymakers to analyze and understand the KNIS based on input-activities and outcomes. The COSTII index, which is published every year, provides national differences among OECD countries with a brief explanation. For example, in 2016, the outcomes of this metric have been ranked as low at 9th place among OECD countries, which contrasts with the high rank of R&D investment (2nd).
As described above, policy makers tend to adopt the NIS as an analytical tool for measuring national competitiveness by using indicators such as GDP, R&D investment, and patents, and this phenomenon reflect innovation processes that are considered as aspects connected to the formal process of R&D [30].

3. Historical Background of KNIS

Since the OECD (1997) [15] discussed the NIS concept as an analytical device, member countries have actively developed innovation policies based on the NIS approach, including South Korea. In 1996, South Korea joined the OECD, and the overarching framework for science and technology policies was gradually formulated. In 1998, the concept of NIS was widely introduced by government-funded research institutes [31]. In 2001, the Framework Act on Science and Technology was enacted to pursue the development of the national economy and contribute to the improvement of the quality of life of people, through innovation in science and technology, with the government having a legal obligation to construct the KNIS. In addition, a basic plan for national science and technology was established and promoted based on the KNIS [32].
After the legal basis of the KNIS, the Roh government, which was inaugurated in 2003, envisioned the KNIS as ‘building a science-based society,’ a new paradigm to strengthen growth potential while overcoming the limits of the existing input-driven economy. The Roh government first adopted the perspective of an innovation system to build a new framework for the science and technology policy, which remained as a subpolicy of economic policy. While emphasizing the interactions with the actors involved in innovation systems, the Roh government established policies from developing national strategic science and technology to supporting technology commercialization for Small and Medium-sized Enterprises (SMEs), as shown in Figure 1 [33].
The Lee government inaugurated in 2009 envisioned the KNIS as a ‘leap forward to becoming an advanced country (rich people, warm society, strong nation [34].’ Unlike the Roh government, which viewed science and technology from the perspective of integration with a society, the Lee government under the principle of being ‘small government, big-market (business friendly),’ established the KNIS as being centered on the private sector. It was desirable for the government to play a limited role in remedying market failures [34]. In addition, the 577 Initiative was advocated to systematically promote science and technology policy. The main goal of the 577 Initiative was that the total R&D investment of the country reached 5% of GDP, and that competitiveness of science and technology was improved.
The Park government inaugurated in 2013 established a national vision of ‘opening a new era of hope with creative science and technology,’ which envisioned an KNIS. Unlike the Lee government, which had a passive approach to the KNIS, the Park government advanced the concept of an innovation system and aimed to build an innovative ecosystem. In other words, whereas previous governments had taken the approach of developing and growing the components of innovation systems, the Park government took the approach of activating the organic integration of innovation systems. The definition of NIS mentioned in Section 2.1 makes it clear that the Park government emphasized the interactions and linkages between components. This resulted from the diversification in the expectations of innovative outcomes based on the complexity of economic and social issues that determine innovative environments and the growing maturity of innovative actors. Ultimately, an innovation ecosystem is one where the interactions between components bring forth a virtuous cycle of innovation, and where R&D investment is efficiently aligned with outcomes such as technology transfers, commercialization, and job creation. Five strategic areas were selected for the purpose of establishing the ecosystem. The main strategy was to expand R&D investment, to develop national strategic technology, to strengthen mid- to long-term creative capabilities, to support new industries, and to create jobs [35].
The KNIS has been shaped in accordance to the national vision, which differs from each government as shown in Figure 1. The most recent assessment of COSTII, as shown in Section 2, displayed the science and technology innovation capacity of 30 OECD countries in 2016. South Korea was ranked 5th out of the OECD countries. Under the Roh government, in 2007, South Korea was ranked 12th among OECD countries and this ranking had been maintained until the first term of the Lee government. During the Lee government, the ranking continued to rise to 9th place and South Korea finally established itself as the top five among 30 OECD countries in the Park government.
With regard to the R&D investment displayed in Table 1, since 2006, when the COSTII was first developed and measured, the ranking continued to increase throughout the Roh and Lee governments. After ranking second highest worldwide in 2013, the ranking was maintained through the Park government. The economic outcome ranking, which differs from the R&D investment ranking, stayed constant at around 10th place during the Roh government, but reached 6th place for the first time during the Lee government. This ranking dropped to 7th place during the Park government, staying constant until 2017 when it returned to 6th place again. As of 2017, however, the rankings for R&D investment and economic outcomes still show a large gap compared with those of the leading country. When observing the relative levels of R&D investment and economic outcome in relation with the top-ranking country by setting their levels at 100%, South Korea’s R&D investment was 93% and the country’s economic outcome was 44.9% in 2017 [29]. Similarly, as mentioned in Section 2.2, IMD research findings point out the huge gap between R&D investment and outcomes. The total expenditure on R&D as a percentage of the GDP rank increased from 6th to 5th place during the Roh government, and jumped to 3rd place during the Lee government. Moreover, during the Park government, it alternated between 1st and 2nd place. Competitiveness, by contrast, rose and fell around 30th place during the Roh government, managed to rise to 22nd place during the Lee government, but fell to 29th during the Park government (Table 2) [36].
In sum, the KNIS has achieved remarkable progress in recent decades in terms of quantitative growth of innovative performance. However, several scholars stated that it has also caused a number of mismatches in the KNIS and some weaknesses that may make the KNIS inefficient include the prevalence of uncompetitive SMEs [37], and underdeveloped linkages for the use of knowledge stock among actors [38].

4. Data and Analytical Method

This study aims to examine the changes in the KNIS from the Roh government, which first constructed the policy framework based on the perspective of an innovation system, to the recently completed term of the Park government. For the study to capture how KNIS has evolved for 14 years at macro-levels, we used the dataset of national R&D projects extracted from NTIS.
The point of why we utilized the data is that the KNIS can be explained by input investment and output performance, as shown in Figure 1. In the preceding section, the KNIS was built based on the Framework Act on Science and Technology. Under the Framework Act, the basic plan for national science and technology was established in accordance with the national vision of each government, and in order to promote the plan, the government had constructed the KNIS that would lead to innovative performance by investing national R&D projects. The critical factor representing the KNIS is how much was invested on national R&D projects and how much innovative performance was created.
The other point is that the data allow a sufficient historical character for analysis to identify the changes in the KNIS for 14 years. As previously mentioned, we obtained a dataset of national R&D projects from the Roh government inaugurated in 2003 to the end of the Park government in 2017. NTIS is the first science and technology R&D information service that integrates and provides national R&D projects information. NTIS widely contains information about 540,000 national R&D projects, such as government investment on R&D, the number of projects, and innovative performance.
As shown in Figure 2, we measure the government investment on R&D, the number of national R&D projects for each government, and innovative performance based on the dataset. Indicators for innovative performance which have been defined as outcomes in the KNIS include the number of papers published, the number of patent applications and grants, technology transfers, royalty income from the technology transfers, the number of commercialized projects, sales from the commercialized projects, and the number of jobs created by the commercialized projects. As innovative performance occurs within two years from the start of a national R&D project on average, the latest available data of 470,000 R&D projects are adopted and classified that cover comprehensive information at the start of each project: the Roh government from 2003 to 2007, the Lee government from 2008 to 2012, and the Park government from 2013 to 2016.

5. Analysis and Results

5.1. Inputs in KNIS

The previous sections described research methods adopted for this study. Section 5.1 provides the historical context of innovation systems in South Korea while presenting the data analyzed from NTIS. This includes how much each government invested in R&D in terms of public-private expenditure, number of R&D projects, and average government investment per R&D project. Figure 3 shows the amount invested in national R&D projects for each year from the Roh government to the Park government. Over the course of the three governments from 2003 to 2016, R&D investment in South Korea steadily increased, with the exception of 2016.
According to the results of R&D investment by each government, the Compound Annual Growth Rate (CAGR) of R&D investment was the highest at 14.47% between 2003 and 2007 at the time of the Roh government, compared to 9.50% for the Lee government and 4.22% for the Park government. It seems that the Roh government had originally planned and built the KNIS towards developing a science-based society, followed by designing policies and programs. In a similar vein, the Roh, Lee, and Park governments made an effort towards enhancing national competitiveness by increasing R&D investment.
In particular, the Roh government recognized the need for R&D investment to achieve technological innovation at the level of advanced OECD countries and thus rapidly increased its investments. In terms of aligning various actors in organizing the KNIS, R&D projects began to involve not only the government sector but also the private sector, and the collaboration between two sectors has been steadily increasing, except for during the final year of the Park government. It was observed that there are various actors that the government intends to encourage through the NIS construction process to interact with each other to carry out R&D projects. The government investment and private expenditure decreased at the end of the term of the Park government in 2016 and the trend has not been recovered yet.
Figure 4 indicates the number of R&D projects carried out by each government; it has been steadily increasing since the Roh government. Among all governments, the number of R&D projects showed the highest rate of increase in the Lee government. The first year of the Lee government, the number of R&D projects increased by about 4000 compared to the previous year and the last year of the Lee government, the number of R&D projects increased by 8000 compared with the previous year. Even though the number of R&D projects was increasing in the Park government, the increase rate is significantly lower than that of the other two governments.
Figure 5 compares the number of R&D projects with that of the government investment in R&D. The government investment for each project surpassed KRW 300 million in Lee’s presidency. However, the average government investment for each project decreased in the last year of the Lee government because the total number of R&D projects shown in Figure 4 increased sharply. In the case of Park government, the average government investment for each project was again returned in KRW 300 million. Compared to the government investment shown in Figure 3, the increase of the government investment had gradually been decreasing, while the number of R&D projects had been increasing across all governments. This confirms Joo’s (2015) [39] assertion that the South Korean government was committed to allocating government investment to R&D for as many projects as possible. As a result, 80% of R&D projects remained at a funding low level of KRW 50 million [40]. It is logical to assume that new R&D projects to achieve different national visions designed by each government continued to be created, which may have led to a negative effect on R&D efficiency.

5.2. Outputs in KNIS

This section presents innovative outcomes based on government investment: how many papers published, patents applied or granted, technology transfers and commercialized projects generated, and jobs from the commercialized projects created, as well as how much royalty income from the technology transfers and sales from the commercialized projects was obtained.
The number of papers published in journals based on government investment is shown in Figure 6 and Table 3. At the time of the Roh government, the number of papers published showed a rapid increase both in South Korea and overseas. In order to construct the KNIS, the Roh government figured out the problems of the existing South Korean science and technology administration, following which two major issues were raised; the first was the rational allocation of the R&D budget; and the second was the evaluation of performance. Therefore, the Roh government made efforts to develop an evaluation system to overcome these problems and reflect them in the KNIS, and as a result the number of papers became an evaluation indicator. Along with such efforts to evaluate R&D performance, the number of papers published began to rapidly increase since the end of the Roh government.
Adopting the new evaluation system which included in the number of papers, the number of papers increased the most compared to past years during the first year of the Lee government. Since then, the number of papers published in South Korean journals rose sharply in 2012 at the end of the Lee government, while the number of papers published in SCI journals decreased.
Under the Park government, the number of papers published in South Korean journals declined rapidly, except for in 2014. However, the number of papers published in SCI journals during this time was far greater than during the two previous governments. It seems that world-class research was carried out that led to the publication of papers in SCI journals. In addition to producing the world’s first or highest level of performance, R&D performance indicators had gradually been changed towards the evaluating of papers in qualitative rather than quantitative terms (Ministry of Science, ICT and Future Planning 2016) [41].
Figure 7 shows the number of patent applications and grants domestically and overseas as a result of national R&D projects conducted by each government. Based on the total number of patent applications and grants, there was a steady increase from the Roh government in 2003 to the Park government in 2014. The dramatic decline at the end of the Park government in 2015 and 2016 is owing to the time lag between patent applications and grants, which takes 18 months to process. In other words, the examination duration is approximately one year to 18 months for a patent that did not apply for preferential examination; afterwards, the registration status is decided. In addition, it is mandatory for patent applications to be disclosed automatically after 18 months from their application date. Therefore, because publicized patents were counted based on the aforementioned disclosure, it appears as if the number of patent applications and grants decreased dramatically in 2015 and 2016. By the end of Roh government in 2007, the increase rate of the number of patent applications and grants was significantly higher than in previous years, while the rate gradually decreased since the Lee government. During the Park government, the increase rate continued to decline by 10%, and the rate of patent applications and grants dramatically slowed.
Figure 8 depicts the percentages of patent applications and grants. In the case of the Roh government, the percentage of patent grants was higher than applications, which changed after 2007. In the case of the Lee government, the number of patent applications also increased sharply. It is possible to speculate that researchers tended to focus on increasing the number of patent applications.
Figure 9 compares domestic and overseas patent applications and grants provided by the government. The percentage of overseas patents compared to domestic patents was strikingly low, but the number of overseas patents steadily increased, except for at the beginning of the Lee government. Researchers focused on how R&D performance gained a competitive advantage on the global scale by increasing the number of overseas patent applications and grants. Based on the year-on-year growth rate of domestic and overseas patents, the rate was higher for the domestic patents except for 2009 and 2010 during the Lee government. In 2009 and 2010, the growth rate of overseas patents was 24% and 37%, higher than domestic patents, which was 21% and 13%, respectively. There was obviously a difference between the Lee government and the Park government regarding the growth rate for domestic and overseas patents.
The results in Figure 10 and Figure 11 show the number of technology transfers and the royalty income from utilizing national R&D projects. The striking feature of technology transfers is that the number of technology transfers decreased continuously after the Roh government. During the Lee government, the number of technology transfers tended to decline except for during 2012, and the same results were produced during the Park government. On closer scrutiny of Figure 11, the royalty income decreased, while the number of technology transfers increased in 2005 during the Roh government. In the case of the Lee government, the number of technology transfers increased sharply in 2012 and was about 7000. In contrast, the royalty income did not change significantly compared to 2010, when the number of 4000 technology transfers was recorded. In the period of 2013–2016, the royalty income on each technology transfer continued to decrease. It can be interpreted that the outcomes of technology transfers gradually deteriorated in quality during this time.
According to the report which explains technology transfers policies which is presented in Figure 1, technology transfer in South Korea was the result of a plan initiated by the government which centered on the commercialization of SMEs [42]. In the report written by the South Korean Intellectual Property Strategy Institute (KISPI) (2015) [42], 93% of domestic companies that transferred technology from the public are SMEs and only 5% are major companies. In the case of the US, 47% of technology transfers have been to SMEs, 36% to major companies, and 17% to start-up companies. Japan, one of the Asian countries, has a higher percentage of technology transfer to major companies (55%) than to SMEs (44%). Taking all of this into account, it is possible to forecast that technology transfer in South Korea was enacted from a supplier perspective compared to other countries. In other words, there was no interaction between the public and private sector in technology transfer because it was built around the public for supporting SMEs only.
Figure 12 depicts the number of commercialized projects among all national R&D projects. During the Roh government, the number of commercialized projects increased steadily. In the Lee government, except for in 2009, the number of commercialized projects also increased and a total of 17,835 projects led to commercialization in 2012. This was the highest figure across all governments. Figure 13, which demonstrates the sales from commercialized projects, showed that the Lee government had the largest amount of sales among all governments. Additionally, the number of commercialized projects in 2008 was lower than in 2012, but the sales were higher in 2008. During the Lee government, the number of sales reached KRW 200 billion, representing the most striking performance caused by commercialization. However, beginning during the Park government, this number entered a downward curve and finally the number of commercialized projects which had been in the tens of thousands since 2008, sharply decreased to 7743. Although the number of commercialized projects turned positive in 2016, this was at the same level of outcomes as in the last year of the Roh government. The sales were low as well as the number of commercialization projects. Thus, it can be implied that the KNIS was not functioning properly during the Park government.
Figure 14 illustrates the number of jobs created directly by commercialized projects. There was significant correlation between the number of jobs created and commercialized projects in the Roh government. Both of them indicated the same CAGR of 76%; this was also a very high level compared to for the other two governments. In contrast, the Lee government had not exceeded the number of jobs of the Roh government except for in 2009. These findings conflict with the number of commercialized projects and the sales from the projects. In 2009, the number of jobs increased rapidly, but in the rest of Lee’s presidency, it was less than 50,000 revealed in the last year of the Roh government. The number of jobs declined during the Park government.

5.3. Characteristics of the KNIS

Our main results are reported in Figure 15, which demonstrates innovative performance per government investments of KRW 100 million: the number of papers published domestically and overseas (Papers), the number of patent applications and grants domestically and overseas (Patents), the royalty income from technology transfers (Royalty income), the sales from commercialized projects (Commercialization), and the number of jobs created by commercialized projects (Job creation).
The Roh government constructed the KNIS with a legal obligation and expanded the government investment on R&D with the aim of developing a science-based society. The observable innovative performance came from commercialization and job creation in the Roh government. In 2003, more inputs brought about more outcomes and this tendency seemed to become further strengthened. All indicators of the innovative performance were positive overall in 2006. During the last year of the Roh government, in 2007, the upward trend slowed a little, but the KNIS gradually evolved with the growth of commercialization and job creation.
Most of all, the Lee government reached the highest figures of innovative performance and continued to expand its R&D investment more than the Roh government. The increase rate of innovative performance was lower than the Roh government, except for in the case of commercialization. Interestingly, the overall efficiency of the KNIS was maintained, but the correlation between input investment and output performance became weak in this period.
In the case of the Park government, the level of innovative performance other than the number of papers published showed negative growth, and was lower than the KNIS established in 2003. Although the Park government invested more heavily in R&D than the two previous governments, the R&D investment had not led to increased innovative performance. These empirical results largely confirm that no linear relationship is created between input investment and output performance. The KNIS did not develop satisfactorily (i.e., it declined) during Park’s presidency.
There are two findings to note about the KNIS. One point is that the KNIS has been developing constantly since it was constructed in the mid-1990s. All governments made an effort to develop the KNIS and therefore they invested in national R&D projects with expectation of innovative performance. However, the development level of KNIS was different across each government. It is assumed that there are other factors needed to make the KNIS more dynamic between input investment and output performance. This is linked to the second point. In addition to R&D investment, other socioeconomic and sociocultural factors might have impacts on innovative performance, as displayed in Figure 1. For example, all governments particularly focused on fostering SMEs and accelerating regional innovation although the COSTII did not identify how R&D investment was allocated for SMEs and regional innovation was facilitated in the implementation process.

6. Discussion

Since the NIS concept was introduced in the mid-1990s, it has been a popular device in policymaking contexts in many countries. In South Korea, highlighting R&D investment, the government introduced of a Plan to Construct a National Innovation System in 2004 [43]. Since then, as analyzed in Section 5, the KNIS has evolved and declined over time in performance, although R&D investment remarkably increased for 14 years. Empirical evidence highlights that policies implemented for more than a decade do not effectively link to economic outcomes, resulting in an imbalance between innovation input and innovation output. We argue three policy directions could describe the imbalanced phenomenon of the KNIS.
The first is a mismatch between the national vision and economic performance. As mentioned in Section 3, the Roh, Lee, and Park governments had different national visions, but differences have not yet been revealed in the implementation process of KNIS. At the time of the Roh government, NIS was established as a government-led public sector with a strong supply-oriented dimension, while the Lee government shifted this focus to the market, deciding that public initiative would hinder growth. The Park government advocated the concept of an innovative ecosystem by taking the government one step further from the innovation system. Although there has been a major change in the views of KNIS as presented in previous section, all the governments have implemented and measured the KNIS based on the identical framework (i.e., COSTII) along with the expansion of national R&D investment. This might lead to a gap in the implementation process between the national vision and economic performance and result in some weaknesses of the KNIS (e.g., weak SMEs) as criticized in Section 3.
The second is the greater emphasis on R&D investment as the direct input that brings about economic outcomes. As analyzed in Section 2, it seems that a strong tendency exists among policymakers to consider innovation performance as directly related to enhancing R&D activities through leading scholars [2,6,44]. In a similar vein, all three governments seemed to adopt a linear perspective to set quantitative goals to be allocated first, and then to implement policies through inputs after the establishment of KNIS. It is assumed that an increase of R&D investment contributed to enhancing innovative capabilities to some extent, however R&D input did not always link to expected innovative outcomes according to Section 5.3. The output indicators fluctuated, although the South Korean government made consistent efforts to increase R&D investment in the past 14 years. We argue that policy is likely to focus on limited input and output indicators as well as technical achievements, which seems to overlook sociopolitical and socioeconomic dimensions in spite of growth in size of the KNIS.
Finally, closely related to the above, policymakers tend to measure KNIS only with a statistical index composed of input and output data (i.e., COSTII) that can be quantified in a framework overlooking the discussion of the various scholars who established and developed the concept of NIS. Policymakers developed huge investment science and technology systems and statistical analysis. These developments may have led to the development of much quantitative survey-based innovation research, as described in Section 2.2. Along with the usefulness of NIS as a policy tool, mapping indicators of national performance regarding R&D efforts and innovation have been a major issue in South Korea for more than a decade. As described in Section 2, NIS is more than a framework which emphasizes a dynamic nature reflecting technological, social, and cultural aspects of a nation. As such, understanding actors and institutions, interactions that facilitate knowledge creation, diffusion, and utilization are crucial for analysis and development of the NIS. We argue policymakers may have been misled to focus on static factors and functions at a specific point in time, but the broader concept seems to be useful for shaping future national policy where specific sociocultural factors and groups co-construct and formulate the framework and concept.

7. Conclusions

This study examined the history of KNIS for 14 years based on a dataset of 470,000 national R&D projects. The result shows that the KNIS evolved during the Roh government, persisted through the beginning of the Lee government into 2008, and maintained itself until the end of Lee’s term. Afterwards, it gradually declined during the Park government. South Korea has been steadily expanding government investment in national R&D projects through three governments from 2003 to 2016, but the outputs indicated that innovative performance levels varied among governments. Since the KNIS was established by the Roh government, innovative performance gradually appeared from 2005 and increased significantly in 2006 and 2007. In the Lee government, the highest CAGR for government investment on R&D was recorded, while innovative performance was maintained. During the Park government, the rate of government investment in R&D had stagnated and innovative performance had declined as well. Throughout the three governments, the KNIS has undergone a phase of evolution, maintenance, and decline.
In this study, we suggest the following policy implications. First, according to empirical analysis, policymakers need to consider the dynamic process of NIS in its implementation process, as the expansion of input (i.e., R&D investment) does not necessarily lead to an increase in output (i.e., innovative performance). It seems that the linear based viewpoint, in which increasing R&D investment simply leads to national economic growth, has prevailed in the policymaking arena and might have resulted in quantitative growth of the KNIS.
Second, in order to understand the dynamic and complex nature of NIS and its implementation, policy makers should consider not only quantitative indicators in measuring NIS but also qualitative indicators reflecting sociotechnical aspects. As scholars advocate the concept of NIS from a broad perspective, this study reveals that NIS is not a fixed framework based on historical review of its 14-year existence. Rather, NIS is considered as an entity that interacts continuously and moves dynamically. Therefore, we suggest that future studies should consider explanatory national and sociotechnical characteristics in analyzing and establishing the NIS. Otherwise, the process leading to innovative outcomes remains as a black box.
This study traces back the dynamic history of KNIS and contributes to existing knowledge by analyzing big data that has rarely been examined by NIS scholars. This is in contrast to previous research, which tends to focus on national comparisons of innovative performance or analyzing the current status of NIS. In particular, the concept of KNIS is mostly used in the policymaking arena to measure innovation activities every year, thereby overlooking its history. Although we attempted to examine its evolution process over 14 years, some limitations still remain. First, in addition to the dataset analyses on R&D investment and outputs during each government relevant to the historical background of the KNIS provided in Section 3, it is necessary to promote an understanding of the KNIS by analyzing other indicators that measure NISs from various perspectives (i.e., the number of new startups created in NIS projects). Second, the NIS framework used in this study does not show how actors and institutions, such as firms, research institutes, and ministries carried out their roles and how they were affected by the three different government policies. Third, although NIS scholars have emphasized interaction and learning, this study lacks explanatory interactivity among actors and therefore does not show how this interactivity enhances innovative outcomes, as indicated in Section 2. In this vein, future research would take account of exploring how different actors (e.g., policy makers and scholars) shape and implement KNIS in order to determine the principal cause of any decline in innovative performance.

Author Contributions

E.S.K. conceived the methodology and wrote the first draft. K.J.B. investigated previous research and analyzed data. J.B. developed the overall idea and reviewed the final draft. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Acknowledgments

This research was supported by the Korea Institute of Science and Technology Information (KISTI).

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Freeman, C. The Economics of Industrial Innovation; Pinter Publishers: London, UK, 1982. [Google Scholar]
  2. Freeman, C. Technology Policy and Economic Performance: Lessons from Japan; Frances Pinter: London, UK, 1987. [Google Scholar]
  3. Lundvall, B.-Å. National Systems of Innovation; Pinter: London, UK, 1992. [Google Scholar]
  4. Cooper, R.; Nelson, R.R. National Innovation Systems. Foreign Aff. 1993, 72, 161. [Google Scholar] [CrossRef]
  5. Bank of Korea. Growth National Income. 2017. Available online: http://ecos.bok.or.kr/ (accessed on 10 September 2019).
  6. Lee, H.J.; Kim, T.K.; Huh, S.J. A Study of Reevaluation for Reduction in GDP Volatility; Bank of Korea: Seoul, Korea, 2017. [Google Scholar]
  7. Lundvall, B.-Å. Innovation as an Interactive Process: From User–producer Interaction to the National Innovation Systems. In Technology and Economic Theory; Dosi, G., Freeman, C., Nelson, R.R., Silverberg, G., Soete, L., Eds.; Pinter: London, UK, 1988; pp. 349–369. [Google Scholar]
  8. Metcalfe, J.S. Technology systems and technology policy in an evolutionary framework. Camb. J. Econ. 1995, 19, 25–46. [Google Scholar]
  9. Lundvall, B. Åke Scope, Style, and Theme of Research on Knowledge and Learning Societies. J. Knowl. Econ. 2010, 1, 18–23. [Google Scholar] [CrossRef]
  10. Sharif, N. Emergence and Development of the National Innovation Systems Concept: 1990 to the Present. Res. Policy 2006, 35, 745–766. [Google Scholar] [CrossRef] [Green Version]
  11. Lundvall, B. National Innovation Systems—Analytical Concept and Development Tool. Ind. Innov. 2007, 14, 95–119. [Google Scholar] [CrossRef]
  12. OECD. Dynamising National Innovation Systems; Organisation for Economic Co-Operation and Development (OECD): Paris, French, 2002. [Google Scholar]
  13. Jiang, H.; Gao, S.; Song, Y.; Sheng, K.; Amaratunga, G.A.J. An Empirical Study on the Impact of Collaborative R&D Networks on Enterprise Innovation Performance Based on the Mediating Effect of Technology Standard Setting. Sustainability 2019, 11, 7249. [Google Scholar]
  14. Watkins, A.; Papaioannou, T.; Mugwagwa, J.; Kale, D. National innovation systems and the intermediary role of industry associations in building institutional capacities for innovation in developing countries: A critical review of the literature. Res. Policy 2015, 44, 1407–1418. [Google Scholar] [CrossRef]
  15. OECD. National Innovation Systems; OECD: Paris, France, 1997. [Google Scholar]
  16. Gordin, B. The Knowledge-Based Economy: Conceptual Framework or Buzz Word? J. Technol. Transfer 2006, 31, 17–30. [Google Scholar] [CrossRef]
  17. OECD. Possible Meeting of the CSTP at Ministerial Level: Statistical Compendium, DSTI/EAS/STP/NESTI (98); OECD: Paris, France, 1998. [Google Scholar]
  18. Balzat, M.; Hanusch, H. Recent trends in the research on national innovation systems. J. Evol. Econ. 2004, 14, 197–210. [Google Scholar] [CrossRef] [Green Version]
  19. Boden, M.; FitzGibbon, M. Evaluation of science and technology policy in Ireland. Res. Eval. 1995, 5, 55–62. [Google Scholar] [CrossRef]
  20. Lankhuizen, M.; Woolthuis, R.K. The National Systems of Innovation Approach and Innovation by SMEs (Research Report H200309); Scientific Analysis of Entrepreneurship and SMEs: Zoetermeer, The Netherlands, 2003; pp. 3–4. [Google Scholar]
  21. Schot, J.; Steinmueller, W.E. Three frames for innovation policy: R&D, systems of innovation and transformative change. Res. Policy 2018, 47, 1554–1567. [Google Scholar]
  22. Arocena, R.; Sutz, J. Looking at national systems of innovation from the south. Ind. Innov. 2000, 7, 55–75. [Google Scholar] [CrossRef]
  23. Intarakumnerd, P.; Chairatana, P.-A.; Tangchitpiboon, T. National innovation system in less successful developing countries: The case of Thailand. Res. Policy 2002, 31, 1445–1457. [Google Scholar] [CrossRef]
  24. Marxt, C.; Brunner, C. Analyzing and improving the national innovation system of highly developed countries—The case of Switzerland. Technol. Forecast. Soc. Chang. 2013, 80, 1035–1049. [Google Scholar] [CrossRef]
  25. Chen, C.-P.; Hu, J.-L.; Yang, C.-H. An international comparison of R&D efficiency of multiple innovative outputs: The role of the national innovation system. Innovation 2011, 13, 341–360. [Google Scholar]
  26. Guan, J.; Chen, K. Modeling the relative efficiency of national innovation systems. Res. Policy 2012, 41, 102–115. [Google Scholar] [CrossRef]
  27. Chaminade, C.; Intarakumnerd, P.; Sapprasert, K. Measuring systemic problems in National Innovation Systems. An application to Thailand. Res. Policy 2012, 41, 1476–1488. [Google Scholar] [CrossRef] [Green Version]
  28. KISTEP. Assessment of National Science and Technology Innovation Capability: Composite Science and Technology Innovation Index; KISTEP: Seoul, Korea, 2016. [Google Scholar]
  29. Available online: https://www.k2base.re.kr/costii/ko/home.do (accessed on 30 January 2020).
  30. Jensen, M.B.; Johnson, B.; Lorenz, E.; Lundvall, B.-Å. Forms of Knowledge and Modes of Innovation. In The Learning Economy and the Economics of Hope; Lundvall, B.-Å., Ed.; Anthem Press: London, UK, 2016; pp. 155–180. [Google Scholar]
  31. Science and Technology Policy Institute. National Innovation Systems in Korea; Science and Technology Policy Institute: Seoul, Korea, 1998. [Google Scholar]
  32. Ministry of Education, Science and Technology. Framework Act on Science and Technology. 2009. Available online: https://www.law.go.kr/lsInfoP.do?lsiSeq=165472&lsId=009177&chrClsCd=010202&urlMode=engLsInfoR&viewCls=engLsInfoR#0000 (accessed on 12 September 2019).
  33. Presidential Commission on Policy Planning. National Technology Innovation System (NIS) for Advanced Korea; Presidential Commission on Policy Planning: Seoul, Korea, 2008. [Google Scholar]
  34. Ministry concerned. Science and Technology Basic Plan of the Lee Myung-Bak Government; National Science and Technology Commission: Seoul, Korea, 2008.
  35. Ministry concerned. The 3rd Science and Technology Basic Plan; National Science and Technology Council: Seoul, Korea, 2013.
  36. IMD. World Competitiveness Yearbook. 2006–2017. Available online: http://worldcompetitiveness.imd.org (accessed on 10 October 2019).
  37. MarketLine. South Korea: In-Depth PESTLE Onsights; MarketLine: London, UK, 2013. [Google Scholar]
  38. Cho, H.D.; Lee, B.H.; Sung, T.K.; Kim, S.W. Assessing the Institutional Legitimacy of Research and Technology Organizations in South Korea: A Context Analysis Approach. Sci. Technol. Soc. 2011, 16, 53–73. [Google Scholar] [CrossRef]
  39. Joo, W. Public R&D is the Main Factor of Creative Innovation? Hyundai Research Institute: Seoul, Korea, 2015. [Google Scholar]
  40. National Assembly Budget Office. Assessment of R&D Projects for Supporting Basic Research; National Assembly Budget Office: Seoul, Korea, 2016.
  41. Ministry of Science, ICT and Future Planning. Standards and Guidelines for Assessment of National R&D; Ministry of Science, ICT and Future Planning: Seoul, Korea, 2016.
  42. KIPSI. Patent Performance Analysis for National R&D; KIPSI: Seoul, Korea, 2015. [Google Scholar]
  43. Lim, C. Towards Knowledge Generation with Bipolarized NSI: Korea. In Small Country Innovation Systems: Globalization, Change and Policy in Asia and Europe; Edquist, C., Holmmen, L., Eds.; Edward Elgar Publishing: Cheltenham, UK, 2008; pp. 113–155. [Google Scholar]
  44. Lundvall, B.-Å. User–Producer Relationships, National Systems of Innovation and Internationalisation. In Technology and the Wealth of Nations: The Dynamics of Constructed Advantage; Foray, D., Freeman, C., Eds.; Pinter: London, UK, 1993. [Google Scholar]
Figure 1. The changes in the Korean national innovation systems (KNIS) by governments.
Figure 1. The changes in the Korean national innovation systems (KNIS) by governments.
Sustainability 12 01266 g001
Figure 2. A framework for analysis of the KNIS.
Figure 2. A framework for analysis of the KNIS.
Sustainability 12 01266 g002
Figure 3. Government investment and private expenditure on R&D. Note: The period average exchange rate for 2016 is KRW 1,160.13 = USD 1. * R&D expenditure accounts for private sector contributions by the relevant performing organization and/or the local government.
Figure 3. Government investment and private expenditure on R&D. Note: The period average exchange rate for 2016 is KRW 1,160.13 = USD 1. * R&D expenditure accounts for private sector contributions by the relevant performing organization and/or the local government.
Sustainability 12 01266 g003
Figure 4. Number of R&D projects carried out by government.
Figure 4. Number of R&D projects carried out by government.
Sustainability 12 01266 g004
Figure 5. Average government investment per R&D project.
Figure 5. Average government investment per R&D project.
Sustainability 12 01266 g005
Figure 6. Number of papers published.
Figure 6. Number of papers published.
Sustainability 12 01266 g006
Figure 7. Number of patent applications and grants.
Figure 7. Number of patent applications and grants.
Sustainability 12 01266 g007
Figure 8. Number and percentage of patent applications and grants.
Figure 8. Number and percentage of patent applications and grants.
Sustainability 12 01266 g008
Figure 9. Number and percentage of patent applications and grants domestically/overseas. Note: Domestic patents refer to the number of applications or grants in South Korea and overseas patents refer to patent applications or grants in countries other than South Korea.
Figure 9. Number and percentage of patent applications and grants domestically/overseas. Note: Domestic patents refer to the number of applications or grants in South Korea and overseas patents refer to patent applications or grants in countries other than South Korea.
Sustainability 12 01266 g009
Figure 10. Number of technology transfers.
Figure 10. Number of technology transfers.
Sustainability 12 01266 g010
Figure 11. Royalty income from technology transfers.
Figure 11. Royalty income from technology transfers.
Sustainability 12 01266 g011
Figure 12. Number of commercialized projects.
Figure 12. Number of commercialized projects.
Sustainability 12 01266 g012
Figure 13. Sales from commercialized projects.
Figure 13. Sales from commercialized projects.
Sustainability 12 01266 g013
Figure 14. Number of jobs created by commercialized projects.
Figure 14. Number of jobs created by commercialized projects.
Sustainability 12 01266 g014
Figure 15. The change in path of the KNIS via empirical analysis.
Figure 15. The change in path of the KNIS via empirical analysis.
Sustainability 12 01266 g015
Table 1. COSTII rankings of South Korea in sectors.
Table 1. COSTII rankings of South Korea in sectors.
SectorsSouth Korea (Year/Rank)Relative *
(%)
200620072008200920102011201220132014201520162017
R&D investment77756332222293.0
Economic outcome910111111767777644.9
Note: * This refers to the country’s relative position against the top country placed at 100% in 2017. Source: KISTEP [29].
Table 2. Competitiveness ranking of South Korea in sectors.
Table 2. Competitiveness ranking of South Korea in sectors.
SectorsSouth Korea (Year/Rank)
200620072008200920102011201220132014201520162017
Overall competitiveness322931272322222226252929
Total expenditure on R&D as a percentage of GDP655555331212
Source: IMD [36].
Table 3. Number of papers published (detailed).
Table 3. Number of papers published (detailed).
GovernmentYearPapers Published in South KoreaPapers Published in Overseas (Non-SCI)Papers Published in SCI Journals
Roh government2003378316447
2004123711441366
2005268317093103
200611,20512,58423,445
200713,49614,18018,818
Lee government200831,654705834,583
200931,52410,48639,887
201036,04214,45135,366
201141,59712,73241,725
201268,447693726,518
Park government201341,783758829,942
201462,53529,55266,369
201541,82516,07963,305
201637,31713,48560,166

Share and Cite

MDPI and ACS Style

Kim, E.S.; Bae, K.J.; Byun, J. The History and Evolution: A Big Data Analysis of the National Innovation Systems in South Korea. Sustainability 2020, 12, 1266. https://0-doi-org.brum.beds.ac.uk/10.3390/su12031266

AMA Style

Kim ES, Bae KJ, Byun J. The History and Evolution: A Big Data Analysis of the National Innovation Systems in South Korea. Sustainability. 2020; 12(3):1266. https://0-doi-org.brum.beds.ac.uk/10.3390/su12031266

Chicago/Turabian Style

Kim, Eun Sun, Kuk Jin Bae, and Jeongeun Byun. 2020. "The History and Evolution: A Big Data Analysis of the National Innovation Systems in South Korea" Sustainability 12, no. 3: 1266. https://0-doi-org.brum.beds.ac.uk/10.3390/su12031266

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop