Roadmap Modeling and Assessment Approach for Defense Technology System of Systems

Abstract

Advanced defense technology plays a crucial role in safeguarding national safety and development interests. Aiming to handle the problems of current research and development (R&D) management approaches faced with the rocketing complexities of system of systems, the authors propose a novel roadmap modeling and assessment methodology through studying the driving forces of general technology development and analyzing realistic requirements of defense technology management in this article. First, a requirement decomposition framework is designed based on multi-view theories and text-mining tools are used to construct a multi-layer knowledge-flow network model. Second, the contribution rates of requirement elements at different levels are evaluated using a multi-criteria decision-making approach and the node importance is assessed based on the topological structure of multi-layer network. Third, it is utilized to demonstrate the effectiveness of the proposed approaches that illustrative examples of the technology requirements in maritime security strategy investigating and a dual-layer knowledge-flow network consists of patents that belong to the “Coherent Light Generator (CLC)” classification from the United States Patent and Trademark Office (USPTO) database and the related academic papers from Web of Science. Finally, the contributions, potential applications, and drawbacks of this work are discussed and research outlooks are provided.

1. Introduction

The development of science and technology is a fundamental driving force for the evolution of civilization and social change. A typical example is the reformation of combat patterns originating from the emergence and application of advanced defense technologies. Nowadays, military transformation is drawing massive attentions of all superpowers. As a symbolic feature of warfare in the information age, a critical factor determining the outcomes of battles is the integration of the combat system of systems, which assembles different kinds of interrelated and interactive operational units systematically [1]. Furthermore, disruptive innovations of defense technologies trigger the revolutions of warfare game rules and initiate the transformation of international political structures. The administration of the United States launched the Third Offset Strategy, of which the main objective is to defeat potential adversaries in a long-term competition [2]. Consequently, the research and development (R&D) management of defense technologies plays a critical role in both military force building and national security [2,3,4].

Generally speaking, there are three principle benefits/objectives that are infused in advanced defense technologies. First, it will improve the tactical performance of weapon systems and reinforce the capability of combat system of systems. Second, it will accelerate the structural reformation of command and control (C2) systems as well as the organization of combat units. Third, it will build and strengthen technology deterrence, which contributes to the discourse power in the negotiations of focal international security issues, such as arms control and conflict mediation. On the other hand, the strategies of national security and military modernization provide the requirements for development planning of defense technology. Based on the research achievements of technology roadmaps aiming at maintaining the technical competitiveness for commercial enterprises [5,6,7,8], it is proposed that a novel roadmap modeling and assessment approach for defense technology system of systems in this article. The authors combine two significant driving forces of technology development in the proposed framework, namely requirement-pull and innovation-drive, through surveying the elements in the lifecycle of R&D and their relationships. The approaches can be used to offer an effective support to decision-making in the R&D management of defense technologies.

The remaining components are organized as follows. Section 2 reviews the related literatures and summarizes the advantages and deficiencies of the mentioned research studies. Section 3 demonstrates the framework of roadmap modeling using multi-view theories. Moreover, the authors propose the assessment approaches of the contribution rate and the importance from the perspectives of requirement satisfaction and knowledge diffusion, respectively. In Section 4, two illustrative examples discussing the contribution rates to satisfy requirements and the importance of knowledge diffusion of focused technologies are used to demonstrate the proposed approach. Finally, Section 5 concludes the contributions and drawbacks of this work and notes possible improvements in future research.

2. Literature Review

The traditional methodology of R&D management is now confronted with rocketing complexities stemming from the rapid emergence of novel innovations. The focal problem consists of two main components. The first originates from the diversity of basic scientific fields relating to applied technologies and the pluralism of technology interrelations, which are considered complexities of the nature of technology and the scale of system of systems, respectively. The second concerns the interactions among the significant stakeholders in the lifecycle procedure of R&D [8]. Drawing lessons from commercial enterprise management, the Department of Defense (DoD) developed a requisition management framework of weapon systems based on the multi-view theory [9]. Through collecting and integrating data, information, and expertise from different stakeholders, this framework can be used to calibrated comprehensive and accurate requirement models of defense requisition. Following this research methodology, other scholars discussed “division and integration” approaches for system of systems modeling. Moreover, these methods have been applied and validated in the requirement analyses of military capability [10], weapon systems [11], and defense technology [12].

Patent text is one of the focal data sources in R&D study due to its availability and semi-structured storage format. With text-mining tools, semantic and citation analysis [13,14,15,16,17,18] can be used to measure technology relations through calculating the similarities among abstracted keyword vectors. Differing from the methods processing polysemous keyword vectors, citation analysis focuses on the accumulation and diffusion of innovation knowledge. In addition, it has the advantage of eliminating the ambiguities from authors’ word choices and writing styles to measure the technology relations more objectively [19]. The centrality indices [16] were applied to identify core patents based on citation networks with the analytic network process (ANP) method [15]. Moreover, the division of technology areas in dynamic citation networks and the detection of those focal ones with higher development potential has been discussed through the integration of network analysis, time series models, and cluster analysis [17,18].

Technology roadmaps have been widely applied in different domains of R&D planning, providing flexible information exchange platforms for the stakeholders including researchers, engineers, product managers, suppliers, and consumers [5]. This methodology can be used to identify vital information, including current and future customers’ requirements and technology development opportunities, as well as to support decision-making in programing, executing, and inspecting the R&D schedules [5,20]. Roadmap models involve various elements, such as technologies, products, services, and business/markets [6]. According to the related survey [5], the roadmaps drawing methods can be classified into expertise-based, computer/algorithm-based, and a hybrid category. Furthermore, different approaches of data analysis have been infused into and consummated this modeling framework. Fenwick et al. [21] investigated a value-driven technology roadmap process focusing on customers’ requirements, through integrating multiple related approaches, and demonstrated this methodology using the case study of software service business application. Geum et al. [7] proposed an integrative approach combining technology roadmap and system dynamic simulation to improve the effectiveness of scenario planning. Aiming to cope with the rocketing data volume in drawing technology roadmaps, approaches of association rule mining (ARM) were used to abstract the relations among different elements [8]. In addition, the implementing process of a technology roadmap was demonstrated using applications in the energy service sector [22]. This empirical research was extended through discussing the differences of objectives in national, industry/sector, and organizational levels [20].

As mentioned above, various interactive elements, such as knowledge, technologies, products, and businesses, are required to investigate R&D planning. Consequently, one can use the multi-layer network model to depict technology roadmaps [8]. Wang et al. [23] constructed a dual-layer citation network model involving scientific papers and patents based on open access data. Through analyzing the network structure, one can identify focal knowledge sources and technology fronts. Fang et al. [24] investigated a triple-layer weighted network involving domains, researchers, and knowledge, and a novel method was designed to predict the potential links based on structural similarities between pairs of nodes. Meanwhile, the structure and dynamics of multi-layer networks has become a highlighted research area. Scholars have published many achievements on the theories and approaches on the node centralities, clustering, spreading process, and synchronization [25], which will promote the development of technology roadmap research.

The foregoing literature provides theoretical guidance and useful tools for the research of defense technology roadmaps. Nevertheless, practical modeling and assessment approaches are still insufficient to identify the requirements from various sources and investigate the evolution mechanisms of the focal system of systems accurately. Therefore, this article will focus on three interrelated objectives, including model framework integrating data analysis and expertise, the contribution rate to satisfy external requirements, and the importance of knowledge diffusion. In particular, this article proposes a framework to collect expertise, taking into consideration the viewpoints of different stakeholders to reinforce the quantitative system evaluation based on data analysis of the roadmap model. Thus, the assessment results can be utilized to support decision-making with the assistance of practical experience of R&D managers. The corresponding approaches are proposed in the next section.

3. Roadmap Modeling and Assessment Approaches

As mentioned in Section 1, the effects of requirement-pull and innovation-drive driving forces in the evolution of general technologies have been discussed. More specifically, the former factor emphasizes the economic activities that R&D projects require to input various resources while the outputs can be used to improve production efficiency and create social wealth. Besides, the latter concentrates on the discovery and diffusion of scientific and experiential knowledge, which is considered the basis of innovations. Technologies are created and ameliorated in the process of combining and applying the available knowledge to solve realistic problems.

Figure 1 demonstrates that technologies, stemming from the evolution and organization of knowledges, have been evolving in different routes determined by their application objective. The development of civilian innovation is defined as the “products-service-business/market” path driven by economic interests. In detail, novel technologies are applied to improve the performance of products and to offer better services to clients; meanwhile, corporations can occupy more market shares and obtain business profits with such R&D achievements. On the other hand, the requirements of defense technologies concern with the strategic objectives of national security, which are defined by the “strategies-capabilities-weapons” structure. Similar to their application in a civilian area, defense technologies are expected to enhance/develop advanced weapons and strengthen/create military capabilities in order to reach the strategy objectives. Besides, it is also obvious that applied R&D projects depend on basic research. The knowledge acquired from logical reasoning and experimental observations which are discovered, recorded, and proliferated by scientists, and then selected and organized to cope with problems and obstacles by engineers. Additionally, knowledge can be accumulated, propagated, and reused for new innovations through theoretical discovery and engineering experience from past R&D projects. Consequently, the authors plan to design the model framework and assessment approaches for defense technologies with the viewpoints of requirement analysis and knowledge diffusion. The multi-view theory is utilized to guide the model framework by identifying core stakeholders and collecting related information to their viewpoints, with which one can extract the requirement elements and their interactions with defense technologies. Furthermore, the multi-layer network is used to depict the knowledge diffusion among different kinds of innovation units. With those mentioned models, the appropriate algorithms can be discussed and applied to assess the contribution rate of requirement elements and the importance of innovation units.

3.1. Requirement Model Framework Based on Multi-View Theory

The model framework based on multi-view theory has been validated as an effective tool to collect and integrate the requirement information of defense technologies [12]. Moreover, it is a critical intelligence input for requirement modeling to identify and select various stakeholders of the focal R&D projects. Through integrating the data and intelligence collected from the selected stakeholders, one can calibrate a comprehensive system model that consists of requirement elements at different levels. According to the elements of the defense technology roadmap, as shown in Figure 1, the authors focus on the interactive requirements from the viewpoints of technology researchers, weapon engineers, military officers, and strategic deciders.

The requirement model framework is displayed in

Table 1. Requirement analysis models for defense technologies.

Stakeholders	Related Elements	Requirement Models	Outline
Strategic decider	Security strategy	Strategic game models	This model focuses on hotspot security issues, including strategic objectives, tactics, and capabilities
Military officer	Military capability
Military officer	Military capability	Combat network models	This model consists of different types of nodes representing weapons function and edges referring to their interrelations
Weapons engineer	Weapons
Weapons engineer	Weapons	Technology impact matrices	This model collects the core functional indices of weapons, and measures the impacts of technology on those selected indices
Technology researcher	Technology

. The strategic game model focuses on the holistic requirements of national security. Based on qualitative analysis with related expertise, one can calibrate the graph model for conflict resolution (GMCR) including decision-makers (DMs), options, game state, feasible transitions, and relative preferences. Moreover, the potential compromises represented as different types of equilibriums can be identified and the impacts of DMs’ tactics and capabilities can be measured using the proposed approaches in Reference [26]. Then, the authors introduce the combat network model using the concepts of operation loops, in which the involved weapons are decomposed into functional nodes and their interactions are abstracted as different edges linking node pairs [1,27]. By analyzing the combat network model, one can not only survey the interrelations among relevant combat units with the given missions/scenarios, but also quantify the impacts of weapons’ performances on the military capabilities of the focal combat system of systems. The technology impact matrix is an effective tool to measure the benefits to weapons’ performances through infusing new technologies [28]. In this model, the core functional indices are listed in the column heading and the technology alternatives are displayed in the array heading. Furthermore, one can evaluate the expected impact values through integrating the related expertise, prior knowledge, and simulation experimental results [28].

3.2. Multi-Layer Knowledge-Flow Network Models

As the other basic driving force of defense technology development, knowledge diffusion focuses on the features of technology as the carrier and transmitter of detected scientific principles and engineering experience, which differ from the solving-problem oriented attributions in requirement analysis. Besides, the focal technologies are arranged in a fine-grained hierarchy. Based on the data source and modeling approach in References [19,23], the authors selected academic papers to represent basic knowledge and patent documents to indicate applied technologies. Moreover, the citations were used to measure the inner and intra knowledge-flows among the two kinds of elements.

The open access data from the United States Patent and Trademark Office (USPTO) and Web of Science were collected and processed to abstracted the nodes representing papers and patents and edges referring to citations using text-mining tools. Furthermore, the dynamic evolution of the dual-layer citation network was recorded to depict the information of the granted data of patents and the published date of papers. Based on the citation network model, this article proposes a novel approach to assess the importance of focal nodes from the perspective of spreading property in the knowledge-flow network.

3.3. Assessment Approaches

Based on the foregoing models, two assessment approaches, focusing on the contribution rate to satisfy the requirements and the importance of knowledge diffusion, are proposed. First, the analytic hierarchy processes (AHP) method was introduced to evaluate the contribution rate of technology alternatives using the requirement decomposed structure mentioned above [29]. The detail process is displayed as follows.

Step 1:

Investigate the judgement matrix $A_{n\times n}^{ij}$ involving $n$ elements which stay in the $i$th layer and belong to the $j$th father node. By comparing the element pairs with related support intelligence and calibrated expertise, one can measure the relative importance using the 1–9 score rank.

Step 2:

Compute the maximum eigenvalue and its corresponding eigenvector $e^{ij}$ of the judgement matrix $A_{n\times n}^{ij}$.

Step 3:

Calculate the assessment result of the contribute rates. The contribute rate of the $l$th element is defined as $C_l^i=C_l^i+e_l^{ij}\times C_j^{i-1}$ when the value of its father node is $C_j^{i-1}$.

In the algorithm above, the assessment of root node $C_{}^0$ is initialized as 1 and others are set as 0. Additionally, one should not update the values of the elements in the $i$th layer until the assessment of their father nodes has been completed.

Second, the authors investigated the knowledge diffusion among the dual-layer citation network. Based on the network topological structure, an approach to assess the node importance of spreading property combining the eigenvector centrality [30] and PageRank value [31] is put forth. The detail process is displayed as follows.

Step 1:

Compute the eigenvector centrality vector $x$: $x={\lambda }^{-1}{A}^{a}x$ of the sub-network in the article layer. The $A^a$ refers to the adjacency matrix of the mentioned network and the $\lambda$ is its maximum eigenvalue.

Step 2:

Assign the values of two parameters, namely the inter-layer and intra-layer diffusion strength factors $\alpha$ and $\beta$. Then, one can construct the universal citation network $A^u$, where $A_{ij}^u=\alpha (A_{ij}^a+A_{ij}^p)+\beta A_{ij}^{a-p}$, by extending and integrating the article-layer sub-network $A^a$, article-patent inter-layer sub-network $A^{a-p}$, and patent-layer sub-network $A^p$.

Step 3:

Initialize the node importance values. The initial value of the $i$th article node is set as $P{R}_i(0)=x_i/{\displaystyle \sum _{j=1}^n{x}_j}$ and the patent nodes’ values are assigned as 0.

Step 4:

Calculate the elements of the revised row stochastic matrix $R_{ij}^u$ using the algorithm below.

$$R_{ij}^u=\{\begin{array}{l}1/k_i^{out},if{k}_i^{out}>0\wedge A_{{}_{ij}}^u>0\\ 0,if{k}_i^{out}>0\wedge A_{{}_{ij}}^u=0\\ 1/n,if{k}_i^{out}=0\end{array}.$$

(1)

where the $k_i^{out}$ represents the out-degree of the $i$th node in $A^u$ while $n$ refers to its node count.

Step 5:

Select a parameter $s$ to measure the randomness in knowledge diffusion. Then, compute the assessment values of node importance using the following iterative algorithm, where $k$ is the given iteration number.

$$P{R}_i(k)=s{\displaystyle \sum _{j=1}^n{R}_{ji}^{u}P{R}_j(k-1)}+(1-s)/n.$$

(2)

4. Illustrative Examples

Two illustrative examples are demonstrated to validate the proposed modeling and assessment approaches. In the first case, the authors investigated the maritime security strategy and related combat system of systems. Based on the hierarchical decomposition structure of military requirements, the contribution rates of focal defense technologies were evaluated. Then, a dual-layer knowledge-flow network model was constructed using the patent documents belonging to the “Coherent Light Generator (CLC)” classification which are collected from the USPTO database and the related academic papers from Web of Science. Then, the node important in knowledge diffusion was computed in the second example.

Maritime security is a highlighted component in the system of national security, which plays a key role in exploiting natural resources and maintaining trade routes. Through summarizing research achievements in the related literature [32,33], the authors abstracted three focused objectives at the strategy level, including a fast response to potential invasions, patrolling in territorial water/airspaces, and maritime search and rescue, as outlined in

Table 2. Requirement elements at the strategy level.

ID	Requirement Name	Outlines
ST-1	Fast response to potential invasions	Intercepting, tracking, monitoring, and evicting the military objectives invading into the territorial water/airspaces
ST-2	Patrolling in territorial water/airspaces	Maintaining the normal operation of resource exploitation and commercial transportation in the target area
ST-3	Maritime search and rescue	Maintaining the risk monitoring and information communication, locating and rescuing persons, watercrafts, and aircrafts in distress

.

Based on the identified strategic requirements mentioned above, the related elements were decomposed layer by layer including seven military capabilities, 10 weapon systems, and nine defense technologies. Using the multi-view model framework in Section 3.1, the requirement mapping model was constructed; the outline information of the elements in each layer of the model are displayed in

Table 3. Information of elements in the requirement mapping model.

Military Capabilities		Weapon System		Defense Technology
ID	Name	ID	Name	ID	Name
MC-1	Information acquisition	WS-1	Destroyer-052C	DT-1	C2 technology
MC-2	Command and control (C2)	WS-2	Coast Guard-31	DT-2	Vertical launching equipment technology
MC-3	Communication	WS-3	Destroyer-052D	DT-3	Sonar system technology
MC-4	Fast response	WS-4	Medium-Sized Coast Guard Boat	DT-4	Radar system technology
MC-5	Attack	WS-5	Kilo-Class Diesel Attack Submarine	DT-5	Reconnaissance system technology
MC-6	Patrolling coverage area	WS-6	Maritime Helicopter	DT-6	Inshore defense system technology
MC-7	Patrolling endurance	WS-7	J-11 Fighter	DT-7	Combat system technology
-	-	WS-8	Maritime Geographic Information System Platform	DT-8	Torpedo launching equipment technology
-	-	WS-9	KJ-500 Airborne Early Warning Plane	DT-9	High-quality engine technology
-	-	WS-10	Command & Control Center	-	-

and the interrelations among them are demonstrated in Figure 2.

According to the assessment approach for the contribution rate in Section 3.3, the authors constructed the judgement matrices using the element mapping relations and computed the scores of relative importance with collected objective data and relevant expertise. Through updating the contribution rates layer by layer, the nondimensionalized values of focal defense technologies in satisfying military requirements were evaluated; they are demonstrated on the vertical axis of Figure 3.

The knowledge-flow network model consists of patents representing applied technologies and articles referring to basic scientific research achievements. First, the authors collected 21417 patent documents belonging to the CLC classification from USPTO database. The patent-layer network model was constructed through abstracting the citation information using text-mining tools. Then, the highlighted nodes of which the in-degree or out-degree are among the top 1% highest were selected. Third, the authors selected365 articles cited by the patents corresponding to the highlighted nodes and integrated the two kinds of elements to complete the dual-layer citation network. The network model is demonstrated in Figure 4, where the red nodes represent articles and blue ones indicate patents.

According to the assessment process of node importance in Section 3.3, the eigenvector centrality vector of the sub-network in the article layer $x$ was calculated. Then, the authors assigned the values of two diffusion strength factors $\alpha =\beta =1$ and integrated the adjacency matrix of the universal citation network $A^u$. Finally, the assessment results were computed with the parameter conditions of $s=0.85$ and $k=100$ using the proposed algorithm. The top 10 articles and patents with the highest importance values are displayed in

Table 4. Assessment results of node importance in the dual-layer knowledge-flow network.

Patent ID	Node Importance Score	Rank	Article Index No. (Web of Science)	Node Importance Score	Rank
8693517	2.5831 × 10−4	1	000246625500003	1.7036 × 10−5	1
8542713	1.2497 × 10−4	2	000259170400009	8.7287 × 10−6	2
8878119	1.0491 × 10−4	3	000172960800007	8.2545 × 10−6	3
7949215	0.8403 × 10−4	4	000299134400006	7.1009 × 10−6	4
8908727	0.8099 × 10−4	5	000238669800012	6.8828 × 10−6	5
7794159	0.8062 × 10−4	6	000265372900015	6.7064 × 10−6	6
8279517	0.7722 × 10−4	7	000252284200058	6.4555 × 10−6	7
8548017	0.7669 × 10−4	8	000227734400001	6.3947 × 10−6	8
8867586	0.7625 × 10−4	9	000246345800028	6.1771 × 10−6	9
8509575	0.7594 × 10−4	10	000231184400003	6.1709 × 10−6	10

. The nodes with higher scores in knowledge diffusion analysis can be considered as innovation sources and the knowledge chunks which contain more innovation sources should be paid more attention in assessing development potential [18].

5. Conclusions

This work investigates the driving forces of general technology development and proposes roadmap modeling and assessment approaches for defense technology system of systems. Additionally, two illustrative examples, focusing on requirement satisfaction and knowledge diffusion, are utilized to demonstrate and validate the proposed approaches. Through analyzing the assessment results in the abovementioned examples, one can observe the technology alternatives and knowledge chunks which should be paid more attention in R&D management from the perspectives of both the contribution rate and node importance. The accumulation and diffusion of innovation knowledge is used to measure the development potential and the contributions to the realistic problems are selected to qualify the benefits for R&D project/portfolio alternatives. Furthermore, the integration of the benefits, intelligence, and expertise of the stakeholders using a model framework based on multi-view theory is considered as a data basis for better decision-making in R&D planning. Specifically, the calibrated expertise in this model framework can be used to not only reinforce the quantitative assessment process, but also to support R&D decision-making in combination with the analysis results of the technology roadmap.

The authors only provide a conceptual modeling framework for a defense technology system of systems in this research. The proposed assessment approaches seem a little naïve and the illustrative examples were not discussed comprehensively to simulate realistic R&D issues. Accordingly, some notes are given for the future research. First, the information of the expected budget and duration will be involved in the roadmap model, which will be used to examine the cost resource constraints of R&D project portfolio alternatives. Second, the assessment approach for the contribution rate of defense technology portfolios will be investigated by analyzing the structure evolution of the dynamic combat network for specific warfare scenarios. Third, the development potential of focal applied technology/basic research areas will be discussed using the trend forecasting approaches based on multi-layer citation network analysis. Finally, the authors plan to develop a more effective methodology for examining and scheduling R&D projects using the multi-objective optimization algorithm, with the aim of providing high-value supporting intelligence for R&D decision-making.