A Brief Taxonomy of Hybrid Intelligence

Abstract

As artificial intelligence becomes ubiquitous in our lives, so do the opportunities to combine machine and human intelligence to obtain more accurate and more resilient prediction models across a wide range of domains. Hybrid intelligence can be designed in many ways, depending on the role of the human and the algorithm in the hybrid system. This paper offers a brief taxonomy of hybrid intelligence, which describes possible relationships between human and machine intelligence for robust forecasting. In this taxonomy, biological intelligence represents one axis of variation, going from individual intelligence (one individual in isolation) to collective intelligence (several connected individuals). The second axis of variation represents increasingly sophisticated algorithms that can take into account more aspects of the forecasting system, from information to task to human problem-solvers. The novelty of the paper lies in the interpretation of recent studies in hybrid intelligence as precursors of a set of algorithms that are expected to be more prominent in the future. These algorithms promise to increase hybrid system’s resilience across a wide range of human errors and biases thanks to greater human-machine understanding. This work ends with a short overview for future research in this field.

1. Introduction

The last decade has seen major advances in computation power and its increased availability at a low-cost due to cloud computing. Advances in machine learning and statistical learning methods, the availability of big data, mobile devices, and social networks has led to an explosion of artificial intelligence and its applications to a wide range of commercial, governmental, and social products and services. Many parallels exist between biological and artificial intelligence. Both neuroscience and artificial intelligence can draw inspiration from each other and generate new theories and hypotheses [1,2,3,4]. However, while artificial intelligence is building momentum, it is also becoming apparent that artificial intelligence and biological intelligence are also characterized by different mechanisms, oftentimes not entirely overlapping. Compared to biological intelligence, artificial intelligence requires much larger training sets to recognize patterns, is less robust to slight variations in the input, does not take into account prior experience, cultural norms, situational or contextual information, and is less able to generalize to new problems and environments [5,6,7]. Thus, rather than using AI to replace human intelligence, researchers have started to investigate how to integrate human and artificial intelligence by creating hybrid human–artificial systems [8,9,10]. Hybrid intelligence is arguably a more effective and nuanced approach to solve complex problems that both humans and machines struggle to solve alone. According to one definition, hybrid intelligence (HI) is “... the combination of human and machine intelligence, augmenting human intellect and capabilities instead of replacing them and achieving goals that were unreachable by either humans or machines [11]”.

As research on hybrid intelligence expands, so does the space of design choices when designing a hybrid system. One must consider several aspects of the problem space, task, sources of error and information, and relation between human and machine elements of the system. For example, a hybrid system aimed at forecasting rare events is likely to employ very different strategies than a system designed to predict frequent events. In this paper, I propose a brief taxonomy of hybrid human–machine systems to help researchers and forecasting practitioners navigate existing options and opportunities in this field. I use the weak version of the term “taxonomy” to broadly indicate soft categories that can cluster classes of algorithms that are observed in the research. As the subject of a perspective paper, these classes should be interpreted as fuzzy categories rather than hard-bounded boxes. Instead of focusing exclusively on forecasting applications, I draw upon research in collective intelligence, human factors, and psychology. Understanding how other disciplines are integrating human and machine intelligence in their design can help forecasting researchers and practitioners by providing a blueprint that can be adapted to the specific necessities of their field. Compared to previous excellent reviews that have been written on the subject of hybrid intelligence [11,12], the present perspective focuses more on papers, results and insights from behavioral disciplines, cognitive science and psychology. This novel lens can better support practitioners interested in designing better human-aware algorithms. The papers in this article were selected by searching various academic repositories, using the terms “hybrid” and “human–machine”.

2. Biological Intelligence

Figure 1 shows the proposed taxonomy, representing the two main axes of variation. The first axis is composed of examples of biological (human) intelligence. It spans from individual problem-solvers to dyads to teams and networks of problem-solvers acting together to solve a specific task. Compared to individual problem-solvers, multiple problem-solvers can benefit from social learning and achieve greater accuracy [13]. Research on joint perceptual tasks has recently unveiled the conditions and cognitive functions underpinning accuracy improvements in joint decision-making [14,15,16,17]. These results are very relevant for studies on forecasting and researchers in forecasting have highlighted similar effects in forecasting tasks. Human forecasting is improved when forecasters can benefit from each others’ estimations, arguments, evidence, and signals of confidence [18,19,20,21]. More recently, research in collective intelligence expanded their focus of analysis from teams to networks of problem-solvers [22,23,24,25]. Although social processes can sometimes lead to collective gains as well as collective losses [26,27], under the right conditions the capacity for collective intelligence tends to scale up with size [28,29,30]. Hybrid human–machine systems in forecasting and decision-making can be built to amplify human intelligence, from individual users to networks of problem-solvers. Thus, an orthogonal axis of variation can be drawn for hybrid intelligence. In the rest of the paper, I describe the various levels of this second axis, as AI moves from being an assistant to human endeavor, to progressively more sophisticated roles as peer, facilitator, and system-level operator.

3. Algorithms as Assistants

Many artificial intelligence use cases nowadays can be categorized as being of external support to human intelligence. At this level, algorithms make it easier for humans to perform complex, lengthy, or repetitive tasks by automating some of the processes that are more costly for humans to perform. The algorithm reduces the cognitive load of forecasters or increases the amount of relevant information available to the human problem-solver. For example, algorithms as assistants can help humans sort and find existing task-relevant information or present information in a more accessible way for humans to digest.

This category may include not just actual digital assistants such as Siri, Alexa, and Cortana [31,32,33,34,35], but also recommender systems [36,37], search engines [38,39,40,41,42,43], data visualization tools [44,45], argumentation mapping tools [46], image captioning [47], and translators [48,49,50]. These algorithms enable human problem-solvers to amplify human intelligence. For example, in forecasting, analysts, and forecasters routinely use algorithms as assistants to filter the relevant data to make accurate predictions [21,51,52]. Also fitting this category are those algorithms that provide the technological medium where interaction between human problem-solvers occurs, such as platforms for crowdsourcing [18,51,53,54,55,56]. In this example, technology assists human interaction by providing the digital infrastructure that facilitates finding better solutions or amplifies collective human intelligence. However, the algorithm (or platform) does not actively try to influence the outcome of human interaction.

Designing algorithms as assistants does not necessarily require understanding the task that the human wants to perform. Nor does it require an understanding of the biases that affect a human performing such a task. For example, when finding relevant web pages, a search engine does not need to know why users need the information they have typed in the search bar. Nor does it need to understand that the user adapts what they read to their own political views or biases. The algorithm requires only knowledge of the information space where the task occurs (e.g., the network of existing web pages that need to be ranked and recommended). For example, algorithms as assistants do not model how information provided by the algorithm is likely to interact with human behavior to produce collective gains or losses [57]. It is known that algorithmic behavior and biases can interact with human biases [21,58,59]. Designing algorithms as assistants does not need to consider this interaction.

4. Algorithms as Peers

The second level of the hybrid intelligence scale is the use of algorithms as human peers. This level is perhaps the first that comes to mind when we think of hybrid intelligence. Fueled by science fiction and humanoid robots populating the collective imagination, this second level of hybrid intelligence involves designing algorithms reaching human-like and better-than-human performance in previously human-only domains.

Examples of this type of algorithm include AI beating humans at human games such as Atari, Poker, and Go [60,61,62,63], and job displacement and replacement due to automation [64,65,66,67]. These algorithms are designed to mimic humans and to perform overlapping tasks with humans’. A better-than-human algorithm may entirely replace human problem-solvers while certain algorithms as peers can also populate hybrid human-algorithm teams to strengthen team resilience, performance and scale [10,68].

In forecasting, human forecasters often partner with machine models performing the same forecasting tasks [69,70,71,72,73]. Assuming data about the problem being solved is available and machine-readable, artificial intelligence can help with feature identification and prediction [74]. Recent work on human-machine ensembles shows that aggregating human and machine forecasts can outperform each individual component [75].

Designing algorithms as peers requires knowledge of the task and problem space, not just the information space where the task occurs. For instance, to play Go, AlphaGo needs to understand both task-relevant information (e.g., the board state) and the game’s rules. Likewise, in forecasting, algorithmic assistants only need to present or visualize relevant information to a human forecaster. However, when acting as peers, forecasting models need to translate task-relevant information into sensible forecasts in the same way a human analyst would. They move from being a support to a (human) forecaster to being a forecaster themselves. Importantly however, designing algorithms as peers do not necessarily require modeling the human collaborator that is part of the hybrid system.

5. Algorithms as Facilitators

A slightly more sophisticated form of hybrid intelligence uses algorithms as facilitators of human intelligence. In this category, the algorithmic agent does not just try to solve the task as a human but uses its model of human behavior to maximize the chances of good human performance. While algorithms as peers act independently from human agents, algorithms as facilitators act as a function of people’s behavior. What the algorithm does is dependent on the cognitive strategies and problem-solving abilities employed by their human counterpart. The algorithm corrects (either explicitly or implicitly) for human cognitive biases so as to facilitate the system’s aggregate performance.

From medical to military, human decisions are known to be affected by a range of biases emerging from cognitive adaptation known as heuristics [76,77,78,79]. Although adaptive under a wide range of domains, these heuristics can, under specific circumstances, result in maladaptive or seemingly irrational behavior [80,81,82,83]. AI algorithms that correct or de-bias individuals and groups of human solvers thus act as facilitators. This is not a novel field of study. Studies on human factors and human-machine interaction have traditionally focused on designing technology and products that incorporate human error and human capability. For instance, in medical and military decisions, people are known to show a confirmation bias that can compromise the objectivity of the decision by neglecting or under-estimating conflicting evidence. User interfaces, human factors and human-centered design can successfully de-bias human decisions [84,85]. More recently, however, researchers have started to play around the idea of artificial players (e.g., bots) playing alongside human problem-solvers using strategies that, although not effective in isolation, are effective when aggregated together with human strategies. For example, the use of locally noisy autonomous agents embedded in a network of human problem-solvers can increase the likelihood of a social system finding a global solution in a network coloring task [86]. This task is known to have many local optima where human-only agents can get stuck. Thus, the addition of artificial agents employing a random noisy strategy facilitated the solution of the task, even if those same artificial agents alone could not have found the global solution by themselves [87]. It has been recently suggested that algorithms with different problem-solving strategies than humans can improve overall performance by increasing the diversity and thus the resilience of social learning [88]. In another study, researchers embedded machines within human groups to moderate team conflict [89]. Algorithms as facilitators cannot solve the task by themselves but require interaction with human solvers.

Designing an algorithm as a facilitator requires knowledge of the information space and task, as the previous level (algorithms as peers). On top of that however, it also requires knowledge of the human problem-solvers performing the task alongside the AI. It requires anticipating human pitfalls and biases to design systems that can be resilient to systematic human failures. AI researchers are increasingly using human-centered design and social intelligence to design algorithms that can better anticipate and react to human errors, biases and irrationalities [12,90,91,92]. Researchers in AI and hybrid intelligence (HI) have called for greater awareness of humans in the loop by modelling their mental states, recognizing their beliefs and intentions, giving cogent explanations on-demand, and fostering trust [11,12,52,93]. The current research agenda in HI is often framed around four central properties that are required for such hybrid intelligent systems, namely being collaborative, adaptive, responsible, and explainable [11]. In line with this research agenda, algorithms as facilitators move the role of algorithms from being merely tools (algorithms as assistants) or human replacements (algorithms as peers) to being collaborative team members that can model (implicitly or explicitly) human partners. This novel hybrid interaction generates a complex dynamic system that can produce novel ideas, patterns, and solutions than humans or algorithms could do in isolation. Notice, however, that algorithms as facilitators still operate within the boundaries of the task being solved. This assumption is relaxed in the following category, namely algorithms as system-level operators.

6. Algorithms as System-Level Operators

The last level of this taxonomy includes a class of algorithms that I believe will be more and more relevant in the future. These algorithms are what I call “system-level operators”. I believe this level of algorithmic interaction with human agents offers the most interesting opportunities for the future of hybrid intelligence but is currently relatively underexplored compared to the other three. Although not enough work has been conducted in this area, I try here to offer a glimpse into how these kinds of algorithms might look in the future. I define an algorithm as a system-level operator if the algorithm in question can observe the full information–task–people–AI system and the reciprocal interactions between its elements. This knowledge is used so that the algorithm can act on the system to maximize the system’s performance. These algorithms are not expected to operate as problem-solvers from within the task boundaries. Instead, they can change the interactions among problem-solvers to maximize overall aggregate performance. For instance, some algorithms in hybrid intelligence are designed to dynamically select crowdsourced human ideas and information to amplify the power of collective intelligence in real-world challenges [55] and open-ended problems [94]. The algorithm actively controlled the flow of information on the platform to help good performance. In these examples, the algorithm does not operate as an artificial problem-solver (i.e., it does not share the same action set as human players). Thus, it does not replace a human or compensate for their bias from within the task. Instead, it acts as an ex machina system controller. In this regard, the algorithm acting as a system-level operator is dynamic and highly adaptive [11]. There are many advantages of algorithms as system-level operators, including adaptability and design. Some preliminary research on these types of algorithms has been recently conducted. In collective intelligence, for example, one study compared different algorithms that dynamically manipulate the network structure of human problem-solvers [95]. In this work, the algorithm acts as an invisible hand that strategically controls the information flow of the system and the relations between human problem-solvers, tasks, and information to achieve greater performance.

Furthermore, I suggest that this type of algorithm will also be more prominent in forecasting. Again, research so far has been sparse but there are reasons to believe that it will increase as interest and theory for human-machine understanding grows [96]. In particular, one study used a dynamic ensemble method to allocate human and algorithmic forecasters to minimize expected errors dynamically [70]. The algorithmic forecasters are algorithms as peers to the human forecasters. However, the algorithm controlling the dynamic allocation of human and machine forecasters is a system-level operator. I propose that one lens that needs to be adopted when designing algorithms to operate in hybrid environments is that of system control. More research is needed to create more theory-driven methods that can dynamically change the parameters of the forecasting system as a function of the amount and type of data available, the uncertainty of the forecasts and new evidence. A better understanding of both human cognitive biases and machine biases will help create better hybrid forecasting systems. A better understanding of the biases affecting each component of the hybrid system can help designing more principled approaches to selecting algorithmic versus human decisions, which will result in more robust forecasting systems.

Designing algorithms as system-level operators is challenging, but I believe it also offers the greatest promises for hybrid intelligence. This is because as interest in algorithmic and human bias grows, there will be an interest from various stakeholders, from companies to policy makers, to develop algorithms that can produce resilient hybrid systems. On top of task and problem-solver knowledge (as in previous levels), designing algorithms as system-level operators also requires knowledge of the interactions between task and problem-solvers over time. It requires sensing and flexibly adapting parameters of the forecasting system to slight variations in the state of the world. In hybrid forecasting, dynamical model update and system performance monitoring allow hybrid systems to control their own performance, e.g., to collect more evidence when uncertain, to self-correct after errors and to learn from past mistakes. Although still underexplored, system-level AI promises to show greater flexibility in the emerging system’s behavior and greater robustness in performance because the system dynamically adapts to human interactions within the system and to change in the environment outside it [11,97].

7. Future Directions

Hybrid forecasting should not be viewed as the silver bullet to successful forecasting [98]. However, as machine learning techniques expand in scope and reach, there will be an increased capacity of artificial intelligence to complement and augment biological and collective intelligence. According to this perspective, increased capacity for hybrid intelligence scales up with the capacity to include more dimensions of a forecasting system. The paper outlined soft boundaries in which artificial intelligence’s ability to achieve better and more robust forecasting will depend on the capacity of algorithms to incorporate knowledge of the problem space, task and the human forecasters acting in it. Importantly, the increasingly sophisticated levels of hybrid intelligence described in this paper should not be seen as hard categorical boundaries. Instead, hybrid systems may show elements of different levels to various degrees. Finally, people’s increasing awareness of how algorithmic and human biases are often intertwined can play a pivotal role in boosting the development of more transparent and robust AI, and better hybrid human–AI intelligence. Rather than restricting algorithmic design to a few stakeholders with niche skill sets, teams should diversify their expertise, including behavioral and social scientists and machine learning practitioners.