Challenges toward Evidence-Based Policymaking Using Agent-Based Modeling for Federal Sports Grants: A Self-Reflection from a Transdisciplinary Project

Abstract

Despite their importance, federal grant systems often need more clarity regarding cost-effectiveness, lack of transparency, and slow feedback cycles. Sports funding systems aimed at improving child health and contributing to sustainable development goals are incredibly challenging due to their heterogeneity in stakeholders and regional aspects. Here, we analyze how we tackled these challenges in a transdisciplinary EU project in Austria, targeting the use of agent-based modeling for evidence-based policymaking in a co-creation process with policy stakeholders in the domain of federal sports grants to improve the health and well-being of children and youth. The initial and executed set of procedures is described, along with lessons learned during the project’s lifetime. These lessons derive a framework that provides an adapted set of processes, supporting methods, and critical decision points for an improved use of transdisciplinarity. In addition, the steps of the developed framework are combined with essential aspects of knowledge integration, following the main phases of the policy cycle and providing suggestions for required skills and competencies for capacity building concerning implementing the developed framework in the public sector. Our results show that the combination of transdisciplinarity, human-centered policymaking, and sports, supported by cutting-edge technologies such as agent-based modeling, can achieve significantly better results than a pure disciplinary approach and generate positive spill-over effects.

1. Introduction

Promoting healthy lives and well-being for citizens of all ages is one of the 17 sustainable development goals (SDGs) that states pledged to work toward in the framework for global sustainability cooperation. Sports [1,2,3] (for a critical perspective, see Coalter in [4]) and human-centered design [5,6] are thereby considered vital measures for sustainable policymaking. While a commendable goal, implementing such a human-centered and sustainable health promotion via organized sports for all is a challenge, even for affluent countries like Austria. The grand challenges society faces today in the form of sustainable development goals require public administrations to undergo a paradigm shift from a traditional input/output orientation toward a focus on societal outcomes and evidence-based policymaking [7,8]. Examples of such a shift come from regulatory impact assessments and performance-informed budgeting. One element in the supply chain of public services is the system of federal grants, a core instrument in steering societal outcomes [9].

Sports policy and sports governance in Europe have been characterized by its multistakeholder governance interactions, ranging from international sports federations, national authorities, local authorities, and national and regional sports federations to clubs, local sports offerings, and sports participants. Austria follows Ireland (EUR 74.7 per capita) as third in Europe in national funding allocated to grassroots sports. In contrast, Austria ranges in the lower third of countries in annual revenue issued to sports by local authorities, with EUR 17.3 per capita [10]. This situation shows that national public sports funding in Austria plays a significant role, while regional state governments are responsible for sports [11]. However, despite their importance, national grant systems often need more certainty in terms of cost-effectiveness and non-transparent and non-explainable assumptions regarding the underlying effectiveness of the used model [12].

One policy field in which this problem is highly significant is the system of state grants to improve child and youth health and physical condition via sports funding. Sports funding systems are incredibly challenging, as they not only include a high level of heterogeneity in terms of stakeholders but also introduce substantial local and regional aspects, as well as the necessity to work in an interdisciplinary way from a science perspective. The high level of heterogeneity originating from the stakeholder composition poses challenges of various sorts, such as knowledge integration, collaboration, and non-limiting non-functional interaction, hence opposing and challenging established hierarchy, role, or rank [13]. Due to this governance complexity, the aim for evidence-based and human-centered policymaking becomes increasingly complex, with policymakers neither having direct access to the necessary information nor being able to steer the required policy impact directly. Considering these circumstances, a transdisciplinary process that unites researchers, policymakers, and sports stakeholders is necessary to deal with this complexity [14].

The Research Question

This paper aims to answer the following research question: How can the added value of applied transdisciplinarity in sports policy be improved? Transdisciplinarity in the form of close collaboration between research and practice to assess the impact of policy steering in sports funding and to better understand the effect of sports grants on young people’s health is indispensable due to the complexity of interactions between policymakers, governance stakeholders, and policy recipients. A human-centered policy design toward sustainable sports governance needs access to (better) data, a model that allows turning these data into meaningful information, and policymakers willing to engage in such a human-centered mode of policy innovation.

This paper reports on a transdisciplinary EU project in Austria, targeting the use of agent-based modeling for evidence-based policymaking in federal sports grants to improve the health and well-being of children and youths. Based on this project, we conducted a self-reflection process, complementing leaders’ views in science and practice on the added value of transdisciplinarity at the intersection of finance policy and sports. The initial and executed procedures within the project are described, along with lessons learned during the project’s lifetime. These lessons derive a transdisciplinary co-producing process framework, which provides an adapted set of processes, supporting methods, and critical decision points. In addition, the steps of the developed framework are combined with essential aspects of knowledge integration, following the main phases of the policy cycle and providing suggestions for required skills and competencies for capacity building to ensure the smooth implementation of the developed framework in the public sector.

The results of our research show that achieving these three core elements on a sufficient level of quality is difficult, but without it, a human-centered and sustainable policymaking tool to improve young people’s health through sports is not feasible. While stakeholders involved in this project in Austria were all motivated and committed to a transdisciplinary approach, the participation of different types of stakeholders—though essential—is not sufficient. For transdisciplinarity to have a positive effect, stakeholders need to adjust their processes, expectations, and collaboration methods.

2. Related Work

The reported transdisciplinary project builds upon three types of scholarly work. First, the paper is informed by the literature on how the health of children and adolescents can be improved via sports funding Secondly, we draw on insights from agent-based models (ABMs) as formal tools to quantify the impact of interventions on systems. Thirdly, we discuss the role of transdisciplinarity in sports science. In the following, we first synthesize the state of the art in those four bodies of literature, discuss their strengths and weaknesses, and finally identify the gap in the literature that this paper addresses.

2.1. How Can Sports Funding Promote the Health of Children and Adolescents?

Improving child and adolescent health through sports is considered a sustainable investment in citizens’ physical and mental health and improving social justice and inequalities. In particular for youth and adolescents, sports teaching is vital in fostering regular participation in sports and physical activity [13]. It is assumed that physical fitness positively impacts young people’s physical and mental health, and local interventions matter for fitness levels and, thus, healthy lifestyles for adolescents.

These assumptions are also well-established in Austria and Europe [14]. The public agency to promote sports activities in Austria, the “Fonds Gesundes Österreich” emphasizes that sports and physical activity have various positive effects on the individual’s physical health, particularly lowering risks for depression, improving school performance, and supporting the healthy development of children and youths [15]. Austria’s National Plan on Physical Activity [16], the Austrian Recommendations on Physical Activity [15], and a study on the comparison between physical activity in EU member states [17] each provide recommendations on improving the individual’s health by practicing sports and being physically active. Children and youths, in general, are advised to exercise at moderate intensity for 60 min each day and do muscle-strengthening exercises three times a week, do small exercises after 60 min of sitting, and perform muscle-strengthening activities three times a week [16]. Children between three years and school age are advised to be physically active for at least three hours a day. Children above six and youths are advised to be physically active for at least an hour a day and practice endurance training at least three times a week [15]. Austrian data also confirms that being an active member of a sports club can improve children’s self-esteem, particularly girls [18].

The challenge for theory and practice emerges from inconclusive study results and underexplored causal relationships. Dozens of interacting factors and intervening variables (such as regional variations [19], gender [20], education [21], or income [15]) have been identified in how institutional sports funding affects the individual health of young people. However, there is also insufficient insight into the mechanisms and processes determining these causal effects. Bjørnarå et al. concluded that already at a more general level, the “causal relationships between organized child and adolescent sports participation and health remain uncertain” ([22], p. 1), and Weed suggests that “sport stakeholders [...] do not accept, that the effectiveness of sport as a public health intervention is not proven” ([23], p. 561). Furthermore, we know that despite their importance, national grant systems would need more certainty in terms of cost-effectiveness and non-transparent and non-explainable assumptions regarding the underlying effectiveness of the used model [12].

While Austrian public actors, in their attempts to promote youth and adolescent sports activity, are aware of this complex interplay of variables and the shacky theoretical ground of a causal relationship between sports funding and youth health, public interventions are restricted to the available tool of sports funding. For the presented project to succeed, it needed an up-to-date assessment of how or under what conditions the existing funding frameworks in Austria achieved the set goals and objectives of improving child and adolescent health through sports. With this goal in mind, it was also clear from the outset of the project that it would need the close collaboration of the funder (ministry), funding recipients (sports stakeholders), and researchers to overcome the existing gap in the literature. While the original project put this interaction of a diverse set of actors center-stage, this paper showcases that the presence of such diverse agents alone is necessary but insufficient to ensure that transdisciplinary collaboration’s added value emerges.

2.2. How Agent-Based Models Can Foster System-Understanding of the Effect of Sports Funding

Secondly, we draw on the literature about the development and use of agent-based models (ABMs). Our project employed agent-based modeling to understand the complex organizational and structural system, providing policymakers with a tool for evidence-based decision-making. ABMs are usually developed for a specific purpose, such as predicting a particular system behavior, supporting scenario planning, or optimizing current allocations of assets [24]. Therefore, the scope of an ABM is defined by its use case, i.e., the model should be as complex as the use case requires but as parsimonious as possible elsewhere. Formally, ABMs are based on a set of agents and a set of interaction rules for these agents. Agents can be thought of as sets of entities (citizens, companies, governmental units), with each group being defined by a given set of properties (e.g., a citizen being defined by age, sex, area of residence, or income). These properties can change over time. They might either change due to the interaction of one agent with one or several others or through changes in environmental conditions. The rules that prescribe how and under which conditions the states of the agent change are referred to as interaction rules. An ABM is therefore given by two sets of elements, agents, and their interaction rules.

ABMs were initially introduced in the 1980s as computational tools to study systems and their behavior on a conceptual level [25]. ABMs were primarily developed to deal with systems whose behavior results from multiple interactions between highly heterogeneous actors, mainly if the set of actors and their interactions changes over time. For this reason, ABMs soon established themselves in a variety of policy-relevant contexts as tools to assess quantitatively, for instance, economies [26], the impact of disasters or other adverse events on critical infrastructures [27], or business cycles [28]. Utilizing large-scale datasets from, e.g., business information, national accounts, census data, or open datasets, it has now become possible to develop ABMs from conceptual into data-driven tools by simulating millions of agents. These agents represent practically every natural person or legal entity in an entire national economy in a way that the properties of the agents are calibrated to the data. This approach has resulted in a new generation of agent-based models that go beyond stylized relationships and aim at building “digital twins” of the existing system under study [26,27].

In the realm of evidence-based policymaking, ABMs have been developed to assess the impact of public funding or subsidies on different policy sectors. For example, studies have used ABMs to investigate disruptive changes and their effects on health systems [29], to assess the system risk in financial networks [30], or to evaluate alternative co-payment scenarios for contributing to health systems financing [31]. Lempert suggests that ABMs are particularly helpful for policy advice because they allow to “connect heterogeneous microbehavior to different patterns of macrobehavior”([32], p. 7195). Choi and Park also confirm that ABMs “can help researchers to advance theories by means of sophisticated thought experiment which is not possible by formal modeling and verbal reasoning.” ([33], p. 614). However, ABMs have not been used yet to assess the impact of public funding on the sports sector. In sports research, ABMs have been used to explore the impact of individual and environmental interventions on reducing inequalities in sports participation [34].

Further examples include studies on the effects of interventions on children’s physical activity in an urban environment [35], research on the impact of social networks on increasing physical activity in children [36], or studies on the interaction of factors influencing leisure-time physical activity [37]. Blok et al. [34] also suggest that ABMs should be used more commonly to explore the complexity of the social–environmental system and that “ABMs may eventually become useful tools to support decision-makers in answering questions in public health arising from complex interactions” ([34], p. 10). Yet, to our knowledge, no studies have been conducted that use ABMs to understand the effect of sports funding on the promotion of the health of individuals.

One of the core challenges this project aimed to tackle is that agent properties and their interactions can only be meaningfully represented in a model if the information is available regarding the actual value of these properties or the strengths of the interactions [38]. Such data is necessary for the usefulness of ABMs to improve, and the model results might then heavily depend on more or less arbitrary assumptions regarding the actual value of the unobserved properties or interactions. Hence, to harness the potential of ABMs (or other data-intensive decision support tools) to inform policymaking, accuracy, completeness, and availability of timely data is crucial [39]. Furthermore, as Savin et al. conclude, “ABMs require simplifications of reality to limit model complexity” ([40], p. 6). They suggest that, so as not to lose crucial details, ABM assumptions and outputs should be discussed with policy stakeholders, pilot studies should be undertaken, and policy outcomes should be monitored closely.

While the usefulness of ABMs for assessing the effectiveness of public spending is thus generally shared, there are only a few studies that attempt to engage with the empirical complexity and the organizational necessities of such a multi-stakeholder endeavor. McComb and Jablokow attribute this gap in the literature “to the lack of practical and theoretical bridges between researchers possessing complementary skills“ ([41], p. 2). Additionally, a further complication is that such a research project not only needs the smooth collaboration of stakeholders from practice and science but also requires researchers who will be comfortable working across computational, quantitative, and qualitative methods.

2.3. How to Use Transdisciplinarity in Researching Sports Funding

Lastly, we consult the scholarly work on transdisciplinarity and transdisciplinarity in sports research. Transdisciplinarity is commonly defined as a method of mutual learning processes between science and practice. The paradigm of transdisciplinarity has already established itself within the sports and health research domain. One key aspect of transdisciplinarity is developing a common semantic understanding of the challenge at hand [42]. Understanding behavior and ways of thinking is essential. This circumstance is also stressed in the work of Rothwell et al. [43], where the authors argue for the need for theoretical frameworks to integrate concepts and tools from both sides, sciences and practice, pushing the effectiveness of implementation without sacrificing scientific rigor. Following this argumentation, Otte et al. [44] argue for the benefit of a multi-scale perspective (i.e., micro, meso, macro perspective) to capture interdependencies between different ecosystem sectors under analysis, including the multi-directional exchange of critical actors. It is precisely this exchange of key actors across disciplines and sectors that is crucial to providing “[…] many important findings and some cautionary tales [as well as] insightful reflections and learnings from their transdisciplinary health projects, sharing the challenges, strengths, and strategies devised along the way” ([35], p. 362).

However, cooperation across disciplines is hard, even for those with high levels of commitment on a topic of shared interest [45]; hence, one particular phrase to stress here is “along the way”. In [46], Woods et al. follow this idea of gathering information and planning for the upcoming actions, acknowledging the high degree of uncertainty and sometimes even chaotic environment researchers and practitioners must orient themselves. The authors further argue that the underlying back and forth between new information and redirection takes the shape of an iterative process that repeats itself yet constantly leads to charting previously unknown territory, continuously opening up paths for rethinking established patterns and thus allowing for innovation.

This interpretation and adoption of the idea of transdisciplinarity embrace aspects of complexity science within sports research [47], allowing for the required more profound understanding of human behavior in the context of today’s grand societal challenges [48], also referred to as wicked problems [49]. Within the sports policy domain, these wicked problems present their challenges. In [50], Sam discusses the tension between the field of sports organizations in their social responsibility to efficiently provide services to the general public while, on the other side, finding the right balance between state grants and commercial monetarization. He argues that while both financing approaches have their benefits, either can trigger side effects that need to be considered. However, these unintended side effects can take time to anticipate due to the complexity of the ecosystem where the sports organizations reside. Hence, an approach is required that iteratively works toward a joint solution, representing a practical and working compromise acceptable by all relevant stakeholders based on the current common understanding of the problem at hand.

With this backdrop in mind, we have embedded our study within the complex and challenging procedure of design and execution of transdisciplinary projects in the domain of sports policy. Building on the previous work discussed above, we pursue our study in a policy area with a high societal impact, e.g., the health promotion of children and adolescents. In particular, we target the diffuse impact of public funding, i.e., state sports grants and their governance. Our project employed agent-based modeling to understand the complex organizational and structural system, providing policymakers with a tool for evidence-based decision-making. We report on important lessons learned and practical implications concerning the application of transdisciplinarity in the sports policy domain and the resulting framework for enabling transdisciplinary processes in the ABM context. We reflect on what aspects worked and which did not work and offer reflections on how to overcome the identified hurdles and roadblocks with concrete actions along the way of the heterogenous, highly dynamic stakeholder process necessary.

3. Conceptual Approach, Methods, and Data

There are three possible analytical pathways for this project. The first pathway would be the actual impact of the agent-based model by redistributing sports funding, translating this redistribution into a change in sports governance and associated actions. These actions would then result in the changed behavior of children and youths, which ultimately should improve health. While this is an interesting perspective and worthwhile investigating, it is also a long-term process, which, at this point, is out of the scope of our project and this paper, besides associated challenges of the effects of measures and causality.

The second analytical pathway approaches the project through the lens of computer science, i.e., the analysis and evaluation of the actual mathematical model underneath the AMB, including its realization in terms of software engineering. While this perspective is exciting, it is not the main focus from a process and policy perspective, as it is not technology-agnostic.

Hence, we focus in our analysis on the third pathway, i.e., the applied transdisciplinary process of the project, how it can add value to sports policy, and how it can be further improved. In order to analyze and assess the applied transdisciplinary process, we employ a combination of action(-based) research with a transdisciplinary framework. McCutcheon and Jung define action research as “[…] systematic inquiry that is collective, collaborative, self-reflective, critical and undertaken by the participants of the inquiry. The goals of such research are the understanding of practice and the articulation of a rationale of philosophy of practice in order to improve practice.” ([51], p.148).

This approach suits our employed transdisciplinary process well, with its joint efforts of leaders from science and practice. Following the suggestion of Herr and Anderson [52] concerning a spiral of action cycles, we adapted the proposed four steps, i.e.,

• Improve current processes in place via a dedicated action plan;
• Implement the plan;
• Observe the effects in the given context;
• Reflect on the results to foster and improve development and associated actions.

Step 1 is realized via the submitted and later funded project proposal, including the foreseen action plan via its six main activities (see Section 4.3 for more details). Step 2 is realized by the actual execution of the project. Step 3 then reflects on the observations of all relevant aspects of the transdisciplinary process during the project. Finally, Step 4 comprises the self-reflection process, gained insights, and lessons learned, reflected in this paper.

To establish the detailed process for Step 4 (self-reflection), we build on Binder et al.’s transdisciplinary framework [53], which in turn took the work of Walter et al. [54] as their foundation and extended it by combining it with the three-step method proposed by Riley-Douchet and Wilson [55]. The three steps are (i) critical appraisal, (ii) peer group discussion, and (iii) self-awareness/self-evaluation.

The first step uses individual documentation of project partners concerning events, individual progress, and situational assessment.

The second step focuses on feedback and input from peers, e.g., from the project partner’s institute, other project partners, external experts, and the leaders of practice, i.e., public administration and sports organizations.

The final step, again, refers to the individual reflection on the knowledge acquired and lessons learned. While in Riley-Douchet and Wilson’s work, these steps are sequential, we applied them in iterations, sometimes also alternating between steps, e.g., 1 and 2, depending on the project’s current state.

To document our empirical approach transparently, we have to clarify the separation between the ‘scientific part’ and the ‘practice part’. Different models reflect the interdependencies and connections between sectors and their inherent stakeholders, such as the triple, quadruple, or quintuple innovation model [56]. In this paper, we refer to science as the involved disciplines, i.e., sports sciences, public management (incl. finances), e-governance, computer sciences, and complexity science. We refer to practice as the domain of application and impact, thus, the public administration and sports associations as proxies of society.

The following stakeholders from science and practice were involved during the project and the self-reflection process:

• Division III/C/9—“Strategic Performance Management and Administrative Innovation” and Division II—“Sports”, Austrian Federal Ministry of Arts and Culture, Civil Service, and Sport (BMKÖS);
• Department for E-Governance and Administration and Department for Knowledge and Communication Management, University for Continuing Education Krems;
• Complexity Science Hub Vienna;
• Team for Artificial Intelligence at the Austrian Federal Computing Centre;
• Institute of Sport Sciences, University of Vienna;
• Austrian sports organizations—subsuming federal branches (alphabetical order):
• Committee on Sports and Physical Activity in Austria (ASKÖ);
• Sports Association at large in Austria (ASVÖ);
• Federal Sports Company (BSG);
• European Paralympic Committee (EPC);
• Austrian Organization of Sports for the Disabled (ÖBSV);
• Austrian Ski Association (ÖSV);
• SportsEconAustria (SpEA);
• Sport Austria (Representation of interests and service organization of the organized sports in Austria);
• Sport Union;
• Network of Austrian’s Alpine Associations (VAVÖ).

To facilitate peer review and self-reflection, we had foreseen several review mechanics during the project, such as bi-weekly exchange meetings between the ABM implementation team and the public administration; a feedback session with external experts from science and practice via so-called sounding boards for data management, implementation, and knowledge integration; as well as the project officers from the European Commission (EC). In addition, we also presented the project’s results at scientific conferences and forums of public administration and politics.

Concerning the content-wise analytical process, we drew on numerous documents from the project, such as:

• The original project proposal;
• The deliverables/outputs of all six activities (work packages);
• The individual notes of the project partners;
• The “project handbook” in the form of a living documentation of the project’s development and progress by the ministry;
• The results from the peer-review session, e.g., with the sounding boards, the EC, or internal reflection workshops.

In addition, we conducted a small (n = 23) qualitative survey among the participating sports organizations to gain insights into their experience regarding the transdisciplinary process of the project. The survey consisted of twelve questions covering the following aspects:

• The focus of work, either on a local/regional or national level;
• Category of organization: sports clubs, regional associations, national associations;
• Position/field of work within the organization;
• Information about which project activities/events (talks, workshops) the representatives of the respective sports organizations have attended;
• Information about the role the representatives had during the project activities/events, how they perceived themselves, and how they would like to be perceived;
• Rating of the events on a Likert scale from 1 (very satisfied) to 5 (not satisfied);
• Perception of the representatives if and to what degree their inputs were heard and considered during the project activities/events;
• Most liked aspects, processes, and approaches during the project activities/events;
• Potential for improvement;
• Request to name other individuals/organizations that should be included in future actions in addition to the already onboarded individuals/organizations;
• Request to state specific aspects/topics that should be investigated in future actions;
• Request to note any outstanding issues that should be resolved in terms of aspects, processes, or approaches.

For data collection purposes, we adapted the framework of Binder et al. [53] to our project. The starting point of transdisciplinary projects can differ quite a lot. However, all start with the triangle of public discourse, scientific interest, and the necessity at some point for decision-makers to act upon facts and knowledge in the interest of the people they represent.

In the middle of this meeting point, where the ideal transdisciplinary process is happening, we can derive different result sets at the end of the journey for both science and practice, reflected in outputs, impacts, and outcomes. Following the work of Binder et al. [53] closely, Figure 1 depicts the structure of outputs, impacts, and outcomes for both science and practice, which we will describe in the upcoming part of the paper.

In terms of outputs from the project, we can distinguish between tangible and intangible outputs. Tangible outputs include all produced documents, such as output deliverables for the European Commission, publications, workshops, associated documentation, meetings, protocols, sports-relevant data sets, developed prototypes, or source code. Concerning intangible outputs, we can identify three main process-related types of outputs. The first type of output refers to methodological outputs concerning procedures that helped us to align the heterogeneous stakeholders involved in the project (e.g., scientists, public servants, or sports association representatives). These procedures were necessary to capture all particularities concerning the different cultural perspectives of the various sectors and organize organizations as their domain-specific vocabulary, working styles, and preferred (scientific) methods. The second type of intangible output is related to organizational aspects, such as improving the planning, managing, structuring, and execution of transdisciplinary projects, especially within a European context; in this context, there is the individual and organizational realization of the actual status quo from the ‘thought to be’ status quo, including insights into where those deviances occurred.

The last type of output focuses on actor-specific interaction within their environment and organization, as well as from a cross-boundary perspective. We have experienced these outputs as one of the most fundamental ones, as the trust-building process took a long time and proved crucial for the project’s success. This trust concerns the provision of documentation and data and opens up and provides insights into thinking patterns, attitudes, and organizational memories, all aspects fundamental to modeling actions.

Next to the project’s generated impacts, these can be seen as intermediate effects. Examples can be found in the derived plans, including decisions, actions, and measures to identify if these actions resulted in the desired outcomes. This outcome also includes the (co-)production of knowledge during the project. One type of knowledge is system knowledge, i.e., about the sports system and the associated federal grant system. This knowledge included, besides other aspects, details concerning the underlying structure, dependencies, inherent processes, and actor-specific (perceived) problems and challenges that needed to be addressed to improve the efficiency and effectiveness of grants. Another type of knowledge refers to the overall goal of the project, i.e., to trigger transformational processes in the public sector and within the policy domain. These processes include especially normative aspects in terms of required regulations and law changes; in this project’s case, for example, the “Bundes-Sportförderungsgesetz 2017 (BSFG2017)” (the Federal Sports Promotion Act 2017). Lastly, transformational knowledge was gained as the project realized open issues and gaps that needed to be bridged between the status quo and the desired state of the sports grant system. Examples of this transformational knowledge can be found in the current technical implementation for data governance; political commitment; economic conditions surrounding the sports sector in Austria; or cultural patterns and habits between the state representatives, the sports associations, and the sports clubs. When it comes to intangible knowledge, this refers in some part to the already mentioned intangible output, i.e., individual and organizational memories. This knowledge is hard to capture, as it is usually not explicitly documented and is highly dependent on the involved individual. Thus, once these individuals leave the organization, this knowledge goes with them. Hence, it was essential to capture as much of this knowledge during the modeling workshops; for example, in particular system change priorities.

The last part, reflected in the framework in Figure 1, comprises the project’s outcomes. These are somewhat difficult to capture, as they include short-term and long-term outcomes. The long-term outcomes are explicitly challenging, as related outcomes may lie years in the future and do not necessarily primarily depend on the project but have other contributing factors as well [53,57]. Nevertheless, we will try to provide the intermediate outcomes as far as they were revealed after the project, providing hints toward future developments, using the created outcomes as foundations.

4. The Case Study

This paper aims to assess how to improve the added value of applied transdisciplinarity in sports policy by reporting a transdisciplinary EU project in Austria, targeting the use of agent-based modeling for evidence-based policymaking in federal sports grants to improve the health and well-being of children and youths. In the following, we will provide the necessary background about the project, its goals, its set-up, and activities before we report on the results in the next part.

4.1. Study Area and Domain

Austria aims to improve its capacity for evidence-based policymaking, in line with the Better Regulation framework promoted by the European Union [58]. Better regulation in this context means devising more effective and more efficient policy measures and other regulatory aspects through state-of-the-art (electronic) decision aids and incorporating behavioral insights. However, many policy decisions tend to be more politically driven than evidence-based. The Austrian administration is aware of this lack of innovative tools to enhance the evidence base to influence the political level to make regulatory policy decisions that have a better social and economic impact. Due to this EU policy context, the action “Better Regulation in Austria—Phase II” has been brought to life as a project under the Structural Reform Support Programme (SRSP) by DG Reform.

For this project, the primary beneficiary from the policy domain was the BMKÖS, including its sports section. The service providers were the University for Continuing Education Krems and the Complexity Science Hub Vienna. While the university partner was responsible for the coordination of service providers, in addition to the social science aspects, such as the meta-studies, stakeholder mapping, transdisciplinary system modeling, as well as up-scaling and skills and competencies development, the Complexity Science Hub Vienna focused on the data governance aspects of the project, the development of the underlying statistical model, the development of the ABM prototype, as well as policy recommendations concerning data management and the usage of ABMs within grant management of sports funding.

4.2. Motivation and Goals

Since 2013, the Austrian public administration has been undergoing a paradigm shift from a traditional input/output orientation to a focus on societal outcomes. Two internationally renowned and outcome-focused instruments have been established to support this organizational and cultural change: regulatory impact assessments and performance-informed budgeting. The structural implementation process was successful and enabled the Austrian administration to expand the system toward an integrated outcome-oriented supply chain of public services. One element in the supply chain of public services is the system of state grants, a core instrument in creating societal outcomes. Despite its importance, there are several issues regarding the system of state grants that need to be addressed:

• It is not clear whether specific state grants constitute the most cost-effective pathway to reach the stated impact/goals;
• The underlying assumptions for modeling the effectiveness of state grants are not sufficiently transparent and explainable;
• In the current system, the feedback loops to correct and adjust measures on the fly are not quick enough.

One policy field in which this problem is highly significant is the system of state grants to improve child health and physical condition. Regional aspects contribute significantly to the challenges in evaluating the impact in this field. Furthermore, the stakeholders in this policy field—schools, national and regional sports organizations, and the children—exhibit a complex interactive way. By focusing on the system of state grants to improve child health and physical condition, the project tackles the described problems of the impact measurement of grants in a multistakeholder environment in a new and innovative way. By using an agent-based simulation approach, the project aimed to achieve the following:

• Develop a simulation model of the impact of grants on the health and physical condition of Austrian children;
• Implement this model and run simulations of different grant systems;
• Deliver a blueprint to expand the approach to different policy fields, primarily focusing on the following issues:

  (a)

  Explainability/transparency of assumptions (basic assumptions are hard to explain and only implicit);

  (b)

  Use of a large amount of synthesized data (to avoid privacy problems);

  (c)

  Clear accountability among different grant recipients.

Additionally, those simulation models increase decision-making capabilities by providing precise and visual ad hoc information about the impact of respective decisions. Thus, such models could foster evidence-based decision-making within public administration.

4.3. Project Organization and Activities

The kick-off commenced in September 2020 and thus coincided with the first fall wave of COVID-19 in Austria. This situation led to a slower start and shift in the original timeline. Figure 2 shows the updated timing, including all main activities and milestones.

As it was the project’s goal to co-create policy advice in the domain of sports funding via agent-based models, it was decided to adopt the framework of [59] and hence build up the project based on six main activities (see Figure 3).

Activity A1 was concerned with overall project management, including financial reporting, progress reporting, risk management, and quality assurance. In addition, A1 also handled the alignment and communication.

Activity A2 covered all activities concerning requirements elicitation, data acquisition, inspection, and key stakeholder analysis. It focused on analyzing relevant meta-studies concerning sports, youth, physical activity, health, and associated support actions. Based on this analysis, key concepts, interdependencies, and relationships were extracted to be used in the upcoming stakeholder interactions (activity A3) and agent-based modeling (activity A4), as well as for laying the fundament for the functional fitness and up-scaling recommendations (activity A5).

Activity A3 comprised all relevant activities concerning stakeholder knowledge management, including expert roundtables and workshops with practitioners for establishing the foundations for the agent-based simulations in terms of a common system understanding and system model. It further included the development of a socio–technical process to assess and codify the knowledge and expertise of the identified key stakeholders, which, combined with the joint system model (understanding of interdependencies), would serve as input for activity A4 and the ABM.

Activity A4 encompassed all relevant activities for creating the agent-based simulation, including a spatio–temporal snapshot of the current funding landscape and mechanism in Austria in sports funding, data preparation, model design, and the implementation of a first technical and procedural non-final prototype, including an appropriate visualization.

Activity A5 designed a concept for implementation procedures concerning the use of ABMs and thus covered essential aspects, such as competence models, courses, training, administrative regulations, and politics. It addressed the necessary feedback loop of the results out of activity A4 regarding the process of up-scaling within the sports sector administration under direct cooperation with the project beneficiary, i.e., BMKÖS.

Activity A6 finally addressed the final dissemination of the project results, the lessons learned summary, and all required reporting and associated actions to close the project. It targeted the preparation and holding of a final event for presenting the results from the project to the relevant stakeholders, the project owner, and the European Commission. The activity also comprised all actions to formally close the reporting for the project, including project progress, outcomes, outputs, expenses, billing, and evaluation of the results by the project beneficiary.

5. Results

Following the suggested structure of the self-reflection framework of Binder et al. [53] outlined in Section 3, we describe the product and process-related effects concerning outputs, impacts, and outcomes in the following.

5.1. Product-Related Effects

Several reports were provided through the project’s six main activity phases, capturing the core results of all actions during each phase. Activity A1 provided a report on implementing management and communication structures and processes between the service providers, the beneficiary (BMKÖS), and the EC.

Activity A2 delivered a report on a concept for an agent-based simulation aimed at supporting decision-making regarding sports funding in Austria in an evidence-based way. Furthermore, the Austrian mass sports funding system was conceptualized as a geographically embedded simulation of citizens accessing sports clubs and facilities in their geographic proximity. In addition, a systematic strategy was developed to collect, scrape, and acquire the data necessary to represent the identified agents in sufficient detail and describe how we process these data to realize the model. Moreover, a comprehensive report on the stakeholder landscape was created, comprising all relevant key actors within sports funding in Austria, including their personae, responsibilities, areas of work, and interdependencies. This vast landscape was then used to create the initial concept for the agent-based simulation and to target a suitable subset of key stakeholders for the joint workshops on system modeling.

Following activity A2, activity A3 developed a suitable process description for stakeholder management concerning the system modeling activities. After that, interviews, workshops, and expert roundtables were conducted to extract key stakeholders’ inherent system and process knowledge within the sports domain. The extracted knowledge and system components were further used during the design and implementation of the agent-based modeling prototype.

Activity A4 provided comprehensive documentation of the agent-based model and how to operationalize the prototype. This model consists of three types of agents (sports clubs, population agents, funders) and their interactions, both of which were identified as being necessary to represent the Austrian sports funding system during the expert interactions. A user manual of the prototypical agent-based model accompanied the model. This manual described the interaction with the visual component of the model so that non-technical experts of the ministry can use it. Furthermore, a data-driven visualization of the status quo of the current data landscape was designed and implemented. Regarding software, the prototype, including a gamification visualization of the sports funding system to better communicate to non-experts how changes in the funding structure impact the underlying system, was created.

Concerning activity A5, a report was provisioned about the readiness for upscaling within BMKÖS. Here, the extent to which individual components of the agent-based model can be calibrated to real-world data was described. The readiness was assessed by categorizing the extent to which relevant real-world evidence for each vital model element (agents and their interactions) is available. Based on these model element readiness levels, an overall readiness level of the Austrian sports funding system to deploy ABMs or other quantitative decision support systems could be defined. In addition, a user-centric implementation guide was created, including a reflective evaluation of the original transdisciplinary process framework and a newly designed and modified version based on the lessons learned (see Section 6).

Furthermore, required skills, training, and competencies were identified along the policy cycle for implementing agent-based modeling for evidence-based policymaking (see Section 6). Moreover, supporting recommendations for capacity-building actions were delivered, paired with recommendations concerning the current legal landscape, i.e., the BSFG 2017. Lastly, suggestions to improve the readiness of the Austrian sports funding system to deploy ABMs for decision support purposes have been provided. These recommendations include, e.g., Austrian-specific suggestions following the OECD Digital Government Policy Framework and the European Interoperability Framework.

Finally, in activity A6, a position paper in the form of a white paper toward “Evidence-based policymaking through agent-based modeling to improve the impact of funding mechanisms for sports grants on the example of youth sports in Austria” was prepared. Concerning dissemination and outreach, the project was presented during different events, such as the European Forum Alpbach 2021 (24 August–3 September 2021), the 1st Global Transdisciplinary Conference (27–29 September 2021), the presentation of the results of the sports section of BMKÖS (22 November, 2021), as well as discussion rounds concerning the results in several sub-sounding board meetings (data management, implementation, and knowledge integration) from November 2021 to February 2022, with stakeholders from academia as well as the public sector.

5.2. Knowledge Production Cycle within the Project

During these activities, the project underwent different intensity levels concerning knowledge production. Based on the suggestion of [60], we assessed the intensity level of knowledge creation, originating from different perspectives, i.e., disciplinary, interdisciplinary, and transdisciplinary, along with the execution of the individual activities (see Figure 4).

During the initial planning phase, not only were different scientific disciplines such as complexity science, computer science, and administrative science involved, but the public sector initiated the project’s initial concept for realizing the project, i.e., BMKÖS. However, as we discovered later in the project, a broader onboarding and consultation process might have been beneficial to avoid pitfalls that materialized later on in the project (Section 6.1.1).

Similar observations could be made during the research of existing meta-studies in the domain of health and sports, especially regarding the need for a more intense inclusion of sports sciences. In hindsight, it would have also been beneficial to complement the disciplinary and interdisciplinary analysis of previous research already with in-depth interviews of practitioners and domain experts, e.g., from the sports associations or clubs (see Section 6.1.2).

During the system modeling phase, while the level of transdisciplinary knowledge creation was high due to the joint work of researchers, practitioners, and domain experts, the level of intensity could have been kept more elevated for a more extended period if the experts had been held in the transformational loop. For example, from the model to the actual ABM, including multiple reflection rounds and hands-on experiences, would have also significantly raised the demand for necessary resources (see Section 6.1.3 and Section 6.1.4).

When it came to the work concerning up-scaling and recommendations for the implementation and operationalization of the ABM, paired with suggestions for skills and competence development, the interaction between the different disciplines was relatively high. Additionally, the involvement of the three sounding boards (implementation, data management, and knowledge integration) in reflecting upon the results from their respective perspectives significantly improved the quality of the recommendations in terms of formulation and scope. That being said, it would have been interesting to observe to what degree the continuous involvement of the sounding boards during all phases might have changed the course of the project. Again, this is a resource issue primarily, but also an organizational and political one, as partitions and experts might feel cajoled by the boards (see Section 6.1.5).

5.3. Process-Related Effects

A frequently discussed theme during our project was data management and creating a wide-ranging database. The participation of civil society and the use of data by civil society is not explicitly considered in this paper, as the project’s focus was the use of data by scientific organizations and public sector units. Data are the basis for learning and producing evidence about programs and policies and shall be available in the right quality to provide the basis for evidence-based policymaking, implying that data can be used without legal barriers. The public sector is a data steward and data owner of a vast data treasure, which can improve the quality of decisions and give researchers insights into different questions, which can function as a guideline for decision-makers.

Status Quo. The government is crucial in opening data treasures for science and civil society. In the past, many scientific institutions have complained about the data management of state institutions. Especially, the so-called transparency portal in Austria has been criticized as very ineffective. The transparency portal (https://transparenzportal.gv.at, accessed on 30 November 2022) is a public information service that gives citizens and legal entities a general overview of the cash flow financed by the public sector quickly and free of charge.

Nevertheless, the transparency portal was identified during our project as ineffective. Additionally, we faced the need for more data due to the regulations for receivers of sports subsidies. Institutions that grant (sports) subsidies currently need to request sufficient data to analyze the subsidy’s effect (achievement of specific goals), and grant receivers must comply with specific formal requirements. Still, they do not have to transfer data about the performance the receivers achieve with the subsidies, which would be essential to use the taxpayer’s money most effectively.

Data Principles. When managing data, key principles like trustworthiness; responsibility; availability and accessibility of data, open data, interoperability, exchange of data, the robustness of data, up-to-date data infrastructure, standardization of data, readability of data (for AI algorithms), and the possible solutions data can provide for occurring challenges shall be considered. Scientists demand that the government engage more in open data so the public and science can access non-sensitive governmental data in usable formats. The data flows from the highest level of administration—the federal state—to the provincial administrative level, and the administrative level of the municipality is not transparent. It is vital that it is made easier for the public and science to analyze what government data are collected, stored, and accessible.

In our project, accessing data was quite challenging; in many cases, essential data were unavailable. Furthermore, interoperability is critical to the exchange of data. The central question is which standards will be used to exchange data. As diverse data systems are used, an essential task is to consolidate all systems. Another option would be to create new data systems from scratch. In terms of trustworthiness, the protection of sensitive data is vital. Here, a transparent system shall be established so that the safety of the data is not affected but researchers can still use the data effectively. In terms of standardization, it is essential to point out that the data provided shall be prepared in a way so that it can be analyzed automatically by software solutions and that different data sources are connected.

Another challenge is the coordination of different ministries. Currently, various ministries gather and manage their own data in their own data structures (silos). What is necessary is an exchange across different ministries, hence bridging the silos. Apart from this coordination, all ministries are responsible for their data. From a technical perspective, it also became clear that investments shall be made into modernizing the data infrastructure (for example, we received some data in a pdf format).

Another significant point we came across during the project was the responsibility for data. In many instances, it was unclear which institution or person within this particular institution was responsible for the data. Hence, organizations shall appoint Chief Digital Officers (CDOs) accountable for managing data and facilitating access to the data. Furthermore, leaders can support institutions’ paths of becoming insight creators using big data by analyzing the necessities to apply big data analysis and communicate to policymakers and by managing data risks like confidentiality of data.

Room for improvement. Due to the significant differences in data storage across different ministries, an open data best practice/guidance shall be provided on managing data. Additionally, the state shall ask the receiver of subsidies to deliver the data needed for measuring effectiveness. These data requirements shall be part of the grant agreement in which the state grants subsidies. Furthermore, the secondary use of data shall be considered, and data shall be coded so scientific institutions and civil society can use the data.

Moreover, the data coded shall be made available in a particular data quality. Some criteria for data quality are granularity, data-sharing, trust, and up-to-datedness of data. Furthermore, the possibility of randomized measures shall be enabled, especially for experimental trials.

One particular aspect discussed within the public administration after the project ended relates to new ways of knowledge transfer. It became apparent during the different phases and involvement of the various stakeholders that much knowledge already exists yet might either lie dormant as it is not exchanged due to cultural barriers or because there are no means in place to actually institutionalize and also, to a certain degree professionalize the exchange of knowledge. Hence, several ideas for designing future ways of knowledge transfer were born:

• Science Exchange Program: The concept of dispersed expertise within the administration includes the designation of scientific experts (for example, a data scientist) for a limited time of their science career track into the administration, where they learn the requirements of the other system and transfers the problem framing of an administrative problem into the frame of their specific science expertise. One disadvantage is that a single “foreigner” might be kept on the outside or „swallowed“ by the leading organization so that they cannot perform their „bridging task“ effectively;
• Expert teams within the administration: The project results suggest that small expert teams can be founded within the administration to keep a critical mass of expertise. Teams for data science exist in different ministries for specific purposes. It seems that these groups develop a shared understanding of the usage of scientific methods in their respective fields of administration; they remain highly focused and isolated and are not very apt to respond to new concerns or new scientific developments. These assumptions should be further researched and tested;
• General awareness through training: To foster the acceptance of scientific advice and the potential use of scientific methods applied to governmental issues, broad training in the specific methods serves well. A comprehensive understanding of scientific methods through the administration enables public officers to recognize the potential of new methods and to frame the issue appropriately. This momentum fosters the use of scientific advice since the prerequisites for applying a specific scientific approach are met. For example, the availability of data and access to detailed information can be evaluated and secured upfront. An example of this integration of science within the policy process and the optimization of administration is the Austrian Federal Academy of Public Administration (VAB) (https://www.oeffentlicherdienst.gv.at/vab/seminarprogramm/index.html, accessed on 30 November 2022). Internationally the Intergovernmental Panel on Climate Change (IPCC) (https://www.ipcc.ch/, accessed on 30 November 2022) is frequently mentioned as an example of a boundary institution. It provides guidelines for policy decisions through balancing science policy interests.

6. Framing the Lessons Learned

As the empirical evaluation demonstrates, the initial setup plays a critical role. It needs to be considered an integrated part of the entire process, as it provides the foundations and specifications necessary to develop solutions to the specified problem in the given domain. We now present the lessons learned from the transdisciplinary modeling process, depicting its different phases and their crucial decision points before we deduct concrete recommendations for training in the context of knowledge integration. In doing so, we perform the fourth step of our conceptual action-based transdisciplinary approach, which, according to Herr and Anderson [52]—as discussed above in the conceptual approach—contains the reflection on the results to foster and improve development and associated actions.

6.1. Re-Designed Transdisciplinary Modeling Process

The newly designed process framework is presented (see Figure 5). This framework does not replace but complements the steps taken within the project, keeping in mind lessons learned and pointing out critical decision points for the individual phases.

The overall framework consists of five main phases, i.e., Phase 0—Design and Onboarding; Phase 1—Stakeholder Landscape and Transformative Mapping; Phase 2—Capacities, Ideas, and Chances; Phase 3—Developing Transformative Strategy; and Phase 4—Competencies, Processes, and Up-scaling. We are going to present these steps now in more detail.

6.1.1. Phase 0—Design and Onboarding

This phase is one of the most crucial, as it provides the direction of the overall project. In the first sub-phase 0A, “problem identification,” the problem space in which the project will operate is defined. It is imperative to explicitly focus on the problem definition originating from the perspective of the stakeholders, not only the view of the project owners. There exist many ways to consult stakeholders. A common approach is a combination of a systematic review of the literature [61], including scientific literature and other relevant documents such as policy reports. This systematic analysis is accompanied by focus groups and interviews [62] with domain experts in the identified relevant areas.

The second sub-phase, 0B, “prioritization,” takes over all identified problem fields and starts prioritizing them. This prioritization is crucial, as the number of problems from the initial screening of the general problem domain is usually much too high. Of course, the selection of the problem can be made top-down. However, it is strongly suggested to use a more participatory and transparent approach in ranking the issues in terms of severity. A potential solution to this can be a balanced scorecard [63]. While the attributes within the scorecard to describe the problems can be selected on any suitable basis, it is suggested to build it around the 5W2H approach [64], as seen in

Table 1. Problem definition matrix per individual problem using the 5W2H approach.

What	Why	Who (Type)	Where	When	How	How Much
Problem 1
Problem 2
…
Problem n

.

Depending on the domain applied, additional aspects can be introduced, such as the number of affected people, the amount of existing funding, or political priority. In addition, it is possible to raise weights to reflect the importance of the selected dimension for the group that is rating the problems at hand. Creating problem-specific personas [53] can increase involvement and commitment to make the situation more tangible.

The third sub-phase 0C, “guiding question”, condenses the prioritization from the previous phase into a concrete and unambiguous focus concerning the core issue that should be addressed during the entire process. This guiding question needs to be so clear that it can be described within a short paragraph, ideally, within one sentence. If there are multiple problems, each should be distinguishable via its dedicated guiding question.

In the fourth and last sub-phase 0D, “ownership”, the idea is to again reflect with key stakeholders on the provided prioritization and guiding question. Doing so makes it possible to gather additional information, such as potential conflicts that might later occur during the other phases of the process. In addition, this sub-phase allows all stakeholders to reflect and formulate the desired outcomes in their own words, which creates identification and ownership. In order to support the exchange throughout the different domains, it is suggested to have groups of specific domain experts work and reflect on the current setting together and then distribute it to the other groups along the Jigsaw method [65] to mix the expertise, opinion, and reflections.

6.1.2. Phase 1—Stakeholder Landscape and Transformative Mapping

This phase is initiated after the core problem to be addressed is identified and formulated via a concrete guiding question and commitment. In addition, ownership by the involved stakeholders has been ensured. In the first sub-phase, “stakeholder landscape”, the identified and delimited problem domain is analyzed deeper to assess particular functions, interactions, dependencies, and relationships of the involved stakeholders and associated entities. This analysis can be performed via classical desk research, including scientific literature, policy documents, tech reports, and laws.

The second sub-phase 1B, “lessons learned”, covers a follow-up of the identified roles and entities from the sub-phase before. Detailed information can be attached to the personas created in sub-phase 0B. Once this enrichment step has been finalized, deep-dive interviews are conducted with representative stakeholders for each relevant organization and essential individuals.

The third sub-phase 1C, “meta-review”, is intended to identify supplement information from sub-phase 1B with in-depth knowledge from scientific studies explicitly targeted to the identified problem domain and focus. In addition, statistical data and policy reports can supplement the information base, which helps to identify already key correlations and core concepts that can be used as a starting base in the modeling process (see sub-phase 2B). In addition, existing data sets, e.g., statistics or even original data from scientific studies, can be used to develop, support, and evaluate the model in the later stage (see phase 3).

6.1.3. Phase 2—Capacities, Ideas, and Changes

Within the first sub-phase 2A, “stakeholder management”, co-leadership and co-ownership play a crucial role, as introduced during sub-phase 0D. Depending on the problem domain and the concretely selected guiding question (see sub-phase 0C), a designated key stakeholder representative should be approached to jointly prepare the interaction and further onboarding phase for the modeling. While this might have some limited side-effects that some stakeholders might not want to participate due to the co-leader for political or other reasons, having a domain expert alongside the scientific partner will significantly help to lower the entrance barrier. Furthermore, having a partner who can use the proper wording and domain vocabulary helps reduce ambiguity and misunderstandings. However, this stakeholder management does not only focus on key domain stakeholders but also the involved scientists and researchers. While interdisciplinarity is vital to jointly working toward a solution to a given problem, the disciplines also introduce different opinions, methods (e.g., qualitative vs. quantitative methods), norms, and values. These need to be taken into consideration and need to be aligned toward the joint goal. Hence, everybody should be aware of the shared mission, purpose, and the agreed way to achieve this goal.

The next sub-phase, 2B, “system modeling”, then engages with the key stakeholders to iteratively build sub-systems and interconnect them to a “bigger picture”. The approach selected in the project of this paper is suitable for creating this system understanding, based on inputs of sub-phases 1B and 1C, concerning lessons learned from the institutional memory and key concept correlations from scientific studies. During the process, individual and group activities alternate. Thus, using digital tools, e.g., miro (https://miro.com/, accessed on 30 November 2022) and a mental modeler [66], has proven suitable. However, the initial starting point with these software tools is not always intuitive for all participants. Thus, a “gamified” round to get comfortable with the tool can help to reduce frustration.

Another important aspect is to realize that the modeling, even for a specific sub-system, might need several rounds (over several days or weeks), as the stakeholders have to reflect on the generated common knowledge. The energy curve of participants falls quickly in challenging tasks, primarily when the workshop is held online or in a hybrid format. Furthermore, the modeling activity should always be accompanied by experts who later transfer the model into the ABM. By doing so, ambiguity about used concepts, terms, and modeled relations can be clarified on the spot and appropriately documented.

The third sub-phase 2C, “identification of data sources”, is an integral part of the modeling process of the ABM, as the structure of the model, together with its strengths and limitations, are defined by the availability of data to calibrate and validate the underlying assumptions (see also sub-phase 1C). Often, stakeholders are unaware that their daily business information might benefit the task at hand. Likewise, stakeholders often have no concrete idea of what is meant when the developers talk about “data”. The introduction of examples might help to convey better what is required and what can accelerate the modeling and development of the simulation, as well as to avoid investing resources into building a model that becomes impossible to validate.

6.1.4. Phase 3—Developing Transformative Strategy

In phase 3, the developed model and all identified data sources are then prepared to be incorporated into the agent-based model. This integration is an iterative process in which the ABM is refined to match the available data. New datasets are continually collected and assessed for sufficient data quality to inform the modeling process further. What is described as the sub-phases 3A–C (data preparation, ABM modeling, model verification) should therefore be considered an iterative process in which several cycles through these sub-phases will most likely be required before the final model is derived. In particular, in sub-phase 3A, “data preparation”, the pipeline from the initial data to the point where it is used within the simulation must be designed and implemented. The so-called “Extract-Transform-Load (ETL)” [67] requires not only a suitable transformation of data but—and this is most important—also an assessment concerning the overall data quality. While there already exists a plethora of data quality metrics [68], it might also make sense to define individual criteria in the concrete context of the problem domain. This definition is necessary to decide to what extent existing data can be used within the simulation. At the same time, this assessment can help identify weak spots in current data collection and refinement processes.

Once the ETL pipeline is implemented and relevant data are identified, sub-phase 3B, “agent-based modeling,” starts the actual development of the agent-based simulation. Here, based on the environment the simulation should be embedded into, requirement engineering and proper documentation of the implementation via, e.g., UML [69] or other descriptive approaches, directly support the integration process into existing infrastructure.

After the development has been concluded, sub-phase 3C, “model verification”, targets the overall evaluation of to what degree the current simulation is accurate. Historical data can be used to perform this assessment, as it provides the actual outcomes, which can then be compared to the simulation results. That being said, this step becomes more complex if historical data are not available. The results may only be interpreted based on conclusions and results known by scientific studies and other best practices or institutional memory. While this can still work, the accuracy level is lower than in the first case.

6.1.5. Phase 4—Competencies, Processes, and Up-Scaling

Once the model verification has been completed, it is possible to enter the last phase of the overall process model. In this phase, three main goals are included: the general feedback of the entire process model, the joint creation of decisions based on the generated model, and the identification of required competencies for realization.

In the first sub-phase 4A, “process feedback”, all stakeholders of the process come together to reflect on the overall setup process and its execution. The idea is to analyze each phase and sub-phase to identify methodological weak spots and barriers that were identified during the traversal of the process. Furthermore, identified best practices should be gathered, formalized, and documented so they can be reused in future iterations and shared with other stakeholders in related or utterly different problem domains. Method-wise, various options or combinations are possible, including interviews, focus groups, or surveys. The critical aspect is that the retrieved patterns can be easily found and freely accessed.

In the second sub-phase 4B, “policy co-creation”, the focus is on using the agent-based model. Together with the well-described limitations (see sub-phase 3C), several what-if scenarios can be created and discussed with the stakeholders of the problem domain. Depending on the intended style, solutions can be designed based on different scenarios, or several mixed groups (Jigsaw method) work on different scenarios and then exchange between them. Suitable paths for development can be found, e.g., via the application of back-casting [70] approaches. Once a set of commonly agreed directions is derived, these can be again checked against the original problem focus and its priorities in the last step.

In the third sub-phase, 4C, “competencies and skill”, the identified issues and potential solutions in scenarios and inherent steps are reflected if the participating stakeholders and organizations can execute them. If not, the opening gaps concerning competencies and skills need to be formalized and collected. Depending on the overall timeframe of the project, it might be necessary to address some skills via short-term solutions, such as skill workshops or seminars. On the other side of the spectrum, we have the more extensive structural changes, which require a well-composed set of skills reflected in holistic training programs or curricula. Another important aspect at this point is also to decide whether the training will happen externally or if capacities for in-house training and train-the-trainer activities should be established to ensure multiplication within the ministries and related organizations.

6.2. Recommendations for Training in the Context of Knowledge Integration

During the project, the complexity of using disruptive technologies, i.e., agent-based modeling for evidence-based policymaking, in the context of sports grants and beyond became more than obvious. To cope with this high level of complexity, more focus on the training and capacity of concrete technology is needed. Still, planning capacity building from a holistic point of view is strategically crucial. Therefore, our step 4 reflection would not be complete without addressing the training areas and capacity-building requirements from the policy cycle [70] and governmental decision-making progress perspective. The policy cycle and critical aspects of sustainable knowledge integration will be described in detail in the remainder of this section, along with the explanations of Hitziger et al. [71], complemented by specific training recommendations derived from the lessons learned from this paper’s project.

Knowledge integration plays a crucial role in this context, specifically in the three main stages of the cycle, i.e., policy formulation, implementation, and evaluation. The policy formation phase comprises two key aspects: thinking and planning. In consideration, the inclusive design process becomes a pivotal element. Including multiple perspectives and different and complementary understandings of the system characteristics enable a 360-degree view of the problem domain. During the translational process of the environment, its problems, and potential solutions, prioritization becomes possible to identify solutions with the highest leverage potential. In this sense, design thinking and design research training can contribute significantly already during the early stage of designing policy actions.

Training Recommendations based on re-designed modeling process: educational training and courses in design thinking and design research (phase 0), particularly toward policy formation (phase 0 and phase 4), system modeling (phase 2), as well as in environments with a high level of heterogenous stakeholder groups (phases 0–4).

Concerning the second key aspect, i.e., planning, stakeholders’ early-on careful selection and involvement are essential. It needs to be decided which sectors and disciplines to involve, and at what stage of the planned action specific stakeholder groups are required for inputs or even for taking the lead. This decision requires high reflexivity and adaptiveness, especially when assessing one’s competencies, skills, and overall available resources. It is essential to understand, express, and formalize limitations in the current team constellation and how these might impact the envisioned actions and associated objectives/goals.

Training Recommendations based on re-designed modeling process: educational training and courses in agile paradigms for project management in the public sector [72] (phase 0–4, with particular emphasis on phase 0), as well as stakeholder engagement [73] (phase 0–4, with specific emphasis to phase 0 and 4), and training in problem-based learning, focused on setting up transdisciplinary projects and inherent processes (especially phase 0 and phase 2) in the domain of impact assessment and performance benchmarking (phase 4).

The following two key aspects are covered in the policy implementation phase, i.e., organization and working. As for the organizational aspect, team building and structure are critical points to introduce the required stability into the project. Understanding the level of mutual support and the existing relationships among the team members is essential. It is crucial to align every member to the overall objectives and key results (OKRs) [74] that should be achieved. In turn, these need to be aligned with each individual’s individual goals and actions. However, not only internal alignment is necessary but also external alignment (this includes not only stakeholders outside of the ministry but also stakeholders of other departments or sections) as well must be considered. Here, it is vital to set the right frequency of interaction, which might—in addition—be necessary to be adapted several times during the runtime of a project. Lastly, the existing knowledge needs to be identified and bridged to provide access and not reinforce existing silos within and outside the ministry. Here, it must be understood that knowledge can take various forms and, thus, requires a set of methods and approaches to tap into it, distill it, and share it across teams and organizational boundaries.

Enengel et al. [75] summarize the core knowledge categories and their logical positioning against each other as follows: Context-specific knowledge refers to case-specific knowledge, often bound by particular time and spatial aspects. Concerning this project, these are the Austrian specifics regarding sports funding and the system’s logical and organizational structure. The generalized knowledge should—in theory—be universally applicable. However, it needs to be translated into a specific context. This translation, in turn, can lead to missing gaps within the knowledge and potential conflict in terms of contradicting information. In the context of this project, this knowledge could be referred to normative aspects from legal aspects and organizational assumptions, which might differ from real contexts on federal state-level due to their autonomy and sovereignty in certain areas. Phenomenological knowledge often refers to local knowledge about a particular environment, social constructs, and conditions. In the context of this project, this would refer to knowledge of sports clubs, local trainers, or other active sports area organizations. Strategic knowledge addresses structural knowledge and knowledge about network key players, their agendas, relationships, and interactions. In the context of this paper’s project, this would refer to the exchange of the ministry, associated organizations, umbrella organizations, regional organizations, and sports clubs. The final two kinds of knowledge complement each other from a level of formalism. Experimental knowledge refers to implicit or tacit knowledge. This knowledge is seldom formalized and tells stories about the “freedom to move” within existing processes or lived shortcuts. Scientific knowledge, on the contrary, is highly standardized and is based on empirical evidence and theories. In the context of this project, empirical knowledge refers to, e.g., statistical data available in the sports domain. In contrast, tacit knowledge is covered by telling key stakeholders about daily business, non-public strategic priorities, or animosities between individuals and organizations.

Training Recommendations based on the re-designed modeling process: educational training and courses in co-design and co-creation (especially phase 4), as well as training in knowledge management (especially phase 0 and 4). Here, it is recommended to focus on overarching (ministry) strategies, taxonomies, and ontologies for semantic content and platform management.

The second key aspect in the policy implementation phase is working. This aspect includes, besides other criteria, the criterion of power distribution. Especially in environments with a high level of hierarchy and formalism, such as ministry and the public sector in general, decision power is well-defined. That said, it is only sometimes directly available in a project or innovation action when needed. Furthermore, different high-power levels can collide, e.g., through the argument of responsibility and jurisdiction concerning domain areas with fuzzy boundaries. In addition, these power balance issues are more likely to pop up between the ministry and state levels due to the paradigm of Federalism in Austria. Thus, leadership from an enabling perspective is crucial to properly support the team and empower its members to fulfill the key objectives they have been tasked with. In addition, leadership should also actively seek and address conflict resolution. Conflicts should be understood as learning opportunities, which, in turn, require a high level of reflection, as pointed out in the discussion before. As conflict resolution can be a timely process, resilience is vital.

Training Recommendations based on the re-designed modeling process: educational training and courses in management and leadership (phase 0); training in mediation, communication, conflict resolution; as well as systemic coaching and supervision (especially phase 0 and phase 2).

In the last phase of the policy cycle, i.e., evaluation, the two key aspects are sharing and learning. Sharing covers all aspects of information exchange. Here, it is essential to implement and frequently evaluate processes to ensure that each stakeholder receives the information required for proper decision-making and timely action and in a suitable presentation. This implementation and evaluation require a fitting infrastructure that supports these processes and foresees a way of sustaining the derived information. This process, in turn, enables a method of analysis and distillation via multiple perspectives and stakeholders to finally create knowledge toward an institutional memory. Yet, before information can be gathered, data are required. As we have seen during the runtime of this project, data availability and the willingness of data sharing are crucial aspects and provide frequent hurdles for policy actions. It thus requires a set of preconditions in the domain of data. These preconditions include, but are not limited to, knowledge of data quality, data standards, data semantics, linked data, open data, data processing, data access and storage, data modeling, data analytics, or data visualization. Yet, besides these aspects, data governance or data excellence [76] should be remembered and fostered by dedicated processes for data stewardship [77].

Training Recommendations based on re-designed modeling process: educational training and courses in information systems, data quality, data governance, data excellence, data standardization, and open and linked data (especially in phase 3).

The final aspect of the phase of evaluation in the policy cycle is the learning aspect. Here, individual learning meets team learning, which serves as a mediator and proxy toward the organization and, in the case of this project, toward the ministry. The lessons learned by individuals are translated into processes and routines, which in turn are embedded into the organization. This embedding applies not only to operational aspects but also to a code of conduct. Thus, the overall organizational culture and the learning experience build social capacity and cohesion, represent the knowledge obtained, and encapsulate solutions. These practices and blueprints have high value for exchange within the ministry and beyond its organizational and normative boundaries. This aspect is especially important in terms of upscaling. On the examples of agent-based modeling in mass sports, the best practices learned can be used to transfer proven processes to other sports areas and the agent-based modeling approach to other policy domains.

Training Recommendations based on re-designed modeling process: educational training and courses in team-based learning methodologies using real-world use-cases and a general problem-based learning approach (phase 0 and phase 4).

As seen throughout the journey of knowledge integration for agent-based modeling in the context of the policy cycle, the topic ABM, system modeling, and integration in the policy setting cannot be solved within one or a handful of courses. Instead, it becomes clear that a comprehensive educational curriculum is required in the domain of data-driven decision support for evidence-based policymaking. It is recommended to rethink the revealed requirements and properly institutionalize associated training and educational programs in a dedicated environment. This rethinking is necessary, as the before-mentioned real-world context and use cases should be kept as the fundamental core of education. By following this problem-based learning approach, the training can follow the continuous development of requirements, e.g., data, modeling, and agent-based modeling. By doing so, the ministry can, over time, build the required capacity for embedding ABMs in their organization and develop cases that can be shared as knowledge items and lessons learned with other public sector institutions.

7. Discussion

This paper used an action-based and transdisciplinary conceptual framework to report on an EU project in Austria that aimed to assess the impact of policy steering in sports funding and to understand better the effect of sports grants on young people’s health. We now return to our research question on how to improve the added value of applied transdisciplinarity in sports policy and draw inferences for the three types of theoretical literature our paper contributes to.

First, our research problematized once again that establishing a clear causal relationship between public funding for sports organizations and youth health promotion is still difficult, primarily due to the lack of data. Even if an ABM model has been developed to assess the individual impact of institutional funding, the scarcity and gaps in the available data provide a challenge that is difficult to overcome. Transdisciplinary approaches that offer a collaboration environment for researchers and stakeholders from practice are necessary but not sufficient to harness the essential collaboration effects in terms of data availability.

Secondly, our project confirmed the suitability of ABM models while showcasing the necessity for simplification and, thus, close collaboration between research and practice. Our research insights confirm McComb and Jablokow’s [41] assessment that the literature gap is not because of a lack of transdisciplinary collaboration opportunities. However, these transdisciplinary projects also need to pay particular attention and allocate time to build bridges and common understanding between researchers from different disciplines and practice stakeholders.

Thirdly, our paper refines our understanding of the use of transdisciplinarity in researching sports funding and its effect on youth health. The interaction model and its decision points are exposed to several challenges that transdisciplinary processes must engage with, also because the way they are handled might highly impact the successful progression of the project. Therefore, we have identified critical questions along the model phases, which we present in

Table 2. Overview of key questions for successful transdisciplinarity.

Phase 0—Design and Onboarding
	Do stakeholders have heterogenous expectations, or is the system too complex?
	How can the right balance be found between ownership and consensus of stakeholders?
Phase 1—Stakeholder Landscape and Transformative Mapping
	How do you balance learning from the past and being stuck in past experiences? How to effectively identify and build on best practices?
Phase 2—Stakeholder Management
	How much iteration is possible and necessary?
	What do partners or stakeholders need to know, when, and how?
Phase 3—Developing Transformative Strategy
	How do you find the balance between predefining modeling phase and accounting for the required flexibility?
	When can the developed model be verified?
Phase 4—Competencies, Processes, and Up-scaling
	Why is it essential to focus not only on content but also processes?

. These questions provide necessary questions that project teams should consider when planning future transdisciplinary research projects, which we discuss in more detail in the following.

7.1. Phase 0—Design and Onboarding

Do stakeholders have heterogenous expectations, or is the system too complex?

Investing time and resources in the design and onboarding phase is indispensable for transdisciplinary processes. Project partners come to the project with heterogenous expectations, and the complexity of the broader field is likely to complicate further the finding of clear priorities and parsimonious questions to pursue. It is vital here that all involved partners display a degree of openness to collaboration and question existing structures. The collaboration of distinct types of partners will sometimes feel uncomfortable. Therefore, involved stakeholders must be willing to invest in relationships that will not only need their involvement but often also reconsider their daily work identity or work processes.

How can the right balance be found between ownership and consensus of stakeholders?

Despite the added value from the heterogenous backgrounds of the project partners, the project will need to carefully balance ownership by project participants and the efficient finding of consensus. Project partners need to identify themselves with the project sufficiently, but they also need conscious decisions about trade-offs and the most appropriate balance for the project team. Responsibilities for these decisions need to be communicated, and these decisions need to be made explicitly and consciously to ensure the smooth and targeted running of the project.

7.2. Phase 1—Stakeholder Landscape and Transformative Mapping

How do you balance learning from the past and being stuck in past experiences? How to effectively identify and build on best practices?

In this second phase, identifying and approaching stakeholders is key. Again, the right balance needs to be navigated in getting access and involvement from all crucial stakeholders while keeping the necessary critical distance for reflective analysis. It should be an essential endeavor for any transdisciplinary project to learn from the past and past experiences of stakeholders but sticking too much with the past can also hinder innovation and new perspectives. A further complication here is that some best practices are implicit knowledge, where stakeholders need to be aware of their insights’ added value, and this tacit knowledge needs to be carefully brought to the surface.

What is the right level of granularity?

In mapping and engaging stakeholders, it is helpful to define at the initial stages of the project what level of empirical granularity is needed for answering the jointly defined research questions. Most project partners are likely to have similar expectations here, but the engagement with stakeholders should be adjusted to the kind of empirical insights and data the project needs.

7.3. Phase 2—Stakeholder Management

How much iteration is possible and necessary?

In this second phase of stakeholder management, the main challenge is that the focus of inquiry needs to be on the subsystems, which only becomes apparent throughout the research and stakeholder management process. The project team, therefore, needs to have a preliminary plan. However, it also needs to keep flexibility and space for iterations in the stakeholder identification and engagement process to adjust research processes after better coming to terms with the systematic realities in the field. The number of iterations that are possible and necessary needs to be decided on a case-to-case basis, and it helps if the project team right from the outset clearly and explicitly communicates the mode of decision-making on when sufficient iterations have been repeated and when the team feels it is ready to move on to the next stage.

What do partners or stakeholders need to know, when, and how?

Any successful transdisciplinary project needs to define reliable communication structures between the project partners and between the project partners and the other stakeholders. Again, it will facilitate the smooth running of the project to clearly define what or how much stakeholders and project partners need to know and when. The project team should ensure the highest amount of transparency, but it also needs to establish decision-making processes. Everyone should be involved in the project, but only some people always need to be included in making the necessary decisions. These governance structures should be defined and agreed upon by the project team and carefully communicated to all stakeholders involved.

7.4. Phase 3—Developing Transformative Strategy

How do you find the balance between predefining modeling phase and accounting for the required flexibility?

The goal of phase 3 is to develop an agent-based model based on all identified data sources. The challenge here is that the model development needs to adjust to the available data and data availability. Still, the latter will only become apparent in phase 2 of the transdisciplinary project. The agent-based modeling process thus needs to be planned and given some structure. Still, it also needs to include sufficient flexibility to adjust the data realities discovered in phase 2 of the transdisciplinary project. The specific decisions in this stage are also highly dependent on how much generic and domain-specific data quality can be assessed before the start of the project or can only once be assessed when the stakeholders provide the required information while the project is conducted. Here it is also recommendable for the project team to plan early on how to close data holes actively. This activity should best be factored into the project plan right from the beginning, even if details about the exact necessities might still be vague.

When can the developed model be verified?

Given the complexity of the modeling process and the multiple activities necessary to account for in a transdisciplinary project, it is highly likely that the model verification is only possible over time, which might also mean after the formal end of the project. To avoid unrealistic expectations, the project team must manage stakeholder expectations accordingly and communicate clearly what the involved stakeholders can expect as the project’s output.

7.5. Phase 4—Competencies, Processes, and Up-Scaling

Why is it essential to focus not only on content but also processes?

The last phase of our transdisciplinary model focuses on joint assessment and reflection. In this critical self-reflection phase, the project members must focus on the project content and assess and reflect on the project processes. What needs adjustment? What could be done better/differently/more effectively? Such reflection processes are time-intense, and even more importantly, they need ownership from all stakeholders to work. Therefore, it is, on the one hand, highly recommendable that such reflection moments are factored into the project timetable right from the start.

On the other hand, it is also essential that the results of these process reflections can be implemented within the project without endangering the overall project timetable. For example, suppose the result of a reflection phase is that it would be necessary to re-do a whole phase again. In that case, the project plan should include sufficient flexibility in time/resources to allow for such an iteration.

How to ensure stakeholders account for their buy-in in terms of skills development?

Collaborating in a transdisciplinary project does not only mean that one shares experiences, expertise, and data, but it also needs the development of competencies and skills of stakeholders. Again, communication needs to manage expectations carefully: stakeholders are involved in most projects and must be willing to adjust their skills/competencies in the process.

7.6. Future Directions

Next to the questions systematically presented before that any transdisciplinary project will need to consider, our project results also guide relevant and promising future research avenues. The current framework only considers the Austrian national context in its data collection. To what degree could the framework incorporate ecosystem analytics and data collection in the European domain?

When discussing cross-border and potentially cross-domain integrations, several additional challenges arise that need to be reflected in the overall process cycle. First, while already challenging on a national level concerning the alignment of data governance strategies—if they already exist—taking this to the European level represents a huge step. This step forward resembles very much the challenges given in the context of the once-only principle. These challenges start from infrastructure compatibility, over the definition of roles and responsibilities, up to semantic foundations for metadata mapping and data fusion processes. One strategic solution that the European Commission currently pursues is the establishment of data spaces in different domains [70]. An exciting aspect of the extension of the provided model could be the interlinking of existing data spaces, such as the health data space [78] combined with data from the sports sector. This interlinkage would provide stimulating opportunities, as, for example, regional cooperation does not have to stop at political borders concerning the domain of sports and health promotion [79,80]. This cross-border approach, both from an organizational point of view as well as from a geopolitical point of view, would also foster activities on a larger scale, i.e., the combination of data and sports to engage and develop activities around the increase of inclusion, following the overarching agenda of the sustainable development goals [2,3,81,82,83].

Structural and cultural constraints of organizations often slip into projects and hinder openness and innovation. In what way could transdisciplinary projects circumvent these issues?

During the project, we experienced the stakeholders were often torn between their institutional tangible and intangible rules and their own opinions, feelings, and attitudes. Hence, it is essential to account for structure and agency when designing the surroundings and embedding transdisciplinary projects [80]. One solution to ease this situation might be a change of scenery for the participants. However, this does not imply only leaving their familiar surroundings for a workshop or so but putting the overall project into a completely different environment. While this comes with its own challenges, such as financial or time constraints, it has been proven that this change can significantly impact the overall project results and their impact. One way of creating such a different environment is the creation of so-called real-world labs, which in particular, have successfully demonstrated their benefits to transdisciplinary projects in various applications and domains [84,85]. Yet, one of the significant hurdles remains here, i.e., who is officially “running the lab”. As with many things in the policy context, this is often a political decision, which can be a deal breaker, no matter how well-designed the underlying project is.

8. Conclusions

Promoting health and physical activity throughout society is increasingly important considering the shifting demographic curve toward longer lifespans. The provision of suitable offerings and services, including proper infrastructure, is imperative to improve the situation sustainably. In this context, public funding of sports plays an important role. That being said, transparency, efficiency in funding allocation, and measuring the achieved impact vs. the desired effects are essential for decision-makers. While technology and data can support policymaking in this regard, they alone cannot guarantee inclusive, long-term decisions, as well as acceptance and ownership of key stakeholders involved. Therefore, science and practice have to work together—on eye level—to achieve this ambitious goal; transdisciplinary projects are one possible solution to this challenge.

In this paper, we have analyzed a transdisciplinary EU project in Austria, targeting agent-based modeling for evidence-based policymaking in federal sports grants to improve the health and well-being of children and youth. We discussed the initial background of the project, its structure, involved stakeholders, as well as its inherent activities and outcomes. We also analyzed the different types of knowledge creation levels compared to disciplinary, interdisciplinary, and transdisciplinary interaction.

We used a framework of action-based transdisciplinarity to synthesize the results of this reflection into a revised process framework based on the initially employed framework of the project. We accompanied this modified version with specific lessons learned and decision points for each phase included. Complementary to this, we have derived recommendations for training in the context of knowledge integration. Furthermore, we also reflected critically on our revised framework’s current limitations and discussed options for future work.

Overall, the combination of transdisciplinarity, policy, and sports, supported by cutting-edge technologies such as ABMs, has proven that it can achieve significantly better results than a pure disciplinary approach and generate positive spill-over effects. These effects in cross-cutting areas, such as data management, knowledge integration, and implementation processes for new technologies, will ultimately enable future sustainable digital transformation in the public sector.