NASA’s Early Adopter Program Links Satellite Data to Decision Making

Abstract

As the cost of satellite missions grow, government agencies are working to increase the relevance and usefulness of the Earth science data that they produce. The US National Aeronautics and Space Administration’s (NASA) early adopter program seeks to formalize partnerships with users of data early within the satellite planning process to ensure the greatest value of the mission. Scientists who are designing the satellite observing system, those who are developing mission data products, and the community who will use them need to work together to determine how satellite data can better inform decision making. NASA is working to initiate engagement with data users much earlier in the satellite program life so that the user needs can be prioritized, continue to engage with users throughout the development cycle, and then include analysis on societal impact after nominal mission life. The Early Adopter Program allows for broad and early engagement with decision makers by encouraging individuals and institutions with clear uses of data to engage with the mission. Early adopters facilitate feedback on mission products during pre-launch and accelerate the utilization of these products post-launch. This paper provides background and detail on how NASA is working to engage with early adopters, describes the partnership between early adopters and NASA missions, and connects the program to improved societal impacts of its satellite missions.

1. Introduction

The US National Aeronautics and Space Administration’s (NASA) Earth Science Division (ESD) delivers a wide suite of datasets derived from a sophisticated national and international infrastructure of space-based observing systems, scientific research, and applications. A central part of this program is the multiple use nature of all its investments, where observations are designed to serve curiosity-based science, applications-oriented science, and societal benefits at the same time [1].

The applied sciences program within NASA’s Earth Science Division promotes efforts to discover and demonstrate innovative and practical uses of Earth observations [2]. The program funds applied science research and applications projects to enable near-term uses of Earth observations, formulate new applications, improve practitioners’ decision-making, and transfer of the applications. Application projects are carried out in partnership with public- and private-sector organizations to achieve sustained use and sustained benefits from the Earth observations. As part of this program, NASA has focused on developing its knowledge of future applications and diversified engagement with users for upcoming satellites through the concept of an early adopter (EA) [3,4].

The EA program provides a structured way to foster unfunded partnerships between new NASA Earth instrument projects before they launch and external organizations who are motivated to work with them. Each EA project has a principal investigator (PI) who is typically a scientist, manager, or individual in an institution who has a direct or clearly defined use for the new or emerging satellite data observation. After a new mission launches, early adopters integrate the new satellite data into an application that supports improved decision making with societal relevance and reports the impacts and lessons learned back the mission and to NASA.

The EA program allows for direct mission engagement with interested institutions and individuals throughout the technology development process [4]. The EA program is flexible, extensible, and provides useful and sustained engagement to mission scientists and expertise, as well as to decision makers who might use the satellite data. Although partnerships are by no means cost-free, they can be an extremely effective way to engage with a wide variety of institutions who may value working with NASA but typically cannot compete for direct funded grants [5]. At the time of this writing, there are two active EA programs, with several others being planned.

This paper focuses on describing the early adopter program’s ability to foster satellite data applications within a wide variety of organizations and institutions. The 2007 and subsequently the 2017 National Research Council’s Earth Science Decadal Survey reports emphasized the importance of supporting both science and applications within the NASA program. Increasing technology costs and relatively flat budgets mean that accelerating engagement and use of data is increasingly important in a world where hazards and weather extremes are becoming commonplace. Partnerships and networking across organizations and institutions can lead to market-driven learning and engagement that benefits all participants [5].

Early Adopter Program for Missions

As a result of the success of Soil Moisture Active Passive (SMAP) with early adopters, NASA headquarters agreed to implement an early adopter program across all decadal survey missions and make applications activities a formal requirement for directed and competed missions, supported by NASA’s flight program [1]. The Directive on Project Applications Program presents guidance to all future directed NASA missions, stating that Earth science satellite missions entering development after the 2017 Decadal Survey will include an applications program element and an early adopter program as part of the mission life cycle. This is intended to increase awareness and improve the potential impact of future missions through gathering the user communities’ input and feedback during the development of the instrument, while the architecture and requirements for the missions are being defined. This approach has been proven to accelerate data uptake and utilization after launch, as seen with SMAP and the Ice, Cloud and Land Elevation Satellite-2 (ICESat-2). The applications element and especially the EA program allows NASA to extend its reach to new areas of societal utilization while gaining perspective of the challenges and advantages certain architecture development decisions will have on the end product. Thus, instrument and dataset development can be informed by user needs as a part of the mission development process at inception of a mission’s life cycle.

One of the most important aspects of the EA program is developing relationships among the mission, science team and the early adopters. The personal aspect of information provision and partnership is often overlooked in light of the dramatic development of technology that facilitates the ease of locating and obtaining data [6]. Using data and transforming it into relevant and valuable information requires sustained learning and personal engagement. EAs and the mission scientists have the opportunity to learn directly from each other’s expertise, and regular interaction keeps the work of incorporating new data into the system at the forefront of the missions’ efforts. Mission scientists benefit greatly from Early Adopter feedback, gaining perspective on the utility and challenges of the mission data to users while they are still in development. The mission’s data center also gain perspective on best practices for serving data and providing user services. Both NASA and early adopter institutions gain the unique opportunity to develop relationships that help identify the impact of science and implement lessons learned that reveal products that inform societally relevant applications.

2. Early Adopters (EA) and National Aeronautics and Space Administration (NASA) Mission Development

NASA’s Earth sciences programs are made up of a suite of scientific and applications objectives, which can be met through a set of observation capabilities that can be acquired from space. NASA organizes these capabilities into ‘missions’ either through a competitive proposal process or via directed programs. Each mission typically begins with a design phase that requires engineers and scientists to explore the feasibility of different observation approaches, technology, instruments and orbits resulting with a discrete observation system that can be funded. Once selected, a satellite mission has a set budget and much more limited flexibility to change fundamental aspects of the observations promised after launch. Engaging with potential stakeholders from the onset of the mission development allows for the participation of data users to inform the development decision points of the mission where the evaluation of mission observation tradeoffs are determined. Being inclusive of the user needs and requirements in this process results in potentially more impactful data. Early engagement with the broader community is the beginning of discussions of the utility of data with institutions that will later in the mission become early adopters.

EAs can propose projects that are either applications-oriented science or science that supports societal benefits [1]. Moran et al. [4] describes applications research as research that would provide fundamental knowledge of how soil moisture and freeze–thaw information from SMAP could be used for decision making within their institutions. In some cases, this took the form of data-denial experiments to determine how soil moisture information improves the accuracy of climate-informed models. Many EA projects focus on this kind of publishable research that estimates how much more accurate a particular measurement is, and how this improved accuracy may affect the users of their system.

Other EA projects focus on incorporating the new satellite data into existing decision support systems. NASA, along with the Group on Earth Observations (GEO), has established a process for identifying critical Earth observation priorities common to many of the nine GEO societal benefit areas, involving scientific and technical experts, taking account of socio-economic factors, and building on the results of existing systems’ requirements development processes [7]. The nine societal benefit areas defined by GEO are agriculture, biodiversity, climate, disasters, ecosystems, energy, health, water, and weather [8]. Although institutions working to provide services across these areas know about satellite data, integrating new observations or new scientific approaches that support these benefit areas requires the engagement and partnerships like those fostered by the early adopter program.

NASA has developed Formulation Authorization Direction (FAD) language for Earth science missions directed by the Decadal Survey, which directs that Earth science satellite projects evaluate and propose a project-specific applications program. The program will be funded as part of the project development life cycle and that every Earth science mission include an early adopter program. Although the guidelines may be tailored to accommodate project focus, community interest, funding and/or schedule considerations, projects that fail to propose a sufficiently robust applications program may be less competitive than others during selection process. Given this new initiative, we fully expect more early adopter programs to be initiated, and the engagement of NASA’s stakeholders to increase.

Process of EA Selection and Lifecycle

Historically, NASA has begun early adopter programs during Phase C of the mission when the science definition team (SDT) has been selected, and a clear articulation of the data products that are being planned have been published. Because there are no funds exchanged, early adopters’ projects are selected via an informal nomination process where the prospective principal investigator (PI) completes a nomination form that describes the potential applications research and the associated user community whom it may serve. The EA program is widely publicized and is open to everyone who fills out the nomination form. The form is reviewed by the mission’s applications team to ensure that each EA project is scientifically and societally robust and has a clear connection to a stakeholder or user of satellite data and is relevant to the mission’s objectives.

The nomination form requests the following information:

• Title of project that provides a brief description of the application;
• Name and contact information of primary responsible person, identified as the PI;
• Name of mission personnel or science team member affiliate who will serve as primary contact person;
• Description of system or model, including data currently used and ancillary data needs, and how the institution will improve their system with the new data;
• Foreseeable requirements for pre-launch simulated data products and plans for field experiment demonstration, if any;
• The decisions and societal impacts their research intend to support
• Milestones and quantitative metrics that will assess impacts of satellite data products on the application during the pre-launch phase; and
• Post-launch implementation strategy, if known.

All EA proposals are reviewed for their practicality and their focus on stakeholder needs, and their connection to mission and science leadership. Basic science research projects with no clear connection to data use or practical application are typically not accepted. However, in most cases the mission works with the institution submitting the nomination to ensure a clear description of user engagement and how the new satellite data will change the current way of working within the institution and improve overall outcome.

3. Valuing Satellite Data in Decision Making

The data products with the highest impact are those that are developed with knowledge of user requirements in mind. Working with data users from the start of science and instrument development allows scientists to develop results that are both usable and socially robust [9]. User-driven science thrives when institutions find common priorities that incentivize the process accordingly. The EA program provides a structured opportunity for participants in NASA missions and programs to engage with users early in the technology development process.

When scientists provide useful information more effectively to decision makers, science benefits as much as the decision maker [1]. Describing the characteristics of a new dataset from a new sensor in a way that clearly communicates how it can be used is challenging and requires one-on-one engagement with users. If scientists involved in a satellite mission have a better perspective of the uses and needs of the user community, they can better communicate the relevance and impact of their ideas and data for the instrument architecture. Applications may achieve success and continuity long after the mission has been launched. Better messaging allows for more accessible research papers and reports for a wider community. It also allows scientists to be better teachers and mentors for the next-generation. This takes significant effort and time, which needs to be included in grants and proposal opportunities provided by funders [10].

There is a growing body of literature on how Earth observations are useful to individuals, businesses and society [11,12]. For example, Booz Allen Hamilton [13] worked on the European Space Agency’s Global Monitoring for Environmental Security (GMES), now known as Copernicus, and estimated that the benefit of investing in Earth observations is 10:1, where there is a $10 return on every dollar spent. The USGS investigated the value of the Landsat satellite archive after it had become cost-free for northeastern Iowa, to prevent groundwater contamination from agricultural activity. They found that using 2013 dollars, the value of Landsat data for this use was around $38.1 billion ± $8.8 billion, a value that would continue to increase into the foreseeable future [14]. These studies focus on assessing the value of satellite data in particular contexts, using economic valuations of the benefit of improved information. This kind of assessment and knowledge now continues as an integrated part of mission development through the new ESD Directive on Project Applications.

Assessing the Impact of Satellite Data

The theory of change describes how and why a desired change is expected to happen in a particular context. It is focused on mapping out what a program or change initiative does, its activities or interventions, and how these lead to desired goals being achieved. It does this by first identifying the desired long-term goals and the value of these goals to the institution, and then works backwards from these goals to identify all the conditions (outcomes) that must be in place (and how these related to one another causally) for the goals to occur. For each institution, successfully integrating new satellite data will be different and result in application-specific metrics that have meaning only for that institution or application [15]. Although NASA does not provide funding for formal impact assessment, it encourages all EA institutions to articulate their organization’s impact and value by describing how their decision-making changed while participating in the EA program.

To provide an understanding about what evaluation entails, early adopters are invited to a benchmarking meeting at the start of Phase E after the successful launch of the satellite. The objective of the benchmarking activity is for each organization to compare their pre-data integration activities to those that of the post launch data integration. Each organization establishes a metric to be used in benchmarking their effort that is unique to the EA organization’s goals and context. The metric should capture a change in the process, decision making system, or model output between the use of mission data versus never having used it [16]. Examples of how EA programs identify decisions that have been affected by new satellite data include:

• The US Geological Survey is incorporating ICESat-2 data into its LANDFIRE program to better understand the three-dimensional structure of vegetation and how it relates to the likely intensity of fires if they were to break out [17]. This data is used to advise governments and firefighters on the probable fire intensity of particular forest patch were it to burn.
• The Naval Research Lab is incorporating ICESat-2 into the Los Alamos Sea Ice Model (CICE) to facilitate detailed comparisons between measured and modeled sea ice freeboard in Earth system models, including in the Regional Arctic System Model (RASM) and Community Earth System Model (CESM). The project will improve sea ice freeboard and snow cover data to insure high quality information for safety warnings and advisory in northern shipping lanes, which are increasingly available for commercial use [18].
• Applications research on the improvement that can be achieved in flood warning in Italy and in the United States due to the incorporation of soil moisture data from SMAP [19]. The more saturated soil is, the more likely additional rainfall will runoff and cause a flood in sensitive areas.
• Incorporation of operational SMAP data into the US Department of Agriculture (USDA) Foreign Agricultural Service (FAS)’s global crop assessment decision support system, which provides operational yield and food production information for a wide variety of institutions [20,21].

The feedback and results described change from before data integration to after launch with specific details regarding improvements or challenges related to better service, performance changes, conservation of resources, and delivery processes. By planning for a benchmark metric before the new satellite data is available, each early adopter will be prepared to conduct a meaningful evaluation of the difference in their process and decision making with and without the new satellite data after the data is available.

4. Policy Implications of Partnerships

The EA program seeks to provide a structure through which NASA can invest in partnerships that benefit both satellite missions and programs, as well as the early adopter. Figure 1 shows the elements of the partnership between NASA and the EA participants, where inputs, processes, outcomes and impacts are analyzed and the characteristics of successful partnerships are presented [22].

4.1. Partnership Inputs

Many of the key aspects of strong partnerships are iterative and cumulative [23]. If partnerships are to be successful, they need for participants to be able to articulate the value of the work beyond an assumption that partnerships are “a good thing”. Asthana et al. [22] found that the development of meaningful partnerships rests on the availability of key inputs: the institutional need to collaborate to extend knowledge or capability, the availability of sufficient resources for participants, the scientific expertise to follow through on the collaboration, and the institutional recognition for the need for the new observation data type. Although expertise and time to engage in the EA program is a necessary input, it is not sufficient to ensure sustained collaboration through the mission lifecycle.

Commitment to the EA program activity on both the part of the institution and the individual PI participant are both necessary, and the ability to lead the integration from vision during the pre-launch period to actual application in the post-launch period are central. Many EA PIs fail to remain engaged with the mission during the entire period from application to decommissioning because of lack of institutional resources, changing positions, or organizational challenges which enable their institutional application through the entire process of incorporating the new data. Although there are officially 21 ICESat-2 early adopters, it is likely only 9 will conduct benchmarking activities and report back to NASA on the usability of the data within their system. This does not mean those that do not participate in benchmarking won’t ultimately use the satellite data and derive value from it, but that the project PI was unable to engage with the program over the long mission development period, or that the data was determined not to be very valuable for that application. Without some analysis that shows the value of the data within the institution, the value of the information will not be documented.

4.2. Partnership Context

The context in which the EA program is implemented is critical for success. The NASA mission needs to be supportive of each EA project, ensuring that each project has been assigned a contact person with appropriate scientific expertise from either the science definition team (SDT) or the mission leadership itself. These engagements need to be valued, with NASA providing its personnel the time and funding to ensure that they can engage with EA PIs throughout the life of the mission.

The selected EA PI needs to engage with the NASA mission scientists and learn about the new mission and communicate the value of the data to the larger organization. They also need to be appropriately positioned in their institution to affect the data incorporation or applications research they have proposed. A good example of this is the engagement of the SMAP mission within the European Centre for Medium-Range Weather Forecasts (ECMWF), which is an independent intergovernmental organization supported by most of the nations of Europe. As a premiere weather modeling group, the potential societal benefits for effective use of soil moisture data from SMAP in their weather models are substantial. However, the EA PI needed to understand the data, engage with multiple parts of the organization, and sustain engagement for over six years. Experiments were conducted within the organization to demonstrate the utility of SMAP data to improve model outcomes through estimating initial conditions. Through participation with the SMAP science team, and engagement with the Early Adopter program, SMAP data was evaluated and integrated into the system.

NASA’s mission applications lead needs to identify incentives that will continue to foster a vibrant and supportive social network among all the EA organization and that will encourage each participant and institution to remain engaged for the period over which NASA develops missions. For example, it can be a decade or more from the funding of a mission formulation study through to when the data become regularly available post-launch. In many cases, it is several decades from initial concept to launch if NASA’s priorities change or technological maturity is low. A good example of this long and varied instrument history is the Vegetation Canopy Lidar (VCL). NASA funded the project in 1997 but after a series of delays and technological problems, the project was canceled in 2000. The sensor concept was recently once again selected as the Global Ecosystems Dynamics Investigation Lidar (GEDI) via NASA’s Earth Ventures Instrument 2 (EVI-2) competition and installed on the International Space Station in December 2018. Research conducted from the mid-1990s through to the present continue to document the critical importance of three-dimensional structure of the Earth’s forests for environmental and policy decisions [24]. Had there been EAs projects for VCL, they would need to be patient and engaged for over two decades with NASA on multiple projects before GEDI actually provided them data to integrate into their application.

4.3. Partnership Outcomes

The outcome of the partnership between NASA and each PI institution needs to be articulated if at all possible. This will allow for an evaluation of the benefit the institution has derived from the long engagement with NASA during pre-launch, as well as ensuring that the effort put into incorporating the satellite data is valuable to decision making. Benchmarking improved decision making and documenting the expanded networks of people affected by those decisions requires serious investment and understanding of how to evaluate social science systems [25]. This requires substantial effort and funding that is not provided under most science activities.

NASA Earth science is increasingly focused on documenting and quantifying the impact of Earth science data and observations. The recently funded NASA Valuation of Applications Benefits Linked with Earth Science (VALUABLES) consortium and their Value of Information (VOI) assessment framework contributes to this activity. The key objective of the consortium is to advance analytic techniques to quantify the impacts (in economic and social terms) from uses of Earth observations in management, policy, and business decisions and activities. By quantifying the socioeconomic benefits of satellite data applications, we can demonstrate return on investments in satellites and satellite data.

Articulation of benefits to stakeholders enables the prioritization of investments of satellite observations and capacity to maximize societal benefits. Descriptions of the value of satellite information that are being used by stakeholders provide an effective way to communicate the value of investments in satellite instruments to policymakers and the public [11]. The EA program provides a framework for quantifying value of satellite data can help Earth scientists design projects and data applications with an eye toward how they will benefit society.

4.4. Principles of EA Partnerships

A no-funds-exchanged partnership allows institutions with the capacity to incorporate new satellite remote sensing information into their decision process to participate. For-profit companies, international institutions, foreign universities, students, and non-profit institutions can apply and be included. This inclusive network allows for open partnerships where information is exchanged and learning is shared across institutions. Because the entire program is non-competitive, the program focuses on sharing of expertise and experience to bring the most value to all participants.

Including all these diverse institutions can reduce the effectiveness of the program, however. Hudson et al. [26] argues that good partnership depends on limiting the number of parties involved in the collaboration, as the number of “members cannot be so great that the process of partnership becomes unmanageable”. This implies that the process of partnership should involve exclusion, which has important implications for principles of access, representation and power [5]. In particular because the EA program is unfunded, and the selection process is open, different missions have taken a different approach on the number of EA projects they accept and support. ICESat-2, for example, selected approximately half as many EA projects as SMAP, simply by not soliciting proposals as frequently. If the mission personnel and science team members are to effectively support each EA project and engage with them one-on-one throughout the mission life-cycle, careful selection of projects to ensure representation and ability to sustain engagement is necessary.

Another principle of partnership is representation, both from the user community to the mission, and from the mission to the user community. Although useful, the EA program will not provide an effective way for the broader user community to be represented to the mission at every stage of development. This is because as the mission matures, the EA PIs will become extraordinarily well informed about how the new satellite will work, the characteristics of the data, and its utility for their application. If a mission relies only upon the EA program institutions as information about how well they are doing popularizing their data and the level of understanding of how to use it in the community, they will get a biased view. Everyone will be assumed to have the same level of expertise as the EA PIs and their colleagues.

The SMAP mission applications program provides a good example. In 2011, SMAP had 270 members of its applications list-serve, which it grew to over 600 individuals at launch along with its 37 EA institutions [4]. Both the applications user community and the Early Adopter community grew substantially in the four years before the launch of the sensor in 2015 through an active applications program that focused on representation and inclusion. By holding workshops, events at larger meetings, co-hosting meetings at user organizations and engaging with online media, the mission was able to continue to grow the broader community beyond the EA program.

Engagement with stakeholders during the mission formulation period supports the Project’s awareness of the needs of these communities and characterization of the mission’s applications value [27]. Through physical and virtual means, engagement involves speaking with individuals and institutions beyond the immediate research community and across sectors, such as resource managers, policy analysts, commercial organizations, non-profits, and government officials. A key purpose of the EA program is to characterize and understand the disciplines from which the communities originate. Identification of decision-making tools and models that the project data may interact with are parts of the community assessment.

The final principle is to share expertise. Satellite applications tend to be extremely diverse, with users of satellite data continuously coming up with new and totally unexpected uses of satellite data [28]. User institutions and scientists therefore are likely to have very diverse skill sets, which NASA scientists and managers can learn from. Similarly, mission experts share their specific sensor and product expertise with all who are in the EA program, and to all users via the applications program. Because the EA program is open to all, NASA commits to sharing its expertise with all participants, and expects the same from their partners.

4.5. Impacts

The impact of the EA program depends on the success of the partnerships it creates. As the 2017 Earth Science decadal survey points out, society is increasingly dependent on Earth information to support decisions on our daily lives, our businesses, and our government policies. Supporting day-to-day decisions require ongoing investments in the observation, understanding, and prediction of Earth’s environment and an awareness of how it is changing. NASA’s EA program focuses on enabling the usability of new science data investments across multiple institutions and programs.

Although in the past, NASA has not focused on evaluating user communities, the new Directive on Project Applications, coupled with recently funded projects like the Consortium for the Valuation of Applications Benefits Linked with Earth Science (VALUABLES), will heighten the ability of NASA to understand and articulate how satellite data is used and place a value on the data. The focus of the EA program is to engage with institutions over long periods, both encouraging the use of data as well as learning more about how the data has made a difference in decision making and what data are needed in the future. Having NASA missions and institutions engage in EA partnerships in diverse ways will ensure that benchmarking occurs in as many projects as possible and will change the community’s knowledge of the benefits of EA partnerships for new satellite data programs. By telling clear stories and including applications in mission goals and deliverables, NASA will be able to document how its data affects society while it continues to support high quality science.

5. Conclusions

The early adopter program has provided a valuable opportunity for the mission scientists and the users to increase awareness of how the needs of science and technology change over time. Missions take years to develop and the speed of technology moves faster than that of the mission development, thus adaptation to emerging technology needs and societal challenges are critical and cost effective.

Partnerships can bring enormous value to both NASA and program participants if there is alignment on purpose, common understanding of data uses, incentives, and investment in time. The EA program has enabled applications to provide value to individuals, businesses, the nation and the world, as they are an essential information infrastructure element for society [1]. It is only through ongoing engagement and incentives through partnerships that applications programs can bring institutions together to benefit all.