Submit to Remote Sensing Review for Remote Sensing Propose a Special Issue

Journal Browser

► Journal Browser

On Board Artificial Intelligence: A New Era for Earth Observation Satellites

Print Special Issue Flyer
Special Issue Editors
Special Issue Information
Keywords
Published Papers

A special issue of Remote Sensing (ISSN 2072-4292). This special issue belongs to the section "AI Remote Sensing".

Deadline for manuscript submissions: closed (15 April 2022) | Viewed by 28341

Share This Special Issue

Special Issue Editors

Dr. Massimiliano Pastena

E-Mail Website
Guest Editor

European Space Agency/ESTEC, Keplerlaan 1, 2201 AZ Noordwijk, The Netherlands
Interests: earth observation; CubeSats; NewSpace; earth science remote sensing
Special Issues, Collections and Topics in MDPI journals

Prof. Dr. Luca Fanucci

E-Mail Website
Guest Editor

Department of Information Engineering, University of Pisa, Pisa, Italy
Interests: HW accellerators for ML on board satellite

Special Issue Information

Dear Colleagues,

AI is a general-purpose technology already transforming the global economy but still largely untapped potential for Earth Observation (EO) technology. As described in the “AI for the Earth” report presented at the World Economic Forum in 2018, AI research is the “new electricity” fuelling the fourth Industrial revolution. In the past, the main innovation in EO were derived by AI applications on ground data leveraging on large scale computing capability e.g. Cloud computing or GPU architectures.

Thanks on the advances in microelectronics of space grade AI hardware accelerator, today we have the possibility to exploit AI directly on board opening a new era for EO satellites where feature extraction and decision making is performed directly on-board thus reducing unnecessary data exchanged between satellite sensors and ground.

Exploiting this “back to the edge” paradigm, EO could obtain huge advantages increasing satellite operativity and improving capabilities and performance.

In particular the current Special Issue invites contributions on innovative model concepts or improvements of existing AI/ML techniques for space missions, promoting new on-board architectures, or sensors that allow to improve the science of Earth observation through the use of the new hardware accelerators directly on-board AI, such as Intel Movidius Myriad-2 or space-grade FPGA.

Potential topics for this Special Issue include but are not limited to the following:

High-resolution multi-classes segmentation on-board
High performances CNN on-board
Recurrent network on-board
Disaster detection through AI on-board satellite (fire, hurricane, flooding, etc.)
Vehicle detection through AI on-board satellite (plane, vessel, boat, etc.)
Generative network for dataset creation
ML model reduction/quantization techniques

Dr. Massimiliano Pastena
Prof. Dr. Luca Fanucci
Guest Editors

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Remote Sensing is an international peer-reviewed open access semimonthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 2700 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Keywords

AI4EO
On board data processing
Machine learning for remote sensing
GAN for EO
Quantization method for ML
Innovative remote sensing technique
On-line information extraction
Edge computing
HPC

Published Papers (7 papers)

Download All Papers

Research

Jump to: Other

21 pages, 25223 KiB

Open AccessArticle

CloudSatNet-1: FPGA-Based Hardware-Accelerated Quantized CNN for Satellite On-Board Cloud Coverage Classification

by Radoslav Pitonak, Jan Mucha, Lukas Dobis, Martin Javorka and Marek Marusin

Remote Sens. 2022, 14(13), 3180; https://0-doi-org.brum.beds.ac.uk/10.3390/rs14133180 - 02 Jul 2022

Cited by 11 | Viewed by 3222

Abstract

CubeSats, the nanosatellites and microsatellites with a wet mass up to 60 kg, accompanied by the cost decrease of accessing the space, amplified the rapid development of the Earth Observation industry. Acquired image data serve as an essential source of information in various disciplines like environmental protection, geosciences, or the military. As the quantity of remote sensing data grows, the bandwidth resources for the data transmission (downlink) are exhausted. Therefore, new techniques that reduce the downlink utilization of the satellites must be investigated and developed. For that reason, we are presenting CloudSatNet-1: an FPGA-based hardware-accelerated quantized convolutional neural network (CNN) for satellite on-board cloud coverage classification. We aim to explore the effects of the quantization process on the proposed CNN architecture. Additionally, the performance of cloud coverage classification by biomes diversity is investigated, and the hardware architecture design space is explored to identify the optimal FPGA resource utilization. Results of this study showed that the weights and activations quantization adds a minor effect on the model performance. Nevertheless, the memory footprint reduction allows the model deployment on low-cost FPGA Xilinx Zynq-7020. Using the RGB bands only, up to 90% of accuracy was achieved, and when omitting the tiles with snow and ice, the performance increased up to 94.4% of accuracy with a low false-positive rate of 2.23% for the 4-bit width model. With the maximum parallelization settings, the hardware accelerator achieved 15 FPS with 2.5 W of average power consumption (0.2 W increase over the idle state). Full article

(This article belongs to the Special Issue On Board Artificial Intelligence: A New Era for Earth Observation Satellites)

► Show Figures

Graphical abstract

18 pages, 11335 KiB

Open AccessArticle

Satellite On-Board Change Detection via Auto-Associative Neural Networks

by Giorgia Guerrisi, Fabio Del Frate and Giovanni Schiavon

Remote Sens. 2022, 14(12), 2735; https://0-doi-org.brum.beds.ac.uk/10.3390/rs14122735 - 07 Jun 2022

Cited by 6 | Viewed by 2199

Abstract

The increase in remote sensing satellite imagery with high spatial and temporal resolutions has enabled the development of a wide variety of applications for Earth observation and monitoring. At the same time, it requires new techniques that are able to manage the amount of data stored and transmitted to the ground. Advanced techniques for on-board data processing answer this problem, offering the possibility to select only the data of interest for a specific application or to extract specific information from data. However, the computational resources that exist on-board are limited compared to the ground segment availability. Alternatively, in applications such as change detection, only images containing changes are useful and worth being stored and sent to the ground. In this paper, we propose a change detection scheme that could be run on-board. It relies on a feature-based representation of the acquired images which is obtained by means of an auto-associative neural network (AANN). Once the AANN is trained, the dissimilarity between two images is evaluated in terms of the extracted features. This information can be subsequently turned into a change detection result. This study, which presents one of the first techniques for on-board change detection, yielded encouraging results on a set of Sentinel-2 images, even in light of comparison with a benchmark technique. Full article

(This article belongs to the Special Issue On Board Artificial Intelligence: A New Era for Earth Observation Satellites)

► Show Figures

Graphical abstract

24 pages, 3955 KiB

Open AccessArticle

Self-Organizing Maps for Clustering Hyperspectral Images On-Board a CubeSat

by Aksel S. Danielsen, Tor Arne Johansen and Joseph L. Garrett

Remote Sens. 2021, 13(20), 4174; https://0-doi-org.brum.beds.ac.uk/10.3390/rs13204174 - 18 Oct 2021

Cited by 8 | Viewed by 2865

Abstract

Hyperspectral remote sensing reveals detailed information about the optical response of a scene. Self-Organizing Maps (SOMs) can partition a hyperspectral dataset into clusters, both to enable more analysis on-board the imaging platform and to reduce downlink time. Here, the expected on-board performance of the SOM algorithm is calculated within two different satellite operational procedures: one in which the SOM is trained prior to imaging, and another in which the training is part of the operations. The two procedures are found to have advantages that are suitable to quite different situations. The computational requirements for SOMs of different sizes are benchmarked on the target hardware for the HYPSO-1 mission, and dimensionality reduction (DR) is tested as a way of reducing the SOM network size. We find that SOMs can run on the target on-board processing hardware, can be trained reasonably well using less than 0.1% of the total pixels in a scene, are accelerated by DR, and can achieve a relative quantization error of about 1% on scenes acquired by a previous hyperspectral imaging satellite, HICO. Moreover, if class labels are assigned to the nodes of the SOM, these networks can classify with a comparable accuracy to support vector machines, a common benchmark, on a few simple scenes. Full article

(This article belongs to the Special Issue On Board Artificial Intelligence: A New Era for Earth Observation Satellites)

► Show Figures

Figure 1

24 pages, 12323 KiB

Open AccessArticle

Benchmarking Deep Learning for On-Board Space Applications

by Maciej Ziaja, Piotr Bosowski, Michal Myller, Grzegorz Gajoch, Michal Gumiela, Jennifer Protich, Katherine Borda, Dhivya Jayaraman, Renata Dividino and Jakub Nalepa

Remote Sens. 2021, 13(19), 3981; https://0-doi-org.brum.beds.ac.uk/10.3390/rs13193981 - 05 Oct 2021

Cited by 14 | Viewed by 4392

Abstract

Benchmarking deep learning algorithms before deploying them in hardware-constrained execution environments, such as imaging satellites, is pivotal in real-life applications. Although a thorough and consistent benchmarking procedure can allow us to estimate the expected operational abilities of the underlying deep model, this topic remains under-researched. This paper tackles this issue and presents an end-to-end benchmarking approach for quantifying the abilities of deep learning algorithms in virtually any kind of on-board space applications. The experimental validation, performed over several state-of-the-art deep models and benchmark datasets, showed that different deep learning techniques may be effectively benchmarked using the standardized approach, which delivers quantifiable performance measures and is highly configurable. We believe that such benchmarking is crucial in delivering ready-to-use on-board artificial intelligence in emerging space applications and should become a standard tool in the deployment chain. Full article

(This article belongs to the Special Issue On Board Artificial Intelligence: A New Era for Earth Observation Satellites)

► Show Figures

Graphical abstract

22 pages, 2690 KiB

Open AccessArticle

Oil Spill Identification from SAR Images for Low Power Embedded Systems Using CNN

by Lorenzo Diana, Jia Xu and Luca Fanucci

Remote Sens. 2021, 13(18), 3606; https://0-doi-org.brum.beds.ac.uk/10.3390/rs13183606 - 10 Sep 2021

Cited by 9 | Viewed by 2515

Abstract

Oil spills represent one of the major threats to marine ecosystems. Satellite synthetic-aperture radar (SAR) sensors have been widely used to identify oil spills due to their ability to provide high resolution images during day and night under all weather conditions. In recent years, the use of artificial intelligence (AI) systems, especially convolutional neural networks (CNNs), have led to many important improvements in performing this task. However, most of the previous solutions to this problem have focused on obtaining the best performance under the assumption that there are no constraints on the amount of hardware resources being used. For this reason, the amounts of hardware resources such as memory and power consumption required by previous solutions make them unsuitable for remote embedded systems such as nano and micro-satellites, which usually have very limited hardware capability and very strict limits on power consumption. In this paper, we present a CNN architecture for semantically segmenting SAR images into multiple classes. The proposed CNN is specifically designed to run on remote embedded systems, which have very limited hardware capability and strict limits on power consumption. Even if the performance in terms of results accuracy does not represent a step forward compared with previous solutions, the presented CNN has the important advantage of being able to run on remote embedded systems with limited hardware resources while achieving good performance. The presented CNN is compatible with dedicated hardware accelerators available on the market due to its low memory footprint and small size. It also provides many additional very significant advantages, such as having shorter inference times, requiring shorter training times, and avoiding transmission of irrelevant data. Our goal is to allow embedded low power remote devices such as satellite systems for remote sensing to be able to directly run CNNs on board, so that the amount of data that needs to be transmitted to ground and processed on ground can be substantially reduced, which will be greatly beneficial in significantly reducing the amount of time needed for identification of oil spills from SAR images. Full article

(This article belongs to the Special Issue On Board Artificial Intelligence: A New Era for Earth Observation Satellites)

► Show Figures

Graphical abstract

20 pages, 917 KiB

Open AccessArticle

An FPGA-Based Hardware Accelerator for CNNs Inference on Board Satellites: Benchmarking with Myriad 2-Based Solution for the CloudScout Case Study

by Emilio Rapuano, Gabriele Meoni, Tommaso Pacini, Gianmarco Dinelli, Gianluca Furano, Gianluca Giuffrida and Luca Fanucci

Remote Sens. 2021, 13(8), 1518; https://0-doi-org.brum.beds.ac.uk/10.3390/rs13081518 - 14 Apr 2021

Cited by 33 | Viewed by 5300

Abstract

In recent years, research in the space community has shown a growing interest in Artificial Intelligence (AI), mostly driven by systems miniaturization and commercial competition. In particular, the application of Deep Learning (DL) techniques on board Earth Observation (EO) satellites might lead to numerous advantages in terms of mitigation of downlink bandwidth constraints, costs, and increment of the satellite autonomy. In this framework, the CloudScout project, funded by the European Space Agency (ESA), represents the first time in-orbit demonstration of a Convolutional Neural Network (CNN) applied to hyperspectral images for cloud detection. The first instance of this use case has been done with an INTEL Myriad 2 VPU on board a CubeSat optimized for low cost, size, and power efficiency. Nevertheless, this solution introduces multiple drawbacks due to its design not specifically being for the space environment, thus limiting its applicability to short-lifetime Low Earth Orbit (LEO) applications. The current work provides a benchmark between the Myriad 2 and our custom hardware accelerator designed for Field Programmable Gate Arrays (FPGAs). The metrics used for comparison include inference time, power consumption, space qualification, and components. The obtained results show that the FPGA-based solution is characterized by a reduced inference time, and a higher possibility of customization, but at the cost of greater power consumption and a longer Time to Market. As a conclusion, the proposed approach might extend the potential market of DL-based solutions to long-term LEO or interplanetary exploration missions through deployment on space-qualified FPGAs, with a limited cost in energy efficiency. Full article

(This article belongs to the Special Issue On Board Artificial Intelligence: A New Era for Earth Observation Satellites)

► Show Figures

Figure 1

Other

Jump to: Research

16 pages, 12030 KiB

Open AccessTechnical Note

On-Board Volcanic Eruption Detection through CNNs and Satellite Multispectral Imagery

by Maria Pia Del Rosso, Alessandro Sebastianelli, Dario Spiller, Pierre Philippe Mathieu and Silvia Liberata Ullo

Remote Sens. 2021, 13(17), 3479; https://0-doi-org.brum.beds.ac.uk/10.3390/rs13173479 - 02 Sep 2021

Cited by 27 | Viewed by 3947

Abstract

In recent years, the growth of Machine Learning (ML) algorithms has raised the number of studies including their applicability in a variety of different scenarios. Among all, one of the hardest ones is the aerospace, due to its peculiar physical requirements. In this context, a feasibility study, with a prototype of an on board Artificial Intelligence (AI) model, and realistic testing equipment and scenario are presented in this work. As a case study, the detection of volcanic eruptions has been investigated with the objective to swiftly produce alerts and allow immediate interventions. Two Convolutional Neural Networks (CNNs) have been designed and realized from scratch, showing how to efficiently implement them for identifying the eruptions and at the same time adapting their complexity in order to fit on board requirements. The CNNs are then tested with experimental hardware, by means of a drone with a paylod composed of a generic processing unit (Raspberry PI), an AI processing unit (Movidius stick) and a camera. The hardware employed to build the prototype is low-cost, easy to found and to use. Moreover, the dataset has been published on GitHub, made available to everyone. The results are promising and encouraging toward the employment of the proposed system in future missions, given that ESA has already moved the first steps of AI on board with the Phisat-1 satellite, launched on September 2020. Full article

(This article belongs to the Special Issue On Board Artificial Intelligence: A New Era for Earth Observation Satellites)

► Show Figures