LunarSim: Lunar Rover Simulator Focused on High Visual Fidelity and ROS 2 Integration for Advanced Computer Vision Algorithm Development

Abstract

Autonomous lunar exploration is a complex task that requires the development of sophisticated algorithms to control the movement of lunar rovers in a challenging environment, based on visual feedback. To train and evaluate these algorithms, it is crucial to have access to both a simulation framework and data that accurately represent the conditions on the lunar surface, with the main focus on providing the visual fidelity necessary for computer vision algorithm development. In this paper, we present a lunar-orientated robotic simulation environment, developed using the Unity game engine, built on top of robot operating system 2 (ROS 2), which enables researchers to generate quality synthetic vision data and test their algorithms for autonomous perception and navigation of lunar rovers in a controlled environment. To demonstrate the versatility of the simulator, we present several use cases in which it is deployed on various efficient hardware platforms, including FPGA and Edge AI devices, to evaluate the performance of different vision-based algorithms for lunar exploration. In general, the simulation environment provides a valuable tool for researchers developing lunar rover systems.

1. Introduction

Exploring the Moon and other planets has been a long-standing goal of space agencies and private companies around the world. The number of space launches has seen a steady increase in recent years. Lowering costs and the broader availability of key technologies have driven progress and made space exploration more accessible [1,2]. However, the growth in this sector may slow down because of issues related to the scalability of current technology. Although space-related innovations are certainly at the forefront of technical and scientific advancement, most missions still rely on human supervision and intervention. To overcome this issue, we can increase the degree of autonomy by implementing more advanced and independent systems. Autonomy is finding its way into satellites, which require relatively little operator attention. One can observe a trend of making the satellite hardware relatively more and more self-contained, with on-board processing and control capabilities [3,4]. The next logical step is applying autonomy to in-orbit operations and space robotics [5,6]. Complex in-orbit operations, or robot steering, require significant human involvement when carried out completely manually. With the increasing number of launches, one can expect the number of devices and vehicles performing such complex tasks to grow. Without some degree of autonomy, the sheer amount of equipment that needs to be supervised and tended to by qualified staff will impose serious limitations on the scalability of such deployments. Coupled with additional limitations for manual control, such as communication delays, this poses a problem that needs to be addressed—with new paradigms, methods, and algorithms facilitating automation and autonomy in space.

The number of space applications on the verge of disruption by increased autonomy is vast, including, among numerous others, robotic rovers. The number of deployments of such robots has increased significantly as of late [7,8,9,10]. To prepare for these missions, simulating the environments of these extraterrestrial bodies has become an essential part of the development of robotic technologies. Although many such solutions have been developed over the years, most of them are not up to the task of addressing the challenges of modern-day space autonomy. The recent wide adoption of deep learning [11,12] and the increased use of industry-standard robotic software [13,14,15] are slowly making their way into space-related applications. Moreover, space applications often use dedicated and heterogeneous computational resources, which creates the need for hardware-in-the-loop (HIL) simulation, allowing the testing of the complex interplay of hardware and software under possibly realistic conditions [16].

In this paper, we present LunarSim, a space simulator that is tightly integrated with ROS 2 [17], space-ROS [15], and micro-ROS [18], and is capable of performing simulations, using a wide range of hardware (programmable logic, embedded microcomputers, and microcontrollers). A sample view of its environment is demonstrated in Figure 1. The main motivation for the simulator’s development was the insufficient visual fidelity of the images generated by other simulators and/or their limited availability. The focus of the majority of simulators is on terramechanics, as they were mostly developed to analyse the problem of terrain traversal, with the quality of visualisations, the capabilities of HIL simulation, and software integration not being the focus of the developers. According to the scale proposed in [19], they could be classified as high fidelity in terms of terramechanics and physical and terrain simulation, but as low to medium categories of visual fidelity level. The presented simulator aims at a high fidelity level, in terms of visualisation, and a medium fidelity level of physical simulation. Phenomena such as slip, inertia, gravity, collisions, traces left by the moving robot, etc., are simulated, so that robot traversal is realistic enough to test mapping and navigation algorithms. We do not model soil compacting or settlement, as it is not crucial for vision-based situational awareness. In a similar way, terramechanics simulators do not provide configurable, photorealistic rendering, hardware-in-the-loop or standard software integration capabilities, as they are not relevant to their focus area.

The simulator can generate any quantity of ground truth data for training and testing the onboard machine learning and deep learning image processing (segmentation, object detection, depth maps) paired with realistic images. It can also support the development of vision-based navigation—visual odometry (VO) and simultaneous localisation and mapping (SLAM)—paving the way to increased AI-assisted rover autonomy. The simulator offers a range of desirable properties that are typically only partially present in other solutions.

The principal contributions of this paper are outlined as follows:

• Introduction of a simulator and toolkit achieving a high degree of rendered scene realism, which is demonstrated by improved performance of the segmentation neural network model, using generated synthetic data for transfer learning when compared to an alternative solution.
• Interface-level compatibility with ROS 2, an industry-standard robotics software platform enabling rapid development and testing of applications, which is shown by running and evaluating the visual odometry engine, using simulated data.
• Compatibility with a wide range of hardware and simulation scenarios, which is demonstrated using deep learning image analysis, sensor data processing, and navigation functions using real devices, ranging from embedded devices with integrated AI accelerators to complex system-on-chip devices, including custom, programmable hardware.

2. Related Work

The potential of simulation as a tool for aiding the development of space robots has long been recognised, with examples such as SpaceDyn [20], ROAMS [21], and 3DROV [22] paving the way for future, more advanced solutions. In their survey [23], Ding et al. gave an overview of simulators, almost a decade later. They noted the emergence of follow-up solutions, such as ROSTDyn [24] or the use of commercial tools for multibody simulation [25] integrated within more extensive toolsets [26]. A large part of the functionality of both of the aforementioned simulators is focused on accurately modelling the physical properties of the rover and the environment, such as rover–surface soil interactions, slip dynamics, thermal models, power system models, and characteristics of the rover’s frame, sensors, and actuators. The gathered information is very valuable: high-fidelity wheel–soil interaction and rover mobility simulations across various conditions have been crucial for developing successful motion control algorithms, facilitating extraterrestrial exploration in multiple missions.

With the first steps already taken, it was evident that future missions would require increased situational awareness and autonomy, including VO and SLAM. The feasibility of such an approach was proven in large-scale analogue missions [27]. Analogue missions are considered the most valuable source of information, with more recent examples being [28,29]. However, gathering real-world data is a time-consuming, laborious process, which is limited, in terms of the number of scenarios and control over physical world parameters. Aside from SLAM and VO, recent advances in machine learning for visual information processing have created the need for high-fidelity visual information generation in large volumes, preferably with automatically generated and associated ground truth. The ability to use widely adopted, standard software for control and broad hardware support is also welcome.

A game-engine-based simulator introduced by NASA [30] offers excellent visual fidelity in replicating lunar surface images, but focuses primarily on topography design tools, which lack machine learning support components. It is, however, compatible with an in-house tricklibrary for experiment simulation control and HIL capability. With the advent of Space ROS [15], compatibility with the ROS framework will be a significant advantage for robot rover simulators. The Gazebo simulator, typically paired with ROS [31], has been used successfully in various applications and provides seamless integration, but it was not designed with visual fidelity or machine learning support in mind. The research presented in [32] used Gazebo with custom extensions, to improve the image quality of the rendered scene. However, achieving an acceptable level of visual fidelity and performance required extensive modifications to the rendering engine, which were proprietary. The recently proposed Martian terrain simulators [33,34] also used ROS and Gazebo, but were mostly directed at the realistic simulation of the rover and soil contacts and terra-dynamics. Modern simulation systems enable automated training data generation and integration with ROS: for example, Nvidia Isaac Sim [35], which enables accurate simulation with realistic visuals and ROS support. However, it lacks space environments (the existing environments are primarily focused on robotic manipulation and robot warehouses) and is a partially closed, company-developed solution that requires specific hardware and infrastructure. It is important to note that none of these simulators are specifically designed for the development of computer vision algorithms in the lunar environment.

Illustrative frames extracted from our simulator, juxtaposed with sample images generated by prior solutions, are shown in Figure 2.

3. Overview of the Solution

The presented simulator allows researchers to quickly bootstrap a lunar environment, to develop and test algorithms for autonomous rovers. This technology has the potential to significantly accelerate the development of lunar exploration systems by allowing researchers to generate synthetic data and efficiently test rover algorithms in a controlled environment. This is particularly relevant given the substantial costs and logistical complexities associated with real-world lunar missions or even analogue missions. Visual data also empower researchers to explore the potential impact of external factors on algorithmic performance. For instance, lighting conditions, visibility limitations, and variations in terrain textures can all influence the outcomes of the rover’s perception and decision making algorithms. By manipulating these visual variables in the simulator, researchers can assess the robustness of their algorithms and devise strategies to mitigate the adverse effects of real-world uncertainties. The block diagram of the complete system is shown in Figure 3.

To address concerns about the theoretical underpinning of the simulated lunar environment, our approach relies on the meticulous integration of empirical data and established lunar knowledge. The simulator’s fidelity is achieved by incorporating satellite imagery and images from lunar missions. Moreover, to establish the simulator’s theoretical validity, we have aligned its parameters with known lunar properties, such as gravitational conditions, illumination angles, and illumination temperature. The rover’s physical attributes, including its size, weight, speed, and sensor capabilities, have been derived from real-world technologies and devices. These parameters were not only inspired by existing technology but also tuned to ensure that the use of the simulated rover would mirror sensors in the real world. This integration of established knowledge not only reinforces the simulator’s authenticity but also provides a foundation for its representation of the lunar surface.

The Unity real-time development platform was selected as a simulation engine. The engine enables realistic scene rendering and accurate physical simulation out of the box. An artificial lunar landscape was built, using proprietary assets such as textures and models, as the game engine facilitates the use and import of models, objects, textures, and material data from various sources, both free and nonfree. The terrain contains some craters, hills, and flatlands, forming an area 950 × 950 m large. It is highly inspired by the Moon’s surface but does not imitate any true part of the Moon. Similarly, rock models were created, textured, and manually placed in the terrain. A blending strategy was employed for the albedo (colour) maps and normal textures, making the integration of the lunar soil and other objects more natural. Additional post-processing effects, such as colour grading, ambient occlusion, contact shadows, and screen space reflections, were used, to further improve photorealism. It is worth noting that the scene’s lighting can also be easily controlled, to simulate realistic effects like shadows, glare, lens flares, etc. The physical forces of the lunar environment are simulated and controlled by the NVIDIA PhysX engine [37]. The ability to freely model the lighting conditions is an important step in providing the necessary level of realism. Moreover, the ability to change the lighting-related parameters of the atmosphere and materials enables simulation in a range of realistic conditions, improving the robustness of the resulting solution. Dynamic lighting conditions and high dynamic range are significant problems in robot navigation [38]. Capturing environments under a range of lighting conditions is a crucial element of computer vision system evaluation in many applications [39]. Introducing realistic textures is also an important step towards enabling robot situational awareness and navigation capabilities, using simulated input image data. Feature detection and tracking is a prerequisite and first step in almost all visual odometry and SLAM algorithms, and lack of texture might not only degrade the faithfulness of the simulation, it can also make testing the vision-based navigation algorithms impossible. Recent studies have highlighted the importance of textures in human perception and computer vision [40]. Moreover, the presence of sufficiently textured terrain has been highlighted as a necessary condition for Martian navigation [41].

In addition, numerous aspects of the simulation can be configured, using YAML configuration files. Specifically, various camera properties can be set to mimic real cameras, including focal length, sensor size, image resolution, noise, distortion, etc. The configuration also allows for switching simulation between the modes of generating data (Section 3.1) and ROS-connected (Section 3.2). The functionalities use built-in, easy-to-use, well-documented features of the utilised game engine, enabling easy, rapid changes and development. Having the configuration stored in files facilitates experimental result replication.

Finally, the simulator is containerised, using Docker software (ver. 24.0.5), and it is provided as a single image, to allow rapid use without the need to satisfy the dependencies, making it more accessible and portable for users. As shown in Figure 3, the containerised simulation is controlled by and capable of generating ROS-2-compatible signals, whose source or destination can be a variety of hardware platforms, which is demonstrated in detail by examples in Section 4. Containerisation also enables the lawful sharing of scenes and terrains containing not only free but also proprietary resources.

3.1. Dataset Generator Mode

The simulation can be configured to run in the data generation mode. A perception [42] library is used to capture the corresponding RGB and semantic segmentation frames with the information of the bounding boxes. Additionally, the simulation uses a simulated depth camera aligned with the left camera of the rover and saves depth information.

The semantic segmentation ground truth capture is configurable and can contain rocks, terrain, and rover parts masks. However, the bounding boxes are captured for rocks only. Both functions can be easily extended to process other objects of interest. All cameras simulate physical devices with configurable parameters. For the depth image, the clipping planes’ settings are exposed, to enable the selection of depth range. This, however, can also be achieved when postprocessing the data. Another settable parameter is the registration frequency.

Moreover, the simulator’s versatility is further enhanced by the incorporation of additional camera configurations, allowing it to generate essential visual ground-truth masks for intricate tasks, like bird’s-eye-view (BEV) analysis. The capabilities of the simulator are showcased in Figure 4, in which a bird’s eye view of the rover, with automatically generated semantic segmentation masks for the rocks and the rover, is presented. BEV algorithms for perception tasks are trending and drawing extensive attention both from industry and academia [43].

The data acquired using the simulator are synchronised and can later be used to train various machine learning algorithms. These include semantic scene segmentation, rock detection, monocular and stereo depth estimation, and optical flow or interest point location estimation. Additionally, we publish sample datasets, ready to use for the aforementioned tasks.

3.2. Ros-2-Connected Mode

In the ROS-connected mode, the simulation does not save the cameras’ data nor the ground-truth annotations to disk. Instead, ROS 2 for the Unity [44] library connects the application to the ROS system. It enables the simulator to receive and publish information simultaneously, emulating sensors and effectors from a real-world rover platform and accepting commands sent by ROS to control the rover.

ROS 2 provides a modular and distributed architecture that integrates different components and sensors, making it a flexible platform for building complex robotic systems. It offers a rich set of libraries, tools, and algorithms for various tasks, such as navigation, perception, manipulation, and planning, which can significantly reduce development time.

The simulation has been integrated with ROS 2 Humble, but there are currently no obstacles to upgrading it to newer versions in the future. This distribution was selected as the most recent version supporting modern Linux operating systems. It is also the basis of the recently proposed Space ROS system dedicated to space applications [15].

The rover accepts steering information as standard geometry_msgs.msg.Twistmessages and, therefore, can be controlled using any capable ROS nodes. The publishing system sends the IMU (inertial measurement unit), stereo camera, depth sensor, and ground truth pose information with a configurable frequency, allowing the development and testing of perception and control algorithms. To make integration with other nodes straightforward, common ROS message types—such as sensor_msgs.msg.Imu, sensor_msgs.msg.Image or sensor_msgs.msg.CompressedImage, and geometry_msgs.msg.PoseStamped—are used. As mentioned before, the number, placement, and parameters of the sensors can be freely configured.

4. Real-World Use Cases with Sample Results

In the following section, we present examples of computer vision applications that can interact with the simulation. As shown in Figure 3, the simulator and ROS interface support a wide range of heterogeneous computational platforms. To demonstrate this capability, the applications were executed on efficient and low-power embedded hardware, facilitating the deployment of applications in resource-constrained environments, such as mobile and aerial robots. Please note that the presented results are not meant to set new standards in the presented use scenarios, but rather to comprehensively demonstrate the capabilities of the simulator on a broad range of target applications, using a broad range of hardware. The ability to integrate various commercial off-the-shelf (COTS) devices with the simulator aligns with the principles of the New Space paradigm [45]. The evolution of this paradigm necessitates the creation of versatile tools capable of accommodating a wide range of hardware. The simulator can also easily be adapted to other devices, thanks to the use of industry-standard ROS interfaces.

We based this research on the artificial lunar landscape dataset (ALLD) [36]. It provides real lunar (ALLD (real)) and rendered (ALLD (synthetic)) images with ground truth segmentation masks. Similarly, we generated images from the simulator (our (synthetic)) and included a real subset of images collected at an analogue lunar research station (our (real))—the LunAres Research Station [46], which is a simulation facility for studying the challenges and requirements of space missions. The images were collected in various lighting conditions, using a camera mounted on the rover and directed 45 degrees down.

To test the simulator’s capability for rendering realistic images, a simple test was performed. An image descriptor, in the form of embeddings extracted by a convolutional neural network, was used to encode the images into vectors [47]. Generally, the closer the embedding vectors of two images lie in the embedding vector space, as measured by some metric, the more similar the images are. Using embeddings from single images would not be a good representation of a whole dataset, so after the embeddings were extracted, they were averaged, to represent all the images in a given dataset. The base MobileNetV2 model was used to generate the embedding vectors [48]. The model was pre-trained, using data from the ImageNet dataset [49]. The cosine distances of the averaged embeddings of the LunarSim and ALLD synthetic part to the average embedding of the ALLD real part were 0.347 and 0.360, respectively. This confirms that the LunarSim rendered images are a closer representation of the real images.

4.1. Object Detection—Embedded Device

Object detection is one of the fundamental tasks in the design of autonomous robots. It can provide information about ”what objects are where”, providing a foundation for understanding the visual scene [50].

Although, given a well-performing scene segmentation algorithm, one can view object detection as redundant, it is often used alongside semantic segmentation, serving as a safety mechanism that provides results decoupled from the segmentation pipeline. Alternatively, it may be used to locate objects that are rare or small and, therefore, easily omitted by other approaches. Sometimes, object detection can also be used as a preprocessing step for semantic segmentation by first detecting the region of interest and then segmenting objects more accurately.

The presented simulator can be instructed to generate object detection annotations for any objects present in the scene, but it primarily generates them for rocks. This is mainly due to the lack of any significant objects of interest present in the lunar landscape. However, rock avoidance can be viewed as a crucial task for autonomous lunar rover operation or sample extraction. The size characteristics of the rocks are randomised, to generate diverse instances with varying sizes.

To demonstrate the accuracy and feasibility of the automatically generated object annotations, YOLOv7 [51] (tiny version) was trained, using simulation data. This specific algorithm was selected as it achieves State-of-the-Art results while maintaining an inference speed sufficient for on-board processing. Moreover, we quantised the model weights and benchmarked its accuracy and performance on the Edge AI device, Jetson Xavier NX, which is an embedded CPU with a GPU designed by Nvidia for edge computing devices. It is small, lightweight, energy-efficient, and equipped with software tools like TensorRT that make it easy to develop and deploy AI applications—in particular, based on convolutional neural networks (CNN) and computer vision (CV). Performed tests include floating-point, half-floating-point, and eight-bit integer quantisation done through minmax scaling. The most widely used metric, mean average precision (mAP), was applied. The results are shown in

Table 1. Performance and speed comparison with different weights representation precision. YOLOv7 was used as an example object detection model for our synthetic data. Jetson NX was chosen as the reference platform.

Precision	[email protected]	Speed [img/s]
FP-32	0.424	40.27
FP-16	0.432	44.62
INT-8	0.411	48.96

. Then, an ROS-based application was developed. It uses a colour image (/lunarsim/depth_camera/depth topic) to detect rocks and depth information (/lunarsim/left_camera/compressed topic), to estimate their real world size. An example result of node inference is shown in Figure 5. It contains a real image provided by ALLD (A) and our synthetic image from the test set (B).

The comparison shows that the reduced floating point representation improves processing speed and detection accuracy. Speed can be further improved by using the 8-bit integer format but at the cost of degraded detection performance. False negatives are usually the result of a relatively small rock size associated with a large rover-to-rock distance. Performance can be easily improved by using a more complex YOLOv7 variant, but image processing speed will be negatively affected.

4.2. Image Segmentation—FPGA Platform

Field programmable gate arrays (FPGAs) are a common computational platform choice for space robotics because they offer several key advantages over traditional microprocessors, including high performance and relatively low power consumption. They have already been used in space missions and, therefore, radiation-resistant variants are available [52]. The possibilities for using FPGAs in artificial intelligence have broadened with the availability of tools that allow for the quantisation and compilation of deep learning models.

“Identifying and avoiding terrain hazards is crucial for the safety of planetary rovers. [...] The rover must avoid driving over any potential hazards to its wheels, particularly angular embedded rocks”, as described in [53]. Therefore, the key task is terrain analysis, which can be defined as pixel-wise image segmentation. One of the obstacles that occur in the lunar environment is rocks. To demonstrate that our synthetic data are helpful and that the simulator provides meaningful, measurable aid in application development, we cross-validate datasets. We use synthetic data from LunarSum and ALLR to train the neural networks and real ALLD lunar data to validate them. For this purpose, we use the UNet [54] segmentation network, which was created with a resnet10t backend [55] and trained with recent improvements, such as the cosine learning rate schedule and data augmentation. The UNet architecture is a well-known neural network framework, designed to assign class membership to individual pixels within an image. Specifically, we opted for the UNet implementation, which adopts an encoder–decoder structure and leverages skip connections to facilitate learning of spatial correlations across corresponding encoding stages, as detailed in [56].

Using the synthetic data generated using the described simulator resulted in a better F1 score than in the case when the synthetic data was taken from the ALLD dataset. This indicates that the generated synthetic data more closely resembled real-life conditions. This was even more evident when both parts of the real life data set were used for testing, as the difference in the F1 score was even greater.

In the same way, as in [57], we used the Xilinx VCK190 developer kit to perform deep learning inference. We quantised and compiled the models trained using our synthetic data. After quantisation, the models reached the F1 score values of 0.459 and 0.541 for ALLD (real)and ALLD (real)+ our (real). Comparison between data set combinations (

Table 2. Comparison of model accuracy depending on the training and test data set used.

Train Dataset	Test Dataset	F1-Score
ALLD (synthetic)	ALLD (real)	0.431
ALLD (synthetic)	ALLD (real) + our (real)	0.428
Ours (synthetic)	ALLD (real)	0.449
Ours (synthetic)	ALLD (real) + our (real)	0.537

) shows improvement in performance on real images with our synthetic data being used for training. Example predictions of the network in real images are presented in Figure 6a (from the ALLD dataset) and Figure 6b (from the analogue lunar research station). Next, we built an ROS 2 node running on VCK190 that performed inference on the input images (/lunarsim/left_camera/compressed topic) and published a blended image with a heat map overlay for the presence of rocks (Figure 6c).

4.3. Visual Odometry—x64-Based Mini-PC

Accurate localisation of mobile robots (rovers) is one of the most challenging tasks—especially when the target environment is unknown to a large extent. In the case of space applications, it is even more challenging, as the appearance of exoplanetary environments is notably different to Earth environments, making appearance-based navigation implementation and testing significantly more difficult. Visual odometry (VO) enables self-localisation based on a series of images captured by the robot’s onboard camera. It is a relatively low-cost solution compared to other localisation techniques that rely on expensive sensors or external infrastructure. It is lightweight and computationally efficient, making it suitable for real-time applications on mobile robots with limited processing power.

To demonstrate the cooperation between our simulator and visual odometry, the OpenVSLAM framework was used [58]. This framework is compatible with ROS 2, utilises the fast ORB feature extraction, and supports mono, stereo, and depth cameras. In our experiments, a stereovision-based approach was selected (/lunarsim/left_camera/raw and /lunarsim/right_camera/raw topics). The trajectory of movement was generated using a visual odometry algorithm and was compared to the ground-truth trajectory (/lunarsim/gt/pose topic) provided by the simulator (Figure 7). The calculated trajectory reached a mean error of 0.55 m and a maximal error of 1.26 m at the testing distance of 106.31 m. This outcome signified the ability of the VO algorithm to accurately reconstruct the trajectory, even in the face of challenging environmental conditions and a limited number of features, due to the significant self-similarity of the environment. Furthermore, the level of error implied that the simulation environment was just as demanding as real-world outdoor scenarios [58]. This simple proof of concept illustrates the simulator’s capability for running vision-based navigation algorithms and using the generated ground truth information to evaluate their accuracy.

5. Conclusions

In this article, we presented a new space rover simulator and toolkit, achieving a high degree of rendered scene realism, ease of use through the adoption of a popular game engine, and familiar, industry-standard ROS interface compatibility, with integrated support for a wide range of heterogeneous computational hardware. The presented solution allows researchers to develop and test perception and control algorithms for autonomous lunar and exoplanetary rovers based on vision sensors. At its core, it is integrated with ROS 2 while providing superior visual fidelity when compared to other existing approaches. Two modes of operation allow for synthetic data generation and software integration testing. The configuration file can be used to modify the simulation parameters to meet specific usage scenarios, enabling automated testing across a range of parameters, along with easy experiment versioning and replication. Integration with ROS 2 allows for testing a rich set of existing algorithms in exploration scenarios involving broadly understood perception, navigation, and motion planning, using the newest methods based on deep learning.

The simulator displayed favourable performance, in terms of generated image fidelity. It achieved results that surpassed those achieved with the most popular synthetic lunar rock dataset in the transfer learning experiment, in which the test set consisted of real lunar and analogue mission data, confirming its usefulness using objective evaluation. The simulator-generated data were also successfully used for evaluating the object detection and visual odometry algorithms. All the experiments were performed using a range of various computational platforms, including conventional microcomputers and embedded devices with GPU AI accelerators, as well as complex system-on-chip devices with programmable hardware. This confirmed the simulator’s usefulness, in terms of testing using real mission hardware. The simulator is flexible enough to accommodate other devices.

The solution consists of modules based on standardised components and, as such, has very good potential for expansion, in terms of the range of simulated sensors, environments, and other functions or integration as a part of more complex test setups. Overall, the presented simulator significantly extends the capabilities for replicating critical components of space robotics applications in perception and autonomy. The simulator can easily be extended, due to its modular nature, providing support for other hardware, sensors, and terrestrial and extraterrestrial environments. The visual fidelity can also further be extended by using more realistic textures and object models, as well as by the application of domain transfer methods [59]. The authors hope that the presented suite of software will pave the way for further advancements in the field of space robotics and beyond.