How Can Remote Sensing Reduce Required Human Intervention in Robotic Forest Regeneration

Abstract

In this article, we introduce an alternative solution for forest regeneration based on unmanned ground vehicles (UGV) and describe requirements for external data, which could significantly increase the level of automation. Over the past few decades, the global forested area has decreased, and there is a great need to restore and regenerate forests. Challenges such as the lack of labor and high costs demand innovative approaches for forest regeneration. Mechanization has shown satisfactory results in terms of time-efficient planting, although its usage is limited by high operational costs. Innovative technologies must be cost-efficient and profitable for large scale usage. Automation could make mechanized forest regeneration feasible. Forest regeneration operations can be automated using a purpose built unmanned platform. We developed a concept to automate forest planting operations based on mobility platform. The system requires external data for efficient mobility in clear-cut areas. We developed requirements for external data, analyzed available solutions, and experimented with the most promising option, the SfM (structure from motion) technique. Earth observation data are useful in the planning phase. A DEM (digital terrain model) for UGV planter operations can be constructed using ALS (airborne laser scanning), although it may be restricted by the cost. Low-altitude flights by drones equipped with digital cameras or lightweight laser scanners provided a usable model of the terrain. This model was precise (3–20 cm) enough for manually planning of the trajectory for the planting operation. This technique fulfilled the system requirements, although it requires further development and will have to be automated for operational use.

1. Introduction

According to the Food and Agriculture Organization (FAO) [1], the total forested area globally has decreased from 1990 to 2020, from 32.5% to 30.8%. Besides forest management (e.g., harvesting), land-use changes and disturbances have played a significant role in this decrease. Annually, there were roughly 207 million hectares of forest land disturbed between 2003 and 2012 [2]. Due to these substantial amounts of disturbance and harvesting, there is a great need to restore and regenerate forests. Forest regeneration is a cornerstone of sustainable forestry. It is divided into two parts: natural and artificial regeneration. On the one hand, natural regeneration is an inexpensive way to regenerate forests, although it requires time for a stand to develop and tree species composition may not be satisfactory. On the other hand, with artificial regeneration a forest owner can choose the tree species to plant and regeneration is quicker and usually effective. For successful regeneration, there must be quality planting stock and planters. Despite the positive results of artificial regeneration, only 31.5% of the forest area in North Europe is artificially regenerated and 68.5% is left to naturally regenerate [3].

Over the last 50 years, mechanization and automation have been established in forestry and mechanized harvesting has reduced felling costs. Forest regeneration, as with planting and cleaning, still largely depends on a manual work force [4]. Even though new technologies have decreased seedling costs, the labor costs of planting have increased [5]. Mechanization in planting produces high-quality regeneration and is time-efficient but still has low cost-efficiency [6]. In Finland and Sweden, only 1–5% of all seedlings are planted mechanically [6,7].

There are only a few examples of new technological advances in forest regeneration. Since the recent appearance of unmanned aerial systems (UASs) in the civilian market, new options are available to estimate and map forest biophysical properties [8] and to plant trees using unmanned aerial vehicles (UAVs) [9]. Only the Finnish company Rakkatec has developed an unmanned ground vehicle (UGV) for forestry applications [10]. Their modular platform Rakka 3000 can be equipped with a variety of forestry and agricultural machinery and tooling but it is not well suited for forest regeneration purposes, and although remote controlled it still requires human operation. Automation could increase the cost-efficiency of mechanized forest regeneration systems. Automating could decrease the amount of human intervention needed that would reduce operational costs. There are two automation possibilities, using purpose-built machines or automating the current solutions. Both solutions require on-board sensors. However, using only on-board sensors for navigation may not achieve the required reduction in human intervention to make the system economically feasible. Purpose-built autonomous vehicles could be smaller, since no cabin is needed and one operator could control many machines simultaneously, increasing efficiency. There are some automated excavators [11] that could be used for forest regeneration but there are very few purpose-built autonomous vehicles suitable for forest regeneration operations.

In forest regeneration areas, there are obstacles and objects that will affect the planting operation and mobility of the platform. Some of these obstacles and objects could be detected using remote sensing data. Large characteristics (e.g., flooding, drought, branch piles, ditches) must be detected to plan the planting operation and smaller obstacles (e.g., tree stumps, stones, dead wood) must be detected to plan the trajectory for the platform.

In this paper, we discuss how remote sensing can support modern semi-autonomous forest planting systems. We use a commercial off-the-shelf UGV as an example for concept development. Based on the system parameters, we develop requirements for the external data needed to improve system performance. We compare the requirements with remote sensing data currently available from satellite-, aircraft-, and drone-based sensors.

2. Robotic Forest Regeneration

Technological advances have been made in autonomy and machinery development, and some of those can be used for forest regeneration. Purpose-built unmanned ground vehicles designed for rough environments [12] such as forest regeneration areas can carry the necessary payloads and can regenerate forests autonomously. The potential operational concept for mid-sized UGVs with the necessary autonomous functionality to automate planting operations has three main modules:

• Mobility platform (UGV);
• Autonomous functionality (waypoint navigation, obstacle avoidance);
• Payload (mechanized planting payload).

2.1. Mobility Platform

We chose the Milrem Multiscope UGV (Figure 1) as an example of a mobility platform with the necessary functionalities to automate the mobility needed. The mobility platform has two crawler modules that are mechanically and electrically connected. Between the crawlers module there is a free area for the payload. The UGV has a net weight of 1630 kg and payload capacity of 1200 kg. The tracked UGV can move in regeneration areas. The Multiscope UGV is capable of ascending–descending gradients of up to 60% and driving sideways along gradients of up to 30%. It has a specific ground pressure of 16.7 kPa, which is less than one-third of the specific ground pressure exerted by a human foot. This means that damage to soil is minimal, it can operate in wet areas, and it can manage steeper slopes than wheeled machinery [13]. The UGV has a turning radius of 0 m, i.e., the machine can turn itself on the spot. The maximum traction force is 15 kN, depending on the surface. The UGV is a diesel–electric hybrid powered by electric motors (2 × 19 kW) and a diesel generator. The vehicle can be operated from a working station and operations are supported by autonomous functions such as waypoint navigation [14].

2.2. Autonomous Functionality

The crucial part of the automation involves GNSS-based waypoint navigation [15] (Figure 2). This functionality enables the trajectory to be configured through a user interface (UI), as depicted in Figure 2. This enables automation of the mobility part of the planting operation. The UGV will move autonomously on a pre-defined track and conduct a planting operation with a suitable payload.

For the regeneration operation, the trajectory is set by the human operator from the UI; the planting payload then prepares the soil and plants at the required spacing.

2.3. Payload

A planting payload for the UGV is visualized in Figure 3. This was designed to carry up to 380 seedlings and uses spot mounding as the soil preparation technique [5].

In order for the system to become economically feasible the payloads must be modular so that the UGV can be used for other operations, thereby increasing the machine utilization (MU) rate [6]. The payload is powered by a hydraulic power unit.

The forest regeneration service architecture based on the UGV technology is illustrated in Figure 4. At the workstation, the daily operation is pre-planned in terms of the navigation path and service requirements. In this phase, external data are used for pre-planning, including choosing the right tree species. Next, the vehicles are delivered to the production site using conventional trailers. The operator oversees unloading and driving from the unloading point to the production site. When the planting begins, the operator monitors the working systems from a workstation and intervenes when necessary, using tele-operation capabilities. When below-ground obstacles appear and planting is interrupted, the UGV or the planting tube will move 10–20 cm to retry planting. Communication is performed by radio link or a local network. Depending on the specific operation, the operator will continue providing support in terms of refuelling, periodically refilling the seedlings container, and other tasks. The operator at the workstation can provide decision-making support for the entire UGV fleet. During operations, the UGV could gather data regarding the surroundings and information about the condition of each seedling, surrounding trees, soil, and other site characteristics using LIDAR [16], accelerometers, gyroscopes, and video analysis. These data could be used as inputs to a forestry-specific digital twin, for forest modeling, or for training purposes.

3. Requirements for External Data

External data are needed for pre-planning and to set the trajectory [17] for planting operations. In forest regeneration areas, there are obstacles that cannot be surpassed by UGVs, for example tree stumps higher than 60 cm. These obstacles require human interventions, whereby the operator must manually overcome the obstacle or drive around it. The main purpose of using external data is to reduce these human interventions by choosing a trajectory for the UGV that would avoid these obstacles as much as possible. To choose the right trajectory, the locations of the obstacles must be known before planning the operation for the UGV. If this could be done with remote sensing data, it would avoid walking through every operational area and marking these obstacles manually before operation with the UGV begins. If the remote sensing data are usable, the human intervention rate could be lowered to enable economically feasible planting operations with purpose-built unmanned vehicles.

In reforestation, for quality work with UGVs, high-quality information about different obstacles is needed, which may affect the mobility or interrupt mounding and planting activities. The most common obstacles and objects found on the surfaces of clear-cut sites and disturbed areas are stumps, roots, stones, and slash [18,19,20]. Depending on the site, these kinds of objects vary, and there can also be deadwood, holes, roads, ruts, seeds, retention trees, advance regeneration, or other vegetation. For example, a fresh clear-cut site with stumps, slash, retention trees, and advance regeneration is shown in Figure 5 and Figure 6.

When planning the trajectory for the UGV, mechanical constraints must be known and considered. The data for the location, size, and type of obstacle will enable planning of an optimal trajectory that will increase the efficiency of the system [17]. The UGV mobility characteristics are known, which is the baseline for developing high-level system requirements (

Table 1. System requirements that are relevant for defining the need for remote sensing.

The System Shall Be Capable of
Detecting and labeling obstacles and delivering respective mobility decisions.
Assembling a perception model of the mobility environment based on acquired and pre-recorded sensor information.
Augmenting a perception model of the mobility environment based on sensor information.
Classifying objects (branches, stumps, stones, dead wood).
Classifying large excessively moist areas from acquired sensor information.
Classifying positive slopes from acquired sensor information.
Classifying negative slopes from acquired sensor information.
Perceiving no-go zones from data acquired from sensors.
Classifying large excessively dry areas from acquired sensor information.
Classifying large disturbance areas from acquired sensor information.

) for external data concerning platform mobility.

4. Earth Observation Data

Precision driving for planting requires detailed data to detect small and large obstacles that affect platform mobility. The ground is rarely smooth, although by detecting the obstacles, objects, and environmental conditions, one can create a cost map on which the platform can autonomously operate. The forest environment is diverse and changing, so the cost map should be updated or the data should be collected right before the operation. Once the cost map is developed, the trajectory for the mobility platform can be set by human operator or by the mission planning software from the workstation. We summarize different remote sensing methods that can detect and map these objects and obstacles and their accuracy in

Table 2. Remote sensing sensors and platforms providing supportive data for fulfillment of system requirements. ALS—airborne laser scanning; DLS—drone laser scanning; SAR—synthetic aperture radar.

The System Shall Be Capable of	Sensors	Spatial Resolution	Platform
Detecting and labeling obstacles and delivering respective mobility decisions.	LIDAR, RADAR, Visual cameras,	5–50 cm	Drone
Assembling a perception model of the mobility environment based on acquired and pre-recorded sensor information.	Sentinel-2 MSI, Landsat-8 OLI, sparse ALS, dense ALS, SAR	1–30 m	Satellite
Augmenting a perception model of the mobility environment based on sensor information.	LIDAR, RADAR, Visual cameras,	5–50 cm	Drone
Classifying objects (branches, stumps, stones, dead wood).	LIDAR, RADAR, Visual cameras,	5–50 cm	Drone
Classifying large excessively moist areas from acquired sensor information.	Sentinel-2 MSI, Landsat-8 OLI, sparse ALS, dense ALS, SAR	1–30 m	Satellite
Classifying positive slopes from acquired sensor information.	LIDAR, RADAR, Visual cameras,	5–50 cm	Drone
Classifying negative slopes from acquired sensor information.	ALS, DLS	50 cm	Aircraft
Perceiving no-go zones from data acquired from sensors.	ALS, DLS	50 cm	Aircraft
Classifying large excessively dry areas from acquired sensor information.	Sentinel-2 MSI, Landsat-8 OLI, sparse ALS, dense ALS, SAR	1–30 m	Satellite
Classifying large disturbance areas from acquired sensor information.	Sentinel-2 MSI, Landsat-8 OLI, sparse ALS, dense ALS, SAR	1–30 m	Satellite

.

Satellite remote sensing can provide data for pre-planning. Sentinel-2 MSI and Landsat-8 OLI scanners provide multispectral images with spatial resolutions ranging from 10 to 30 m [21,22]. These data are provided for public access free of charge, which is particularly important for cost efficiency. However, such data are only suitable for mapping objects or land patches 50 to 100 m in size [23]; detection of single trees or advance regeneration is not possible. Potential applications are confirmation that a clear-cut event has occurred, estimating the cover of green vegetation and variability of radiance that can be related to variability of the local conditions. Shortwave infrared bands located in the spectral range of 1300 to 2500 nm are sensitive to water and can be used to detect flooding. The data are used to detect disturbances (flooding, pathogens, insects, forest fires) and plan forest management activities.

Aerial images for topographic surveys are taken at altitudes ranging from 2 to 10 km and the spatial resolution may range from 0.1 m to several meters. The cameras usually have four spectral bands (blue, green, red, and near-infrared). Stereoscopic measurements can be made from aerial images if there is sufficient overlap between photos, and then a 3D point cloud can be constructed. By combining spectral signatures and 3D point clouds, it is possible to detect retention trees, skidding roads that are usually filled with cutting slash, and patches of vegetation at a level that may be usable for pre-planning of UGV planter movement.

Airborne laser scanning (ALS) has reached a level of technical readiness for practical applications. The measurements are conducted for topographic surveys for the construction of digital terrain models (DEM), although 3D point clouds also contain information about the plant canopy structure. The ALS data are characterized by laser wavelength, point density per unit area, illuminated footprint size at the position of return, and count of registered returns per emitted pulse [24]. Regular ALS flights are conducted at altitudes that enable 0.1–5 returns per square meter. This is not sufficient to detect tree stumps, although these data are sufficient to construct DEMs for UGV planter operations. A compromise must be made between the cost of airborne laser scanning, achievable point density, and area coverage [25]. Denser point clouds allow analysis of finer details, although costs increase rapidly for large area measurements. In many countries, low-density point cloud datasets are already freely available.

Pre-planning planting operations may be enhanced if positions of stumps, fallen trees, rocks, or other obstacles are known. A custom technique called structure from motion (SfM) was developed to generate high-density 3D maps, from which these small obstacles can be detected. We have experimented with this technique in a relevant environment (Figure 7) using DJI Phantom 4, a single camera feed, and LIDAR. Currently, the SfM photogrammetry is used in forestry for monitoring and it enables surveys requiring low costs and little technical expertise [25]. Single or multiple camera feeds can be merged and optionally enhanced with LIDAR and GPS data to produce both textured polygonal and colored 3D point cloud maps; subsequent steps correlate the new data with existing map layers to improve the overall result. The automatic labeling of surface objects and the properties of the surface are performed to generate extra inputs used for automating cost map generation for UGV path planning.

The desired spatial density of the data is configurable, as the needs vary across applications; practical results for 3 cm and 20 cm precision reconstruction are presented in Figure 7, Figure 8, Figure 9 and Figure 10. The reconstruction and fitting are computationally intensive processes requiring central or cloud resources, although these steps enable the concurrent integration of multiple sources. The achieved level of detail enables precise navigation of UGV platforms in local reference frames, lowering the requirements of global positioning. GPS signals are unreliable close to and under trees, although even here adequate precision was achieved during testing, as the generated detailed map was reliable. Emerging from tree cover, the predicted global position matched with the reacquired GPS position.

5. Conclusions

We have described an alternative planting solution based on a mid-sized UGV with the necessary autonomous functionality to automate forest planting operations. Without pre-mapping, the existing autonomy may not be enough for the human intervention rate to be low enough for the system to operate cost-effectively. The future of automated UGVs for forest regeneration depends on total costs per unit area or per plant. This study shows that one of the important elements that can drive down the regeneration cost is the efficient use of external data in the pre-mapping phase. When combining onboard sensors with external data, one can achieve human intervention rates that are economically feasible for planting operations with purpose-built unmanned vehicles. The cost of the external data must be considered.

Satellite information from Sentinel-2 MSI and Landsat-8 OLI scanners may not fulfil the system requirements for mapping terrain obstacles because of the low spatial resolution. However, information gathered from satellites could be used to confirm that a clear-cut has occurred or to estimate green vegetation cover. There are also commercial satellites with better resolutions, although these may be costly. Free ALS data could be outdated and the spatial resolution may not be sufficient to fulfil the system requirements. Ordering specific overflights for the planting operation does not seem feasible. However, ALS could be used to create elevation maps.

Based on a literature review and an experiment we conducted, the SfM technique seems to be the most suitable option for pre-mapping. This can be done just before the operation, using a drone and sensors that provide up-to-date data. This approach is cost-efficient compared to booking overflights and is the most accurate solution compared to other available solutions. According to the 3D model created with the SfM technique, an operator can plan the trajectory to avoid the obstacles in the path of the UGV. Drone flying, data collection, modeling, and trajectory creation can be automated, making this process cost-effective.