Next Article in Journal
Predicting Soil Organic Carbon and Total Nitrogen at the Farm Scale Using Quantitative Color Sensor Measurements
Next Article in Special Issue
Remote Detection of Growth Dynamics in Red Lettuce Using a Novel Chlorophyll a Fluorometer
Previous Article in Journal
Effects of TaPHS1 and TaMKK3-A Genes on Wheat Pre-Harvest Sprouting Resistance
Previous Article in Special Issue
Evaluating Two Crop Circle Active Canopy Sensors for In-Season Diagnosis of Winter Wheat Nitrogen Status
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Review

Sensing and Automation in Pruning of Apple Trees: A Review

1
Department of Agricultural and Biological Engineering, Pennsylvania State University, University Park, PA 16802, USA
2
Fruit Research and Extension Center, Pennsylvania State University, Biglerville, PA 17307, USA
3
Department of Plant Science, Pennsylvania State University, Pennsylvania State University, University Park, PA 16802, USA
*
Author to whom correspondence should be addressed.
Submission received: 23 July 2018 / Revised: 25 September 2018 / Accepted: 26 September 2018 / Published: 30 September 2018
(This article belongs to the Special Issue Sensing and Automated Systems for Improved Crop Management)

Abstract

:
Pruning is one of the most important tree fruit production activities, which is highly dependent on human labor. Skilled labor is in short supply, and the increasing cost of labor is becoming a big issue for the tree fruit industry. Meanwhile, worker safety is another issue in the manual pruning. Growers are motivated to seek mechanical or robotic solutions for reducing the amount of hand labor required for pruning. Identifying tree branches/canopies with sensors as well as automated operating pruning activity are the important components in the automated pruning system. This paper reviews the research and development of sensing and automated systems for branch pruning in apple production. Tree training systems, pruning strategies, 3D structure reconstruction of tree branches, and practice mechanisms or robotics are some of the developments that need to be addressed for an effective tree branch pruning system. Our study summarizes the potential opportunities for automatic pruning with machine-friendly modern tree architectures, previous studies on sensor development, and efforts to develop and deploy mechanical/robotic systems for automated branch pruning. We also describe two examples of qualified pruning strategies that could potentially simplify the automated pruning decision and pruning end-effector design. Finally, the limitations of current pruning technologies and other challenges for automated branch pruning are described, and possible solutions are discussed.

1. Introduction

The tree fruit industry is an important component in the U.S. agricultural sector, accounting for 26% ($11 billion) of all specialty crop production. Among all crops, apple is one of the most valuable agriculture commodities in the United State [1]. Presently, the majority of tree fruit crop production systems are highly dependent on seasonal human labor. Many critical activities are not only labor intensive, but are also highly time sensitive. This intense labor demand creates a significant risk of growers not having a sufficient supply of labor to conduct seasonal tasks for tree fruit production [2,3]. Going forward, it is critical to minimize dependence on labor for the long-term sustainability of this industry [4].
In recent decades, automation technologies, especially the auto guidance for field tractors has been investigated widely [5,6,7]. The auto guidance technologies were adopted by large industry companies, such as John Deere, to develop autonomous tractors. However, for the specialty crops including tree fruit crops, the application of automation and precision has lagged behind due to the complexity of field operations and inconsistency of crop systems [8]. Studies have shown great potential for using mechanical or robotic system for trees with compatible canopy training, including autonomous assist platform [9,10], and mechanical/robotic harvesting [11,12,13]. Machine vision is one of the core technologies in the automation of tree fruit production. A machine vision system is used to detect and localize the fruit for harvesting, and branch detection and reconstruction for robotic tree training and branch pruning [14].
Proper training through correct pruning is important for a healthy, strong fruit tree [15]. There are many different tree training systems for apple trees (see Table 1 for the detail). Each training system has its own advantages and disadvantages. Many modern tree fruit orchards are planted high-density using dwarfing rootstocks and training systems designed for maximum sunlight interception, higher fruit yields and quality, and easier worker access [16]. New mechanization-friendly orchard architectures with better machine accessibility to either fruits or branches is presenting the promising opportunities for automation in apple production. To achieve targeted tree systems as well as obtain satisfied fruit production, pruning is required in the dormant season and/or summer. Of course, different pruning strategies would be applied to regarding training system for fulfilling the target tree systems.
Dormant pruning of fruit trees refers to removing unproductive parts of trees, and is essential to maintain overall tree health, control plant size, and increase fruit quality and marketable yield. Presently, dormant pruning is still accomplished by field crews either using manual loppers or powered shears, which is a very labor-intensive operation. Pruning is the second largest labor expense for tree fruit field production behind harvesting, accounting for 20% or more of total production cost in the full production years (the fixed investment breakdown for each year, such as land, trellis, irrigation setup and others, is not included) [17,18]. Figure 1 shows an example of Gala apple orchard cost distribution in full production years (reproduced from [17]). Meanwhile, worker safety is another concern in the manual pruning, including falls, lacerations, slips, head/eye injuries, strain to shoulder/back, and repetitive motion injuries [19]. Due to the high cost and declining availability of skilled labor, and the safety issues in the manual pruning, alternative solutions for pruning fruit trees are becoming essential. Automated pruning, including non-selective mechanical pruning and precise robotic pruning, was proposed to address these issues.
Mechanical pruning, mainly referring to hedging, is a non-selective process with a high rate of throughput, which will usually require follow-up hand pruning [20]. Hedging has been tried in the past as a supplement or replacement to selective hand pruning. Ferree and Lakso [21] evaluated hedging for dormant pruning of vigorous semi-dwarf apple trees, and reported negative consequences of low within-canopy light levels and poor fruit color. These conditions resulted from a proliferation of new shoots arising from the numerous non-selective heading cuts made by the hedger. Summer hedging with dormant selective hand pruning, conversely, was shown to be beneficial in creating higher light levels in the lower canopy [22].
Robotic pruning, which is a selective pruning with accurate cuts, typically cuts the branch by an end-effector consisting of a cutting blade and anvil that uses a scissors motion [23]. Before cutting, the challenge is to detect the targeted branches, and identify the pre-determined cutting point. The location, orientation, and dimension of the branch are the critical information for conducting accurate pruning. For cutting itself, it also requires a highly specific degree of accuracy with respect to the placement of the end effector, the jaws of which must be maneuvered into a position over the branch and perpendicular to branch orientation. This level of specificity in the spatial placement of the end effector results in a complex set of maneuvers and slows the pruning process, resulting in low efficiency.
In the following sections, we discuss the tree training systems and strategies for automated pruning, the machine vision sensing technologies for tree branch identification, the current development of automated pruning systems, and the issues and challenges that remain in the procedure.

2. Tree Training Systems

Tree training systems are critical to the success of adopting automation in orchard production. Previously, studies were conducted on free-standing trees on semi-dwarf rootstocks, established at moderate planting density and trained as central leader trees, with numerous scaffolds and a complex branching hierarchy. Modern intensive orchards have a smaller canopy with less branching hierarchy, and are grown at close spacing on size controlling rootstocks that restrict tree vigor, resulting in a smaller simpler canopy. Trellising reduces the variability in canopy shape and position. This serves to make the operation of machinery simpler and less fatiguing to the machine operator and should facilitate more predictable and repeatable results. Table 1 shows the typical tree training systems of apple trees and their specifications.
Detection and accessibility to branches and fruits is a key factor for automating labor-intensive orchard tasks. Establishment of intensive orchards systems at close spacing and using size-controlling rootstocks and training systems is a global trend. Modern intensive orchard systems could provide easier detection and access to both tree canopy and fruits, resulting in higher potential of applying mechanical and robotic technologies [32]. A “Robot Ready” concept was proposed recently to train and manage tree orchards for robotic harvesting [33]. Horticultural advancement along with the attempt of conducting automated activities in tree fruit production has been blooming recently. Various intensive modern tree architecture systems have been developed and tested for production and labor efficiency. Figure 2 shows two examples: a V-trellis fruiting wall system with horizontal branches, and the tall spindle tree system.
For intensive fruit systems, especially those trained to 2D planar fruiting wall, the narrow canopy becomes much simpler and easier to access with machines, which has brought great benefit to tree fruit growers in many aspects. Previous studies have documented the effect of intensive tree architectures in terms of light interception and distribution [34,35], the influence on yield and fruit quality [36,37,38], earlier production and higher returns [39], and compatibility to mechanical solutions such as blossom thinning machines or harvest aids [40,41]. Additionally, mechanical or robotic harvesting is also becoming more promising on intensive trees [12,42,43]. He et al. [42] developed a shake and catch harvesting system for trellis trained V-trellis apple trees. Their results indicated that this tree system provides an opportunity to shake only targeted fruiting limbs and catch the fruit just under those limbs, increasing the potential to keep fruit quality at a desirable level for the fresh market. In [43], the fruit detection and picking rate could reach 100% and 85% for robotic apple picking if working with horizontal branch fruiting wall system. Fewer studies have focused on automated pruning for the intensive architectural trees. With a simplified tree architecture, tree branches could be detected and identified much more easily, by applying simple rules for robotic pruning [44].

3. Pruning Strategies for Automated Pruning

Pruning strategies for fruit trees were mainly determined by certain rules that manage canopy size and shape to improve the light distribution, with the primary goal of improved fruit quality. Another goal for pruning is to remove a certain amount of flower buds to manage crop density [45]. Schupp [46] proposed a severity level pruning strategy for tall spindle apple trees to provide guidelines for determining the cutting threshold for robotic pruning. In robotic pruning, cuts must be precisely defined so the computer can transfer accurate information to the pruner. Researchers have studied on developing simple and quantified pruning rules to increase the feasibility of using robotic and automated pruning [44]. Table 2 summarizes some of the pruning strategies for apple trees with different tree training systems.
In Table 2, for automated pruning, the technique difficulties were mainly on localizing the pruning locations through branch detection and reconstruction. Meanwhile, the accessibility of pruning machine/robot is another challenge, namely the difficulty of cutting end-effector to reach the cutting point. For hedging pruning, the technical critical would be the location identification for precise cutting. To provide more detail information about pruning strategies, two examples of quantified pruning rules are given here, which could be potentially used for robotic pruning systems.
Case 1: Severity pruning levels for tall spindle apple trees [47].
Dr. Schupp and his team have been working on creating an effective pruning strategy for tall spindle apple trees based on severity levels. This study was a component of USDA NIFA project Automation of Dormant Pruning of Specialty Crops. The initial phase of the project established four pruning rules, i.e., the pruning strategies for the pruning task. A pruning severity index, namely limb-to-trunk ratio (LTR), was calculated from the sum of the cross-sectional area of all branches on a tree at 2.5 cm from their union to the central leader divided by the trunk cross-sectional area at 30 cm above the graft union (Figure 3). In the LTR index, a lower value means less limb area relative to trunk area, which represents more severe pruning. Six severity levels ranging from LTR 0.5 to LTR 1.75 have been applied by successively removing the largest branches from the apple trees. The LTR provides a measurable way to define and create different levels of pruning severity and achieve consistent outcomes. This allows a greater degree of accuracy and precision to dormant pruning of tall spindle apple trees. The use of the LTR to establish the level of pruning severity provides a simple and consistent rule for using of autonomous pruning systems.
The rules and pruning strategy generated in this study are very easily implemented, removing only those branches with diameters greater than the setting level. Once the pruning severity level is determined, the maximum allowable branch diameter could be calculated easily based on the calculation. Those with diameters greater than the threshold will be cut from the branch base or about one inch away the trunk depending on the necessary of new branch growth. Therefore, the first critical information needed for automated pruning is to measure the diameter of the trunk as well as the diameter for each individual branch, then the cross-sections and LTR are calculated, and pruning decision are made. This method is suitable for intensively planted trees with minimal branching complexity and no permanent branches, such as the tall spindle or the super spindle. To apply robotic pruning, it will require machine vision to locate branches and map a pruning path.
Case 2: Pruning based on twig length and length/diameter ratio [48].
While the primary goal of this study was to investigate the effect of mechanical harvesting with different pruning treatments, it is still a good model for developing automated pruning, since the proposed rules are simple and measurable. Four different treatments were applied, namely 10 cm, 15 cm, and variable twig length pruning, as well as a control set of 20 cm twig length pruning. In variable length pruning, the diameter to length ratio was set to 0.06 based on our previous study [51]. Figure 4 shows an example of cutting a twig to the length of 15 cm (reproduced from [48]).
The pruning rules proposed in this study are mainly for the trellis trained trees with horizontal permanent branches, the removed part is from the twigs growing from the permanent horizontal branches. Three different pruning treatments with different twig lengths were compared, and results showed that shorter length twigs had higher fruit removal efficiency, while it required more cuts since there were more twigs needed to be cut if the targeted twig length is shorter. Furthermore, by adopting the twig diameter into consideration, an index was proposed based on the ratio of twig diameter and length. With determined index, the twigs with larger diameter could retain longer. The results indicated that the treatment of using an index determined pruning treatment achieved very promising fruit removal as well as fruit yield. The pruning strategy created in this study gained the guidance for the autonomous pruning by providing the specific rules to cut the branches. To apply the created pruning strategy for automated pruning, identifying the twig length as well as the diameter would be essential.

4. Machine Vision Sensing for Branch Detection and 3D Reconstruction

4.1. Tree/Branch Detection/Reconstruction using Machine Vision System

Normally, the first step of automated pruning is to find tree branch and target the cutting location in the branch. A proper sensing technique is essential to detect the branches in the fruit trees, and then select unwanted branches and cutting points based on the desired pruning rules to conduct selective pruning. With the detection of the tree branches, the branches and the location of each branch could be identified. To further obtain branch detail information, such as diameter, orientation, as well as overall tree structure, 3D reconstruction is essential. Machine vision is a system combined with sensors and algorithms to obtain information of the target objects. Machine vision sensing has been used in agricultural application for several decades for various operations [52,53,54,55]. A review summarized the sensors and technologies of 3D imaging systems for agriculture applications [56]. Many different sensors have been used in machine vision system for detection of agricultural objects, e.g., cameras and Lidar sensor. Table 3 lists some past studies using different sensors and techniques for tree/branch detection and 3D structure reconstruction. Among those applications, identifying branches and pruning points for robotic pruning is one.

4.2. Lidar Based Machine Vision System

Lidar based machine vision systems have been mainly used for biomass mapping or individual tree detection, with plenty of them for the forest application [57,58,59]. Especially, Brandtberg et al. [57] developed a 3D tree reconstruction method by fitting cylinders into a point cloud derived from a terrestrial laser-scanned tree. Using the developed method, the coefficient of determination was 0.965 and showed high potential of using for forest inventories. Furthermore, they developed an open source tool named Simple Tree to provide an efficient optimization approach for tree reconstruction [73]. Recently, Li et al. [74] proposed an adaptive extracting method of tree skeleton based on the point cloud data with a terrestrial laser scanner, and obtained consistent tree structure. A Lidar sensor also has been tried for the branch length and diameter identification [75]. In their study, a 3D canopy structure of trees was modeled using Lidar sensor and a reconstruction algorithm, the results indicated that the correlation could be up to 0.78 and 0.99 for branch length and branch diameter, respectively. Mapping the pruning wood for grape vines in the vineyards was another application of laser scanner [62]. Furthermore, in [66], a laser sensor was used to collect observation of fruit trees aiming for automatic dormant pruning, the results showed that the system is able to identify the primary branches with an average accuracy of 98% and estimate their diameters with an average error of 0.6 cm. Even though the current system is too slow for large-scale practice, the study showed the potential of using the proposed approach to develop robotic pruning systems in the near future.

4.3. Camera Based Machine Vision System

Camera based machine vision system has been widely studied and/or applied in agriculture production in the past several decades for many applications. Among those, fruit detection for tree fruit robotic harvesting attracted most attention [76,77,78,79]. There are also a few studies on the branch detection to determine shaking positions using 3D sensing for mechanical massive harvesting [60,61]. Amatya et al. [60] used RGB and 3D cameras to detect and reconstruct the branches that could be used for determining the shaking points for the mechanical sweet cherry harvesting (Figure 5A); and Zhang et al. [61] used a Microsoft Kinect v2 (ToF camera, Microsoft, Seattle, DC, USA) and a convolutional neural network (CNN) deep learning algorithm to detect and model the horizontal branches in the V-trellis fruiting wall apple trees for mechanical harvesting purpose (Figure 5B). Some other applications of 3D tree reconstruction with machine vision system include a vision sensor (stereo camera) based 3D tree reconstruction for automated blossom thinning [80], 3D models developed with source images to reconstruct the overall tree shape [81], and a single range image used to construct a 3D model of the trunk and branches of a real tree [82].

4.4. Branch Detection/Reconstruction for Automated Pruning

Most effort for machine vision towards branch detection for robotic pruning was on grape vines due to its more uniform and organized canopy architecture. For example, some researchers used a single 2D camera and image processing techniques for identification of pruning points in grapevines [64,67,83,84]. Furthermore, a stereo vision system based 3D machine vision system was used and cutting points on the branches were determined with remaining certain length of branches by segmenting the branches and measuring the length of branches [85]. A computer vision system builds a three-dimensional (3D) model of the vines, an artificial intelligence (AI) system decides which canes to prune, and a six degrees of freedom robot arm makes the required cuts [61].
Some progress has been made in the development of robotics technology for pruning more complex canopies such as apple trees. A machine vision system was proposed and evaluated with 3D camera to detect and identify tree branches for pruning for tall spindle apple trees [44,86]. The studies developed an algorithm with two simple rules to determine the pruning points, i.e., branch length and inter-branch spacing. The results showed that the algorithm removed 85% of long branches, and 69% of overlapping branches. A few other studies have also been conducted on tree modeling using different vision sensing for automatic pruning, such as Kinect 2 [68], RGBD [71], depth image [72], and time-of-flight data [67]. Tabb and her collaborator focused on developing a 3D reconstruction of fruit trees (Figure 6, reproduced from [69]) for automatic pruning with identifying the branch parameters such as length, diameter, angle, etc. [69,70,87]. Based on the studies above on the tree branch pruning, the accuracy of branch detection and identification, the efficiency of branch reconstruction as well as the cost of the system would be critical for the success of the robotic tree branch pruning system.

5. Automated Pruning System

As introduced above, manual pruning is a labor intensive and costly operation in the tree fruit production. Mechanical pruning and robotic pruning are two alternative methods to replace the manual pruning. By integrating with autonomous platform, these two methods could accomplish automated pruning task.
Among those commercialized mechanical systems for tree fruit crop production, mechanical pruning system is one of them. Here, mechanical pruning mainly refers to hedging. There are a few types of pruning machine available on the market designed for tree fruit crops, such as disc-type cutter and teeth-type cutter [88]. Depending on the requirement, mechanical pruning could be performed with topping the canopy parallel to the ground, and/or hedging on both sides of canopy [89]. Figure 7 shows a typical mechanical pruning system with saw-tooth cutter.
Hedging is a non-selective mass pruning system in which a cutting tool is run over rows in orchards keeping pre-determined distance from the trees. With this approach, everything beyond certain distance from canopy center and/or above certain height is removed [90]. Hedging pruning has gained extensive application for grape vine pruning [91,92,93]. The application of mechanical pruning for vine grapes showed significant cost saving while maintained yield and fruit quality [94]. For tree fruit crops, non-selective mechanical pruning has been investigated for different crops, such as sweet cherry [95], citrus [88], and olive [96]. Mechanical pruning of citrus obtained increased yields and reduced pruning cost [97]. Mechanical pruning showed certain level of promise for citrus and olive. For example, medium-low vigor olive trees can be successfully controlled by mechanical pruning without compromising their yield [98]. Non-selective pruning systems are limited in their ability to ensure the quality of pruning for some other fruit trees, such as sweet cherry, peach and apple [95], which could result in excessive growth of shoots, leading to reduced fruit quality and yield [99]. On the other hand, inadequate pruning will result in a tree populated with unproductive woods [95,100]. Therefore, mechanical pruning for fruit trees is mainly used for summer pruning with removing some exterior shoots to increase the light interception to the fruits [49]. Dwarf and semi-dwarf trees trained in thin hedgerows can be mechanically pruned more easily than single large trees [20,49]. With intensive tree architectures in apple trees, hedging technology would be beneficial to those trees due to the less possibility of branch regrowth with simple tree structure. Furthermore, automated hedging pruning with precise canopy size control could be considered by estimating the canopy size/shape using sensors such as Lidar.
Robotic pruning is a selective pruning operation, which aims to mechanically prune the tree branches at the same quality and level as human hands. Pruning for tree fruit crops is highly labor intensive, but no work specific to automated pruning has been carried out in the past due to a few challenges. The first challenge is the complex environments of tree canopy/structure. The second challenge is moving robotic parts quickly, efficiently and delicately. Compared to fruit trees, grape vines are relatively more uniform, which benefited from certain amount of studies and field trials.
Sevilla [101] conducted research on a robotic grapevine pruning manipulator with modeling and simulation. Ochs et al. [102] worked on a machine vision system for grapevine pruner. Similarly, Lee et al. [103] reported work in the electro-hydraulic control of a vine pruning robot. Also a manipulator and vision system was developed for multi-purpose vineyard robot [104,105]. Especially, there were two serial robots developed and tested for grape vine pruning; one is from vision robotics Inc. [106], and the other one is from [63]. However, these robotic systems focused on grapevines, which have relatively uniform and organized canopy architecture.
Software framework is another important component for automated systems. Some open source software has been widely used, including PCL (Point cloud library), Open CV (Open Source Computer Vision Library), and ROS (Robot Operating System). Holz et al. [107] illustrated a few examples that used PCL for 3D point cloud processing, including RGB-D point clouds by consumer color and depth camera, high-resolution laser scans from 3D scanners, and low resolution sparse point clouds by a custom lightweight 3D scanner [107]. Within PCL, an open source implementation of the kinectFusion algorithm is included, and can be used to reconstruct a 3D scene using the Kinect sensor like a handheld laser scanner [108]. Open CV has been widely used for the fruit detection for robotic harvesting, such as an apple detection algorithm for robotic harvesting using an RGB-D camera [109]; Open CV was used to test the haze-removal effects in apple harvest robot vision system [110]; and ROS was adopted for a navigation system in a robotic bin handling robot [111]. A set of robotic operating system nodes was created to the sensing and motion control of the robotic harvesting system [112]. In a robotic pruning system, a path planner used RRT-Connect implementation from the Open Motion Planning Library in Robot Operating System (ROS), and a Radial basis function SVM (RBF-SVM) was implemented to label each pixel as foreground, back ground, or wire, based on its color and the color of its neighborhood [63].
Even though no machine has been developed for robotic pruning for tree fruit crops, some progress has been made in the development of robotics technology for pruning more complex canopies such as apple and cherry trees. As discussed above, most studies on robotic fruit tree pruning focus on developing machine vision system for the branch identification and reconstruction, while few studies focus on the robotic arm for simulation of pruning task [113,114]. Furthermore, as plenty of robotic systems have been developed for pruning grape vines as well as picking fruits, it would be possible to develop an effective robotic pruner for tree fruit crops when the tree architecture is getting more uniform. The challenges and solutions are discussed in the following section.

6. Discussion: Challenges and Solutions

As discussed above, tree structures in modern orchard are getting much simpler by adopting the intensive system. Even with these trees, the pruning task is still relatively complex due to the natural biological system. For robotic pruning, the cuts on branches require high precision with a cutting end-effector applied at the right locations and perpendicular to branch orientation. A successful robotic pruning system needs to be c accurate, robust, fast, and even inexpensive. Therefore, the critical points for success of robotic pruning for fruit trees are the accuracy of branch identification/reconstruction, the spatial requirement of pruning end-effector, and the efficiency of pruning operation (time for branch identification and the time for maneuvering the end-effector).
To apply robotic pruning, firstly, the tree branch and cutting location need to be accurately identified. Most studies on automated pruning focus on the tree branch identification and reconstruction using machine vison system, as discussed above. Some studies have been reported on developing algorithms to improve the accuracy of the branch reconstruction [115,116,117]. Most of these studies focus on the tree skeleton from the 3D images, which typically could get the location and the length of the tree branches. It is hard to get other information, such as the diameter and angle of branches. One recent study from Tabb and Medeiros showed the ability to detect and automatically measure the branching structure, branch diameters, branch lengths, and branch angles. That information is required for tasks such as robotic pruning of trees as well as structural phenotyping. At this stage, it takes about 8 min to finish one tree reconstruction, which is too long for practical pruning process [69]. The time for finishing a pruning is determined by both the speed of sensing system and the mechanical/robotic executive system. Both speeds could be potentially improved in the future with the advancement of related technologies. It would be more practical to use stop-and-go since the pruning task for each tree varies, and the time for finish each individual tree also varies. A platform is needed for the automated pruning system, and the platform used in the mechanical harvesting of apples could be an option [48]. With the sensing system mounted on the front of the platform, and the overall pruning system could still be considered as real-time system if the pruning operation could be conducted right after the platform reaches the tree.
Not only branch identification task, but also the accessibility of the robotic manipulator and end-effector is challenging due to complexity and variability of agricultural environment, as well as the required speed of operation. Previously developed pruning robots typically use serial robotic arm with a fix cutter, while this level of specificity in the spatial placement of the end effector results in a complex set of maneuvers and slows the pruning process. Meanwhile, the serial robot arm with an end-effector requires large space for the cutter to engage with the branches. Although it is not for pruning directly, effort has been made to simplify the maneuvers and improve the efficiency of robotic operations in harvesting. Two robotic fruit picking robots have been developed and tested, one is from FFRbotics (Gesher HaEts 12, Benei Dror, Israel) and the other one is from Abundant Robotics (Abundant Robotics, Hayward, CA, USA). These robotic arms are parallel type, which limits the spatial requirement of the picking end-effector. The position of the end-effector could be adjusted at the base of the overall robotic arm, and then the picking end-effector could reach the fruit directly or by extending the rod. Similar robotic arms could be considered for developing the pruning system. There is one thing needs to consider that normally no specific orientation was required for the end-effector to engage fruits. For robotic pruning, the end-effector (cutter) needs not only to reach the right location, but also to be placed perpendicularly to the branch. To be always perpendicular to the branch, as well as using the parallel type robotic arm, the end-effector should be with adjustable orientation [118]. With this kind of end-effector, the cutter itself could be rotated with very small spatial need. Moreover, the cutter could be made of saw blade with no specific orientation constraints.
Finally, the economics of the robotic pruning system also needs to be considered. Typically, the cost for a robotic system is high. With the use of off-the-shelf robotic arms (such as Robolink, Igus, Cologne, Germany) and low cost sensing system (such as Kinect v2, Microsoft), the overall cost of the robotic system could be dramatically decreased. Therefore, with the consideration of labor shortage issue as well as putting effort on building low cost robotic pruning system with off-the-shelf components, the benefit of developing a robotic pruning system would be obvious. Meanwhile, multiple robots could be employed to improve the working efficiency. The cost/benefit ratio of a robotic pruning machine will have to be analyzed after the machine is built.

7. Conclusions

In this study, automated pruning related technologies have been reviewed, from the tree training systems, machine vision sensing, pruning strategies, as well as mechanical and robotic pruning development. Through this comprehensive review and discussion, the following statement could be concluded.
  • Tree architecture is very critical for adopting automated orchard operations such as pruning and harvesting. Intensive tree orchard with narrow tree canopy or even 2D planar fruiting wall would be suitable for fully autonomous pruning system in the future.
  • To develop robotic pruning, simple and quantified pruning rules are the essential of practical pruning strategies.
  • Many studies have focused on the tree branch identification and reconstruction, however the accuracy and efficiency still needs to be improved for practical pruning operation.
  • Robotic pruning technologies have been successfully investigated in some uniformed crops, such as grapevines. With the adoption of intensive tree architecture as well as the improvement of cutting end-effector, it is very promising to have a robotic pruning system for tree fruit crops.

Author Contributions

L.H. led the writing of this manuscript. J.S. was involved in the discussion of the manuscript structure and provided some of the writing materials, especially for the horticultural related sections.

Funding

This research was funded by U.S. Department of Agriculture: Project #PEN04653 and Accession #1016510.

Acknowledgments

This work was supported by the USDA National Institute of Food and Multistate/Regional Research and/or Extension Appropriations under Project #PEN04653 and Accession #1016510. Any opinions, findings, conclusions, or recommendations expressed in this publication are those of the authors and do not necessarily reflect the view of the U.S. Department of Agriculture and Pennsylvania State University.

Conflicts of Interest

The authors declare no conflicts of interest.

References

  1. USDA-NASS. Noncitrus Fruits and Nuts: 2015 Summary. Available online: https://usda.mannlib.cornell.edu/usda/nass/NoncFruiNu//2010s/2016/NoncFruiNu-07-06-2016.pdf (accessed on 6 July 2016).
  2. Fennimore, S.A.; Doohan, D.J. Directions. Weed Technol. 2008, 22, 364–372. [Google Scholar] [CrossRef]
  3. Calvin, L.; Martin, P. The U.S. Produce Industry and Labor: Facing the Future in a Global Economy; Economic Research Services, USDA: Washington, DC, USA, 2010. [Google Scholar]
  4. Gonzalez-Barrera, A. More Mexicans Leaving Than Coming to the U.S. Pew Research Center. 2015. Available online: http://www.pewhispanic.org/2015/11/19/more-mexicans-leaving-than-coming-to-the-u-s/ (accessed on 19 November 2015).
  5. Noguchi, N.; Kise, M.; Ishii, K.; Terao, H. Field automation using robot tractor. In Automation Technology for Off-Road Equipment Proceedings of the 2002 Conference; American Society of Agricultural and Biological Engineers: St. Joseph, MI, USA, 2002; p. 239. [Google Scholar]
  6. Kise, M.; Zhang, Q.; Más, F.R. A stereovision-based crop row detection method for tractor-automated guidance. Biosyst. Eng. 2005, 90, 357–367. [Google Scholar] [CrossRef]
  7. Zhang, Q.; Pierce, F.J. Agricultural Automation: Fundamentals and Practices; CRC Press: Boca Raton, FL, USA, 2016. [Google Scholar]
  8. Karkee, M.; Zhang, Q. Mechanization and automation technologies in specialty crop production. Resour. Mag. 2012, 19, 16–17. [Google Scholar]
  9. Lesser, K.; Harsh, R.M.; Seavert, C.; Lewis, K.; Baugher, T.; Schupp, J.; Auvil, T. Mobile platforms increase orchard management efficiency and profitability. In International Symposium on Application of Precision Agriculture for Fruits and Vegetables; International Society for Horticultural Science: Leuven, Belgium, 2008; Volume 824, pp. 361–364. [Google Scholar]
  10. Hamner, B.; Bergerman, M.; Singh, S. Autonomous orchard vehicles for specialty crops production. In Proceedings of the ASABE Annual International Meeting, Louisville, KY, USA, 7–10 August 2011; American Society of Agricultural and Biological Engineers: St. Joseph, MI, USA, 2011; p. 1. [Google Scholar]
  11. Gongal, A.; Amatya, S.; Karkee, M.; Zhang, Q.; Lewis, K. Sensors and systems for fruit detection and localization: A review. Comput. Electron. Agric. 2015, 116, 8–19. [Google Scholar] [CrossRef]
  12. De Kleine, M.E.; Karkee, M. A semi-automated harvesting prototype for shaking fruit tree limbs. Trans. ASABE 2015, 58, 1461–1470. [Google Scholar]
  13. Davidson, J.R.; Silwal, A.; Hohimer, C.J.; Karkee, M.; Mo, C.; Zhang, Q. Proof-of-concept of a robotic apple harvester. In Proceedings of the 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Daejeon, South Korea, 9–14 October 2016; IEEE: Piscataway, NJ, USA, 2016; pp. 634–639. [Google Scholar]
  14. He, L.; Baugher, T. Engineering opportunities for fruit growers. PA Fruit News 2018, 4, 10–12. [Google Scholar]
  15. Feucht, J.R.; Larsen, H. Training and pruning fruit trees. In Colorado State University Extension; Fact Sheet No. 7.003; Colorado State University: Fort Collins, CO, USA, 2009. [Google Scholar]
  16. WSU Tree Fruit. Pruning and Training Systems. Available online: http://treefruit.wsu.edu/orchard-management/pruning-and-training-systems/?print-view=true (accessed on 21 August 2018).
  17. Gallardo, K.; Taylor, M.; Hinman, H. 2009 Cost Estimates of Establishing and Producing Gala Apples in Washington; Washington State University: Washington, DC, USA, 2011; FS005E. [Google Scholar]
  18. Hansen, M. Fruiting Walls Suit Machinery. Good Fruit Grow. 2011, 62, 24–25. [Google Scholar]
  19. Penn State Extension. Fetzer, L. Orchard Pruning—Safety Tips. Available online: https://extension.psu.edu/orchard-pruning-safety-tips (accessed on 25 December 2017).
  20. Sansavini, S. Mechanical pruning of fruit trees. Symp. High Density Plant. 1976, 65, 183–198. [Google Scholar] [CrossRef]
  21. Ferree, D.C.; Lakso, A.N. Effect of selected dormant pruning techniques in a hedge-row apple orchard. J. Am. Soc. Hort. Sci. 1979, 104, 736–739. [Google Scholar]
  22. Ferree, D.C. Influence of various times of summer hedging on yield and growth of apple trees. Ohio Agric. Res. Dev. Cent. Res. Circ. 1984, 283, 33–37. [Google Scholar]
  23. Lehnert, R. Robotic Pruning. Available online: https://www.goodfruit.com/robotic-pruning/ (accessed on 1 November 2012).
  24. Dorigoni, A.; Lezzer, P.; Dallabetta, N.; Serra, S.; Musacchi, S. Bi-axis: An alternative to slender spindle for apple orchards. Acta Hortic. 2011, 903, 581–588. [Google Scholar] [CrossRef]
  25. Robinson, T.; Hoying, S.; Sazo, M.M.; DeMarree, A.; Dominguez, L. A Vision for Apple Orchard Systems of the Future. New York State Horticultural Society. Available online: http://nyshs.org/wp-content/uploads/2016/10/Pages-11-16-from-NYFQ-Book-Fall-2013-5.pdf (accessed on 3 November 2013).
  26. Hoying, S.; Robinson, T.; Fargione, M. The Tall Spindle Planting System. Available online: https://cpb-us-e1.wpmucdn.com/blogs.cornell.edu/dist/0/7265/files/2016/12/The-Tall-Spindle-Planting-System-2746zv0.pdf (accessed on 21 August 2018).
  27. Weber, M.S. The super spindle system. Acta Hortic. 1998, 513, 271–278. [Google Scholar] [CrossRef]
  28. Lehnert, R. Techniques and Benefits of a Fruit Wall. Available online: https://www.goodfruit.com/dorigoni/ (accessed on 28 January 2014).
  29. Warmund, M.R. The Vertical Axis System: A Training Method for Growing Apple Trees. University of Missouri Extension. Available online: https://extension2.missouri.edu/g6024 (accessed on 15 January 2018).
  30. Hoying, S.; Robinson, T.; Fargione, M. The Vertical Axis Planting System. Available online: https://cpb-us-e1.wpmucdn.com/blogs.cornell.edu/dist/0/7265/files/2016/12/The-Vertical-Axis-Planting-System-u3z03v.pdf (accessed on 21 August 2016).
  31. Anderson, T. Training Fruit Trees Using the Solaxe System. Available online: https://www.fredericknewspost.com/news/environment/training-fruit-trees-using-the-solaxe-system/article_141d613e-c3c6-5ee5-bb1d-e8c6f17d0db4.html (accessed on 17 September 2016).
  32. Dininny, S. Orchard Mechanization Gains Momentum. Available online: https://www.goodfruit.com/mechanization-momentum/ (accessed on 8 August 2017).
  33. DuPont, T.; Lewis, K. Robot Ready Tree Canopies. WSU. Tree Fruit. Available online: http://treefruit.wsu.edu/article/robot-ready-canopies/ (accessed on 2 February 2018).
  34. Willaume, M.; Lauri, P.É.; Sinoquet, H. Light interception in apple trees influenced by canopy architecture manipulation. Trees 2004, 18, 705–713. [Google Scholar] [CrossRef]
  35. Zhang, J.; Whiting, M.D.; Zhang, Q. Diurnal pattern in canopy light interception for tree fruit orchard trained to an upright fruiting offshoots (UFO) architecture. Biosyst. Eng. 2015, 129, 1–10. [Google Scholar] [CrossRef]
  36. Robinson, T.L.; Lakso, A.N.; Carpenter, S.G. Canopy development, yield, and fruit quality of empire and ’delicious’ apple trees grown in four orchard production systems for ten years. J. Am. Soc. Hortic. Sci. 1991, 116, 179–187. [Google Scholar]
  37. Hampson, C.R.; Quamme, H.A.; Brownlee, R.T. Canopy growth, yield, and fruit quality of ’Royal Gala’ apple trees grown for eight years in five tree training systems. Hortic. Sci. 2002, 37, 627–631. [Google Scholar]
  38. Whiting, M.D.; Lang, G.; Ophardt, D. Rootstock and training system affect sweet cherry growth, yield, and fruit quality. Hortic. Sci. 2005, 40, 582–586. [Google Scholar]
  39. Balmer, M.; Blanke, M. Developments in high density cherries in Germany. IV Int. Cherry Symp. 2001, 667, 273–278. [Google Scholar] [CrossRef]
  40. Lyons, D.J.; Heinemann, P.H.; Schupp, J.R.; Baugher, T.A.; Liu, J. Development of a selective automated blossom thinning system for peaches. Trans. ASABE 2015, 58, 1447–1457. [Google Scholar]
  41. Zhang, Z.; Heinemann, P.H.; Liu, J.; Schupp, J.R.; Baugher, T.A. Design and field test of a low-cost apple harvest-assist unit. Trans. ASABE 2016, 59, 1149–1156. [Google Scholar]
  42. He, L.; Fu, H.; Sun, D.; Karkee, M.; Zhang, Q. Shake-and-catch harvesting for fresh market apples in trellis-trained trees. Trans. ASABE 2017, 60, 353–360. [Google Scholar]
  43. Silwal, A.; Davidson, J.R.; Karkee, M.; Mo, C.; Zhang, Q.; Lewis, K. Design, integration, and field evaluation of a robotic apple harvester. J. Field Robot. 2017, 34, 1140–1159. [Google Scholar] [CrossRef]
  44. Karkee, M.; Adhikari, B.; Amatya, S.; Zhang, Q. Identification of pruning branches in tall spindle apple trees for automated pruning. Comput. Electron. Agric. 2014, 103, 127–135. [Google Scholar] [CrossRef]
  45. Robinson, T.L.; Domingurz, L.I.; Acosta, F. Pruning strategy affects fruit size, yield and biennial bearing of Gala and Honeycrisp Apples. N. Y. Fruit Q. 2014, 22, 27–32. [Google Scholar] [CrossRef]
  46. Schupp, J. Pruning by the Numbers. Available online: http://shaponline.org/wp-content/uploads/2012/03/Pruning-by-the-Numbers-Jim-Schupp.pdf (accessed on 15 March 2018).
  47. Schupp, J.R.; Winzeler, H.E.; Kon, T.M.; Marini, R.P.; Baugher, T.A.; Kime, L.F.; Schupp, M.A. A method for quantifying whole-tree pruning severity in mature tall spindle apple plantings. Hortic. Sci. 2017, 52, 1233–1240. [Google Scholar] [CrossRef]
  48. Zhang, X.; He, L.; Majeed, Y.; Karkee, M.; Whiting, M.D.; Zhang, Q. A Study of the influence of pruning strategy effect on vibrational harvesting of apples. In Proceedings of the 2017 ASABE Annual International Meeting, Spokane, Washington, DC, USA, 16–19 July 2017; American Society of Agricultural and Biological Engineers: St. Joseph, MI, USA, 2017; p. 1. [Google Scholar]
  49. Ferree, D.C.; Rhodus, W.T. Apple tree performance with mechanical hedging or root pruning in intensive orchards. J. Am. Soc. Hortic. Sci. 1993, 118, 707–713. [Google Scholar]
  50. Wertheim, S.J. Pruning of slender spindle type trees. Acta Hortic. 1976, 65, 173–180. [Google Scholar] [CrossRef]
  51. He, L.; Fu, H.; Karkee, M.; Zhang, Q. Effect of fruit location on apple detachment with mechanical shaking. Biosyst. Eng. 2017, 157, 63–71. [Google Scholar] [CrossRef]
  52. Chen, Y.R.; Chao, K.; Kim, M.S. Machine vision technology for agricultural applications. Comput. Electron. Agric. 2002, 36, 173–191. [Google Scholar] [CrossRef]
  53. Davies, E.R. The application of machine vision to food and agriculture: A review. Imaging Sci. J. 2009, 57, 197–217. [Google Scholar] [CrossRef]
  54. McCarthy, C.L.; Hancock, N.H.; Raine, S.R. Applied machine vision of plants: A review with implications for field deployment in automated farming operations. Intell. Serv. Robot. 2010, 3, 209–217. [Google Scholar] [CrossRef] [Green Version]
  55. Radcliffe, J.; Cox, J.; Bulanon, D.M. Machine vision for orchard navigation. Comput. Ind. 2018, 98, 165–171. [Google Scholar] [CrossRef]
  56. Vázquez-Arellano, M.; Griepentrog, H.W.; Reiser, D.; Paraforos, D.S. 3-D imaging systems for agricultural applications—A review. Sensors 2016, 16, 618. [Google Scholar] [CrossRef] [PubMed]
  57. Brandtberg, T.; Warner, T.A.; Landenberger, R.E.; McGraw, J.B. Detection and analysis of individual leaf-off tree crowns in small footprint, high sampling density LIDAR data from the eastern deciduous forest in North America. Remote Sens. Environ. 2003, 85, 290–303. [Google Scholar] [CrossRef]
  58. Edson, C.; Wing, M.G. Airborne light detection and ranging (LIDAR) for individual tree stem location, height, and biomass measurements. Remote Sens. 2011, 3, 2494–2528. [Google Scholar] [CrossRef] [Green Version]
  59. Van Aardt, J.A.; Wynne, R.H.; Scrivani, J.A. Lidar-based mapping of forest volume and biomass by taxonomic group using structurally homogenous segments. Photogram. Eng. Remote Sens. 2008, 74, 1033–1044. [Google Scholar] [CrossRef]
  60. Amatya, S.; Karkee, M.; Zhang, Q.; Whiting, M.D. Automated detection of branch shaking locations for robotic cherry harvesting using machine vision. Robotics 2017, 6, 31. [Google Scholar] [CrossRef]
  61. Zhang, J.; He, L.; Karkee, M.; Zhang, Q.; Zhang, X.; Gao, Z. Branch detection with apple trees trained in fruiting wall architecture using stereo vision and regions-convolutional neural network (R-CNN). In Proceedings of the 2017 ASABE Annual International Meeting, Spokane, Washington, DC, USA, 16–19 July 2017; American Society of Agricultural and Biological Engineers: St. Joseph, MI, USA, 2017; p. 1. [Google Scholar]
  62. Tagarakis, A.; Liakos, V.; Chatzinikos, T.; Koundouras, S.; Fountas, S.; Gemtos, T. Using laser scanner to map pruning wood in vineyards. Precis. Agric. 2013, 13, 633–639. [Google Scholar]
  63. Botterill, T.; Paulin, S.; Green, R.; Williams, S.; Lin, J.; Saxton, V.; Mills, S.; Chen, X.; Corbett-Davies, S. A robot system for pruning grape vines. J. Field Robot. 2017, 34, 1100–1122. [Google Scholar] [CrossRef]
  64. Corbett-Davies, S.; Botterill, T.; Green, R.; Saxton, V. An expert system for automatically pruning vines. In Proceedings of the 27th Conference on Image and Vision Computing, Dunedin, New Zealand, 26–28 November 2012; ACM: New York, NY, USA, 2012; pp. 55–60. [Google Scholar]
  65. Naugle, J.A.; Rehkugler, G.E.; Throop, J.A. Grapevine cordon following using digital image processing. Trans. ASAE 1989, 32, 309–0315. [Google Scholar] [CrossRef]
  66. Medeiros, H.; Kim, D.; Sun, J.; Seshadri, H.; Akbar, S.A.; Elfiky, N.M.; Park, J. Modeling dormant fruit trees for agricultural automation. J. Field Robot. 2017, 34, 1203–1224. [Google Scholar] [CrossRef]
  67. Chattopadhyay, S.; Akbar, S.A.; Elfiky, N.M.; Medeiros, H.; Kak, A. Measuring and modeling apple trees using time-of-flight data for automation of dormant pruning applications. In Proceedings of the Applications of Computer Vision 2016 IEEE Winter Conference, Lake Placid, NY, USA, 7–10 March 2016; Institute of Electrical and Electronics Engineers: Piscataway, NJ, USA, 2016; pp. 1–9. [Google Scholar]
  68. Elfiky, N.M.; Akbar, S.A.; Sun, J.; Park, J.; Kak, A. Automation of dormant pruning in specialty crop production: An adaptive framework for automatic reconstruction and modeling of apple trees. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Boston, MA, USA, 7–12 June 2015; Institute of Electrical and Electronics Engineers: Piscataway, NJ, USA, 2015; pp. 65–73. [Google Scholar]
  69. Tabb, A.; Medeiros, H. A robotic vision system to measure tree traits. arXiv, 2017; arXiv:1707.05368. [Google Scholar]
  70. Tabb, A.; Medeiros, H. Automatic segmentation of trees in dynamic outdoor environments. Comput. Ind. 2018, 98, 90–99. [Google Scholar] [CrossRef] [Green Version]
  71. Akbar, S.A.; Chattopadhyay, S.; Elfiky, N.M.; Kak, A. A Novel Benchmark RGBD Dataset for Dormant Apple Trees and its Application to Automatic Pruning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Boston, MA, USA, 8–10 June 2015; Institute of Electrical and Electronics Engineers: Piscataway, NJ, USA, 2016; pp. 81–88. [Google Scholar]
  72. Akbar, S.A.; Elfiky, N.M.; Kak, A. A novel framework for modeling dormant apple trees using single depth image for robotic pruning application. In Proceedings of the IEEE International Conference on Robotics and Automation, Stockholm, Sweden, 16–21 May 2016; Institute of Electrical and Electronics Engineers: Piscataway, NJ, USA, 2016; pp. 5136–5142. [Google Scholar]
  73. Hackenberg, J.; Spiecker, H.; Calders, K.; Disney, M.; Raumonen, P. Simple Tree—An efficient open source tool to build tree models from TLS clouds. Forests 2015, 6, 4245–4294. [Google Scholar] [CrossRef]
  74. Li, R.; Bu, G.; Wang, P. An automatic tree skeleton extracting method based on point cloud of terrestrial laser scanner. Int. J. Opt. 2017. [Google Scholar] [CrossRef]
  75. Bucksch, A.; Fleck, S. Automated detection of branch dimensions in woody skeletons of fruit tree canopies. Photogram. Eng. Remote Sens. 2011, 77, 229–240. [Google Scholar] [CrossRef]
  76. Pla, F.; Juste, F.; Ferri, F.; Vicens, M. Colour segmentation based on a light reflection model to locate citrus fruits for robotic harvesting. Comput. Electron. Agric. 1993, 9, 53–70. [Google Scholar] [CrossRef]
  77. Jiménez, A.R.; Ceres, R.; Pons, J.L. A vision system based on a laser range-finder applied to robotic fruit harvesting. Mach. Vis. Appl. 2000, 11, 321–329. [Google Scholar] [CrossRef] [Green Version]
  78. Hannan, M.W.; Burks, T.F.; Bulanon, D.M. A Machine Vision Algorithm Combining Adaptive Segmentation and Shape Analysis for Orange Fruit Detection. Agric. Eng. Int. 2018, 5, 1281. [Google Scholar]
  79. Silwal, A.; Karkee, M.; Zhang, Q. A hierarchical approach to apple identification for robotic harvesting. Trans. ASABE 2016, 59, 1079–1086. [Google Scholar]
  80. Nielsen, M.; Slaughter, D.C.; Gliever, C. Vision-based 3D peach tree reconstruction for automated blossom thinning. IEEE Trans. Ind. Inform. 2012, 8, 188–196. [Google Scholar] [CrossRef]
  81. Tan, P.; Zeng, G.; Wang, J.; Kang, S.B.; Quan, L. Image-based tree modeling. ACM Trans. Graph. 2007, 26, 87. [Google Scholar] [CrossRef]
  82. Cheng, Z.L.; Zhang, X.P.; Chen, B.Q. Simple reconstruction of tree branches from a single range image. J. Comput. Sci. Technol. 2007, 22, 846–858. [Google Scholar] [CrossRef]
  83. McFarlane, N.J.B.; Tisseyre, B.; Sinfort, C.; Tillett, R.D.; Sevila, F. Image analysis for pruning of long wood grape vines. J. Agric. Eng. Res. 1997, 66, 111–119. [Google Scholar] [CrossRef]
  84. Gao, M.; Lu, T.F. Image processing and analysis for autonomous grapevine pruning. In Proceedings of the 2006 IEEE International Conference on Mechatronics and Automation 2006, Luoyang, China, 25–28 June 2006; Institute of Electrical and Electronics Engineers: Piscataway, New Jersey, USA, 2006; pp. 922–927. [Google Scholar]
  85. Hosseini, S.M.; Jafari, A. Designing an algorithm for pruning grapevine based on 3D image processing. Iran. J. Biosyst. Eng. 2017, 48, 289–297. [Google Scholar]
  86. Karkee, M.; Adhikari, B. A method for three-dimensional reconstruction of apple trees for automated pruning. Trans. ASABE 2015, 58, 565–574. [Google Scholar]
  87. Tabb, A. Three-dimensional reconstruction of fruit trees by a shape from silhouette method. In Proceedings of the ASABE Annual International Meeting 2009, Reno, Nevada, 21–24 June 2009; Paper No. 096138. ASABE: St. Joseph, MI, USA, 2009. [Google Scholar]
  88. Martí, B.V.; González, E.F. The influence of mechanical pruning in cost reduction, production of fruit, and biomass waste in citrus orchards. Appl. Eng. Agric. 2010, 26, 531–540. [Google Scholar] [CrossRef]
  89. Dias, A.B.; Patrocinio, S.; Pereira, S.; Brites, T.; Pita, V.; Mota Barroso, J.M. Evaluation of the use of a disc-saw machine in winter pruning ‘Rocha’ pears orchard—An account of five years. XII Int. Pear Symp. 2014, 1094, 281–288. [Google Scholar] [CrossRef]
  90. Gautz, L.D.; Bittenbender, H.C.; Mauri, S. Effect of mechanized pruning on coffee regrowth and fruit maturity timing. In Proceedings of the ASAE Annual Meeting, Chicago, IL, USA, 28–31 July 2002; Paper No. 021110. ASAE: Washington, DC, USA, 2002. [Google Scholar]
  91. Morris, J.R.; Cawthon, D.L.; Fleming, J.W. Effect of mechanical pruning on yield and quality of ‘Concord’ grapes. Ark. Farm. Res. 1975, 24, 12. [Google Scholar]
  92. Bates, T.; Morris, J. Mechanical cane pruning and crop adjustment decreases labor costs and maintains fruit quality in New York ‘Concord’ grape production. Hortic. Technol. 2009, 19, 247–253. [Google Scholar]
  93. Poni, S.; Tombesi, S.; Palliotti, A.; Ughini, V.; Gatti, M. Mechanical winter pruning of grapevine: Physiological bases and applications. Sci. Hortic. 2016, 204, 88–98. [Google Scholar] [CrossRef]
  94. Cline, H. Mechanical Vine Pruning Dramatically Reduces Costs. Western FarmPress. Available online: https://www.westernfarmpress.com/mechanical-vine-pruning-dramatically-reduces-costs (accessed on 1 February 2002).
  95. Guimond, C.M.; Lang, G.A.; Andrews, P.K. Timing and severity of summer pruning affects flower initiation and shoot regrowth in sweet cherry. Hortic. Sci. 1998, 33, 647–649. [Google Scholar]
  96. Albarracín, V.; Hall, A.J.; Searles, P.S.; Rousseaux, M.C. Responses of vegetative growth and fruit yield to winter and summer mechanical pruning in olive trees. Sci. Hortic. 2017, 225, 185–194. [Google Scholar] [CrossRef]
  97. Moore, P.W. Mechanical pruning for citrus. Calif. Agric. 1958, November. 7–9. [Google Scholar]
  98. Vivaldi, G.A.; Strippoli, G.; Pascuzzi, S.; Stellacci, A.M.; Camposeo, S. Olive genotypes cultivated in an adult high-density orchard respond differently to canopy restraining by mechanical and manual pruning. Sci. Hortic. 2015, 192, 391–399. [Google Scholar] [CrossRef]
  99. Moore-Gough, C.; Gough, R.E. Pruning Fruit Trees in Montana. Available online: http://msuextension.org/publications/YardandGarden/MT199215AG.pdf (accessed on 10 March 2007).
  100. Jensen, F.; Christensen, L.; Beede, R.; Leavitt, G. Effects of mechanical pruning on grapes. Calif. Agric. 1980, 34, 33–34. [Google Scholar]
  101. Sevilla, F. A robot to prune the grapevine. In Proceedings of the Agri-Mation 1 Conference and Exposition, Chicago, IL, USA, 25–28 February 1985. [Google Scholar]
  102. Ochs, E.S.; Gunkel, W.W. Robotic Grape Pruner Field Performance Simulation. In Proceedings of the Annual International Meeting, Chicago, IL, USA, 12–17 December 1993; No. 933528. ASAE: Washington, DC, USA, 1993. [Google Scholar]
  103. Lee, M.F.; Gunkel, W.W.; Throop, J.A. A Digital Regulator and Tracking Controller Design for an Electrohydraulic Robotic Grape Pruner. In Proceedings of the 5th International Conference on Computers in Agriculture, Orlando, FL, USA, 6–9 February 1994; ACM: New York, NY, USA, 1994; pp. 23–28. [Google Scholar]
  104. Kondo, N.; Shibano, Y.; Mohri, K.; Monta, M. Basic studies on robot to work in vineyard 2: Discriminating, position detecting and harvesting experiments using a visual sensor. J. Jpn. Soc. Agric. Mach. 1994, 56, 45–53. [Google Scholar]
  105. Kondo, N.; Shibano, Y.; Mohri, K.; Monta, M. Basic studies on robot to work in vineyard 1: Manipulator and harvesting hand. J. Jpn. Soc. Agric. Mach. 1993, 55, 85–94. [Google Scholar]
  106. Koselka, T. Intelligent Robotic Grapevine Pruning. Vitculture and Enology Extension News, Washington State University. Available online: http://wine.wsu.edu/wp-content/uploads/sites/66/2010/07/2012-Fall-FINAL.pdf (accessed on 12 March 2018).
  107. Holz, D.; Ichim, A.E.; Tombari, F.; Rusu, R.B.; Behnke, S. Registration with the point cloud library: A modular framework for aligning in 3-D. IEEE Robot. Autom. Mag. 2015, 22, 110–124. [Google Scholar] [CrossRef]
  108. Newcombe, R.A.; Fox, D.; Seitz, S.M. Dynamicfusion: Reconstruction and tracking of non-rigid scenes in real-time. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 2015, Boston, MA, USA, 7–12 June 2015; Institute of Electrical and Electronics Engineers: Piscataway, NJ, USA, 2015; pp. 343–352. [Google Scholar]
  109. Nguyen, T.T.; Vandevoorde, K.; Kayacan, E.; De Baerdemaeker, J.; Saeys, W. Apple detection algorithm for robotic harvesting using a RGB-D camera. In Proceedings of the International Conference of Agricultural Engineering, Zurich, Switzerland, 6–10 July 2014; Agricultural Engineering: Zurich, Switzerland, 2014. [Google Scholar]
  110. Zhu, D.; Chen, B.; Yang, Y.; Lianxi, H.; Yang, M.; Qiao, Y. Method of haze-removal based on dark channel prior in visual system of apple harvest robot. Trans. Chin. Soc. Agric. Eng. 2016, 32, 151–158. [Google Scholar]
  111. Ye, Y.; Wang, Z.; Jones, D.; He, L.; Taylor, M.E.; Hollinger, G.A.; Zhang, Q. Bin-dog: A robotic platform for bin management in orchards. Robotics 2017, 6, 12. [Google Scholar] [CrossRef]
  112. Barth, R.; Hemming, J.; van Henten, E.J. Design of an eye-in-hand sensing and servo control framework for harvesting robotics in dense vegetation. Biosyst. Eng. 2016, 146, 71–84. [Google Scholar] [CrossRef]
  113. Korayem, M.H.; Shafei, A.M.; Seidi, E. Symbolic derivation of governing equations for dual-arm mobile manipulators used in fruit-picking and the pruning of tall trees. Comput. Electron. Agric. 2014, 105, 95–102. [Google Scholar] [CrossRef]
  114. Megalingam, R.K.; Vignesh, N.; Sivanantham, V.; Elamon, N.; Sharathkumar, M.S.; Rajith, V. Low cost robotic arm design for pruning and fruit harvesting in developing nations. In Proceedings of the 10th International Conference on Intelligent Systems and Control (ISCO) 2016, Coimbatore, India, 7–8 January 2016; Institute of Electrical and Electronics Engineers: Piscataway, NJ, USA, 2016; pp. 1–5. [Google Scholar]
  115. Krissian, K.; Malandain, G.; Ayache, N.; Vaillant, R.; Trousset, Y. Model-based detection of tubular structures in 3d images. Comput. Vis. Image Underst. 2000, 80, 130–171. [Google Scholar] [CrossRef]
  116. Chuang, J.H.; Tsai, C.H.; Ko, M.C. Skeletonisation of three-dimensional object using generalized potential field. IEEE Trans. Pattern Anal. Mach. Intell. 2000, 22, 1241–1251. [Google Scholar]
  117. Duan, Y.; Yang, L.; Qin, H.; Samaras, D. Shape reconstruction from 3D and 2D data using PDE-based deformable surfaces. In Proceedings of the European Conference on Computer Vision, Prague, Czech Republic, 11–14 May 2004; Springer: Berlin, Germany, 2004; pp. 238–251. [Google Scholar]
  118. He, L. New concept of mechanical/robotic pruning end-effector for fruit trees. Unpublished document. 2018. in preparation. [Google Scholar]
Figure 1. An example of production cost breakdown in percentage for each category for a Gala apple orchard (40 Acres). The fixed cost, such as land, trellis, irrigation setup, etc., is not included [17].
Figure 1. An example of production cost breakdown in percentage for each category for a Gala apple orchard (40 Acres). The fixed cost, such as land, trellis, irrigation setup, etc., is not included [17].
Agronomy 08 00211 g001
Figure 2. Intensive modern fruit tree systems: (A) horizontal branch fruiting wall V-Trellis system in Washington; and (B) tall spindle tree system in Mid-Atlantic fruit region.
Figure 2. Intensive modern fruit tree systems: (A) horizontal branch fruiting wall V-Trellis system in Washington; and (B) tall spindle tree system in Mid-Atlantic fruit region.
Agronomy 08 00211 g002
Figure 3. Pruning the apple trees with the proposed severity pruning levels.
Figure 3. Pruning the apple trees with the proposed severity pruning levels.
Agronomy 08 00211 g003
Figure 4. Pruning based on the length of the twigs and the ratio of length and diameter of twig (from [48], used with permission).
Figure 4. Pruning based on the length of the twigs and the ratio of length and diameter of twig (from [48], used with permission).
Agronomy 08 00211 g004
Figure 5. (A) Branch detection for sweet cherry trees in Y-Trellis (reproduced from [60]); and (B) branch detection for apple trees in V-trellis fruiting wall (from [61], used with permission).
Figure 5. (A) Branch detection for sweet cherry trees in Y-Trellis (reproduced from [60]); and (B) branch detection for apple trees in V-trellis fruiting wall (from [61], used with permission).
Agronomy 08 00211 g005
Figure 6. An example of tree reconstruction with 3D machine vision: (A) RGB image of the test tree; and (B) reconstructed tree [69].
Figure 6. An example of tree reconstruction with 3D machine vision: (A) RGB image of the test tree; and (B) reconstructed tree [69].
Agronomy 08 00211 g006
Figure 7. Mechanical pruning systems for apple trees with saw-tooth cutter driven by a tractor.
Figure 7. Mechanical pruning systems for apple trees with saw-tooth cutter driven by a tractor.
Agronomy 08 00211 g007
Table 1. Typical tree training systems and specifications for apple trees.
Table 1. Typical tree training systems and specifications for apple trees.
Tree Training SystemsSpecificationsPruning ComplexityReferences
Bi-axis- Most in Italy and new to Washington
- Two-stem tree architecture
- High density and trellis required
- Pre-formed and split-branch tree required
- Short shoot length reduced pruning mass
- Well adopted for mechanical harvesting
[24,25]
Tall spindle- High density
- Most common system in Eastern north America
- One leader with narrow canopy
- No permanent branches
- No pruning of leader
- Limb renewal pruning is used
- Summer hedging applicable
[25,26]
Super spindle- Super high density and trellis required
- Low operation cost
- No pruning of leader
- Limb renewal pruning is used
- Summer hedging applicable
[25,27]
V-trellis/Vertical fruiting wall- Fruiting wall (2D)
- Common system in Washington
- Horizontal branches on trellis wire
- Designed for precision pruning
- High accessibility of branches
- Compatible for robotic pruning
[25,28]
Vertical axis- Narrow pyramidal shape
- Central leader with support post
- Support by a trellis
- Lateral branches are renewed periodically[29,30]
Solaxe- Common in south France, part of Spain and Chile
- Extensive limb bending
- Central leader
- Removal of shoot close to trunk and other large shoots
- Limit pruning is required after properly training in first few years
[25,31]
Table 2. Pruning strategies and potential difficulties for automated pruning.
Table 2. Pruning strategies and potential difficulties for automated pruning.
Tree Training SystemPruning RulesTechnical Difficulties for Automated PruningReferences
Tall spindle- Limb renewal pruning: elimination of large branches- Obtain location of each branch
- Identity base diameter of each individual branches
- Machine Accessibility
[45,47]
Tall spindle- Certain branch length and spacing- Measure the length of each branch
- Determine the density of the branches
[44]
Super/tall Spindle- Branch simplification pruning: removing lateral secondary branches- Identify the diameter and length of each secondary branches[45]
Vertical/V-trellis fruiting wall- Keep secondary branches in certain length- Identify the diameter and length of each secondary branches[48]
Trellis training central leader- Hedging and topping- Identify tree structure
- Determine the cutting location
[49]
Slender spindle- Remove tree top
- Remove unwanted shoot
- Identify tree structure
- Determine the cutting location
[50]
Hedgerow- Hedging and topping - Identify tree structure
- Determine the cutting location
[20]
Table 3. Sensors and techniques used for tree/branch detection and reconstruction in different applications.
Table 3. Sensors and techniques used for tree/branch detection and reconstruction in different applications.
ApplicationSensorsTechnique DifficultiesReferences
Forest tree inventoryAirborne laser scanner- Relatively low accuracy
- Difficult to detect small trees
[57,58,59]
Mechanical harvesting3D camera (ToF *)Kinect v2- Leaves/fruits may block the branches
- Affected by the direct sunlight
[60,61]
Robotic grapevine pruningLaser scanner (ToF)- Only for wood estimation
- Efficacy needs to be improved
[62]
3 Grey Grasshopper2 color cameras- Moving canes are not modeled
- Incorrect 3D structure around vine head
- Requires stops at each plant
[63,64]
RGB camera- Not for precise pruning
- Not suit for big distance variation of cane base
[65]
Robotic fruit tree pruningLaser scanner/ToF- Human intervention is necessary for part of registration[66]
3D camera (CamCube 3.0 **)- Overlap branch removal rate is about 70%
- More pruning criteria will be considered
[44]
Kinect v2- Color information is required to improve performance
- Separate two adjacent/close trees
[67,68]
RGB-D camera(ToF)- The operation of 3D reconstruction is offline
- The speed is low
[69,70,71,72]
* ToF camera: Time-of-flight camera; ** CamCube 3.0 is a 3D camera from PMD Tech.

Share and Cite

MDPI and ACS Style

He, L.; Schupp, J. Sensing and Automation in Pruning of Apple Trees: A Review. Agronomy 2018, 8, 211. https://0-doi-org.brum.beds.ac.uk/10.3390/agronomy8100211

AMA Style

He L, Schupp J. Sensing and Automation in Pruning of Apple Trees: A Review. Agronomy. 2018; 8(10):211. https://0-doi-org.brum.beds.ac.uk/10.3390/agronomy8100211

Chicago/Turabian Style

He, Long, and James Schupp. 2018. "Sensing and Automation in Pruning of Apple Trees: A Review" Agronomy 8, no. 10: 211. https://0-doi-org.brum.beds.ac.uk/10.3390/agronomy8100211

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop