Research

Jump to: Review

22 pages, 34729 KiB

Open AccessArticle

From CAD Models to Soft Point Cloud Labels: An Automatic Annotation Pipeline for Cheaply Supervised 3D Semantic Segmentation

by Galadrielle Humblot-Renaux, Simon Buus Jensen and Andreas Møgelmose

Remote Sens. 2023, 15(14), 3578; https://0-doi-org.brum.beds.ac.uk/10.3390/rs15143578 - 17 Jul 2023

Cited by 1 | Viewed by 1203

Abstract

We propose a fully automatic annotation scheme that takes a raw 3D point cloud with a set of fitted CAD models as input and outputs convincing point-wise labels that can be used as cheap training data for point cloud segmentation. Compared with manual [...] Read more.

We propose a fully automatic annotation scheme that takes a raw 3D point cloud with a set of fitted CAD models as input and outputs convincing point-wise labels that can be used as cheap training data for point cloud segmentation. Compared with manual annotations, we show that our automatic labels are accurate while drastically reducing the annotation time and eliminating the need for manual intervention or dataset-specific parameters. Our labeling pipeline outputs semantic classes and soft point-wise object scores, which can either be binarized into standard one-hot-encoded labels, thresholded into weak labels with ambiguous points left unlabeled, or used directly as soft labels during training. We evaluate the label quality and segmentation performance of PointNet++ on a dataset of real industrial point clouds and Scan2CAD, a public dataset of indoor scenes. Our results indicate that reducing supervision in areas that are more difficult to label automatically is beneficial compared with the conventional approach of naively assigning a hard “best guess” label to every point. Full article

(This article belongs to the Special Issue New Perspectives on 3D Point Cloud)

► Show Figures

Figure 1

23 pages, 20182 KiB

Open AccessArticle

A New Approach toward Corner Detection for Use in Point Cloud Registration

by Wei Wang, Yi Zhang, Gengyu Ge, Huan Yang and Yue Wang

Remote Sens. 2023, 15(13), 3375; https://0-doi-org.brum.beds.ac.uk/10.3390/rs15133375 - 01 Jul 2023

Cited by 1 | Viewed by 1414

Abstract

For this study, a new point cloud alignment method is proposed for extracting corner points and aligning them at the geometric level. It can align point clouds that have low overlap and is more robust to outliers and noise. First, planes are extracted [...] Read more.

For this study, a new point cloud alignment method is proposed for extracting corner points and aligning them at the geometric level. It can align point clouds that have low overlap and is more robust to outliers and noise. First, planes are extracted from the raw point cloud, and the corner points are defined as the intersection of three planes. Next, graphs are constructed for subsequent point cloud registration by treating corners as vertices and sharing planes as edges. The graph-matching algorithm is then applied to determine correspondence. Finally, point clouds are registered by aligning the corresponding corner points. The proposed method was investigated by utilizing pertinent metrics on datasets with differing overlap. The results demonstrate that the proposed method can align point clouds that have low overlap, yielding an RMSE of about 0.05 cm for datasets with 90% overlap and about 0.2 cm when there is only about 10% overlap. In this situation, the other methods failed to align point clouds. In terms of time consumption, the proposed method can process a point cloud comprising

10^{4}

points in 4 s when there is high overlap. When there is low overlap, it can also process a point cloud comprising

10^{6}

points in 10 s. The contributions of this study are the definition and extraction of corner points at the geometric level, followed by the use of these corner points to register point clouds. This approach can be directly used for low-precision applications and, in addition, for coarse registration in high-precision applications. Full article

(This article belongs to the Special Issue New Perspectives on 3D Point Cloud)

► Show Figures

Figure 1

18 pages, 4503 KiB

Open AccessArticle

Partial-to-Partial Point Cloud Registration by Rotation Invariant Features and Spatial Geometric Consistency

by Yu Zhang, Wenhao Zhang and Jinlong Li

Remote Sens. 2023, 15(12), 3054; https://0-doi-org.brum.beds.ac.uk/10.3390/rs15123054 - 10 Jun 2023

Viewed by 2023

Abstract

Point cloud registration is a critical problem in 3D vision tasks, and numerous learning-based point cloud registration methods have been proposed in recent years. However, a common issue with most of these methods is that their feature descriptors are rotation-sensitive, which makes them [...] Read more.

Point cloud registration is a critical problem in 3D vision tasks, and numerous learning-based point cloud registration methods have been proposed in recent years. However, a common issue with most of these methods is that their feature descriptors are rotation-sensitive, which makes them difficult to converge at large rotations. In this paper, we propose a new learning-based pipeline to address this issue, which can also handle partially overlapping 3D point clouds. Specifically, we employ rotation-invariant local features to guide the point matching task, and utilize a cross-attention mechanism to update the feature information between the two point clouds to predict the key points in the overlapping regions. Subsequently, we construct a feature matrix based on the features of the key points to solve the soft correspondences. Finally, we construct a non-learning correspondence constraint module that exploits the spatial geometric invariance of the point clouds after rotation and translation, as well as the compatibility between point pairs, to reject the wrong correspondences. To validate our approach, we conduct extensive experiments on ModelNet40. Our approach achieves better performance compared to other methods, especially in the presence of large rotations. Full article

(This article belongs to the Special Issue New Perspectives on 3D Point Cloud)

► Show Figures

Figure 1

20 pages, 1890 KiB

Open AccessArticle

Rotation Invariant Graph Neural Network for 3D Point Clouds

by Alexandru Pop, Victor Domșa and Levente Tamas

Remote Sens. 2023, 15(5), 1437; https://0-doi-org.brum.beds.ac.uk/10.3390/rs15051437 - 03 Mar 2023

Viewed by 1672

Abstract

In this paper we propose a novel rotation normalization technique for point cloud processing using an oriented bounding box. We use this method to create a point cloud annotation tool for part segmentation on real camera data. Custom data sets are used to [...] Read more.

In this paper we propose a novel rotation normalization technique for point cloud processing using an oriented bounding box. We use this method to create a point cloud annotation tool for part segmentation on real camera data. Custom data sets are used to train our network for classification and part segmentation tasks. Successful deployment is completed on an embedded device with limited processing power. A comparison is made with other rotation-invariant features in noisy synthetic datasets. Our method offers more auxiliary information related to the dimension, position, and orientation of the object than previous methods while performing at a similar level. Full article

(This article belongs to the Special Issue New Perspectives on 3D Point Cloud)

► Show Figures

Figure 1

21 pages, 10194 KiB

Open AccessArticle

Enriching Point Clouds with Implicit Representations for 3D Classification and Segmentation

by Zexin Yang, Qin Ye, Jantien Stoter and Liangliang Nan

Remote Sens. 2023, 15(1), 61; https://0-doi-org.brum.beds.ac.uk/10.3390/rs15010061 - 22 Dec 2022

Cited by 4 | Viewed by 2212

Abstract

Continuous implicit representations can flexibly describe complex 3D geometry and offer excellent potential for 3D point cloud analysis. However, it remains challenging for existing point-based deep learning architectures to leverage the implicit representations due to the discrepancy in data structures between implicit fields [...] Read more.

Continuous implicit representations can flexibly describe complex 3D geometry and offer excellent potential for 3D point cloud analysis. However, it remains challenging for existing point-based deep learning architectures to leverage the implicit representations due to the discrepancy in data structures between implicit fields and point clouds. In this work, we propose a new point cloud representation by integrating the 3D Cartesian coordinates with the intrinsic geometric information encapsulated in its implicit field. Specifically, we parameterize the continuous unsigned distance field around each point into a low-dimensional feature vector that captures the local geometry. Then we concatenate the 3D Cartesian coordinates of each point with its encoded implicit feature vector as the network input. The proposed method can be plugged into an existing network architecture as a module without trainable weights. We also introduce a novel local canonicalization approach to ensure the transformation-invariance of encoded implicit features. With its local mechanism, our implicit feature encoding module can be applied to not only point clouds of single objects but also those of complex real-world scenes. We have validated the effectiveness of our approach using five well-known point-based deep networks (i.e., PointNet, SuperPoint Graph, RandLA-Net, CurveNet, and Point Structuring Net) on object-level classification and scene-level semantic segmentation tasks. Extensive experiments on both synthetic and real-world datasets have demonstrated the effectiveness of the proposed point representation. Full article

(This article belongs to the Special Issue New Perspectives on 3D Point Cloud)

► Show Figures

Figure 1

20 pages, 7374 KiB

Open AccessArticle

3D Graph-Based Individual-Tree Isolation (Treeiso) from Terrestrial Laser Scanning Point Clouds

by Zhouxin Xi and Chris Hopkinson

Remote Sens. 2022, 14(23), 6116; https://0-doi-org.brum.beds.ac.uk/10.3390/rs14236116 - 02 Dec 2022

Cited by 3 | Viewed by 3890

Abstract

Using terrestrial laser scanning (TLS) technology, forests can be digitized at the centimeter-level to enable fine-scale forest management. However, there are technical barriers to converting point clouds into individual-tree features or objects aligned with forest inventory standards due to noise, redundancy, and geometric [...] Read more.

Using terrestrial laser scanning (TLS) technology, forests can be digitized at the centimeter-level to enable fine-scale forest management. However, there are technical barriers to converting point clouds into individual-tree features or objects aligned with forest inventory standards due to noise, redundancy, and geometric complexity. A practical model treeiso based on the cut-pursuit graph algorithm was proposed to isolate individual-tree points from plot-level TLS scans. The treeiso followed the local-to-global segmentation scheme, which grouped points into small clusters, large segments, and final trees in a hierarchical manner. Seven tree attributes were investigated to understand the underlying determinants of isolation accuracy. Sensitivity analysis based on the PAWN index was performed using 10,000 parameter combinations to understand the treeiso’s parameter importance and model robustness. With sixteen reference TLS plot scans from various species, an average of 86% of all trees were detected. The mean intersection-over-union (mIoU) between isolated trees and reference trees was 0.82, which increased to 0.92 within the detected trees. Sensitivity analysis showed that only three parameters were needed for treeiso optimization, and it was robust against parameter variations. This new treeiso method is operationally simple and addresses the growing need for practical 3D tree segmentation tools. Full article

(This article belongs to the Special Issue New Perspectives on 3D Point Cloud)

► Show Figures

Figure 1

22 pages, 9517 KiB

Open AccessArticle

Digital Data and Semantic Simulation—The Survey of the Ruins of the Convent of the Paolotti (12th Century A.D.)

by Sara Gonizzi Barsanti, Santiago Lillo Giner and Adriana Rossi

Remote Sens. 2022, 14(20), 5152; https://0-doi-org.brum.beds.ac.uk/10.3390/rs14205152 - 15 Oct 2022

Cited by 1 | Viewed by 1428

Abstract

In the presence of architecturally significant ruins, restoring and disseminating the idea of a testimony that has survived the destructive work of time is a cultural and social necessity that the use of advanced methods and tools allows to communicate in a timely [...] Read more.

In the presence of architecturally significant ruins, restoring and disseminating the idea of a testimony that has survived the destructive work of time is a cultural and social necessity that the use of advanced methods and tools allows to communicate in a timely and comprehensive manner. The integration of 3D surveying techniques and digital information production and management processes (graphic and alphanumeric, i.e., geometric information) makes it possible to put in place multifaceted and effective strategies. The article aims at describing the process of data acquisition (using applied photogrammetry) of the remains of a medieval cloister located on the outskirts of ancient Oppido Mamertina (RC, Italy). The use of the acquired point cloud, cleaned and optimised, made it possible to extract suitable orthophotos from which to derive the matrix profiles of the vaulted roof system. The information organisation of the model, which can be queried on time despite the generic level of detail, leads us to meditate on the change taking place in the field of documentation for urban environmental design and maintenance. Full article

(This article belongs to the Special Issue New Perspectives on 3D Point Cloud)

► Show Figures

Graphical abstract

21 pages, 28793 KiB

Open AccessArticle

KASiam: Keypoints-Aligned Siamese Network for the Completion of Partial TLS Point Clouds

by Xinpu Liu, Yanxin Ma, Ke Xu, Ling Wang and Jianwei Wan

Remote Sens. 2022, 14(15), 3617; https://0-doi-org.brum.beds.ac.uk/10.3390/rs14153617 - 28 Jul 2022

Viewed by 1302

Abstract

Completing point clouds from partial terrestrial laser scannings (TLS) is a fundamental step for many 3D visual applications, such as remote sensing, digital city and autonomous driving. However, existing methods mainly followed an ordinary auto-encoder architecture with only partial point clouds as inputs, [...] Read more.

Completing point clouds from partial terrestrial laser scannings (TLS) is a fundamental step for many 3D visual applications, such as remote sensing, digital city and autonomous driving. However, existing methods mainly followed an ordinary auto-encoder architecture with only partial point clouds as inputs, and adopted K-Nearest Neighbors (KNN) operations to extract local geometric features, which takes insufficient advantage of input point clouds and has limited ability to extract features from long-range geometric relationships, respectively. In this paper, we propose a keypoints-aligned siamese (KASiam) network for the completion of partial TLS point clouds. The network follows a novel siamese auto-encoder architecture, to learn prior geometric information of complete shapes by aligning keypoints of complete-partial pairs during the stage of training. Moreover, we propose two essential blocks cross-attention perception (CAP) and self-attention augment (SAA), which replace KNN operations with attention mechanisms and are able to establish long-range geometric relationships among points by selecting neighborhoods adaptively at the global level. Experiments are conducted on widely used benchmarks and several TLS data, which demonstrate that our method outperforms other state-of-the-art methods by a 4.72% reduction of the average Chamfer Distance of categories in PCN dataset at least, and can generate finer shapes of point clouds on partial TLS data. Full article

(This article belongs to the Special Issue New Perspectives on 3D Point Cloud)

► Show Figures

Figure 1

Review

Jump to: Research

24 pages, 2719 KiB

Open AccessReview

Review on Deep Learning Algorithms and Benchmark Datasets for Pairwise Global Point Cloud Registration

by Yang Zhao and Lei Fan

Remote Sens. 2023, 15(8), 2060; https://0-doi-org.brum.beds.ac.uk/10.3390/rs15082060 - 13 Apr 2023

Cited by 4 | Viewed by 2490

Abstract

Point cloud registration is the process of aligning point clouds collected at different locations of the same scene, which transforms the data into a common coordinate system and forms an integrated dataset. It is a fundamental task before the application of point cloud [...] Read more.

Point cloud registration is the process of aligning point clouds collected at different locations of the same scene, which transforms the data into a common coordinate system and forms an integrated dataset. It is a fundamental task before the application of point cloud data. Recent years have witnessed the rapid development of various deep-learning-based global registration methods to improve performance. Therefore, it is appropriate to carry out a comprehensive review of the more recent developments in this area. As the developments require access to large benchmark point cloud datasets, the most widely used public datasets are also reviewed. The performance of deep-learning-based registration methods on the benchmark datasets are summarized using the reported performance metrics in the literature. This forms part of a critical discussion of the strengths and weaknesses of the various methods considered in this article, which supports presentation of the main challenges currently faced in typical global point cloud registration tasks that use deep learning methods. Recommendations for potential future studies on this topic are provided. Full article

(This article belongs to the Special Issue New Perspectives on 3D Point Cloud)

► Show Figures

Figure 1

Journal Menu

Journal Browser

New Perspectives on 3D Point Cloud

Share This Special Issue

Special Issue Editors

Special Issue Information

Keywords

Related Special Issue

Published Papers (9 papers)

Research

Review

Further Information

Guidelines

MDPI Initiatives

Follow MDPI