Sensors

Research

18 pages, 6597 KiB

Open AccessArticle

A Performance Comparison of 3D Survey Instruments for Their Application in the Cultural Heritage Field

by Irene Lunghi, Emma Vannini, Alice Dal Fovo, Valentina Di Sarno, Alessandra Rocco and Raffaella Fontana

Sensors 2024, 24(12), 3876; https://0-doi-org.brum.beds.ac.uk/10.3390/s24123876 - 15 Jun 2024

Viewed by 278

Abstract

Thanks to the recent development of innovative instruments and software with high accuracy and resolution, 3D modelling provides useful insights in several sectors (from industrial metrology to cultural heritage). Moreover, the 3D reconstruction of objects of artistic interest is becoming mandatory, not only [...] Read more.

Thanks to the recent development of innovative instruments and software with high accuracy and resolution, 3D modelling provides useful insights in several sectors (from industrial metrology to cultural heritage). Moreover, the 3D reconstruction of objects of artistic interest is becoming mandatory, not only because of the risks to which works of art are increasingly exposed (e.g., wars and climatic disasters) but also because of the leading role that the virtual fruition of art is taking. In this work, we compared the performance of four 3D instruments based on different working principles and techniques (laser micro-profilometry, structured-light topography and the phase-shifting method) by measuring four samples of different sizes, dimensions and surface characteristics. We aimed to assess the capabilities and limitations of these instruments to verify their accuracy and the technical specifications given in the suppliers’ data sheets. To this end, we calculated the point densities and extracted several profiles from the models to evaluate both their lateral (XY) and axial (Z) resolution. A comparison between the nominal resolution values and those calculated on samples representative of cultural artefacts was used to predict the performance of the instruments in real case studies. Overall, the purpose of this comparison is to provide a quantitative assessment of the performance of the instruments that allows for their correct application to works of art according to their specific characteristics. Full article

(This article belongs to the Special Issue Stereo Vision Sensing and Image Processing)

► Show Figures

Figure 1

16 pages, 12071 KiB

Open AccessArticle

Importing Diffusion and Re-Designed Backward Process for Image De-Raining

by Jhe-Wei Lin, Cheng-Hsuan Lee, Tang-Wei Su and Che-Cheng Chang

Sensors 2024, 24(12), 3715; https://0-doi-org.brum.beds.ac.uk/10.3390/s24123715 - 7 Jun 2024

Viewed by 249

Abstract

In recent years, with the increasing demand for high-quality images in various fields, more and more attention has been focused on noise removal techniques for image processing. The effective elimination of unwanted noise plays a crucial role in improving image quality. To meet [...] Read more.

In recent years, with the increasing demand for high-quality images in various fields, more and more attention has been focused on noise removal techniques for image processing. The effective elimination of unwanted noise plays a crucial role in improving image quality. To meet this challenge, many noise removal methods have been proposed, among which the diffusion model has become one of the focuses of many researchers. In order to make the restored image closer to the real image and retain more features of the image, this paper proposes a DIR-SDE method with reference to the diffusion models of IR-SDE and IDM, which improve the feature retention of the image in the de-raining process, and then improve the realism of the image for the image de-raining task. In this study, IR-SDE was used as the base structure of the diffusion model, IR-SDE was improved, and DINO-ViT was combined to enhance the image features. During the diffusion process, the image features were extracted using DINO-ViT, and these features were fused with the original images to enhance the learning effect of the model. The model was also trained and validated with the Rain100H dataset. Compared with the IR-SDE method, it improved 0.003 in the SSIM, 0.003 in the LPIPS, and 1.23 in the FID. The experimental results show that the diffusion model proposed in this study can effectively improve the image restoration performance. Full article

(This article belongs to the Special Issue Stereo Vision Sensing and Image Processing)

18 pages, 8250 KiB

Open AccessArticle

Single-Shot 3D Reconstruction via Nonlinear Fringe Transformation: Supervised and Unsupervised Learning Approaches

by Andrew-Hieu Nguyen and Zhaoyang Wang

Sensors 2024, 24(10), 3246; https://0-doi-org.brum.beds.ac.uk/10.3390/s24103246 - 20 May 2024

Viewed by 465

Abstract

The field of computer vision has been focusing on achieving accurate three-dimensional (3D) object representations from a single two-dimensional (2D) image through deep artificial neural networks. Recent advancements in 3D shape reconstruction techniques that combine structured light and deep learning show promise in [...] Read more.

The field of computer vision has been focusing on achieving accurate three-dimensional (3D) object representations from a single two-dimensional (2D) image through deep artificial neural networks. Recent advancements in 3D shape reconstruction techniques that combine structured light and deep learning show promise in acquiring high-quality geometric information about object surfaces. This paper introduces a new single-shot 3D shape reconstruction method that uses a nonlinear fringe transformation approach through both supervised and unsupervised learning networks. In this method, a deep learning network learns to convert a grayscale fringe input into multiple phase-shifted fringe outputs with different frequencies, which act as an intermediate result for the subsequent 3D reconstruction process using the structured-light fringe projection profilometry technique. Experiments have been conducted to validate the practicality and robustness of the proposed technique. The experimental results demonstrate that the unsupervised learning approach using a deep convolutional generative adversarial network (DCGAN) is superior to the supervised learning approach using UNet in image-to-image generation. The proposed technique’s ability to accurately reconstruct 3D shapes of objects using only a single fringe image opens up vast opportunities for its application across diverse real-world scenarios. Full article

(This article belongs to the Special Issue Stereo Vision Sensing and Image Processing)

► Show Figures

Figure 1

18 pages, 36613 KiB

Open AccessArticle

A Light Multi-View Stereo Method with Patch-Uncertainty Awareness

by Zhen Liu, Guangzheng Wu, Tao Xie, Shilong Li, Chao Wu, Zhiming Zhang and Jiali Zhou

Sensors 2024, 24(4), 1293; https://0-doi-org.brum.beds.ac.uk/10.3390/s24041293 - 17 Feb 2024

Viewed by 649

Abstract

Multi-view stereo methods utilize image sequences from different views to generate a 3D point cloud model of the scene. However, existing approaches often overlook coarse-stage features, impacting the final reconstruction accuracy. Moreover, using a fixed range for all the pixels during inverse depth [...] Read more.

Multi-view stereo methods utilize image sequences from different views to generate a 3D point cloud model of the scene. However, existing approaches often overlook coarse-stage features, impacting the final reconstruction accuracy. Moreover, using a fixed range for all the pixels during inverse depth sampling can adversely affect depth estimation. To address these challenges, we present a novel learning-based multi-view stereo method incorporating attention mechanisms and an adaptive depth sampling strategy. Firstly, we propose a lightweight, coarse-feature-enhanced feature pyramid network in the feature extraction stage, augmented by a coarse-feature-enhanced module. This module integrates features with channel and spatial attention, enriching the contextual features that are crucial for the initial depth estimation. Secondly, we introduce a novel patch-uncertainty-based depth sampling strategy for depth refinement, dynamically configuring depth sampling ranges within the GRU-based optimization process. Furthermore, we incorporate an edge detection operator to extract edge features from the reference image’s feature map. These edge features are additionally integrated into the iterative cost volume construction, enhancing the reconstruction accuracy. Lastly, our method is rigorously evaluated on the DTU and Tanks and Temples benchmark datasets, revealing its low GPU memory consumption and competitive reconstruction quality compared to other learning-based MVS methods. Full article

(This article belongs to the Special Issue Stereo Vision Sensing and Image Processing)

► Show Figures

Figure 1

16 pages, 11264 KiB

Open AccessArticle

Calibration-Free Mobile Eye-Tracking Using Corneal Imaging

by Moayad Mokatren, Tsvi Kuflik and Ilan Shimshoni

Sensors 2024, 24(4), 1237; https://0-doi-org.brum.beds.ac.uk/10.3390/s24041237 - 15 Feb 2024

Viewed by 991

Abstract

In this paper, we present and evaluate a calibration-free mobile eye-traking system. The system’s mobile device consists of three cameras: an IR eye camera, an RGB eye camera, and a front-scene RGB camera. The three cameras build a reliable corneal imaging system that [...] Read more.

In this paper, we present and evaluate a calibration-free mobile eye-traking system. The system’s mobile device consists of three cameras: an IR eye camera, an RGB eye camera, and a front-scene RGB camera. The three cameras build a reliable corneal imaging system that is used to estimate the user’s point of gaze continuously and reliably. The system auto-calibrates the device unobtrusively. Since the user is not required to follow any special instructions to calibrate the system, they can simply put on the eye tracker and start moving around using it. Deep learning algorithms together with 3D geometric computations were used to auto-calibrate the system per user. Once the model is built, a point-to-point transformation from the eye camera to the front camera is computed automatically by matching corneal and scene images, which allows the gaze point in the scene image to be estimated. The system was evaluated by users in real-life scenarios, indoors and outdoors. The average gaze error was 1.6∘ indoors and 1.69∘ outdoors, which is considered very good compared to state-of-the-art approaches. Full article

(This article belongs to the Special Issue Stereo Vision Sensing and Image Processing)

► Show Figures

Figure 1

24 pages, 132107 KiB

Open AccessArticle

TranSpec3D: A Novel Measurement Principle to Generate A Non-Synthetic Data Set of Transparent and Specular Surfaces without Object Preparation

by Christina Junger, Henri Speck, Martin Landmann, Kevin Srokos and Gunther Notni

Sensors 2023, 23(20), 8567; https://0-doi-org.brum.beds.ac.uk/10.3390/s23208567 - 18 Oct 2023

Viewed by 1059

Abstract

Estimating depth from images is a common technique in 3D perception. However, dealing with non-Lambertian materials, e.g., transparent or specular, is still nowadays an open challenge. However, to overcome this challenge with deep stereo matching networks or monocular depth estimation, data sets with [...] Read more.

Estimating depth from images is a common technique in 3D perception. However, dealing with non-Lambertian materials, e.g., transparent or specular, is still nowadays an open challenge. However, to overcome this challenge with deep stereo matching networks or monocular depth estimation, data sets with non-Lambertian objects are mandatory. Currently, only few real-world data sets are available. This is due to the high effort and time-consuming process of generating these data sets with ground truth. Currently, transparent objects must be prepared, e.g., painted or powdered, or an opaque twin of the non-Lambertian object is needed. This makes data acquisition very time consuming and elaborate. We present a new measurement principle for how to generate a real data set of transparent and specular surfaces without object preparation techniques, which greatly reduces the effort and time required for data collection. For this purpose, we use a thermal 3D sensor as a reference system, which allows the 3D detection of transparent and reflective surfaces without object preparation. In addition, we publish the first-ever real stereo data set, called TranSpec3D, where ground truth disparities without object preparation were generated using this measurement principle. The data set contains 110 objects and consists of 148 scenes, each taken in different lighting environments, which increases the size of the data set and creates different reflections on the surface. We also show the advantages and disadvantages of our measurement principle and data set compared to the Booster data set (generated with object preparation), as well as the current limitations of our novel method. Full article

(This article belongs to the Special Issue Stereo Vision Sensing and Image Processing)

► Show Figures

Figure 1

16 pages, 4046 KiB

Open AccessArticle

Eye Segmentation Method for Telehealth: Application to the Myasthenia Gravis Physical Examination

by Quentin Lesport, Guillaume Joerger, Henry J. Kaminski, Helen Girma, Sienna McNett, Mohammad Abu-Rub and Marc Garbey

Sensors 2023, 23(18), 7744; https://0-doi-org.brum.beds.ac.uk/10.3390/s23187744 - 7 Sep 2023

Cited by 1 | Viewed by 1563

Abstract

Due to the precautions put in place during the COVID-19 pandemic, utilization of telemedicine has increased quickly for patient care and clinical trials. Unfortunately, teleconsultation is closer to a video conference than a medical consultation, with the current solutions setting the patient and [...] Read more.

Due to the precautions put in place during the COVID-19 pandemic, utilization of telemedicine has increased quickly for patient care and clinical trials. Unfortunately, teleconsultation is closer to a video conference than a medical consultation, with the current solutions setting the patient and doctor into an evaluation that relies entirely on a two-dimensional view of each other. We are developing a patented telehealth platform that assists with diagnostic testing of ocular manifestations of myasthenia gravis. We present a hybrid algorithm combining deep learning with computer vision to give quantitative metrics of ptosis and ocular muscle fatigue leading to eyelid droop and diplopia. The method works both on a fixed image and frame by frame of the video in real-time, allowing capture of dynamic muscular weakness during the examination. We then use signal processing and filtering to derive robust metrics of ptosis and l ocular misalignment. In our construction, we have prioritized the robustness of the method versus accuracy obtained in controlled conditions in order to provide a method that can operate in standard telehealth conditions. The approach is general and can be applied to many disorders of ocular motility and ptosis. Full article

(This article belongs to the Special Issue Stereo Vision Sensing and Image Processing)

► Show Figures

Figure 1

12 pages, 2838 KiB

Open AccessArticle

Research on 3D Reconstruction of Binocular Vision Based on Thermal Infrared

by Huaizhou Li, Shuaijun Wang, Zhenpeng Bai, Hong Wang, Sen Li and Shupei Wen

Sensors 2023, 23(17), 7372; https://0-doi-org.brum.beds.ac.uk/10.3390/s23177372 - 24 Aug 2023

Cited by 3 | Viewed by 1723

Abstract

Thermal infrared imaging is less affected by lighting conditions and smoke compared to visible light imaging. However, thermal infrared images often have lower resolution and lack rich texture details, making them unsuitable for stereo matching and 3D reconstruction. To enhance the quality of [...] Read more.

Thermal infrared imaging is less affected by lighting conditions and smoke compared to visible light imaging. However, thermal infrared images often have lower resolution and lack rich texture details, making them unsuitable for stereo matching and 3D reconstruction. To enhance the quality of infrared stereo imaging, we propose an advanced stereo matching algorithm. Firstly, the images undergo preprocessing using a non-local mean noise reduction algorithm to remove thermal noise and achieve a smoother result. Subsequently, we perform camera calibration using a custom-made chessboard calibration board and Zhang’s camera calibration method to obtain accurate camera parameters. Finally, the disparity map is generated using the SGBM (semi-global block matching) algorithm based on the weighted least squares method, enabling the 3D point cloud reconstruction of the object. The experimental results demonstrate that the proposed algorithm performs well in objects with sufficient thermal contrast and relatively simple scenes. The proposed algorithm reduces the average error value by 10.9 mm and the absolute value of the average error by 1.07% when compared with the traditional SGBM algorithm, resulting in improved stereo matching accuracy for thermal infrared imaging. While ensuring accuracy, our proposed algorithm achieves the stereo reconstruction of the object with a good visual effect, thereby holding high practical value. Full article

(This article belongs to the Special Issue Stereo Vision Sensing and Image Processing)

► Show Figures

Figure 1

Journal Menu

Journal Browser

Stereo Vision Sensing and Image Processing

Share This Special Issue

Special Issue Editor

Special Issue Information

Keywords

Published Papers (8 papers)

Research

Further Information

Guidelines

MDPI Initiatives

Follow MDPI