Video Distance Measurement Technique Using Least Squares Based Sharpness Cost Function

Abstract

A wide range of precision applications requires video measuring systems that achieve a large number of successive measurements and deliver fast results. Their efficiency is essentially given by the technical performances of the used equipment and by the measurement technique on which they operate. In order to enhance the reliability of such a system, the paper presents a new method of measuring the distance with a single video camera intended to assess the distance at which the object of interest to the camera is located. The technique makes use of a least squares-based sharpness cost function and determines the distance between the camera and the object of interest by minimizing the least squares deviation of the current sharpness values from the sharpness values obtained by calibration. It involves the current sharpness calculation phase, the normalization phase, the phase of calculating the deviations of the current sharpness from the dependencies obtained by calibration and the phase of determining the minimum deviation index.

1. Introduction

Optical distance determination can be performed by active and passive methods [1]. Active emitter-receiver methods are based on computing the time of flight for a light, ultrasound or radio beam emitted to the object of interest, which after reflection, is received by the video camera [2]. The beam contains a known pattern, and the method of determining the distance is based on measuring the deformation of the pattern [3,4]. There were several products developed that were subjected to this active method, each with its complexity and hardware limits: ARCore, a Google platform for building augmented reality [5], Occipital’s Structure Sensor for 3D scanning [6], Kinovea analysis software for sport and medical professionals [7] or Microsoft’s Kinect, a motion controller for gaming peripherals [8]. Research and advances in optical systems [9,10,11] regarding optical transmitter-receiver application in video communication are also acquainted [12,13,14] but preserve the potential confusion of echoes from previous or subsequent beam pulses or from other systems. Moreover, these applications require expensive and specialized equipment to operate, depending on calibration accuracy [15,16] and a limit of 4 m distance to be measured [17].

Passive methods for distance measuring are based on cameras and computer vision techniques, generally using two-chamber binocular systems between which the distance is accurately known [18]. These methods are based on information about the object’s position, thus requiring a precise alignment of the two cameras’ captured images to identify the correspondence between the object of interest’s images [19,20,21]. The issue of disparity between corresponding pixels in the two images was removed through the adoption of deep convolutional neural networks, whose gain depends on the images’ resolution and on the number of iterations the application requires [22,23]. Another approach for image stitching is a feature-based technique, which extracts point-by-point characteristics from two or more images using different algorithms to compose a panoramic image. They compare all pixels’ intensities with each other or determine a correspondence between images through distinct features extracted from the processing [24,25]. One popular IT device built on this principle is Dell Venue 8 7000 Series, whose stereoscopic cameras allow measuring length, width, and height and calculating particular areas through photos. A derived variant of the binocular method uses a single camera moving over a known distance, obtaining the binocular effect. This concept finds most of its applications in the optical 3D shape reconstruction field [26,27,28] or for obstacle avoidance [29], with the downside of a large amount of data that have to be captured and processed. The precision and test–retest reliability of image reconstruction have been improved by distance measurement methods based on the correspondence between the position of the focus lens and the distance the object is from the camera, known as shape from focus or depth from focus [30,31,32]. To accurately generate a map of the scene, the distance from the camera must be calculated for every point by measuring its relative degree of focus in the images where the point appears [33], so the performance of using the focus in determining the distance depends on the number of positions that the focus lens can have. In shape from focus applications, the process of Gaussian interpolation for creating new data points using a given discrete set of data has the widest practice. Gaussian interpolation uses three focus values (the maximum and two adjacent values) and has been proposed since 1994 [34]. To some extent, linear and quadratic interpolation are adopted for measuring processing [35], with the advantage of speed and simplicity.

A new passive method for visual distance measurement with a single camera is proposed in this paper. It requires a fixed camera and involves the processing of all the images of the object of interest (for each position of the focus lens). In order to overcome the high computational demand of the shape from focus and derived methods, the quality of the image is evaluated globally and not on the pixel granularity level. Since sharpness and contrast define the quality of a perceived image [36,37], a least squares-based sharpness cost function is minimized, yielding the distance to the object of interest.

2. Materials and Methods

2.1. Method Description

The proposed distance measurement method, making use of a least squares-based sharpness cost function, comprises two steps: a calibration step and a measurement one. Calibration is performed only once for a camera type and involves:

• establishing the dependence of the sharpness, for each position of the lens, on the position of the object of interest. A (n,m) mapping matrix is obtained with n and m being the number of the positions of the focus lens and the number of positions of the object of interest, respectively;
• approximating each dependency by a polynomial function S_i and identification of the function coefficients.

Measuring the distance to the object of interest involves:

• taking a set of images of the object (stack) and calculating the sharpness for each position of the lens, S_{i_measured};
• calculating the Cost Function as being the square deviations between the measured sharpness and the sharpness obtained by calibration for each of the m possible positions of the object;
• $CF\left(j\right)={{\displaystyle \sum }}_{i=0}^{n-1}{\left(Si\left(xj\right)-Si\_measured\right)}^2$;
• establishing the index of the minimum of the square deviations, which yield the calculated distance;
• $estimated index=\underset{j=1;m}{\min }CF\left(j\right)$.

2.2. Experimental Setup

For testing, the proposed measurement technique used was a Logitech Pro C920 webcam equipped with Carl Zeiss^® optics, with a 4-bit focus, equivalent to 16 distinct lens positions. The autofocusing process is completed within a fraction of a second (0.38 s in good light; 0.89 s in low light conditions), set by default responsive, suitable for short or medium distances (this mode is useful when aiming to capture a quick-moving visual task or when aiming to capture multiple visual tasks while switching the focus continually from one to another). The webcam is integrated on an experimental linear displacement stand that allows object movement relative to the camera with a high resolution. Structurally, the stand (Figure 1) is composed of an axis for linear motion with a slider (1), a camera test chart/array of 140 colors (2), a single-ended optical shaft encoder with 0.0235 mm resolution (4096 counts per revolution in quadrature mode/1024 lines per revolution), used for position feedback (3), a DC brushed micromotor, with 0.7 Nm precision planetary gearheads (4), single channel linear voltage-controlled power amplifier (5), a data acquisition board (6) with closed-loop control, connected with (3), and the webcam (7). In order to drive the system toward a target position, a feedback loop with a PID controller is implemented. LabView 2021 software is used for command, control, data acquisition and image processing with Vision Development Module 2021 National Instruments toolkit.

3. Results

3.1. Calibration Stage

First, to determine the evolution of the clarities according to displacement, the calibration phase is performed, as rendered in Figure 2. Two cycles are involved: the cycle of the object of interest (1) and the cycle of the focus lens (2). The lens cycle is performed only once for each position (3) of the object cycle and leads to obtaining the image set (stack) on which the focus is established, acquiring the image with the best sharpness. The object cycle involves moving the object of interest (3) along the axis of symmetry of the camera, with a resolution equal to the desired resolution to be obtained when measuring. For each position of the object, the sharpness values (4) obtained from the lens cycle are normalized (5). From the matrix of the obtained data, the dependence of the sharpness on the position of the object (6) is extracted for each position of the lens, and each dependence is polynomially approximated (7).

From step (6) the dependencies of the sharpness on the position of the object were approximated for each of the 16 positions of the focus lens by a polynomial function:

S_i = a₀ + a₁x + a₂x² + a₃x³ + a₄x⁴ + a₅x⁵ + a₆x⁶

(1)

where i represents the lens’ position, i = 0…15, x is the objects’ position and S_i is the sharpness value. The coefficients of the 16 associated polynomial functions of 6 degree are given in

Table 1. Polynomial coefficients determined for each lens position.

Lens’ Position (i)	Polynomial Coefficients
	a0	a1	a2	a3	a4	a5	a6
0	1.00242	−3.41879 × 10−3	−292321 × 10−4	4.05166 × 10−6	−2.16663 × 10−8	5.22761 × 10−11	−4.74343 × 10−14
1	0.898721	6.36388 × 10−3	−4.76928 × 10−4	5.53824 × 10−6	−2.7626 × 10−8	6.40214 × 10−11	−5.65194 × 10−14
2	0.802894	1.44673 × 10−2	−5.99068 × 10−4	6.25795 × 10−6	−2.94182 × 10−8	6.53407 × 10−11	−5.57859 × 10−14
3	0.704034	1.85123 × 10−2	−5.59871 × 10−4	5.0166 × 10−6	−2.07422 × 10−8	4.09188 × 10−11	−3.11744 × 10−14
4	0.672847	1.47299 × 10−2	−3.34911 × 10−4	2.04935 × 10−6	−4.44685 × 10−8	3.8416 × 10−11	6.60522 × 10−15
5	0.684477	6.57673 × 10−3	−2.49564 × 10−5	−1.41218 × 10−6	1.24868 × 10−8	−3.79673 × 10−11	3.95845 × 10−14
6	0.707126	−1.54146 × 10−3	2.0778 × 10−4	−3.24671 × 10−6	1.81618 × 10−8	−4.42197 × 10−11	3.99154 × 10−14
7	0.745179	−9.34635 × 10−3	3.99719 × 10−4	−4.82945 × 10−6	2.47073 × 10−8	−5.8496 × 10−11	5.28991 × 10−14
8	0.831732	−2.12925 × 10−2	6.80613 × 10−4	−7.49486 × 10−6	3.79759 × 10−8	−9.16576 × 10−11	8.50388 × 10−14
9	0.706765	−4.9561 × 10−3	1.48598 × 10−4	−1.44829 × 10−6	7.39629 × 10−9	−1.89172 × 10−11	1.82406 × 10−14
10	0.704984	−6.61971 × 10−3	1.71546 × 10−4	−1.6138 × 10−6	7.1569 × 10−9	−1.44787 × 10−11	1.04764 × 10−14
11	0.690874	−4.97126 × 10−3	1.07935 × 10−4	−8.17178 × 10−7	2.66015 × 10−9	−2.95716 × 10−12	−3.20389 × 10−16
12	0.687389	−4.74209 × 10−3	9.63782 × 10−5	−6.6829 × 10−7	1.82623 × 10−9	−8.70554 × 10−13	−2.20633 × 10−15
13	0.682394	−4.54304 × 10−3	8.85567 × 10−5	−5.72535 × 10−7	1.29942 × 10−9	4.56257 × 10−13	−3.44156 × 10−15
14	0.677695	−4.47291 × 10−3	8.77463 × 10−5	−5.72904 × 10−7	1.33118 × 10−9	3.31304 × 10−13	−3.29169 × 10−15
15	0.674817	−4.50933 × 10−3	8.79664 × 10−5	−5.6798 × 10−7	1.27092 × 10−9	5.60098 × 10−13	−3.57643 × 10−15

and their graphical approximation functions to the data obtained by calibration are rendered in Figure 3. For method accurately accomplishment, all decimals must be used.

3.2. Measurement Stage

The measurement phase, performed following the operations from Figure 4, involves taking the image set (stack) through the lens cycle (2), calculating the sharpness corresponding to each position of the lens (4) and normalizing the sharpness values (5). Then, one calculates the deviations of the current clarities from the dependencies resulting from the calibration for each position of the object (8), and the index of the minimum deviations is determined (9), which will correspond to the measured distance.

For an object placed, for example, at a distance of 104 mm away, the set of 16 images is taken for which the normalized clarities are calculated (Figure 5). For each possible position of the object, the deviation between the current clarities and the ones obtained by calibration is calculated (Figure 6). The minimum deviation is set, which will correspond to a measured distance of 103 mm (Figure 7).

4. Discussion

The object has been displaced in the range of [25 mm; 620 mm], passing through 341 positions. For each position, the relative error was calculated as the difference between the prescribed distance (d) and the distance calculated by the proposed method (D). A graphical render of their values is shown in Figure 8. Moreover, the relative error to the prescribed distance was calculated and displayed in Figure 9.

A qualitative analysis of the obtained results reveals that the relative error falls within a narrow range of values, both for small and large distance values. This is a great advantage when dealing with industrial applications where the measurement range is wide.

A further quantitative evaluation of the reported errors is somehow difficult to be done since there are no established benchmarks for video distance measurement. Moreover, the results are strongly determined by the optical system performance and properties, as highlighted in [38].

A comparative analysis with the best results reported in mainstream publications is presented in

Table 2. The relative error in video distance measurement.

Distance [mm]	Zuckerman M., et al. 2017 [39]	Yankin C., et al. 2017 [40]	Hahne C., et al. 2014 [41]	Setyawan R.A., et al. 2018 [42]	Megalingam R.K., et al. 2016 [43]	Dragne C., et al. 2022 [44]	Proposed Method
50					6%		2%
100	23 %				6.3%		1%
150	0.2%				2.7%	10.6%	0.7%
200	7%	1.5%	4.7%		3.5%	2.72%	2.5%
250	7.2%					5.3%	1.2%
300	11%	1.7%	5.9%			6.22%	0.4%
350	7%						1.5%
400	2%						3.7%
450	1.3%	2%	6.7%				0.3%
500	2%			2.5%			4%
550	6.1%						0.3%
600	9%	2.4%	7%	2.1%			1.7%
650	5.3%

.

By using the proposed method, improved accuracy is obtained compared to the methods that used dedicated and expensive equipment such as plenoptic (array of microlenses) cameras [40,41] and stereo vision cameras [42,44], or make use of a known dimensional reference in the optical path [43].

5. Conclusions

In this study, a new distance measurement method is proposed, with a single video camera for assessing the distance of an object from the camera. The method comprises a calibration step and a measurement step.

The first step contains the cycle of the object of interest (involving the movement of the object of interest along the axis of symmetry of the camera) and the cycle of the focus lens (performed only once for each position in the object cycle). For each position of the object, the sharpness values obtained from the lens cycle are normalized. The dependence of the sharpness on the position of the object is extracted from the bidimensional data matrix for each lens’ position and polynomially approximated.

In the measurement step, for a certain position of the object, a set of images is taken characteristic of the lens cycle, and the sharpness corresponding to each position of the lens is determined and normalized. A least squares-based sharpness cost function is then computed and minimized, yielding the distance to the object of interest.

In terms of accuracy, the proposed method gives similar or better results when compared with reported methods making use of dedicated and expensive equipment such as plenoptic or stereo setups.