A Novel Patient-to-Image Surface Registration Technique for ENT- and Neuro-Navigation Systems: Proper Point Set in Patient Space

Abstract

Patient-to-medical image registration is a crucial factor that affects the accuracy of image-guided ENT- and neurosurgery systems. In this study, a novel registration protocol that extracts the point cloud in the patient space using the contact approach was proposed. To extract the optimal point cloud in patient space, we propose a multi-step registration protocol consisting of augmentation of the point cloud and creation of an optimal point cloud in patient space that satisfies the minimum distance from the point cloud in the medical image space. A hemisphere mathematical model and plastic facial phantom were used to validate the proposed registration protocol. An optical and electromagnetic tracking system, of the type that is commonly used in clinical practice, was used to acquire the point cloud in the patient space and evaluate the accuracy of the proposed registration protocol. The SRE and TRE of the proposed protocol were improved by about 30% and 50%, respectively, compared to those of a conventional registration protocol. In addition, TRE was reduced to about 28% and 21% in the optical and electromagnetic methods, respectively, thus showing improved accuracy. The new algorithm proposed in this study is expected to be applied to surgical navigation systems in the near future, which could increase the success rate of otolaryngological and neurological surgery.

1. Introduction

An image-guided surgery system visualizes the three-dimensional position of a surgical tool using preoperative medical images and provides the location of lesions and surrounding areas in real time [1,2]. The imaging information of this type of system allows surgeons to perform minimally invasive surgery and plan a path to the lesion in advance, thereby improving the quality of the surgery and reducing the operating time [3]. Therefore, this technology is widely used for different types of neurosurgery, such as tumor biopsy and resection, craniotomy, and deep brain stimulation [4], as well as in head and neck surgery, including sinusitis surgery [5]. Spatial registration, one of the core elements of image-guided ENT- and neurosurgery systems, is the process of aligning the coordinates of a medical image taken before surgery with the spatial coordinates of the patient acquired during surgery in the same space [6]. If the registration between the two spaces is not accurate, a gap arises between the position of the actual surgical tool and that of the virtual surgical tool displayed on the image. Therefore, the registration technology is a crucial factor that affects the accuracy of an image-guided ENT- and neurosurgery system [3,4,7].

Patient-to-medical image registration methods can be divided into point registration and surface registration methods [8]. A surface registration method finds the transformation matrix based on a point cloud in the patient space and the medical image space, and does not require attachment of specific fiducial markers [9,10]. Therefore, this type of registration method is free from problems such as skin swelling due to attachment of fiducial markers [11], and additional medical imaging is unnecessary, thereby eliminating cost and hassle [2]. In surface registration, also called marker-less registration, many corresponding points in each space are matched up through a sequential process of coarse and precise registration [12]. Coarse registration, which is rough and relatively inaccurate, is performed using the initial anatomical landmarks of each space, then precise registration is performed by applying an iterative closest point (ICP) algorithm. ICP is an optimization method that iteratively finds the medical image coordinates that satisfy the minimum distance from the point cloud in the patient space, and a transformation matrix is acquired when the sum of the Euclidean distances between corresponding points is minimal [13,14]. Unfortunately, despite the various advantages of surface registration, its accuracy continues to be controversial. Several studies reported that the accuracy of the technique is similar to that of point registration [15], but many other studies reported significantly lower accuracy [2,16]. Therefore, despite the various advantages of the surface registration technique, concerns about its accuracy remain.

The acquisition and processing of the point sets of each space are essential to improving the accuracy of a surface registration method that is based on the point sets in the patient and medical image space [12]. While medical images such as CT often show excellent resolution and accuracy [3,17], most previous studies focused on the extraction of a point cloud in patient space; these studies showed that accuracy can be affected by such factors as equipment performance or clinical proficiency [2,9,12,18]. The approaches to extraction of the point cloud in patient space can be divided into contact-type approaches that extract points by tracing probes on the surface of the face and non-contact approaches that collect points using additional scanner devices [6]. In the latter, a scanner device can be used to avoid contact with the surface of the face, thereby reducing the error due to deformation of soft skin when obtaining the point cloud. Furthermore, this method has the advantage of being able to obtain more point sets, which increases the accuracy of registration. Recent studies reported on methods to reduce target registration error (TRE), which utilize mobile 3D scanners to collect point clouds across the whole head [4,7]. However, scanners with guaranteed precision are generally very expensive [3]. In addition, since the device differs from that used for commercial image-guided surgery systems, an additional coordinate transformation process must be performed, which may result in accumulation of errors [19].

Therefore, the commercial image-guided surgery systems that are currently in use extract point clouds in patient space through contact with the surface of the face. Accordingly, in neurosurgery or head and neck surgery, clinicians perform surface registration by placing a probe on the patient’s face as a convenient method of extracting the point cloud; several clinical case reports of the surgical effects of this method have been conducted [5,20,21,22]. However, in all of these studies, only the original patient space coordinates obtained through facial surface tracing were used, and no attempts have been made to improve the point cloud in order to reduce the surface registration error. Therefore, the main purpose of this study is to propose a novel registration protocol that extracts the point cloud in patient space using the contact approach to increase the accuracy of surface registration of an image-guided surgery system. The proposed registration protocol consists of a two-step process: augmentation of the point clouds and creation of a proper point cloud. The secondary purpose of this study is to validate the proposed registration protocol using a hemisphere mathematical model and a plastic facial phantom.

2. Proposed Registration Strategy

The surface registration method used for image-guided surgery systems refers to a process of matching point clouds between preoperative medical image space and intraoperative patient space. The 3D structure is first reconstructed based on pre-operative tomographic medical images such as CT, and the point cloud on the facial surface is then extracted. In addition, the point cloud of the facial surface is acquired using a probe or scanner during surgery. Points in different spaces are used to perform coarse registration using three or four representative fiducial markers. The final process is a precise registration process that aims to minimize the distance between the corresponding points. In this study, a novel protocol is used to create a point cloud in patient space between the initial coarse registration stage and the precise registration stage. The detailed process is illustrated in Figure 1.

2.1. Registration Optimization Phase 1: Augmentation of the Number of Points in Patient Space

In general, medical images include a large number of points due to their high resolution. However, the point cloud in patient space is determined by the duration of the facial tracing and the available camera samples of the surgical probe. Due to practical limitations, it is difficult to obtain a large number of points in real surgical situations. Therefore, starting with the point cloud obtained by tracing, the point cloud is augmented using an interpolation technique and the surface registration performance is evaluated according to the precise registration process (Figure 1). Here, the point cloud is increased by 5% compared to the previous step. Furthermore, the precision registration method utilizes the ICP method, which is the most widely utilized method (see Section 2.3 for details). The surface registration error (SRE) is evaluated according to augmentation of the point cloud, and the termination criteria are set when the absolute value of the difference in SRE between two consecutive ICPs in the current and previous steps is within 0.1%.

2.2. Registration Optimization Phase 2: Proper Point Set in Patient Space

In the second step, the proper point cloud is extracted to match the medical image-based spatial point cloud using the augmented point cloud in patient space (Figure 2). An interpolation polynomial is extracted based on the point cloud, and a virtual point cloud that satisfies the minimum distance to each point in the corresponding medical image space in the previous step (registration optimization phase #1) is extracted. In this study, the piecewise cubic Hermite interpolation method is used, which maintains a polynomial form consisting of coordinate points with less overshoot and undershoot [23]. The process of converting the interval of $[x_k,x_{k+1}]$ to intervals of $\left[0,1\right]$ and obtaining the coefficients $a_0$, $a_1$, $a_2$, $a_3$ of the arbitrary cubic polynomial $p\left(u\right)$ is shown below [24].

$$P\left(u\right)=a_0+a_{1}u+a_2{u}^2+a_3{u}^3$$

(1)

The above cubic polynomial consists of four interpolation conditions, corresponding to two function values and two differential values at both endpoints, as follows.

$$P\left(0\right)=P_0, P\left(1\right)=P_1, P{}^{\prime }\left(0\right)=\nabla P_0, P{}^{\prime }\left(1\right)=\nabla P_{0 }$$

(2)

Polynomials that meet the above conditions can be expressed in the form of a matrix, as follows.

$$P(u)=[u^3 u^2 u 1]\left[\begin{array}{cccc}2& -2& 1& 1\\ -3& 3& -2& -1\\ 0& 0& 1& 0\\ 1& 0& 0& 0\end{array}\right]\left[\begin{array}{c}{P}_0\\ P_1\\ \nabla P_0\\ \nabla p_1\end{array}\right]$$

(3)

2.3. Final ICP Refinement

The extracted medical images and patient spatial point clouds are finally matched by applying the ICP algorithm [15]. The ICP algorithm calculates and matches rotational and translation matrices such that the mean difference between two spatial points is minimized, and is defined by the following formula:

$$\mathrm{f}(R,t)=argmi{n}_f{\displaystyle \sum }_{i=1}^N\parallel (R{s}_i+t)-c_i\parallel {}^2$$

(4)

Here, $R$ and $t$, respectively, refer to the rotational and translation matrices that can minimize the positional error of the two-point cloud. $s_i$ and $c_i$ denote the point cloud in patient space and medical image space, respectively. Optimization processing is done as follows. The distance from one point in the reference space to all points in the transform space is calculated, and the closest point is set as the corresponding point. Corresponding points are extracted by repeating the above process for all points in the reference space, and rotation and translation transformation matrices between the two point sets of the corresponding relationship are calculated. Coordinate transformation is performed through the transformation matrix, and the final transformation matrix is iteratively calculated until the criteria are satisfied. The algorithm iterates until it reaches one of two convergence criteria: either the absolute value of the difference in SRE between two consecutive iterations is below 0.001 or the number of maximum iterations is reached (set as 30 iterations) [25].

3. Validation Study Based on Hemisphere Model

3.1. Hemisphere Modeling

To evaluate the performance of the proposed surface registration algorithm, a hypothetical hemisphere model [26] was created using MATLAB R2018b (MathWorks, Inc., Natick, MA, USA). The surface points of the hemisphere were constructed so that the distribution of points was uniform (Figure 3A). In order to uniformly distribute the points on the surface of the hemisphere, the outer area of a hemisphere with a radius ($r$) of 1 was divided by the number of points $N$. A square was obtained, with area $A$ and side length $d$.

$$A=\frac{4\pi r^2}{N}$$

(5)

The unit angle $M_{\theta }$ was calculated by dividing a semicircle with the arc $\pi$ belonging to the hemisphere by the length ($d$) of one side of the square ($A$) and rounding up the result.

$$M_{\theta }=round\left(\frac{\pi }{d}\right)$$

(6)

The length $\pi$ of the semicircle with a radius ($r$) of 1 was divided by the unit angle $M_{\theta }$ to calculate the gap $d_{\theta }$ between circles with a constant latitude, and the width of the square $A$ was divided by $d_{\theta }$ to calculate the gap ($d_{\phi }$) between points with constant hardness.

$$d_{\theta }=\frac{\pi }{M_{\theta }} and d_{\phi }=\frac{A}{d_{\theta }}$$

(7)

The set of points was placed at constant intervals ($d_{\phi }$) in circles with constant latitudes and $d_{\phi }$ values, using the previously calculated values of $d_{\theta }$ and $d_{\phi }$. The radius of the hemisphere was set to 100 mm considering the size of the human face, and it was modeled for eight cases, ranging from 1527 points to 12,100 points, to evaluate the effectiveness of the number of medical image point clouds. The point cloud in the patient space represented the patient’s face traced in a left-to-right motion (Figure 3B). The point sets were constructed in the same form as the hemisphere model, and the number of point clusters was limited to 200.

The six target positions (3 rows × 2 columns) were located inside the hemisphere model to evaluate the TRE. The interval between targets was set at 50 mm. The coordinates of each target were [0,0,25], [0,−50,25], [0,−100,25], [0,0,−25], [0,−50,−25], and [0,−100,−25].

3.2. Performance Evaluation of the Proposed Registration Strategy

The performance of registration was evaluated by SRE and TRE. The SRE was obtained by calculating the average distance between the point cloud in the medical image space $\left(i=1\dots .n\right)$ and the point cloud that was augmented and registered with the proposed strategy $\left(i=1\dots .n\right)$. TRE was obtained by calculating the distance between the target location in the medical image space and the target location in the transformed patient space by applying the transformation matrix obtained through the ICP algorithm [27].

$$SRE=\frac{{{\displaystyle \sum }}_{i=1}^n\parallel CT poin{t}_i-Patient poin{t}_i \parallel }{n}$$

(8)

$$TRE=\parallel Targe{t}_{CT}P-Targe{t}_{Camera}P \parallel$$

(9)

Here, $CT poin{t}_i$ denotes the ith point of the medical image and $Patient poin{t}_i$ denotes the ith point of the patient after the transformation is applied. In addition, $Targe{t}_{CT}P$ denotes the location of the target measured in the medical image space, and $Targe{t}_{Camera}P$ denotes the points that were generated after applying the transformation matrix of the target measured in the patient space.

3.3. Results of the Proposed Registration Strategy Using the Hemisphere Model

To evaluate the proposed registration method, the initial condition (coarse registration effect) was set to coincide with the origin of the hemisphere model composed of the point sets in the medical image and patient space. The rotation matrix was calculated using the ICP optimization process due to the mismatch between the points in each space. The SRE and TRE values were calculated by transforming the points in patient space (Figure 4 and Figure 5). Compared to the initial condition, the SRE was reduced by approximately 1.3% and 33% for the conventional and proposed method, respectively. As the number of points in the medical image increases, the overall trend of SRE decreases. Overall, the error when the proposed algorithm is applied is reduced by an average of about 0.4 (0.2) mm compared to the error when the conventional registration method is used. When the registration results of the proposed method are visualized, the number of points in patient space increases, but the overall error value decreases (Figure 4).

TRE using the conventional registration protocol and the proposed registration protocol is demonstrated in Figure 5. Depending on the number of points in the medical image space (8 cases), the TRE varies by about 0.6 mm for the conventional method, and the registration using the proposed protocol was found to be 0.2 (0.1) mm smaller on average. In addition, as the target was located at an inferior position, the conventional registration method showed a larger TRE. However, there was almost no difference in TRE (0.075–0.083 mm) according to the target location.

4. Application to Plastic Facial Phantom

4.1. Design of the Plastic Facial Phantom

To evaluate the applicability of the proposed registration strategy, a plastic facial phantom was created. The basic structure of the phantom consisted of the upper part of a mannequin (the upper part of the chest, the neck, and the head; Figure 6B) and a frame to show the location of the lesion to be inserted inside the mannequin (Figure 6C). The lesion location was printed using an industrial 3D printer (ZPrinter 650, 3Dsystems, Rock Hill, SC, USA) and consisted of three rows in the center on the coronal plane. To evaluate the TRE, 5 lesion targets were placed from the surface of the face towards the back of the head at 20 mm intervals, and a frame with a total of 15 lesion targets (5 targets × 3 rows) was fabricated.

4.2. Apparatus and Experimental Protocol

CT was performed using medical imaging equipment (Brivo CT 385, General Electronics Medical Systems, Milwaukee, WI, USA) to acquire the location information of the phantom surface. Cross-sectional CT images were taken at 0.5 mm intervals (resolution, 512 × 512 pixels). In addition, an optical tracking device and an electromagnetic tracking device, which are commonly used in clinical practice, were used to acquire the points in the patient space. For the optical devices, three optical cameras (Optitrack Flex 3, Natural points, Corvallis, OR, USA), like those that are generally used for motion capture systems [28,29], and a passive surgical probe (Northern Digital Inc., Waterloo, ON, Canada) were used (Figure 6A). The 3D coordinates of four infrared markers attached to passive probes were obtained using the optical system. The optical system consisted of the three optical cameras, the data acquisition S/W, and the surgical probe. An Aurora electromagnetic tracking system (Northern Digital Inc., Waterloo, ON, Canada) was used to acquire the point cloud of the facial surface (Figure 6B). The system consisted of an electromagnetic field generator, a tracker attached to the surface of the forehead, and a pointer tool to extract the point cloud of the facial surface.

The Medical Imaging Interaction Toolkit (MITK) library was used to acquire 2D tomographic images and the 3D shape of the facial phantom from CT DICOM files. The 3D point sets of the facial surface were extracted using the marching cube technique, and the acquisition range of the 3D points was limited in range from the lip area to the top of the forehead. Patient space coordinates were obtained by tracing the surface of the face using the optical and electromagnetic equipment. The tracing time did not exceed 20 s in order to accurately reflect real clinical situations.

4.3. Experimental Results of the Proposed Registration Strategy

For initial registration (coarse registration) of the point cloud acquired from the medical image and patient space, four coordinate points on the facial surface that could easily be identified were roughly selected. The locations of the four feature points in this study were the tip of the nose, the tips of both eyes, and the glabellar. Subsequently, the transformation matrix (rotation and translation) of the two spaces was extracted using the singular value decomposition method to perform the initial registration. Figure 7 illustrates a typical case of initial registration and shows the difference with ICP precision registration. The difference in the position of the point cloud between the initial registration and the ICP registration in all experimental cases using the plastic facial phantom was found to be about 2 mm on average. Successful initial registration was performed, leading to ICP refinement optimization without local minima.

Figure 8 shows a comparison of the results of the conventional and proposed registration methods using experimental data from the plastic facial phantom. For the optical tracking system, the average SRE value using the conventional method was 1.36 mm, while the proposed registration method showed a reduced average SRE value of 1.07 mm and an increase in accuracy of about 21% (p < 0.01). Additionally, the SRE value in the proposed registration method decreased by about 15% compared to the conventional registration method, and the difference was significant (p < 0.01).

Figure 9 visually illustrates the difference in SRE between the conventional and proposed registration methods for representative experimental data from 20 replications. The colormap shown on the right side of the figure indicates that the SRE increases as the color goes from blue to red (i.e., blue represents zero). Both the optical and electromagnetic tracking systems produced a wide distribution of blue across the facial surface using the proposed registration method compared to the conventional method, and a significant reduction in errors (red). In the rest of the experiments, both the optical and electromagnetic systems produced similar results (color distributions).

Table 1. Comparison of TRE between the conventional and proposed registration strategies and between target locations in the optical and electromagnetic systems.

	Optical System		Electromagnetic System
	Conventional	Proposed	Conventional	Proposed
Front	3.10 (0.8)	1.79 (0.6)	4.23 (0.8)	3.37 (1.0)
Middle	3.55 (0.7)	2.65 (0.3)	4.44 (0.9)	3.58 (1.1)
Rear	4.16 (0.6)	3.32 (0.5)	4.90 (1.0)	3.78 (1.0)

shows a comparison between the registration methodologies (conventional method vs. proposed method) and the TRE value according to the location of the diseased area. Compared to the existing method, the proposed registration method in this study showed reduced TRE values regardless of the location of the target.

5. Discussion

Various attempts have been made to improve the surface matching performance of surgical navigation systems. Recently, in an attempt to reduce the TRE, point clouds in patient space have been extracted using a non-contact method via scanners and other devices [2,7,9,12]. Cao et al. (2008) extracted patient space coordinates using a commercial laser range scanner. The facial surface and the cortical surface of the brain were measured using different cradles, and the size of the scanner device was large and cumbersome, which limited task performance. Recently, Fan et al. (2014, 2017) attempted to easily extract point clouds in patient space using a mobile scanner (Go!SCAN scanner, Sense 3D scanner). The authors were able to extract a point cloud by scanning up to the back of the actual head, resulting in improved matching accuracy. However, since scanners with guaranteed precision are generally expensive and are not integrated with commercial navigation systems [3], their use necessitates an additional coordinate transformation, and there is an increased possibility that cumulative errors will occur in this process. Therefore, the objective of this study is to propose a new protocol that can reduce the registration error by utilizing the face surface tracing method that is currently in widespread clinical use.

In the registration result of the hemisphere model, the SRE tended to decrease as the number of points increased, and the accuracy of the proposed registration protocol was excellent for all cases (Figure 4 and Figure 5). Previous studies have attempted to increase the accuracy of registration by increasing the number of points acquired in the patient space, and the highest accuracy was achieved at 300,000 points [18]. Similar studies also showed that when the number of points increases (up to 40,000), the registration error (about 2 mm) decreases rapidly [7]. This is because the larger the number of points, the higher the positional accuracy of the corresponding points in different spaces. The more points that are located within the same area, the smaller the distance between the points in the corresponding space, thus reducing SRE [10]. However, since the acquisition of many points requires a lot of time and processing steps [7], an appropriate number of points must be determined. The first strategy proposed in this work is based on the results of the existing studies mentioned above. It aims to improve accuracy by performing the matching process in a stepwise manner and establishing appropriate thresholds of differences from previous-stage errors.

The optimal point cloud extraction strategy in patient space that is proposed in this study showed improved performance compared to the conventional registration method in all cases (hemisphere and plastic facial phantom) (Figure 4, Figure 5, Figure 8, and Figure 9 and Table 1). The proposed registration protocol reduces SRE by generating patient space points that correspond as closely as possible to the medical image data cluster through the first point augmentation strategy. In this process, the target position error is decreased by reducing the residual rotation and translation error. Then, a new point cloud is generated in patient space by interpolation of the second strategy, in addition to the medical image and the point cloud in the patient space that has been mapped. In this process, the points in patient space that significantly affect residual rotation and translation error are eliminated and a new point cloud in patient space that best corresponds to the one in medical image space is created. Therefore, we believe that additional reductions in SRE and TRE are possible. As discussed by Yoo et al. (2020b), the registration accuracy can be improved in some cases by acquiring new point sets in patient space through least squares projection. We believe that the registration performance was improved in the present study for similar reasons. In addition, we expect that further improvements in registration accuracy can be achieved by combining the proper point set in patient space proposed in this study.

The limitations of this study are as follows. First, since only a few lesion locations were used for TRE analysis, the accuracy of registration for the locations of various actual lesions could not be conclusively evaluated. In this study, TRE was evaluated using only six lesion locations in the hemisphere model and 15 target lesion locations in the plastic facial phantom. However, it was possible to confirm the superiority of the registration accuracy of the proposed algorithm despite the limited set of lesion locations, and we found that the error rate was reduced by 20% even when the locations of the target lesions were very deep as compared to the existing algorithm. In the future, CT images of actual human faces must be used to evaluate the error rate, given an accurate reflection of human anatomical structure. Second, as the skin tissue of the plastic facial phantom is hard, it was not possible to study the soft tissue of actual human skin and utilize the new technique in actual clinical practice. When obtaining a point cloud in patient space, there are inherent errors in the data due to the flexibility of actual human skin during probe tracing. Future research is required to verify the new registration algorithm, including further experiments on phantoms with skin-like soft tissue and, ultimately, actual clinical validation experiments.

In conclusion, in this study, a new surface registration protocol was proposed to improve the accuracy of a surgical navigation system. To extract the optimal point cloud in patient space before registration, we propose a multi-step registration protocol consisting of augmentation of the point cloud and creation of an optimal point cloud in patient space that satisfies the minimum distance from the point cloud in medical image space. Compared with the conventional method of surface registration, the new protocol showed improvements in SRE and TRE of about 30% and 50%, respectively. In addition, a plastic facial phantom was designed, which was used to verify the accuracy and usefulness of the proposed registration method. The point cloud on the facial surface was obtained in the patient space using optical and electromagnetic systems. As a result of registration, TRE was reduced to about 28% and 21% in the optical and electromagnetic systems, respectively, thus showing improved accuracy. The proposed algorithm is expected to be applied to surgical navigation systems in the near future, which could increase the success rate of otolaryngological and neurological surgery.