Next Article in Journal
Detection of Motion on a Trampoline with a Smartwatch
Next Article in Special Issue
Enabling Fine Sample Rate Settings in DSOs with Time-Interleaved ADCs
Previous Article in Journal
Wearable Technology to Increase Self-Awareness of Low Back Pain: A Survey of Technology Needs among Health Care Workers
Previous Article in Special Issue
Systematization and Comparison of the Binary Successive Approximation Variants
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Sensors
Volume 21
Issue 24
10.3390/s21248405
sensors-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

The Use of Smartphone Photogrammetry to Digitise Transtibial Sockets: Optimisation of Method and Quantitative Evaluation of Suitability

by
Sean Cullen
1,*,
Ruth Mackay
1,
Amir Mohagheghi
2 and
Xinli Du
1
1
Department of Mechanical and Aerospace Engineering, College of Engineering Design and Physical Sciences, Brunel University, Kingston Lane, Uxbridge UB8 3PH, UK
2
Sport, Health & Exercise Sciences, College of Health, Medicine and Life Sciences, Brunel University, Kingston Lane, Uxbridge UB8 3PH, UK
*
Author to whom correspondence should be addressed.
Sensors 2021, 21(24), 8405; https://0-doi-org.brum.beds.ac.uk/10.3390/s21248405
Submission received: 28 October 2021 / Revised: 28 November 2021 / Accepted: 12 December 2021 / Published: 16 December 2021
(This article belongs to the Collection Instrument and Measurement)
Browse Figures
Versions Notes

Abstract

:
The fit of a lower limb prosthetic socket is critical for user comfort and the quality of life of lower limb amputees. Sockets are conventionally produced using hand-crafted patient-based casting techniques. Modern digital techniques offer a host of advantages to the process and ultimately lead to improving the lives of amputees. However, commercially available scanning equipment required is often expensive and proprietary. Smartphone photogrammetry could offer a low cost alternative, but there is no widely accepted imaging technique for prosthetic socket digitisation. Therefore, this paper aims to determine an optimal imaging technique for whole socket photogrammetry and evaluate the resultant scan measurement accuracy. A 3D printed transtibial socket was produced to create digital and physical twins, as reference models. The printed socket was photographed from 360 positions and simplified genetic algorithms were used to design a series of experiments, whereby a collection of photos were processed using Autodesk ReCap. The most fit technique was used to assess accuracy. The accuracy of the socket wall volume, surface area and height were 61.63%, 99.61% and 99.90%, respectively, when compared to the digital reference model. The scanned model had a wall thickness ranging from 2.075 mm at the top to 7.758 mm towards the base of the socket, compared to a consistent thickness of 2.025 mm in the control model. The technique selected did not show sufficient accuracy for clinical application due to the degradation of accuracy nearer to the base of the socket interior. However, using an internal wall thickness estimation, scans may be of sufficient accuracy for clinical use; assuming a uniform wall thickness.

1. Introduction

Obtaining a reliable and comfortable socket fit remains one of the largest areas of dissatisfaction amongst lower limb amputees [1,2,3]. Traditionally, transtibial sockets are produced from handmade plaster casts of an individual’s residual limbs, these casts are then manipulated to produce a rectified copy of the original patient’s limb. The rectified cast is subsequently used as a mould to form the socket material [4,5,6,7]. When the socket fit changes the socket is discarded and a new socket is produced. This is often a lengthy and wasteful process, especially if only minor revisions are needed. Digital technologies can replace stages in the traditional socket manufacturing process, such as rectification, mould production, and final socket fabrication. Digital techniques offer a host of advantages over conventional techniques, such as reproducibility, design history, and rapid prototyping. The ability to digitise an existing socket without destroying it could in turn create sockets that are more comfortable for the end user, leading to an increased quality of life, whilst reducing pain and suffering.
Within limb centres one of the critical advantages of digital socket production is the ability to utilise centralised fabrication facilities, avoiding much of the advanced machining costs whilst reducing workshop man hours, and potentially decreasing delivery times to patients [8,9,10,11,12]. There are techniques that use direct limb casting to laminate the socket onto the user, allowing for socket delivery within one session [13]. Whilst these methods have increased satisfaction for transfemoral amputees, the authors are not aware of any literature on reliably successful use with transtibial amputees.
To fabricate prosthetics using computer aided manufacturing (CAM) techniques, the patient’s biological data require digitising. A popular method for digitising is the use of hand held 3D scanners that use lasers or structured light to determine the surface topography and volume of a patient’s residual limb, residuum cast, or existing socket [14,15]. The scanner is moved by the clinician around the subject to capture the entire surface. The topography point cloud generated by the scanner is then converted into a mesh, and finally a 3D object that can be manipulated via computer aided design (CAD) software to create the desired object, such as a socket mould. One of the main barriers to accessing digital techniques is the equipment cost. Commercially available prosthetic 3D scanners, such as the WillowWood Omega scanner, are often proprietary and can cost in excess of £5,000. This is significant when compared to the cost of casting consumables, in the region of <£50; re-training and upskilling in the use of 3D scanners should also be taken into consideration [16,17]. Therefore, the use of low-cost scanning equipment could be the key to more widespread use of digital socket manufacturing, especially in low- and middle-income countries (LMICs) where traditional fabrication equipment is not established. Van der Stelt et al., who used a 3D scanner and 3D printing to produce transtibial prosthetics, required €15,000 alongside other significant donations, including the time of a researcher, that were required to produce 3D printed sockets in Sierra Leone [18].
Photogrammetry is a digitisation technique in which a collection of 2D photographs from different viewpoints of an object are analysed with specialised software. Key elements are identified that can be seen in multiple images, allowing points to be located in a 3D space relative to each other, forming a 3D surface mesh [19,20,21].
Solav et al., explored using photogrammetric techniques for residuum scanning applications with an array of cameras centred around an enclosed scanning volume [22]. This method cost $2,000, which was low when compared to 3D scanners; however, it was specialised which would result in increased cost if commercialised, affecting accessibility. An alternative solution to specialised equipment for digital scanning was explored by Hernandez and Lemaire. Their work utilised a readily available smartphone camera and patterned tape to digitise prosthetic sockets [23]. Modern smartphones can cost up to £1,000, but are something that most prosthetists will have access to.
Another important factor to consider for scanning techniques is accuracy. It was noted by Sanders and Fatone, during a literature review, that a general residuum volume change of −5% or +2.5% could lead to a clinically significant change in fit, with volume changes of −10% to +5% leading to an unacceptable socket fit [24]. These volumes were taken from changes in sock ply and converted to an equivalent volume percentage. Therefore, inaccurate residuum scans can lead to a socket with poorer fit. Hernandes and Lemaire compared the scan volumes and surface measurements for the internal socket wall. The percentage volume difference in the study was reported between 2% and 10% [23]. By comparison, a low cost (~£500) structured light 3D scanner (iSense) was shown to have errors of 3–6% in a study by Armitage et al. [25]. Laser scanners can achieve even higher accuracies, with one study reporting accuracies of 0.8% in volume [26]. Whilst the smartphone photogrammetry errors are higher, the study by Hernandes and Lemaire imaged an internal socket wall which has a limited number of viewing angles, and is therefore more challenging to scan when compared to the other studies [23]. A team in Diponegoro that used photogrammetry techniques, with both smartphones and conventional DSLR cameras, to produce a transtibial socket, was unable to fairly evaluate the scan accuracy for the socket as the rectifications were different for the hand-casted model. The team were able to determine that the photogrammetry scans had an accuracy of over 95% against uniform reference objects [27].
The usability of each type of scanning method should also be considered alongside the cost. Structured light scanners are handheld and portable, and provide real-time scanning, whereas laser scanners can be static with a set measurement volume, such as the one in the study mentioned previously [26]. Davide et al. investigated the difference between high- and low-cost scanning equipment; this highlighted that whilst high and low cost fixed scanners may have increased accuracy, the set up and scan time would limit their applications in prosthetics when compared to hand held devices for direct limb scanning [28]. Smartphone photogrammetry can be considered handheld as it uses a mobile device with no fixed measurement volume; currently, however, there are no real-time scanning options available on common smartphones suitable for prosthetic applications.
Determining the topological differences that cause volume change are critical for clinical use in socket reproduction, as certain residuum regions are more sensitive to increased pressures created by the socket wall topography [29]. It should be noted that many of the studies discussed used residuum casts or substitutes for evaluation and that additional errors will be present when using the techniques on patient’s residuums. One literature review concluded that there are limited papers on using digital scanning techniques on amputees, making it impossible to determine if digital techniques are reliable and valid when compared to residuum volume calculations using hand- or water-based measurement techniques [30]. However, another paper reports that high end 3D scanners (Omega, WillowWood, Mt. Sterling, OH, USA) can be more reliable and accurate than traditional techniques [31].
One of the main challenges of using photogrammetry is the refinement of the photographing technique for accuracy and reliability. Whilst the general principal is to ensure a 60% overlap between adjoining images, there are near infinite possibilities to the number of photos taken and their relative position to the subject [32]. In other engineering disciplines, problems with multiple variables, such as this, have utilised genetic algorithms to reduce the computational requirements, reducing the time taken to find an optimum solution [33,34]. For example, genetic algorithms treat the experiments akin to evolving organisms, breeding and mutating gene information to find an experiment with the highest fitness characteristics (accuracy) based on a predetermined fitness equation iteratively. Unlike systematic experiment design methods, genetic algorithms search a wide array of possible experiments quickly via random sampling; these are refined to find the optimal solution. This method has limitations; not all possible experiments can be trialled due to the infinite number of inputs that could be used, therefore, there is no guarantee that the final experiment is the global optimal, however it does achieve a local optimum with significantly reduced computational time [35].
This paper aimed to determine an optimal arrangement of imaging (photograph) positions for a transtibial socket to give the highest level of photogrammetry scan accuracy for an entire socket. This was conducted by prioritising reduced photo count, using simplified genetic algorithms, and quantitively evaluating the accuracy of the resultant scans. This aim was with considerations for using low cost and widely available technologies, such as a smartphone camera, as well as falling within the limits of using Autodesk ReCap Photo software (Autodesk, San Rafael, CA, USA), to help overcome some of the challenges facing digital integration in prosthetic limb centres.

2. Materials and Methods

2.1. Control Model Creation and Preparation

It should be noted that these experiments were conducted during the COVID-19 pandemic when access to laboratories was prohibited.
An existing transtibial socket was marked with pattered tape (unicorn-patterned duct tape, Shurtape, Hemel Hempstead, Hertfordshire, UK) allowing for easier identification of overlap within images by the photogrammetry software. Whilst the pattern was repeating, slight inconsistencies in application on the socket as well as background objects were sufficient for point mapping.
This approach is similar to one outlined in the literature, including the marker dots for scaling, see Figure 1a,b [23]. Initial tests to find suitable ranges for camera positioning were conducted, resulting in a scan that resembled the socket surface both internally and externally. This was achieved by photographing the socket 384 times using a preliminary version of the technique described in more detail using cylindrical co-ordinates (Section 2.2). The photos were manually selected on a trial-and-error basis to achieve a scan that resembled the socket. After six attempts, a scan using 96 images taken at 30 degree increments at the following heights (h) relative of the base of the socket and radial distances (d) was chosen: h = 10 cm at r = 50 cm; h = 20 cm at r = 100, 50, and 30 cm; h = 50 cm at r = 100, 50, 30, and 10 cm. The achieved scan contained deformations, so the internal socket wall was removed from the digital model. The wall was replaced with a uniform offset of 2.6 mm from the external surface geometry, to create an approximated digital socket model. This model was then 3D printed (Creality Cr-10, Longhua Dist, Shenzhen, China) using PLA (Polylactic Acid) with a 0.4 mm nozzle at 0.2 mm layer height to create a near identical physical and digital model (digital twin), which is referred to as the control socket, Figure 1c. The digital and physical control socket had a known volume and surface area, that could be used for comparison with scanned files, based on the object file parameters.
The 3D printed control socket was then marked with patterned tape (flamingo-patterned duct tape, Shurtape, Hemel Hempstead, Hertfordshire, UK), with an additional two bands of patterning on the external socket wall (dog-patterned duct tape, Shurtape, Hemel Hempstead, Hertfordshire, UK), see Figure 1d. As the control model was significantly lighter than the original carbon fibre socket, a 300 g adaptor was added to avoid movements in the model position whilst the photographs were acquired. To achieve full socket coverage, the taping process took approximately 10 min. The representative steps are shown in Figure 1.

2.2. Equipment and Structure for Photo Capture

In order to photograph the control model, the socket was placed on top of a flat plate on a raised platform in the centre of a printed protractor indicating the angle increments. The smartphone (iPhone XR, Apple, Cupertino, CA, USA) was attached to a tripod with the flash turned off, in a portrait position using automatic focusing. All settings were automatic and the standard photo mode was used, to represent how this technique is likely to be carried out by clinicians. To reduce time between photos, a remote Bluetooth trigger was used. A sample photo taken from the experiment shows the model arraignment in Figure 2a.
The positions for the camera were defined using a cylindrical coordinate system, with the base of the socket as the origin. The photos were split into a series of set heights (h; −10, 20, 50, and 80 cm) and radial distances (r; 0, 10, 30, 50, 100, and 150 cm), with each series consisting of 24 photos taken at 15° increments to allow for analysis of four angle increments (Ө = 15, 30, 45, and 60°); the 3D representation is shown in Figure 2b. These parameters were chosen based on the initial testing. Increasing the number of discrete options for r, h, and Ө would make it difficult to achieve an accurate measurement and would lead to a large increase in the total number of photos required. Some photo series were not feasible for low r values, as the camera was too close to view the entire socket at low heights. Larger r and h values were omitted as they provided no increased accuracy benefits during initial testing.
The tripod was used to fix the camera at the designated height, and the radial distance was checked every second photo with the use of a measuring ruler marked with each key distance. The angle was visually identified using the camera screen and the pre-set marks on the protractor. The expected error in positioning for each photo is ±5° angle, ±5 cm in radial distance, and ±2.5 cm in height. In total, 360 photos were taken across an approximately one-hour timespan. As photography was conducted outside, the ambient light levels were recorded at the start of each series, with indirect sunlight being preferred in order to avoid shadows and lens flaring. The ideal light level was 21–25 Klux, corresponding to overcast and evenly diffused lighting, and was measured between each photo series. If the measured levels were outside this range, photography was halted until light levels returned. At light levels outside this range the internal surface of the socket was not as visible or contained shadows. These inconsistencies would affect scan quality and be of detriment to the optimisation process.

2.3. Genetic Algorithm Definition

The design of the experiments used for this optimisation was simplified genetic algorithms constructed with a total of 30 gene bits with two bits per photo series. The first bit designated by “A” refers to the angle of photos for that series (15, 30, 45, 60), and the second bit “G” determines if the photo series is included or not (1, 0). In the example shown in Table 1, the gene series generated created an experiment consisting of 76 photos across the 1st, 2nd, 4th, 8th, 9th, 14th, and 15th photo series at angles of 45°, 15°, 30°, 60°, 60°, 30°, and 45°, respectively, with all other photos being excluded.
The chance of a specific angle being chosen for a gene was equal at 25% across the four discrete angle options, selected randomly for each bit pair. There was a 40% chance that any pair would be active (1) as opposed to excluded (0). The increased weighting of inactive genes was used as a fair way to reduce the photo count of a randomised gene set. As Autodesk ReCap Photo had a limit of 100 photos, any gene sets above this were regenerated. Before any gene set was included it was checked against the existing gene sets that had been tested to ensure there were no duplicates. During initial testing Autodesk ReCap and Meshrooms were trialled, ReCap was chosen as the output scans were more consistent and allowed for simultaneous solving, reducing the overall solver time for the optimisation generations. For this project, access to Autodesk ReCap was free under an educational licence.
For this method of optimisation, both gene mutation and breeding were used. Gene breeding consisted of crossing the last five bit pairs of the two most fit experiments from a given generation. The eleventh bit pair was chosen as the crossover point, as it denoted the photos of the control model that focused on the internal socket surface. An example of two bred genes is shown in Table 2.
For the mutation of genes, there was a 10% chance that any given bit could change value by one increment, i.e., inactive pairs could become active and active pairs could become inactive. If the angle bit changed, there was an even chance that the increment would be positive or negative unless this was at the end of the discrete set, i.e., angles of 45° could become 30° or 60°, but an angle of 60° could only become 45° through mutation. Table 3 shows a mutated gene example with the orange cells indicating mutations. Some mutations led to no change in the experiment, such as the angle change on the inactive gene 2 set.
For this study, five generations were used, each consisting of 15 gene sets. The first generation consisted of 15 randomly generated gene sets. Each subsequent generation consisted of the two most fit gene sets from the previous generation; the two gene sets created from breeding; six mutated gene sets (three from each carried forward); and five additional randomly generated gene sets. After the five generations, the most fit gene series was identified. Each gene pair was reduced by two further increments and retested. This allowed the minimum number of photographs required to be determined without compromising accuracy. To rank each gene series from a generation, Equation (1) was used to compare each gene set to one another:
Rank = AVG(Rank(Volume), Rank(Surface Area), Rank(Number of Photos), Rank (Model Height))
The metric scores for volume, surface area, and height were calculated by determining the percentage difference between the scan data and the digital control model. If two gene sets received the same calculated rank, the one with the closest volume to the control model was selected as the most fit gene within the series.

2.4. Model Preparation for Analysis

Following a successful scan, the socket models were isolated by removing the surrounding mesh with the slicing tool, thus leaving only the socket and flat plate and removing any residual data. The model was then scaled using a point-to-point distance, based on the longest clear mm markings of the metal ruler (commonly 300 mm). Using two points on the ruler furthest apart reduced the effect of errors in point selection. Some scan results had poorly meshed photos, specifically of the ruler where the beginning or end of the ruler was obscured. A final cut is made parallel to the plate to the base of the socket, to remove the model data for the pyramid adaptor. The built-in ReCap Photo mesh report was used to determine the model volume, surface area and height for comparison with the equivalent properties of the control socket file.
A selected model mesh could then be exported from ReCap as an object file for further analysis in Autodesk Fusion 360, using a 10k Quad based model; this was found to be the most reliable file type for export and import between the software used. Once imported the scan model was aligned in the digital space with the control model, using socket reference points (base of socket, patella tendon bar, trim line, fibial head relief area). A series of cuts were made at 10 mm increments concurrently on both models; the cuts were made parallel to the base of the control model, and at 60° sections about the center axis of the socket, as illustrated in Figure 3. The volume, top surface area, and external edge length of each vertical slice was extracted for both the control and scanned model for direct comparison. The internal edge length of each slice at the 60° angle was also extracted to indicate wall thickness. This data extraction provides information on both the external topography and wall thickness of the socket files.

3. Results

The final selected experiment photo set referred to as “22FBG3-14” was identified during the third generation and achieved the highest fitness score throughout the remaining generations. Gene set 22FBG3-14 was comprised of 82 photos and had an accuracy in terms of socket wall volume, surface area and height of 61.63%, 99.61% and 99.90%, respectively. The volume bound by the internal socket wall was 6.76% smaller than that of the control model. Therefore, the accuracy of reproduction rectified moulds from the scan would be 93.24%. The model properties of the two most fit gene sets in each series are shown in Figure 4, showing a slight reduction in photo count through the generations. The measured volume and surface area for the control model are 196.4 cm3 and 1790.9 cm2 respectively, based on the digital mesh.
In total, 75 unique simulations were run, resulting in 32 successful socket meshes. The remaining models were rejected due to incomplete models being formed. Incomplete models showed errors, including filled internal volume, large or numerous holes in the socket wall or surface fragmentation and anomalies. Across all successful experiments the average accuracies measured against the control model were 36.13%, 99.17%, and 99.50% for volume, surface area, and height, respectively, with an average photo count of 81.
The camera position cylindrical coordinates for the final gene are shown in Table 4:

Analysis

In order to understand the large difference in volume but minimal difference in surface area, as previously discussed, the digital scan was exported as a 10k quad based object file into Fusion 360. No additional surface smoothing was used. The digital scan file had some slight pitting and mesh defects; these were rectified using star point surface fills. Both the digital control model and scan model were split into 29 horizontal slices, each slice being 10 mm thick. Each section was then further divided into 60° sections around the central vertical axis, as per the methodology. An additional model was created by replacing the internal socket wall from the scan mesh with a uniform offset of 2.2 mm from the external wall. From these models the following data could be extrapolated: slice volume, slice top surface area, and slice top surface loop length (face outer contour length), as shown in Figure 5.
Each of the graphs follows the expected pattern of a quick increase in volume, surface area, and loop length in the first 50 mm at the base of the socket, followed by a steady increase through the main portion of the socket as the circumference remains similar. There is a quicker increase followed by sudden drop in all metrics at the ~200 mm point, which is representative of the posterior flair (above the popliteal depression), followed by the side guards (lateral and medial supracondylar sections). Following this, there is a decrease in metrics as the sideguard reduces in size, followed by a sharp decline at the tip of lateral sideguard. A visual depiction using the digital model cross sections can be seen in Figure 6.
From the six sections of each slice the horizontal socket wall thickness was extracted. The average thickness for each slice is shown in Figure 7. Similar to the trend seen in regard to the slice volume, the average wall thickness at the base of the socket is significantly higher in the scanned mesh. The scan wall thickness also increases nearer the brim of the socket, reducing the accuracy in this region.
The addition of tape to the 3D printed socket caused a slight increase in wall thickness to 2.657 mm +0.42–0.53, averaged across 10 points at the brim of the socket compared to 2.235mm from the control digital model. The increase depended on the number of layers of tape, as there were some areas of overlap. Whilst this addition may account for some of the scan increases in wall thickness, the amount is minimal compared to the range in the measured wall thicknesses (2.075 mm to 7.758 mm).

4. Discussion

The aim of this study was to identify an optimised photogrammetry technique for accuracy, whilst reducing the number of required photos. Overall, the optimisation process led to a clear most-fit gene set (22FBG3-14). Further reductions in photo count from this gene set led to significantly larger inaccuracies in volume, suggesting that the gene was a local optimum. With the large number of potential gene sets, it is difficult to say if the optimal solution discovered is the global or a local optimum. Determination of the actual global optimal gene set would require significantly more computational time.
The fitness equation used did achieve the target of reducing the number of photos; this may have put more weighting on the surface area of the models than needed, as the lowest recorded accuracy was 97.30% with an average of 99.16%, leading to a very small range. Instead, the use of an exclusion criteria for experiments with and accuracy below 98.50% may have allowed for the selection of a lower number of photographs, or more accuracy in terms of the volume of the gene sets. By contrast the highest accuracy in volume for the socket was 67.69% and a photo count of 100, compared to 22FBG3-14, which had a percentage volume difference of 61.63% and a photo count of 82.
The number of rejected experiments—32 out of 75—and large percentage volume differences from the photogrammetry scans suggest that the method is not reliable for transtibial socket applications; this is in contradiction to previously published literature [23]. A socket resembles a tall cylinder, which restricts the number of viewing angles where the base is still visible on the internal wall. The small changes in the viewing angle led to minimal point to point changes, making depth perception for the software difficult. In addition, light levels at the base of the socket were visibly lower than the socket exterior, whilst additional light may benefit photogrammetry, an increase in reflections was found to reduce scan quality during initial experimentation. This could be improved with brighter diffused lighting; however this was unavailable at the time due to COVID-19 restrictions. Reliability may also be improved by scanning a mould of the internal surface of the socket, either through an alginate copy of the socket or using the Plaster of Paris (PoP) mould prior to socket lamination. Whilst not in the scope of this paper, further study should be conducted into the accuracies and repeatability of scanning positive socket moulds.
A common observation across all of the gene sets was the significantly larger scan volumes compared to the control model. Previously, this issue was determined to be a scaling issue [23]. When combined with the surface area and height metrics, scaling would lead to additional distortions. Instead, the comparison of the scan wall thickness showed that the wall accuracy deteriorated closer to the base of the socket. It is reasonable to assume that the internal socket wall degradation is the main source of the volume discrepancy, as the external loop length of each slice is near identical to the control model (2.64% average difference). The ruler-based scaling method could be considered more accurate than the measured distance between marker dots used in the literature, as it negated the need to measure the distance between reference points. It should be noted, however, that having multiple reference objects for scanning provides redundancy if the ruler is obscured, as was noted on some scanning experiments.
The results achieved in this paper are in line with the existing literature when considering the volume bound by the socket walls, which is the equivalent of the rectified residuum cast. The final model had an internal volume that was 6.76% smaller than the control model, which would be clinically noticeable to the wearer of a recreated socket. The non-uniform reduction in volume is critical for socket reproduction, as the volume reduction is heavily distributed towards the base of the socket. By contrast, when using an extrapolated internal socket wall from the external surface, the internal volume difference lowered to 3.2% smaller. Critically, however, this decrease in volume was uniformly distributed through the socket; thus a better fitting socket may be created if CAM manufacturing is used. This method may not be as effective if there is additional material lamination added during the socket construction, as the distal end of the scan would become enlarged. However this could be remedied by adding material into the base of any reproduced sockets.
The digital twin technique for scan accuracy evaluation used in this paper assumed that the physical 3D model and digital model were identical. Papers from different disciplines have used a meteorological grade 3D scanner to determine a reference topography that scans from other devices that can be compared with [36,37,38,39]. Both methodologies have merit, and inherent error in either production or digitisation. The meteorological scanners were prohibitively expensive and do not fit with the low-cost principals of the study. Further study may be beneficial to determine the true error inherent in the digital twin technique and form a ‘gold standard’ for model evaluation for prosthetic applications. These issues are inherent to photogrammetry, leading to reproduced sockets that would have a significantly restricted distal end to any sockets produced. This could result in uncomfortable reconstructions that will ultimately affect the clinical applicability. However, the technique is non-destructive, unlike traditional PoP copying techniques. The level of distortion observed could be comparable to distortions found in other non-destructive copying methods, such as alginate, and further study should be used to make further comparisons.
Photogrammetry may still be of clinical benefit for sockets with near uniform wall thickness as the external surface scan can provide a reasonable approximation for the entire socket, however this will come with increased computation cost and skill requirements, due to the post scan manipulations required. It is also worth noting that the optimisation discussed in this paper was for an entire socket scan, and further study should be conducted to optimise photo positions for external surface only scans.
Despite the challenges raised in this paper, photogrammetry still offers an inexpensive alternative to conventional optical scanners, with the largest cost being attributed to software—Autodesk ReCap Photo used in this instance. However, freeware alternatives are available. In addition to the base cost for photogrammetry, this study only required patterned tape and a suitable imaging environment.
The low-cost nature of the method proposed in the present study should be weighed against the computation time to generate the mesh scans. Autodesk ReCap utilises cloud computing and, depending on workload, there can be a queue for solving meshes. It was not possible to record the exact solver time but, on average, the process took between half an hour and several hours. When trialling photogrammetry freeware, such as Meshrooms (Alice Vision, France), the solving process took a few hours for each scan, which would vary depending on the computer hardware. The other consideration for Autodesk ReCap is that multiple scans can be solved simultaneously, without compromising solver time, reducing delay for high scan loads. Other authors have noted that there is a large discrepancy in the quality of scans produced from different photogrammetry software packages, which should be investigated prior to use [40].
Further consideration should be made to the continual improvement and development in both the photogrammetry and smartphone industry, meaning that it could become more accessible and relevant as its popularity grows [41].

5. Conclusions

This paper presents a low-cost smartphone photogrammetry technique for scanning entire prosthetic sockets. This technique was identified and refined using genetic algorithm optimisation. The evaluation was conducted using a low-cost alternative to the conventional meteorological scanning technique, using digital and physical twins, and 3D printing.
Whilst the optimisation process was considered successful, the model analysis indicated that the specific technique identified for direct photogrammetry of transtibial sockets was not a suitably accurate method for clinical use.
For some applications a modified external socket surface scan would be sufficient to recreate a socket profile, but this incurs increased computational cost and skill requirements.
The optimal gene set simulated led to a socket scan with 38.37% volume difference to the control model across 82 photos. This scan had an average wall thickness of 3.30mm ranging from 2.075 mm to 7.758 mm. In contrast, the socket constructed from the external scan wall had a volume percentage difference of 3.37%, still using 82 photos but with additional computation time required for mesh manipulation.
Further study should be conducted to determine the optimal camera position for external scanning, as well as potentially simpler patterning techniques. A scan of the cast used to form a socket could also be useful for clinical applications and is worth further exploration.
The evaluation of this technique was based on the existing literature for accuracy requirements; clinical trials of the scanned socket would create a better evaluation of socket fit.

Author Contributions

Conceptualisation, S.C.; Data curation, S.C.; Formal analysis, S.C.; Investigation, S.C.; Methodology, S.C.; Project administration, S.C.; Resources, S.C.; Software, S.C.; Supervision, R.M., A.M. and X.D.; Visualization, S.C.; Writing—original draft, S.C.; Writing—review & editing, S.C., R.M., A.M. and X.D. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by EPSRC, through the studentship funding for S.C.

Institutional Review Board Statement

Ethical review and approval were waived for this study by the Brunel Ethics team. The socket used is a decommissioned socket belonging to one of the authors.

Conflicts of Interest

The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.

References

  1. Berke, G.M.; Fergason, J.; Milani, J.R.; Hattingh, J.; Mcdowell, M.; Nguyen, V.; Reiber, G.E. Comparison of satisfaction with current prosthetic care in veterans and servicemembers from vietnam and OIF/OEF conflicts with major traumatic limb loss. J. Rehabil. Res. Dev. 2010, 47, 361–372. [Google Scholar] [CrossRef]
  2. Sanders, J.E.; Youngblood, R.T.; Hafner, B.J.; Cagle, J.C.; McLean, J.B.; Redd, C.B.; Dietrich, C.R.; Ciol, M.A.; Allyn, K.J. Effects of socket size on metrics of socket fit in trans-tibial prosthesis users. Med. Eng. Phys. 2017, 44, 32–43. [Google Scholar] [CrossRef]
  3. Legro, M.W.; Reiber, G.; Del Aguila, M.D.; Ajax, M.J.; Boone, D.A.; Larsen, J.A.; Smith, D.G.; Sangeorzan, B. Issues of importance reported by persons with lower limb amputations and prostheses. J. Rehabil. Res. Dev. 1999, 36, 155–163. [Google Scholar] [PubMed]
  4. Colombo, G.; Filippi, S.; Rizzi, C.; Rotini, F. A new design paradigm for the development of custom-fit soft sockets for lower limb prostheses. Comput. Ind. 2010, 61, 513–523. [Google Scholar] [CrossRef]
  5. Mehmood, W.; Abd Razak, N.A.; Lau, M.S.; Chung, T.Y.; Gholizadeh, H.; Abu Osman, N.A. Comparative study of the circumferential and volumetric analysis between conventional casting and three-dimensional scanning methods for transtibial socket: A preliminary study. Proc. Inst. Mech. Eng. Part H J. Eng. Med. 2019, 233, 181–192. [Google Scholar] [CrossRef] [PubMed]
  6. Afiqah Hamzah, N.; Razak, N.A.A.; Sayuti Ab Karim, M.; Gholizadeh, H. A review of history of CAD/CAM system application in the production of transtibial prosthetic socket in developing countries (from 1980to 2019). Proc. Inst. Mech. Eng. Part H J. Eng. Med. 2021, 1–16. [Google Scholar] [CrossRef]
  7. Suyi Yang, E.; Aslani, N.; McGarry, A. Influences and trends of various shape-capture methods on outcomes in trans-tibial prosthetics: A systematic review. Prosthet. Orthot. Int. 2019, 43, 540–555. [Google Scholar] [CrossRef] [Green Version]
  8. Nayak, C.; Singh, A.; Chaudhary, H. Customised prosthetic socket fabrication using 3D scanning and printing. In Proceedings of the Conference Additive Manufacturing Society of India, Bangalore, India, 1–2 September 2014. [Google Scholar]
  9. Dean, D.; Saunders, C.G. A Software Package for Design and Manufacture of Prosthetic Sockets for Transtibial Amputees. IEEE Trans. Biomed. Eng. 1985, BME-32, 257–262. [Google Scholar] [CrossRef]
  10. Torres-Moreno, R.; Saunders, C.G.; Foort, J.; Morrison, J.B. Computer-aided design and manufacture of an above-knee amputee socket. J. Biomed. Eng. 1991, 13, 3–9. [Google Scholar] [CrossRef]
  11. Sanders, J.E.; Rogers, E.L.; Sorenson, E.A.; Lee, G.S.; Abrahamson, D.C. CAD/CAM transtibial prosthetic sockets from central fabrication facilities: How accurate are they? J. Rehabil. Res. Dev. 2007, 44, 395–405. [Google Scholar] [CrossRef]
  12. Stenvall, E.; Flodberg, G.; Pettersson, H.; Hellberg, K.; Hermansson, L.; Wallin, M.; Yang, L. Additive Manufacturing of Prostheses Using Forest-Based Composites. Bioengineering 2020, 7, 103. [Google Scholar] [CrossRef]
  13. Marable, W.R.; Smith, C.; Sigurjónsson, B.; Atlason, I.F.; Johannesson, G.A. Transfemoral socket fabrication method using direct casting: Outcomes regarding patient satisfaction with device and services. Can. Prosthetics Orthot. J. 2020, 3. [Google Scholar] [CrossRef]
  14. Paternò, L.; Ibrahimi, M.; Rosini, E.; Menfi, G.; Monaco, V.; Gruppioni, E.; Ricotti, L.; Menciassi, A. Residual limb volume fluctuations in transfemoral amputees. Sci. Rep. 2021, 11, 12273. [Google Scholar] [CrossRef]
  15. Kofman, R.; Beekman, A.M.; Emmelot, C.H.; Geertzen, J.H.B.; Dijkstra, P.U. Measurement properties and usability of non-contact scanners for measuring transtibial residual limb volume. Prosthet. Orthot. Int. 2018, 42, 280–287. [Google Scholar] [CrossRef] [PubMed]
  16. Aniwaa Commercially Avaliable 3D Scanning Equipment. Available online: https://www.aniwaa.com (accessed on 19 June 2020).
  17. Oldfrey, B.; Miodownik, M.; Barbareschi, G.; Williams, R.; Holloway, C. Digital Fabrication of Lower Limb Prosthetic Sockets. In Proceedings of the AT2030 Innovation Insights, London, UK, 16 December 2020. [Google Scholar]
  18. van der Stelt, M.; Grobusch, M.P.; Koroma, A.R.; Papenburg, M.; Kebbie, I.; Slump, C.H.; Maal, T.J.J.; Brouwers, L. Pioneering low-cost 3D-printed transtibial prosthetics to serve a rural population in Sierra Leone—an observational cohort study. EClinicalMedicine 2021, 35, 100874. [Google Scholar] [CrossRef] [PubMed]
  19. Yilmaz, M.H.; Yakar, M.; Yildiz, F. Digital Photogrammetry in Obtaining of 3D Model Data of Irregular Small Objects. Int. Arch. Phototgrammetry Remote Sens. Spat. Inf. Sci. 2008, 37, 125–130. [Google Scholar]
  20. Barbero-García, I.; Lerma, J.L.; Marqués-Mateu, Á.; Miranda, P. Low-Cost Smartphone-Based Photogrammetry for the Analysis of Cranial Deformation in Infants. World Neurosurg. 2017, 102, 545–554. [Google Scholar] [CrossRef]
  21. Ismail, R.; Taqriban, R.B.; Ariyanto, M.; Atmaja, A.T.; Sugiyanto; Caesarendra, W.; Glowacz, A.; Irfan, M.; Glowacz, W. Affordable and faster transradial prosthetic socket production using photogrammetry and 3d printing. Electronics 2020, 9, 1456. [Google Scholar] [CrossRef]
  22. Solav, D.; Moerman, K.M.; Jaeger, A.M.; Herr, H.M. A Framework for Measuring the Time-Varying Shape and Full-Field Deformation of Residual Limbs Using 3-D Digital Image Correlation. IEEE Trans. Biomed. Eng. 2019, 66, 2740–2752. [Google Scholar] [CrossRef] [Green Version]
  23. Hernandez, A.; Lemaire, E. A smartphone photogrammetry method for digitizing prosthetic socket interiors. Prosthet. Orthot. Int. 2017, 41, 210–214. [Google Scholar] [CrossRef]
  24. Sanders, J.E.; Fatone, S. Residual limb volume change: Systematic review of measurement and management. J. Rehabil. Res. Dev. 2011, 48, 949–986. [Google Scholar] [CrossRef]
  25. Armitage, L.; Kwah, L.K.; Kark, L. Reliability and validity of the iSense optical scanner for measuring volume of transtibial residual limb models. Prosthet. Orthot. Int. 2019, 43, 213–220. [Google Scholar] [CrossRef] [PubMed]
  26. Johansson, S.; Öberg, T. Accuracy and precision of volumetric determinations using two commercial CAD systems for prosthetics: A technical note. J. Rehabil. Res. Dev. 1998, 35, 27–33. [Google Scholar] [PubMed]
  27. Taqriban, R.B.; Ismail, R.; Ariyanto, M.; Putra, A.F.Y.S. 3D Model of Photogrammetry Technique for Transtibial Prosthetic Socket Design Development. In Proceedings of the 2019 2nd International Seminar on Research of Information Technology and Intelligent Systems, ISRITI 2019, Yogyakarta, Indonesia, 5–6 December 2019. [Google Scholar] [CrossRef]
  28. Redaelli, D.F.; Barsanti, S.G.; Biffi, E.; Storm, F.A.; Colombo, G. Comparison of geometrical accuracy of active devices for 3D orthopaedic reconstructions. Int. J. Adv. Manuf. Technol. 2021, 114, 319–342. [Google Scholar] [CrossRef]
  29. Armitage, L.; Buller, A.; Rajan, G.; Prusty, G.; Simmons, A.; Kark, L. Clinical utility of pressure feedback to socket design and fabrication. Prosthet. Orthot. Int. 2020, 44, 18–26. [Google Scholar] [CrossRef] [PubMed]
  30. Armitage, L.; Kark, L.; Czerniec, S.; Kwah, L.K. Reliability and Validity of Measurement Tools for Residual Limb Volume in People With Limb Amputations: A Systematic Review. Phys. Ther. 2019, 99, 612–626. [Google Scholar] [CrossRef]
  31. Dickinson, A.S.; Donovan-Hall, M.K.; Kheng, S.; Bou, K.; Tech, A.; Steer, J.W.; Metcalf, C.D.; Worsley, P.R. Selecting Appropriate 3D Scanning Technologies for Prosthetic Socket Design and Transtibial Residual Limb Shape Characterization. JPO J. Prosthetics Orthot. 2020, 283. [Google Scholar] [CrossRef]
  32. Kraus, K. Photogrammetry-Geometry from Images and Laser Scans; De Gruyter: Boston, MA, USA, 2007; ISBN 9788578110796. [Google Scholar]
  33. García-Martínez, C.; Rodriguez, F.J.; Lozano, M. Genetic algorithms. In Handbook of Heuristics; Springer: Cham, Switzerland, 2018; ISBN 9783319071244. [Google Scholar] [CrossRef]
  34. Guo, P.; Wang, X.; Han, Y. The enhanced genetic algorithms for the optimization design. In Proceedings of the 2010 3rd International Conference on Biomedical Engineering and Informatics, BMEI 2010, Yantai, China, 16–18 October 2010. [Google Scholar] [CrossRef]
  35. Dumitrescu, D.; Lazzerini, B.; Jain, L.C.; Dumitrescu, A. Evolutionary Computation; Taylor & Francis Group: Abingdon, UK, 2000; ISBN 9780849305887. [Google Scholar]
  36. Grant, C.A.; Johnston, M.; Adam, C.J.; Little, J.P. Accuracy of 3D surface scanners for clinical torso and spinal deformity assessment. Med. Eng. Phys. 2019, 63, 63–71. [Google Scholar] [CrossRef] [PubMed]
  37. Ten Harkel, T.C.; Speksnijder, C.M.; Van Der Heijden, F.; Beurskens, C.H.G.; Ingels, K.J.A.O.; Maal, T.J.J. Depth accuracy of the RealSense F200: Low-cost 4D facial imaging. Sci. Rep. 2017, 7, 16263. [Google Scholar] [CrossRef]
  38. Yang, Y.; Xu, J.; Elkhuizen, W.S.; Song, Y. The development of a low-cost photogrammetry-based 3D hand scanner. HardwareX 2021, 10, e00212. [Google Scholar] [CrossRef]
  39. Guidi, G.; Micoli, L.L.; Gonizzi, S.; Brennan, M.; Frischer, B. Image-based 3D capture of cultural heritage artifacts: An experimental study about 3D data quality. In Proceedings of the 2015 Digital Heritage, Granada, Spain, 28 September–2 October 2015; Volume 2, pp. 321–324. [Google Scholar] [CrossRef] [Green Version]
  40. Waltenberger, L.; Rebay-Salisbury, K.; Mitteroecker, P. Three-dimensional surface scanning methods in osteology: A topographical and geometric morphometric comparison. Am. J. Phys. Anthropol. 2021, 174, 846–858. [Google Scholar] [CrossRef] [PubMed]
  41. Ortiz-Sanz, J.; Gil-Docampo, M.; Rego-Sanmartín, T.; Arza-García, M.; Tucci, G. A PBeL for training non-experts in mobile-based photogrammetry and accurate 3-D recording of small-size/non-complex objects. Meas. J. Int. Meas. Confed. 2021, 178, 109338. [Google Scholar] [CrossRef]
Sensors 21 08405 g001 550
Figure 1. Socket and model preparation stages: (a) socket; (b) taped socket; (c) 3D printed control socket; (d) taped 3D printed control socket.
Figure 1. Socket and model preparation stages: (a) socket; (b) taped socket; (c) 3D printed control socket; (d) taped 3D printed control socket.
Sensors 21 08405 g001
Sensors 21 08405 g002 550
Figure 2. Illustration of experiment set up; (a) taped socket positioned on platform with ruler; (b) cylindrical co-ordinate system used for camera positioning.
Figure 2. Illustration of experiment set up; (a) taped socket positioned on platform with ruler; (b) cylindrical co-ordinate system used for camera positioning.
Sensors 21 08405 g002
Sensors 21 08405 g003 550
Figure 3. A digitised and imported socket in Autodesk Fusion 360: (a) socket scan 22FBG7-1 segmented around central axis, showing alternate segments; (b) vertical cut lines on with both sockets aligned and visible with the white model being the scan and orange being the control model; (c) sample slice with top surface highlighted; (d) sample slice with loop length highlighted; (e) sample slice with perpendicular wall thickness of single segment.
Figure 3. A digitised and imported socket in Autodesk Fusion 360: (a) socket scan 22FBG7-1 segmented around central axis, showing alternate segments; (b) vertical cut lines on with both sockets aligned and visible with the white model being the scan and orange being the control model; (c) sample slice with top surface highlighted; (d) sample slice with loop length highlighted; (e) sample slice with perpendicular wall thickness of single segment.
Sensors 21 08405 g003
Sensors 21 08405 g004 550
Figure 4. Comparison of the two most fit gene sets from each generations; volume and surface area to control values, including linear approximation of trend in number of photos.
Figure 4. Comparison of the two most fit gene sets from each generations; volume and surface area to control values, including linear approximation of trend in number of photos.
Sensors 21 08405 g004
Sensors 21 08405 g005a 550Sensors 21 08405 g005b 550
Figure 5. Analysis of scan model and reconstructed model vertical slices, compared to digital control model: (a) comparison of slice loop length based on external top edge; (b) comparison based on surface area of the top cut surface for each slice; (c) comparison of entire slice volume for each vertical height from socket base.
Figure 5. Analysis of scan model and reconstructed model vertical slices, compared to digital control model: (a) comparison of slice loop length based on external top edge; (b) comparison based on surface area of the top cut surface for each slice; (c) comparison of entire slice volume for each vertical height from socket base.
Sensors 21 08405 g005aSensors 21 08405 g005b
Sensors 21 08405 g006 550
Figure 6. Cross sectional analysis of the socket, including the visual reference points: (a) cross sectional view of the digital socket scan; (b) the digital reference model that was 3D printed.
Figure 6. Cross sectional analysis of the socket, including the visual reference points: (a) cross sectional view of the digital socket scan; (b) the digital reference model that was 3D printed.
Sensors 21 08405 g006
Sensors 21 08405 g007 550
Figure 7. The horizontal wall thickness of the scanned model averaged across six radial cuts for each vertical slice compared to the control model showing a large disparity between the scan and control at the base of the socket.
Figure 7. The horizontal wall thickness of the scanned model averaged across six radial cuts for each vertical slice compared to the control model showing a large disparity between the scan and control at the base of the socket.
Sensors 21 08405 g007
Table
Table 1. Gene code sample.
Table 1. Gene code sample.
Gene Generator
A1G1A2G2A3G3A4G4A5G5A6G6A7G7A8G8A9G9A10G10A11G11A12G12A13G13A14G14A15G15
451151300301300150300601601150150600450301451
Table
Table 2. Gene breeding example.
Table 2. Gene breeding example.
NameGene Breeder
A1G1A2G2A3G3A4G4A5G5A6G6A7G7A8G8A9G9A10G10A11G11A12G12A13G13A14G14A15G15
22FBG12450601300301600451600601300151451451601451150
22FBG5601150150150601301301450451151300150451601450
Child Gene 1450601300301600451600601300151300150451601450
Child Gene 2601150150150601301301450451151451451601451150
Table
Table 3. Gene mutation example.
Table 3. Gene mutation example.
MutationMutator
10%A1G1A2G2A3G3A4G4A5G5A6G6A7G7A8G8A9G9A10G10A11G11A12G12A13G13A14G14A15G15
22FBG3-13450600600300300300601601600301151600451150601
M Gene450450600301300300600601450451151600601150601
Table
Table 4. Final camera positions for most fit gene series (22FBG3-14).
Table 4. Final camera positions for most fit gene series (22FBG3-14).
Distance d (cm)Height h (cm)Angle Increments Ө (°)
1502060
100−1030
1005045
502060
302015
305030
105060
108045
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Cullen, S.; Mackay, R.; Mohagheghi, A.; Du, X. The Use of Smartphone Photogrammetry to Digitise Transtibial Sockets: Optimisation of Method and Quantitative Evaluation of Suitability. Sensors 2021, 21, 8405. https://0-doi-org.brum.beds.ac.uk/10.3390/s21248405

AMA Style

Cullen S, Mackay R, Mohagheghi A, Du X. The Use of Smartphone Photogrammetry to Digitise Transtibial Sockets: Optimisation of Method and Quantitative Evaluation of Suitability. Sensors. 2021; 21(24):8405. https://0-doi-org.brum.beds.ac.uk/10.3390/s21248405

Chicago/Turabian Style

Cullen, Sean, Ruth Mackay, Amir Mohagheghi, and Xinli Du. 2021. "The Use of Smartphone Photogrammetry to Digitise Transtibial Sockets: Optimisation of Method and Quantitative Evaluation of Suitability" Sensors 21, no. 24: 8405. https://0-doi-org.brum.beds.ac.uk/10.3390/s21248405

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop