J. Imaging, Volume 9, Issue 11 (November 2023) – 21 articles

This paper presents a parallel implementation of a non-local transform-domain filter (BM4D). The effectiveness of the parallel implementation is demonstrated by denoising image series from computed tomography (CT) and magnetic resonance imaging (MRI). The basic idea of the filter is based on grouping and filtering similar data within the image. Due to the high level of similarity and data redundancy, the filter can provide even better denoising quality than current extensively used approaches based on deep learning (DL). In BM4D, cubes of voxels named patches are the essential image elements for filtering. Using voxels instead of pixels means that the area for searching similar patches is large. Because of this and the application of multi-dimensional transformations, the computation time of the filter is exceptionally long. The original implementation of BM4D is only single-threaded. We provide a parallel version of the filter that supports multi-core and many-core processors and scales on such versatile hardware resources, typical for high-performance computing clusters, even if they are concurrently used for the task. Our algorithm uses hybrid parallelisation that combines open multi-processing (OpenMP) and message passing interface (MPI) technologies and provides up to 283× speedup, which is a 99.65% reduction in processing time compared to the sequential version of the algorithm. In denoising quality, the method performs considerably better than recent DL methods on the data type that these methods have yet to be trained on. Full article

Retinal imaging is a non-invasive technique used to scan the back of the eye, enabling the extraction of potential biomarkers like the artery and vein ratio (AVR). This ratio is known for its association with various diseases, such as hypertensive retinopathy (HR) or diabetic retinopathy, and is crucial in assessing retinal health. HR refers to the morphological changes in retinal vessels caused by persistent high blood pressure. Timely identification of these alterations is crucial for preventing blindness and reducing the risk of stroke-related fatalities. The main objective of this paper is to propose a new method for assessing one of the morphological changes in the fundus through morphometric analysis of retinal images. The proposed method in this paper introduces a novel approach called the arteriovenous length ratio (AVLR), which has not been utilized in previous studies. Unlike commonly used measures such as the arteriovenous width ratio or tortuosity, AVLR focuses on assessing the relative length of arteries and veins in the retinal vasculature. The initial step involves segmenting the retinal blood vessels and distinguishing between arteries and veins; AVLR is calculated based on artery and vein caliber measurements for both eyes. Nine equations are used, and the length of both arteries and veins is measured in the region of interest (ROI) covering the optic disc for each eye. Using the AV-Classification dataset, the efficiency of the iterative AVLR assessment is evalutaed. The results show that the proposed approach performs better than the existing methods. By introducing AVLR as a diagnostic feature, this paper contributes to advancing retinal imaging analysis. It provides a valuable tool for the timely diagnosis of HR and other eye-related conditions and represents a novel diagnostic-feature-based method that can be integrated to serve as a clinical decision support system. Full article

Accurate diagnosis and timely intervention are key to addressing common knee conditions effectively. In this work, we aim to identify textural changes in knee lesions based on bone marrow edema (BME), injury (INJ), and osteoarthritis (OST). One hundred and twenty-one MRI knee examinations were selected. Cases were divided into three groups based on radiological findings: forty-one in the BME, thirty-seven in the INJ, and forty-three in the OST groups. From each ROI, eighty-one radiomic descriptors were calculated, encoding texture information. The results suggested differences in the texture characteristics of regions of interest (ROIs) extracted from PD-FSE and STIR sequences. We observed that the ROIs associated with BME exhibited greater local contrast and a wider range of structural diversity compared to the ROIs corresponding to OST. When it comes to STIR sequences, the ROIs related to BME showed higher uniformity in terms of both signal intensity and the variability of local structures compared to the INJ ROIs. A combined radiomic descriptor managed to achieve a high separation ability, with AUC of 0.93 ± 0.02 in the test set. Radiomics analysis may provide a non-invasive and quantitative means to assess the spatial distribution and heterogeneity of bone marrow edema, aiding in its early detection and characterization. Full article

To digital grade the staining color fastness of fabrics after rubbing, an automatic grading method based on spectral reconstruction technology and BP neural network was proposed. Firstly, the modeling samples are prepared by rubbing the fabrics according to the ISO standard of 105-X12. Then, to comply with visual rating standards for color fastness, the modeling samples are professionally graded to obtain the visual rating result. After that, a digital camera is used to capture digital images of the modeling samples inside a closed and uniform lighting box, and the color data values of the modeling samples are obtained through spectral reconstruction technology. Finally, the color fastness prediction model for rubbing was constructed using the modeling samples data and BP neural network. The color fastness level of the testing samples was predicted using the prediction model, and the prediction results were compared with the existing color difference conversion method and gray scale difference method based on the five-fold cross-validation strategy. Experiments show that the prediction model of fabric color fastness can be better constructed using the BP neural network. The overall performance of the method is better than the color difference conversion method and the gray scale difference method. It can be seen that the digital rating method of fabric staining color fastness to rubbing based on spectral reconstruction and BP neural network has high consistency with the visual evaluation, which will help for the automatic color fastness grading. Full article

Aortic valve stenosis (AS) is increasing in prevalence due to the aging population, and severe AS is associated with significant morbidity and mortality. Echocardiography remains the mainstay for the initial detection and diagnosis of AS, as well as for grading of severity. However, there are important subgroups of patients, for example, patients with low-flow low-gradient or paradoxical low-gradient AS, where quantification of severity of AS is challenging by echocardiography and underestimation of severity may delay appropriate management and impart a worse prognosis. Aortic valve calcium score by computed tomography has emerged as a useful clinical diagnostic test that is complimentary to echocardiography, particularly in cases where there may be conflicting data or clinical uncertainty about the degree of AS. In these situations, aortic valve calcium scoring may help re-stratify grading of severity and, therefore, further direct clinical management. This review presents the evolution of aortic valve calcium score by computed tomography, its diagnostic and prognostic value, as well as its utility in clinical care. Full article

This work tackles the problem of image restoration, a crucial task in many fields of applied sciences, focusing on removing degradation caused by blur and noise during the acquisition process. Drawing inspiration from the multi-penalty approach based on the Uniform Penalty principle, discussed in previous work, here we develop a new image restoration model and an iterative algorithm for its effective solution. The model incorporates pixel-wise regularization terms and establishes a rule for parameter selection, aiming to restore images through the solution of a sequence of constrained optimization problems. To achieve this, we present a modified version of the Newton Projection method, adapted to multi-penalty scenarios, and prove its convergence. Numerical experiments demonstrate the efficacy of the method in eliminating noise and blur while preserving the image edges. Full article

Connectionist temporal classification (CTC) is a favored decoder in scene text recognition (STR) for its simplicity and efficiency. However, most CTC-based methods utilize one-dimensional (1D) vector sequences, usually derived from a recurrent neural network (RNN) encoder. This results in the absence of explainable 2D spatial relationship between the predicted characters and corresponding image regions, essential for model explainability. On the other hand, 2D attention-based methods enhance recognition accuracy and offer character location information via cross-attention mechanisms, linking predictions to image regions. However, these methods are more computationally intensive, compared with the 1D CTC-based methods. To achieve both low latency and model explainability via character localization using a 1D CTC decoder, we propose a marginalization-based method that processes 2D feature maps and predicts a sequence of 2D joint probability distributions over the height and class dimensions. Based on the proposed method, we newly introduce an association map that aids in character localization and model prediction explanation. This map parallels the role of a cross-attention map, as seen in computationally-intensive attention-based architectures. With the proposed method, we consider a ViT-CTC STR architecture that uses a 1D CTC decoder and a pretrained vision Transformer (ViT) as a 2D feature extractor. Our ViT-CTC models were trained on synthetic data and fine-tuned on real labeled sets. These models outperform the recent state-of-the-art (SOTA) CTC-based methods on benchmarks in terms of recognition accuracy. Compared with the baseline Transformer-decoder-based models, our ViT-CTC models offer a speed boost up to 12 times regardless of the backbone, with a maximum 3.1% reduction in total word recognition accuracy. In addition, both qualitative and quantitative assessments of character locations estimated from the association map align closely with those from the cross-attention map and ground-truth character-level bounding boxes. Full article

Breast cancer’s high mortality rate is often linked to late diagnosis, with mammograms as key but sometimes limited tools in early detection. To enhance diagnostic accuracy and speed, this study introduces a novel computer-aided detection (CAD) ensemble system. This system incorporates advanced deep learning networks—EfficientNet, Xception, MobileNetV2, InceptionV3, and Resnet50—integrated via our innovative consensus-adaptive weighting (CAW) method. This method permits the dynamic adjustment of multiple deep networks, bolstering the system’s detection capabilities. Our approach also addresses a major challenge in pixel-level data annotation of faster R-CNNs, highlighted in a prominent previous study. Evaluations on various datasets, including the cropped DDSM (Digital Database for Screening Mammography), DDSM, and INbreast, demonstrated the system’s superior performance. In particular, our CAD system showed marked improvement on the cropped DDSM dataset, enhancing detection rates by approximately 1.59% and achieving an accuracy of 95.48%. This innovative system represents a significant advancement in early breast cancer detection, offering the potential for more precise and timely diagnosis, ultimately fostering improved patient outcomes. Full article

There has been considerable progress in implicit neural representation to upscale an image to any arbitrary resolution. However, existing methods are based on defining a function to predict the Red, Green and Blue (RGB) value from just four specific loci. Relying on just four loci is insufficient as it leads to losing fine details from the neighboring region(s). We show that by taking into account the semi-local region leads to an improvement in performance. In this paper, we propose applying a new technique called Overlapping Windows on Semi-Local Region (OW-SLR) to an image to obtain any arbitrary resolution by taking the coordinates of the semi-local region around a point in the latent space. This extracted detail is used to predict the RGB value of a point. We illustrate the technique by applying the algorithm to the Optical Coherence Tomography-Angiography (OCT-A) images and show that it can upscale them to random resolution. This technique outperforms the existing state-of-the-art methods when applied to the OCT500 dataset. OW-SLR provides better results for classifying healthy and diseased retinal images such as diabetic retinopathy and normals from the given set of OCT-A images. Full article

In this study, we aimed to enhance the contouring accuracy of cardiac pacemakers by improving their visualization using deep learning models to predict MV CBCT images based on kV CT or CBCT images. Ten pacemakers and four thorax phantoms were included, creating a total of 35 combinations. Each combination was imaged on a Varian Halcyon (kV/MV CBCT images) and Siemens SOMATOM CT scanner (kV CT images). Two generative adversarial network (GAN)-based models, cycleGAN and conditional GAN (cGAN), were trained to generate synthetic MV (sMV) CBCT images from kV CT/CBCT images using twenty-eight datasets (80%). The pacemakers in the sMV CBCT images and original MV CBCT images were manually delineated and reviewed by three users. The Dice similarity coefficient (DSC), 95% Hausdorff distance (HD95), and mean surface distance (MSD) were used to compare contour accuracy. Visual inspection showed the improved visualization of pacemakers on sMV CBCT images compared to original kV CT/CBCT images. Moreover, cGAN demonstrated superior performance in enhancing pacemaker visualization compared to cycleGAN. The mean DSC, HD95, and MSD for contours on sMV CBCT images generated from kV CT/CBCT images were 0.91 ± 0.02/0.92 ± 0.01, 1.38 ± 0.31 mm/1.18 ± 0.20 mm, and 0.42 ± 0.07 mm/0.36 ± 0.06 mm using the cGAN model. Deep learning-based methods, specifically cycleGAN and cGAN, can effectively enhance the visualization of pacemakers in thorax kV CT/CBCT images, therefore improving the contouring precision of these devices. Full article

Optical coherence tomography (OCT) is an emerging imaging tool in healthcare with common applications in ophthalmology for the detection of retinal diseases and in dentistry for the early detection of tooth decay. Speckle noise is ubiquitous in OCT images, which can hinder diagnosis by clinicians. In this paper, a region-based, deep learning framework for the detection of anomalies is proposed for OCT-acquired images. The core of the framework is Transformer-Enhanced Detection (TED), which includes attention gates (AGs) to ensure focus is placed on the foreground while identifying and removing noise artifacts as anomalies. TED was designed to detect the different types of anomalies commonly present in OCT images for diagnostic purposes and thus aid clinical interpretation. Extensive quantitative evaluations were performed to measure the performance of TED against current, widely known, deep learning detection algorithms. Three different datasets were tested: two dental and one CT (hosting scans of lung nodules, livers, etc.). The results showed that the approach verifiably detected tooth decay and numerous lesions across two modalities, achieving superior performance compared to several well-known algorithms. The proposed method improved the accuracy of detection by 16–22% and the Intersection over Union (IOU) by 10% for both dentistry datasets. For the CT dataset, the performance metrics were similarly improved by 9% and 20%, respectively. Full article

The visualization of neuronal activity in vivo is an urgent task in modern neuroscience. It allows neurobiologists to obtain a large amount of information about neuronal network architecture and connections between neurons. The miniscope technique might help to determine changes that occurred in the network due to external stimuli and various conditions: processes of learning, stress, epileptic seizures and neurodegenerative diseases. Furthermore, using the miniscope method, functional changes in the early stages of such disorders could be detected. The miniscope has become a modern approach for recording hundreds to thousands of neurons simultaneously in a certain brain area of a freely behaving animal. Nevertheless, the analysis and interpretation of the large recorded data is still a nontrivial task. There are a few well-working algorithms for miniscope data preprocessing and calcium trace extraction. However, software for further high-level quantitative analysis of neuronal calcium signals is not publicly available. NeuroActivityToolkit is a toolbox that provides diverse statistical metrics calculation, reflecting the neuronal network properties such as the number of neuronal activations per minute, amount of simultaneously co-active neurons, etc. In addition, the module for analyzing neuronal pairwise correlations is implemented. Moreover, one can visualize and characterize neuronal network states and detect changes in 2D coordinates using PCA analysis. This toolbox, which is deposited in a public software repository, is accompanied by a detailed tutorial and is highly valuable for the statistical interpretation of miniscope data in a wide range of experimental tasks. Full article

Developmental dysplasia of the hip (DDH) is a disorder characterized by abnormal hip development that frequently manifests in infancy and early childhood. Preventing DDH from occurring relies on a timely and accurate diagnosis, which requires careful assessment by medical specialists during early X-ray scans. However, this process can be challenging for medical personnel to achieve without proper training. To address this challenge, we propose a computational framework to detect DDH in pelvic X-ray imaging of infants that utilizes a pipelined deep learning-based technique consisting of two stages: instance segmentation and keypoint detection models to measure acetabular index angle and assess DDH affliction in the presented case. The main aim of this process is to provide an objective and unified approach to DDH diagnosis. The model achieved an average pixel error of 2.862 ± 2.392 and an error range of 2.402 ± 1.963° for the acetabular angle measurement relative to the ground truth annotation. Ultimately, the deep-learning model will be integrated into the fully developed mobile application to make it easily accessible for medical specialists to test and evaluate. This will reduce the burden on medical specialists while providing an accurate and explainable DDH diagnosis for infants, thereby increasing their chances of successful treatment and recovery. Full article

A scalable optical convolutional neural network (SOCNN) based on free-space optics and Koehler illumination was proposed to address the limitations of the previous 4f correlator system. Unlike Abbe illumination, Koehler illumination provides more uniform illumination and reduces crosstalk. The SOCNN allows for scaling of the input array and the use of incoherent light sources. Hence, the problems associated with 4f correlator systems can be avoided. We analyzed the limitations in scaling the kernel size and parallel throughput and found that the SOCNN can offer a multilayer convolutional neural network with massive optical parallelism. Full article

Plant-parasitic nematodes (PPN), especially sedentary endoparasitic nematodes like root-knot nematodes (RKN), pose a significant threat to major crops and vegetables. They are responsible for causing substantial yield losses, leading to economic consequences, and impacting the global food supply. The identification of PPNs and the assessment of their population is a tedious and time-consuming task. This study developed a state-of-the-art deep learning model-based decision support tool to detect and estimate the nematode population. The decision support tool is integrated with the fast inferencing YOLOv5 model and used pretrained nematode weight to detect plant-parasitic nematodes (juveniles) and eggs. The performance of the YOLOv5-640 model at detecting RKN eggs was as follows: precision = 0.992; recall = 0.959; F1-score = 0.975; and mAP = 0.979. YOLOv5-640 was able to detect RKN eggs with an inference time of 3.9 milliseconds, which is faster compared to other detection methods. The deep learning framework was integrated into a user-friendly web application system to build a fast and reliable prototype nematode decision support tool (NemDST). The NemDST facilitates farmers/growers to input image data, assess the nematode population, track the population growths, and recommend immediate actions necessary to control nematode infestation. This tool has the potential for rapid assessment of the nematode population to minimise crop yield losses and enhance financial outcomes. Full article

Advertisements have become commonplace on modern websites. While ads are typically designed for visual consumption, it is unclear how they affect blind users who interact with the ads using a screen reader. Existing research studies on non-visual web interaction predominantly focus on general web browsing; the specific impact of extraneous ad content on blind users’ experience remains largely unexplored. To fill this gap, we conducted an interview study with 18 blind participants; we found that blind users are often deceived by ads that contextually blend in with the surrounding web page content. While ad blockers can address this problem via a blanket filtering operation, many websites are increasingly denying access if an ad blocker is active. Moreover, ad blockers often do not filter out internal ads injected by the websites themselves. Therefore, we devised an algorithm to automatically identify contextually deceptive ads on a web page. Specifically, we built a detection model that leverages a multi-modal combination of handcrafted and automatically extracted features to determine if a particular ad is contextually deceptive. Evaluations of the model on a representative test dataset and ‘in-the-wild’ random websites yielded F1 scores of $0.86$ and $0.88$, respectively. Full article

Supervised deep learning models can be optimised by applying regularisation techniques to reduce overfitting, which can prove difficult when fine tuning the associated hyperparameters. Not all hyperparameters are equal, and understanding the effect each hyperparameter and regularisation technique has on the performance of a given model is of paramount importance in research. We present the first comprehensive, large-scale ablation study for an encoder-only transformer to model sign language using the improved Word-level American Sign Language dataset (WLASL-alt) and human pose estimation keypoint data, with a view to put constraints on the potential to optimise the task. We measure the impact a range of model parameter regularisation and data augmentation techniques have on sign classification accuracy. We demonstrate that within the quoted uncertainties, other than ${\ell }_2$ parameter regularisation, none of the regularisation techniques we employ have an appreciable positive impact on performance, which we find to be in contradiction to results reported by other similar, albeit smaller scale, studies. We also demonstrate that the model architecture is bounded by the small dataset size for this task over finding an appropriate set of model parameter regularisation and common or basic dataset augmentation techniques. Furthermore, using the base model configuration, we report a new maximum top-1 classification accuracy of 84% on 100 signs, thereby improving on the previous benchmark result for this model architecture and dataset. Full article

Speckle noise has long been an extensively studied problem in medical imaging. In recent years, there have been significant advances in leveraging deep learning methods for noise reduction. Nevertheless, adaptation of supervised learning models to unseen domains remains a challenging problem. Specifically, deep neural networks (DNNs) trained for computational imaging tasks are vulnerable to changes in the acquisition system’s physical parameters, such as: sampling space, resolution, and contrast. Even within the same acquisition system, performance degrades across datasets of different biological tissues. In this work, we propose a few-shot supervised learning framework for optical coherence tomography (OCT) noise reduction, that offers high-speed training (of the order of seconds) and requires only a single image, or part of an image, and a corresponding speckle-suppressed ground truth, for training. Furthermore, we formulate the domain shift problem for OCT diverse imaging systems and prove that the output resolution of a despeckling trained model is determined by the source domain resolution. We also provide possible remedies. We propose different practical implementations of our approach, verify and compare their applicability, robustness, and computational efficiency. Our results demonstrate the potential to improve sample complexity, generalization, and time efficiency, for coherent and non-coherent noise reduction via supervised learning models, that can also be leveraged for other real-time computer vision applications. Full article

Chest radiography (CXR) is the most frequently performed radiological test worldwide because of its wide availability, non-invasive nature, and low cost. The ability of CXR to diagnose cardiovascular diseases, give insight into cardiac function, and predict cardiovascular events is often underutilized, not clearly understood, and affected by inter- and intra-observer variability. Therefore, more sophisticated tests are generally needed to assess cardiovascular diseases. Considering the sustained increase in the incidence of cardiovascular diseases, it is critical to find accessible, fast, and reproducible tests to help diagnose these frequent conditions. The expanded focus on the application of artificial intelligence (AI) with respect to diagnostic cardiovascular imaging has also been applied to CXR, with several publications suggesting that AI models can be trained to detect cardiovascular conditions by identifying features in the CXR. Multiple models have been developed to predict mortality, cardiovascular morphology and function, coronary artery disease, valvular heart diseases, aortic diseases, arrhythmias, pulmonary hypertension, and heart failure. The available evidence demonstrates that the use of AI-based tools applied to CXR for the diagnosis of cardiovascular conditions and prognostication has the potential to transform clinical care. AI-analyzed CXRs could be utilized in the future as a complimentary, easy-to-apply technology to improve diagnosis and risk stratification for cardiovascular diseases. Such advances will likely help better target more advanced investigations, which may reduce the burden of testing in some cases, as well as better identify higher-risk patients who would benefit from earlier, dedicated, and comprehensive cardiovascular evaluation. Full article

Communication between Deaf and hearing individuals remains a persistent challenge requiring attention to foster inclusivity. Despite notable efforts in the development of digital solutions for sign language recognition (SLR), several issues persist, such as cross-platform interoperability and strategies for tokenizing signs to enable continuous conversations and coherent sentence construction. To address such issues, this paper proposes a non-invasive Portuguese Sign Language (Língua Gestual Portuguesa or LGP) interpretation system-as-a-service, leveraging skeletal posture sequence inference powered by long-short term memory (LSTM) architectures. To address the scarcity of examples during machine learning (ML) model training, dataset augmentation strategies are explored. Additionally, a buffer-based interaction technique is introduced to facilitate LGP terms tokenization. This technique provides real-time feedback to users, allowing them to gauge the time remaining to complete a sign, which aids in the construction of grammatically coherent sentences based on inferred terms/words. To support human-like conditioning rules for interpretation, a large language model (LLM) service is integrated. Experiments reveal that LSTM-based neural networks, trained with 50 LGP terms and subjected to data augmentation, achieved accuracy levels ranging from 80% to 95.6%. Users unanimously reported a high level of intuition when using the buffer-based interaction strategy for terms/words tokenization. Furthermore, tests with an LLM—specifically ChatGPT—demonstrated promising semantic correlation rates in generated sentences, comparable to expected sentences. Full article

Several optical coherence tomography angiography (OCT-A) studies have demonstrated retinal microvascular changes in patients post-SARS-CoV-2 infection, reflecting retinal-systemic microvasculature homology. Post-COVID-19 syndrome (PCS) entails persistent symptoms following SARS-CoV-2 infection. In this study, we investigated the retinal microvasculature in PCS patients using OCT-angiography and analysed the macular retinal nerve fibre layer (RNFL) and ganglion cell layer (GCL) thickness via spectral domain-OCT (SD-OCT). Conducted at the Manchester Royal Eye Hospital, UK, this cross-sectional study compared 40 PCS participants with 40 healthy controls, who underwent ophthalmic assessments, SD-OCT, and OCT-A imaging. OCT-A images from the superficial capillary plexus (SCP) were analysed using an in-house specialised software, OCT-A vascular image analysis (OCTAVIA), measuring the mean large vessel and capillary intensity, vessel density, ischaemia areas, and foveal avascular zone (FAZ) area and circularity. RNFL and GCL thickness was measured using the OCT machine’s software. Retinal evaluations occurred at an average of 15.2 ± 6.9 months post SARS-CoV-2 infection in PCS participants. Our findings revealed no significant differences between the PCS and control groups in the OCT-A parameters or RNFL and GCL thicknesses, indicating that no long-term damage ensued in the vascular bed or retinal layers within our cohort, providing a degree of reassurance for PCS patients. Full article