J. Imaging, Volume 6, Issue 9 (September 2020) – 18 articles

Quantitative phase imaging (QPI) techniques are widely used for the label-free examining of transparent biological samples. QPI techniques can be broadly classified into interference-based and interferenceless methods. The interferometric methods which record the complex amplitude are usually bulky with many optical components and use coherent illumination. The interferenceless approaches which need only the intensity distribution and works using phase retrieval algorithms have gained attention as they require lesser resources, cost, space and can work with incoherent illumination. With rapid developments in computational optical techniques and deep learning, QPI has reached new levels of applications. In this tutorial, we discuss one of the basic optical configurations of a lensless QPI technique based on the phase-retrieval algorithm. Simulative studies on QPI of thin, thick, and greyscale phase objects with assistive pseudo-codes and computational codes in Octave is provided. Binary phase samples with positive and negative resist profiles were fabricated using lithography, and a single plane and two plane phase objects were constructed. Light diffracted from a point object is modulated by phase samples and the corresponding intensity patterns are recorded. The phase retrieval approach is applied for 2D and 3D phase reconstructions. Commented codes in Octave for image acquisition and automation using a web camera in an open source operating system are provided. Full article

Collisionless media devoid of intrinsic stresses, for example, a dispersed phase in a multiphase medium, have a much wider variety of space-time structures and features formed in them than collisional media, for example, a carrier, gas, or liquid phase. This is a consequence of the fact that evolution in such media occurs in phase space, i.e., in a space of greater dimensions than the usual coordinate space. As a consequence, the process of the formation of features in collisionless media (clustering or vice versa, a loss of continuity) can occur primarily in the velocity space, which, in contrast to the features in the coordinate space (folds, caustics, or voids), is poorly observed directly. To identify such features, it is necessary to use visualization methods that allow us to consider, in detail, the evolution of the medium in the velocity space. This article is devoted to the development of techniques that allow visualizing the degree of anisotropy of the velocity fields of collisionless interpenetrating media. Simultaneously tracking the behavior of different fractions in such media is important, as their behavior can be significantly different. We propose three different techniques for visualizing the anisotropy of velocity fields using the example of two- and three-continuum dispersed media models. We proposed the construction of spatial distributions of eccentricity fields (scalar fields), or fields of principal directions of the velocity dispersion tensor (tensor fields). In the first case, we used some simple eccentricity functions for dispersion tensors for two fractions simultaneously, which we call surrogate entropy. In the second case, to visualize the anisotropy of the velocity fields of three fractions simultaneously, we used an ordered array (3-vector) of eccentricities for the color representation through decomposition in three basic colors. In the case of a multi-stream flow, we used cluster analysis methods to identify sections of a multi-stream flow (beams) and used glyphs to visualize the entire set of beams (vector-tensor fields). Full article

Deep learning (DL) convolutional neural networks (CNNs) have been rapidly adapted in very high spatial resolution (VHSR) satellite image analysis. DLCNN-based computer visions (CV) applications primarily aim for everyday object detection from standard red, green, blue (RGB) imagery, while earth science remote sensing applications focus on geo object detection and classification from multispectral (MS) imagery. MS imagery includes RGB and narrow spectral channels from near- and/or middle-infrared regions of reflectance spectra. The central objective of this exploratory study is to understand to what degree MS band statistics govern DLCNN model predictions. We scaffold our analysis on a case study that uses Arctic tundra permafrost landform features called ice-wedge polygons (IWPs) as candidate geo objects. We choose Mask RCNN as the DLCNN architecture to detect IWPs from eight-band Worldview-02 VHSR satellite imagery. A systematic experiment was designed to understand the impact on choosing the optimal three-band combination in model prediction. We tasked five cohorts of three-band combinations coupled with statistical measures to gauge the spectral variability of input MS bands. The candidate scenes produced high model detection accuracies for the F1 score, ranging between 0.89 to 0.95, for two different band combinations (coastal blue, blue, green (1,2,3) and green, yellow, red (3,4,5)). The mapping workflow discerned the IWPs by exhibiting low random and systematic error in the order of 0.17–0.19 and 0.20–0.21, respectively, for band combinations (1,2,3). Results suggest that the prediction accuracy of the Mask-RCNN model is significantly influenced by the input MS bands. Overall, our findings accentuate the importance of considering the image statistics of input MS bands and careful selection of optimal bands for DLCNN predictions when DLCNN architectures are restricted to three spectral channels. Full article

With increased use of light-weight materials with low factors of safety, non-destructive testing becomes increasingly important. Thanks to the advancement of infrared camera technology, pulse thermography is a cost efficient way to detect subsurface defects non-destructively. However, currently available evaluation algorithms have either a high computational cost or show poor performance if any geometry other than the most simple kind is surveyed. We present an extension of the thermographic signal reconstruction technique which can automatically segment and image defects from sound areas, while also estimating the defect depth, all with low computational cost. We verified our algorithm using real world measurements and compare our results to standard active thermography algorithms with similar computational complexity. We found that our algorithm can detect defects more accurately, especially when more complex geometries are examined. Full article

Recently, our world witnessed major events that attracted a lot of attention towards the importance of automatic crowd scene analysis. For example, the COVID-19 breakout and public events require an automatic system to manage, count, secure, and track a crowd that shares the same area. However, analyzing crowd scenes is very challenging due to heavy occlusion, complex behaviors, and posture changes. This paper surveys deep learning-based methods for analyzing crowded scenes. The reviewed methods are categorized as (1) crowd counting and (2) crowd actions recognition. Moreover, crowd scene datasets are surveyed. In additional to the above surveys, this paper proposes an evaluation metric for crowd scene analysis methods. This metric estimates the difference between calculated crowed count and actual count in crowd scene videos. Full article

Current point cloud extraction methods based on photogrammetry generate large amounts of spurious detections that hamper useful 3D mesh reconstructions or, even worse, the possibility of adequate measurements. Moreover, noise removal methods for point clouds are complex, slow and incapable to cope with semantic noise. In this work, we present body2vec, a model-based body segmentation tool that uses a specifically trained Neural Network architecture. Body2vec is capable to perform human body point cloud reconstruction from videos taken on hand-held devices (smartphones or tablets), achieving high quality anthropometric measurements. The main contribution of the proposed workflow is to perform a background removal step, thus avoiding the spurious points generation that is usual in photogrammetric reconstruction. A group of 60 persons were taped with a smartphone, and the corresponding point clouds were obtained automatically with standard photogrammetric methods. We used as a 3D silver standard the clean meshes obtained at the same time with LiDAR sensors post-processed and noise-filtered by expert anthropological biologists. Finally, we used as gold standard anthropometric measurements of the waist and hip of the same people, taken by expert anthropometrists. Applying our method to the raw videos significantly enhanced the quality of the results of the point cloud as compared with the LiDAR-based mesh, and of the anthropometric measurements as compared with the actual hip and waist perimeter measured by the anthropometrists. In both contexts, the resulting quality of body2vec is equivalent to the LiDAR reconstruction. Full article

In this work, a novel algorithmic scheme is developed that processes echocardiogram videos, and tracks the movement of the mitral valve leaflets, and thereby estimates whether the movement is symptomatic of a healthy or diseased heart. This algorithm uses automatic Otsu’s thresholding to find a closed boundary around the left atrium, with the basic presumption that it is situated in the bottom right corner of the apical 4 chamber view. A centroid is calculated, and protruding prongs are taken within a 40-degree cone above the centroid, where the mitral valve is located. Binary images are obtained from the videos where the mitral valve leaflets have different pixel values than the cavity of the left atrium. Thus, the points where the prongs touch the valve will show where the mitral valve leaflets are located. The standard deviation of these points is used to calculate closeness of the leaflets. The estimation of the valve movement across subsequent frames is used to determine if the movement is regular, or affected by heart disease. Tests conducted with numerous videos containing both healthy and diseased hearts attest to our method’s efficacy, with a key novelty in being fully unsupervised and computationally efficient. Full article

The classification of histopathology images requires an experienced physician with years of experience to classify the histopathology images accurately. In this study, an algorithm was developed to assist physicians in classifying histopathology images; the algorithm receives the histopathology image as an input and produces the percentage of cancer presence. The primary classifier used in this algorithm is the convolutional neural network, which is a state-of-the-art classifier used in image classification as it can classify images without relying on the manual selection of features from each image. The main aim of this research is to improve the robustness of the classifier used by comparing six different first-order stochastic gradient-based optimizers to select the best for this particular dataset. The dataset used to train the classifier is the PatchCamelyon public dataset, which consists of 220,025 images to train the classifier; the dataset is composed of 60% positive images and 40% negative images, and 57,458 images to test its performance. The classifier was trained on 80% of the images and validated on the rest of 20% of the images; then, it was tested on the test set. The optimizers were evaluated based on their AUC of the ROC curve. The results show that the adaptative based optimizers achieved the highest results except for AdaGrad that achieved the lowest results. Full article

Convolution neural networks usually require large labeled data-sets to construct accurate models. However, in many real-world scenarios, such as global illumination, labeling data are a time-consuming and costly human intelligent task. Semi-supervised learning methods leverage this issue by making use of a small labeled data-set and a larger set of unlabeled data. In this paper, our contributions focus on the development of a robust algorithm that combines active and deep semi-supervised convolution neural network to reduce labeling workload and to accelerate convergence in case of real-time global illumination. While the theoretical concepts of photo-realistic rendering are well understood, the increased need for the delivery of highly dynamic interactive content in vast virtual environments has increased recently. Particularly, the quality measure of computer-generated images is of great importance. The experiments are conducted on global illumination scenes which contain diverse distortions. Compared with human psycho-visual thresholds, the good consistency between these thresholds and the learning models quality measures can been seen. A comparison has also been made with SVM and other state-of-the-art deep learning models. We do transfer learning by running the convolution base of these models over our image set. Then, we use the output features of the convolution base as input to retrain the parameters of the fully connected layer. The obtained results show that our proposed method provides promising efficiency in terms of precision, time complexity, and optimal architecture. Full article

The measurement of night sky quality has become an important task in night sky conservation. Modern measurement techniques involve mainly a calibrated digital camera or a spectroradiometer. However, panchromatic devices are still prevalent to this day, even in the absence of determining the spectral information of the night sky. In the case of multispectral measurements, colour information is currently presented in multiple ways. One of the most frequently used metrics is correlated colour temperature (CCT), which is not without its limitation for the purpose of describing especially the colour of natural night sky. Moreover, visually displaying the colour of the night sky in a quantitatively meaningful way has not attracted sufficient attention in the community of astronomy and light pollution research—most photographs of the night sky are post-processed in a way for aesthetic attractiveness rather than accurate representation of the night sky. The spectrum of the natural night sky varies in a wide range depending on solar activity and atmospheric properties. The most noticeable variation in the visible range is the variation of the atomic emission lines, primarily the green oxygen and orange sodium emission. Based on the accepted models of night sky emission, we created a random spectral database which represents the possible range of night sky radiance distribution. We used this spectral database as a learning set, to create a colour transformation between different colour spaces. The spectral sensitivity of some digital cameras is also used to determine an optimal transformation matrix from camera defined coordinates to real colours. The theoretical predictions were extended with actual spectral measurements in order to test the models and check the local constituents of night sky radiance. Here, we present an extended modelling of night sky colour and recommendations of its consistent measurement, as well as methods of visualising the colour of night sky in a consistent way, namely using the false colour enhancement. Full article

In the framework of palaeography, the availability of both effective image analysis algorithms, and high-quality digital images has favored the development of new applications for the study of ancient manuscripts and has provided new tools for decision-making support systems. The quality of the results provided by such applications, however, is strongly influenced by the selection of effective features, which should be able to capture the distinctive aspects to which the paleography expert is interested in. This process is very difficult to generalize due to the enormous variability in the type of ancient documents, produced in different historical periods with different languages and styles. The effect is that it is very difficult to define standard techniques that are general enough to be effectively used in any case, and this is the reason why ad-hoc systems, generally designed according to paleographers’ suggestions, have been designed for the analysis of ancient manuscripts. In recent years, there has been a growing scientific interest in the use of techniques based on deep learning (DL) for the automatic processing of ancient documents. This interest is not only due to their capability of designing high-performance pattern recognition systems, but also to their ability of automatically extracting features from raw data, without using any a priori knowledge. Moving from these considerations, the aim of this study is to verify if DL-based approaches may actually represent a general methodology for automatically designing machine learning systems for palaeography applications. To this purpose, we compared the performance of a DL-based approach with that of a “classical” machine learning one, in a particularly unfavorable case for DL, namely that of highly standardized schools. The rationale of this choice is to compare the obtainable results even when context information is present and discriminating: this information is ignored by DL approaches, while it is used by machine learning methods, making the comparison more significant. The experimental results refer to the use of a large sets of digital images extracted from an entire 12th-century Bibles, the “Avila Bible”. This manuscript, produced by several scribes who worked in different periods and in different places, represents a severe test bed to evaluate the efficiency of scribe identification systems. Full article

This paper aims to implement histogram pyramids with marching cubes method for 3D medical volumetric rendering. The histogram pyramids are used for feature extraction by segmenting the image into the hierarchical order like the pyramid shape. The histogram pyramids can decrease the number of sparse matrixes that will occur during voxel manipulation. The important feature of the histogram pyramids is the direction of segments in the image. Then this feature will be used for connecting pixels (2D) to form up voxel (3D) during marching cubes implementation. The proposed method is fast and easy to implement and it also produces a smooth result (compared to the traditional marching cubes technique). The experimental results show the time consuming for generating 3D model can be reduced by 15.59% in average. The paper also shows the comparison between the surface rendering using the traditional marching cubes and the marching cubes with histogram pyramids. Therefore, for the volumetric rendering such as 3D medical models and terrains where a large number of lookups in 3D grids are performed, this method is a particularly good choice for generating the smooth surface of 3D object. Full article

Despite the numerous band selection (BS) algorithms reported in the field, most if not all have exhibited maximal accuracy when more spectral bands are utilized for classification. This apparently disagrees with the theoretical model of the ‘curse of dimensionality’ phenomenon, without apparent explanations. If it were true, then BS would be deemed as an academic piece of research without real benefits to practical applications. This paper presents a spatial spectral mutual information (SSMI) BS scheme that utilizes a spatial feature extraction technique as a preprocessing step, followed by the clustering of the mutual information (MI) of spectral bands for enhancing the efficiency of the BS. Through the SSMI BS scheme, a sharp ’bell’-shaped accuracy-dimensionality characteristic that peaks at about 20 bands has been observed for the very first time. The performance of the proposed SSMI BS scheme has been validated through 6 hyperspectral imaging (HSI) datasets (Indian Pines, Botswana, Barrax, Pavia University, Salinas, and Kennedy Space Center (KSC)), and its classification accuracy is shown to be approximately 10% better than seven state-of-the-art BS schemes (Saliency, HyperBS, SLN, OCF, FDPC, ISSC, and Convolution Neural Network (CNN)). The present result confirms that the high efficiency of the BS scheme is essentially important to observe and validate the Hughes’ phenomenon in the analysis of HSI data. Experiments also show that the classification accuracy can be affected by as much as approximately 10% when a single ‘crucial’ band is included or missed out for classification. Full article

Dynamic and real-time MRI (rtMRI) of human speech is an active field of research, with interest from both the linguistics and clinical communities. At present, different research groups are investigating a range of rtMRI acquisition and reconstruction approaches to visualise the speech organs. Similar to other moving organs, it is difficult to create a physical phantom of the speech organs to optimise these approaches; therefore, the optimisation requires extensive scanner access and imaging of volunteers. As previously demonstrated in cardiac imaging, realistic numerical phantoms can be useful tools for optimising rtMRI approaches and reduce reliance on scanner access and imaging volunteers. However, currently, no such speech rtMRI phantom exists. In this work, a numerical phantom for optimising speech rtMRI approaches was developed and tested on different reconstruction schemes. The novel phantom comprised a dynamic image series and corresponding k-space data of a single mid-sagittal slice with a temporal resolution of 30 frames per second (fps). The phantom was developed based on images of a volunteer acquired at a frame rate of 10 fps. The creation of the numerical phantom involved the following steps: image acquisition, image enhancement, segmentation, mask optimisation, through-time and spatial interpolation and finally the derived k-space phantom. The phantom was used to: (1) test different k-space sampling schemes (Cartesian, radial and spiral); (2) create lower frame rate acquisitions by simulating segmented k-space acquisitions; (3) simulate parallel imaging reconstructions (SENSE and GRAPPA). This demonstrated how such a numerical phantom could be used to optimise images and test multiple sampling strategies without extensive scanner access. Full article

Today, convolutional and deconvolutional neural network models are exceptionally popular thanks to the impressive accuracies they have been proven in several computer-vision applications. To speed up the overall tasks of these neural networks, purpose-designed accelerators are highly desirable. Unfortunately, the high computational complexity and the huge memory demand make the design of efficient hardware architectures, as well as their deployment in resource- and power-constrained embedded systems, still quite challenging. This paper presents a novel purpose-designed hardware accelerator to perform 2D deconvolutions. The proposed structure applies a hardware-oriented computational approach that overcomes the issues of traditional deconvolution methods, and it is suitable for being implemented within any virtually system-on-chip based on field-programmable gate array devices. In fact, the novel accelerator is simply scalable to comply with resources available within both high- and low-end devices by adequately scaling the adopted parallelism. As an example, when exploited to accelerate the Deep Convolutional Generative Adversarial Network model, the novel accelerator, running as a standalone unit implemented within the Xilinx Zynq XC7Z020 System-on-Chip (SoC) device, performs up to 72 GOPs. Moreover, it dissipates less than 500mW@200MHz and occupies 5.6%, 4.1%, 17%, and 96%, respectively, of the look-up tables, flip-flops, random access memory, and digital signal processors available on-chip. When accommodated within the same device, the whole embedded system equipped with the novel accelerator performs up to 54 GOPs and dissipates less than 1.8W@150MHz. Thanks to the increased parallelism exploitable, more than 900 GOPs can be executed when the high-end Virtex-7 XC7VX690T device is used as the implementation platform. Moreover, in comparison with state-of-the-art competitors implemented within the Zynq XC7Z045 device, the system proposed here reaches a computational capability up to 20% higher, and saves more than 60% and 80% of power consumption and logic resources requirement, respectively, using 5.7× fewer on-chip memory resources. Full article

We present a numerical illumination model to calculate direct as well as diffuse or Hapke scattered radiation scenarios on arbitrary planetary surfaces. This includes small body surfaces such as main belt asteroids as well as e.g., the lunar surface. The model is based on the ray tracing method. This method is not restricted to spherical or ellipsoidal shapes but digital terrain data of arbitrary spatial resolution can be fed into the model. Solar radiation is the source of direct radiation, wavelength-dependent effects (e.g. albedo) can be accounted for. Mutual illumination of individual bodies in implemented (e.g. in binary or multiple systems) as well as self-illumination (e.g. crater floors by crater walls) by diffuse or Hapke radiation. The model is validated by statistical methods. A ${\chi }^2$ test is utilized to compare simulated images with DAWN images acquired during the survey phase at small body 4 Vesta and to successfully prove its validity. Full article

The manual delineation of region of interest (RoI) in 3D magnetic resonance imaging (MRI) of the prostate is time-consuming and subjective. Correct identification of prostate tissue is helpful to define a precise RoI to be used in CAD systems in clinical practice during diagnostic imaging, radiotherapy and monitoring the progress of disease. Conditional GAN (cGAN), cycleGAN and U-Net models and their performances were studied for the detection and segmentation of prostate tissue in 3D multi-parametric MRI scans. These models were trained and evaluated on MRI data from 40 patients with biopsy-proven prostate cancer. Due to the limited amount of available training data, three augmentation schemes were proposed to artificially increase the training samples. These models were tested on a clinical dataset annotated for this study and on a public dataset (PROMISE12). The cGAN model outperformed the U-Net and cycleGAN predictions owing to the inclusion of paired image supervision. Based on our quantitative results, cGAN gained a Dice score of 0.78 and 0.75 on the private and the PROMISE12 public datasets, respectively. Full article

Breast cancer is the most frequently diagnosed cancer in woman. The correct identification of the HER2 receptor is a matter of major importance when dealing with breast cancer: an over-expression of HER2 is associated with aggressive clinical behaviour; moreover, HER2 targeted therapy results in a significant improvement in the overall survival rate. In this work, we employ a pipeline based on a cascade of deep neural network classifiers and multi-instance learning to detect the presence of HER2 from Haematoxylin–Eosin slides, which partly mimics the pathologist’s behaviour by first recognizing cancer and then evaluating HER2. Our results show that the proposed system presents a good overall effectiveness. Furthermore, the system design is prone to further improvements that can be easily deployed in order to increase the effectiveness score. Full article