Single-Image Multi-Parametric Representation of Optical Properties through Encodings to the HSV Color Space

Abstract

The visualization of 2D clinical data often relies on color-coded images, but different colormaps can introduce cognitive biases, impacting result interpretation. Moreover, when using color for diagnosis with multiple biomarkers, the application of distinct colormaps for each parameter can hinder comparisons. Our aim was to introduce a visualization technique that utilizes the hue (H), saturation (S), and value (V) in a single image to convey multi-parametric data on various optical properties in an effective manner. To achieve this, we conducted a study involving two datasets, one comprising multi-modality measurements of the human aorta and the other featuring multiple parameters of dystrophic mice muscles. Through this analysis, we determined that H is best suited to emphasize differences related to pathology, while V highlights high-spatial-resolution data disparities, and color alterations effectively indicate changes in chemical component concentrations. Furthermore, encoding structural information as S and V within the same image assists in pinpointing the specific locations of these variations. In cases where all data are of a high resolution, H remains the optimal indicator of pathology, ensuring results’ interpretability. This approach simplifies the selection of an appropriate colormap and enhances the ability to grasp a sample’s characteristics at a single glance.

1. Introduction

Data visualization is at the core of data sharing and clinical assessment, allowing for the easy interpretation of numerical results by end users, such as clinicians. When discussing 2D sets of data, researchers usually represent them as false-color images that are created by mapping some parameter to a color scale, referred to as colormaps. These colormaps and their construction can be defined in terms of certain luminance/lightness, chroma/hue, and saturation/value properties [1], leading to a wide range of options to choose from, which results in a lack of consensus on the best colormap for the representation of different types of data [2].

Color and vision perception are mainly cognitive processes, so the use of different colormaps will lead to different, sometimes more accurate, interpretations of the same results [3]. There are multiple articles that explore the variation in the explainability of the results in terms of different perception biases [4,5,6], such as the dark-is-more bias, where darker colors are believed to represent greater values, or the hotspot-is-more bias, where the localized accumulation of data points is interpreted as greater quantities, even if the color of the hotspot itself is not darker than that of the surrounding ones [4,7]. Further interpretation can be derived from analyzing the influence of colormap properties independently of each other. Recent experiments found that the structures in data (shapes and patterns) are best viewed in luminance variations, since the human capability to resolve chroma variations is worse than that of luminance [8]. These claims lead to the assertion that a rapidly changing (high spatial frequency/high resolution) dataset would benefit from the use of colormaps with perceptually uniform luminance, while a lower-resolution dataset could be better represented from a chroma-variation-first colormap to exploit the dark- or hotspot-is-more biases.

Given the intricate relationship between perception biases and data visualization, numerous studies have explored the impact of different colormaps on the visualization and representation of identical 2D data [9], the creation of task-customized colormaps [10,11,12,13], as well as the best way to build a color-based representation [14,15,16,17]. Conventional colormaps, featuring diverse spectral arrangements, have the potential to elicit varied perceptual responses due to the interplay of color and luminance gradients. Researchers have examined how these colormap properties influence critical factors such as data clarity, interpretability, and the prevention of misinterpretations [18,19,20,21]. The choice of color scale can significantly affect the perceived magnitude and patterns within a dataset, potentially leading to disparities in comprehension. Moreover, the exploration of diverging, sequential, or categorical colormaps adds complexity to the investigation, as each type may excel or fail in different contexts [22].

When dealing with the optical properties of clinical data, more often than not, samples are analyzed with several imaging modalities, which can help to better understand the physiology of different pathologies. However, such imaging devices may yield data with different spatial properties. In such cases, one could provide numerous false-color images for the various parameters, each of which would be better viewed in a different colormap according to their characteristics. Nonetheless, evaluating multiple color scales for the same sample may complicate the interpretability of the dataset or even worsen it [23].

Some imaging modalities, such as polarization imaging, combine different parameters using the hue–saturation–value (HSV) color space, assigning different optical properties to each component in terms of the existing relationship between the physical parameters and the interpretation of the H, S, and V coordinates, but the correction of such parameters to enable better visualization is based on the user input [24]. Others use the addition of color as a semi-transparent overlay on a grayscale image, combined with further manipulation in the H channel to enhance the clinical differences, without variations in S and V [25]. A similar process was used to represent speckle variance over optical coherence tomography images, but the assignment of data to HSV was performed in terms of noise intensity [26].

Overall, having multiple maps for different optical properties is useful but, in some environments where rapid decision making is crucial, having an additional single image that contains all the necessary information might yield faster results. Inspired by the approaches taken in polarization imaging research, we propose a method of encoding the optical properties of biological tissue in the three components of the HSV color space by specifically choosing the parameters based on their visualization properties. This approach allows for the combination of high- and low-resolution data into a single image that conveys more information and possesses better explainability than its individual components, while still being easily interpretable and avoiding the need to select an optimal colormap for each one. The assignment to HSV makes use of the three colorimetric components and is designed to provide a clinical interpretation of the samples through color, with the most clinically relevant optical property being assigned to the H component. The S and V components are chosen based on the data with the highest resolution and spatial features, taking advantage of the human ability to recognize luminance variations as structural differences. The proposed enhancement of HSV is done automatically, taking into account the range of all possible values in a dataset, and thus using only one HSV legend shared by all the samples in it. With this work, the aim is to continue expanding the research on colormaps and color perception in the realm of clinical imaging.

2. Materials and Methods

2.1. Datasets

The proposed method can be applied to any combination of two or three different magnitudes that have complementary information about the data. However, since the focus of our research lies in the optical properties of biological tissues, we recovered two datasets to test the applicability of the proposed visualization methodology in two case studies: one composed of human samples of aortic tissue and another of dystrophic mice muscles.

For the aorta dataset, we used two ex vivo samples of the human ascending thoracic aorta: a healthy one as a control (AR26, healthy transplant donor) and a pathological aneurysmatic one (AR21, with presence of atheroma plaques). Aortas were measured with optical coherence tomography (OCT) to evaluate their structures, and their composition was studied with hyperspectral imaging (HSI). From the OCT measurements, the overall attenuation coefficient (${\mu }_t$) was derived by comparing its response to that of a reference material, as proposed by Scolaro et al. [27]. On the other hand, the model proposed by Jacques et al. [28] was used to obtain an estimation for the elastin, water, and lipids, using the spectra measured by Sekar et al. [29,30] as a reference. Specifically, we used the fraction of elastin $c_e$ as a marker of an aortic aneurysm.

The muscle dataset contained three dystrophic mice muscles of differently aged mice (control, 1-month-old dystrophic and 3-month-old dystrophic mice). The muscles were measured and treated as described in our previous work [31], where we obtained the ${\mu }_t$ with the same technique as for the aortas, but also their birefringence ($\Delta n$) by measuring their polarization responses with polarization-sensitive OCT (PS-OCT). The method proposed to calculate the $\Delta n$ was first described by Chin et al. in [32]. Since both OCT modalities (regular and PS) are captured with the same device, $\Delta n$ and ${\mu }_t$ have the same spatial resolution and are implicitly co-registered.

These optical properties, namely ${\mu }_t$, $\Delta n$ and $c_e$, can be used to characterize the samples from the structural and chemical points of view. Specifically, ${\mu }_t$ is a measurement of how quickly light decays inside the sample; therefore, smaller ${\mu }_t$ values indicate that light can travel to further tissue depths without being completely attenuated. Given that ${\mu }_t$ is dominated by scattering, its value can be an indicator of the cell size, and therefore of the tissue morphology, which might change with different tissue structures. Additionally, $\Delta n$ is affected by tissue anisotropy, with highly anisotropic structures (e.g., collagen/fiber bundles) yielding higher values of $\Delta n$. The number of tissue chromophores (collagen, elastin, fat), and specifically $c_e$, at different points on the sample can complement the information given by ${\mu }_t$ and $\Delta n$, since many structural alterations are caused by the sub- or overexpression of some tissue components, such as elastin. Combining structural-/chemical-based information can be extremely useful for the understanding of multiple pathologies.

The derivation of the attenuation coefficient ${\mu }_t$ and the birefringence $\Delta n$ and the chromophore modeling are beyond the scope of this work and have been thoroughly described in the literature [27,28,29,30,31,32]. The measurement procedure, on the other hand, does depend on the type of sample and, specifically, on its size. Our OCT system has an FOV of 10 × 10 mm², which makes the imaging of the entire aortas impossible in a single measurement. To maintain a spatial reference between measurements and instrumental systems, a 9-by-9 square grid with squares of 10 × 110 mm² was designed and 3D-printed for the posterior mosaicking of the aorta measurements. We refer to each of the squares in the grid as a region of interest (ROI). The PS-OCT measurement of the mice muscles was performed with the same OCT device, but, in this case, only one measurement was needed to obtain an image of the whole muscle due to their reduced footprint. Aorta HSI was performed with a rotating-mirror-based system [33] that captured wavelengths between 1000 nm and 1700 nm, which is the range in which the most relevant chromophores of tissue are detectable. The field of view was much wider than for the OCT, namely 15 × 15 cm²; therefore, the whole aorta was imaged in a single HSI data cube by keeping the squared grid for co-registration with the OCT data.

2.2. Color Transformation

In the HSV color space, the hue (H) indicates the color portrayed in a cyclic fashion, ranging from red to green to blue and back to red. Usually, it is represented as an angle between $0^{\circ }$ and ${360}^{\circ }$, but it can be remapped to the $[0,1]$ range [34]. The cyclic nature of H can be avoided by cropping the H values to [0.3, 1], which would remove the first third of colors corresponding to red–orange–yellow while maintaining the last red corresponding to the highest H values. The saturation (S) of a color represents its distance from its grayscale equivalent; therefore, highly saturated images will provide vibrant, colored images, while saturation values close to zero will yield a fully grayscale image. For each individual H and S, the value (V) represents its shade or tint. For example, a vibrant red that has a fixed H and S differs from a brownish red only in its V component [35].

It is possible to use the properties of the HSV formalism to create a single representation of up to three magnitudes by assigning each of them to one coordinate of the HSV space. The proposed procedure is summarized in Algorithms 1 and 2. Our aim is to keep the transformation constant for each dataset, so the same interpretation, scale and legends can be applied for all samples in the same dataset. Since HSV magnitudes are defined in the range $[0,1]$, we need to scale all data to this range. Let $y_i$ be a 2D numerical map that represents some property of a sample (i). Let $Y=y_1,\dots ,y_i,\dots ,y_N$ be the set of $y_i$ for all the samples in the dataset (N). Then, each $y_i$ can be scaled to the range $[0,1]$ while keeping the relationship between all $y_i$ by making use of the range defined by $min\left(Y\right)$ and $max\left(Y\right)$ (function ScaleData in Algorithm 1).

Algorithm 1 Scaling and equalization functions

However, having data in the $[0,1]$ range does not imply that the visualization of maps in HSV will improve the overall representation. This is of particular importance when scaling $y_i$ with respect to Y, since the range is not defined by each $y_i$, but by Y. If there is one $y_j$ with a particularly high maximum value (outliers), all of the remaining scaled $y_i$ will have their values collapse to a small dynamic range. If this happens for the magnitude that we later assign to H, this would mean that all samples would essentially have the same “color”, resulting in insignificant visualization improvements. To compensate for this effect, histogram equalization can be performed prior to assigning Y to any HSV coordinate, as proposed in [26]. However, we propose improving the equalization process by considering the histogram of the dataset Y as a whole, as well as its cumulative distribution function ($CDF$), to non-linearly scale each $y_i$ with $CDF\left(Y\right)$, so that those ranges where data are heavily compressed can be expanded to occupy the full $[0,1]$ range. This additional equalization (function EqualizeHue in Algorithm 1) is applied to the magnitude representing H only, since our aim is to use color as an immediate means to distinguish between sample types. Using a global instead of a local equalization allows for the use of only one HSV legend to compare between images. Note that histogram equalization is only necessary if the values of the dataset collapse to a small range inside the wider $[0,1]$ range in such a way that the distribution hinders visualization (i.e., uniform H, S or V).

The magnitude-to-HSV encoding itself is easily implemented (Algorithm 2). After obtaining the limits and equalization parameters of Y, the only remaining step is to choose which $y_i$ will be assigned to H, S or V for each sample in the dataset. Then, the HSV vector can be constructed and transformed to RGB for visualization.

The choice of which optical parameter is assigned to each HSV dimension must be carefully considered. Since our datasets comprised clinical information, we chose to assign the most clinical relevance to H, which will vary between datasets. In this case, the term “clinically relevant” refers to the ability of Y to distinguish between control and pathological samples. Specifically, we choose the Y values associated with the control samples to be displayed in green, while those of pathological ones are displayed in red. For S and V, we choose to assign those Y that have strong spatial features, since these types of variations are better visualized through luminance/value/saturation changes than through color ones.

Algorithm 2 HSV transformation. Note that this algorithm refers to the functions described in Algorithm 1

3. Results

3.1. HSV for Multimodal Imaging: Degradation of Aortic Tissue

To test the combination of optical properties obtained from multiple imaging systems, we used one control and one pathological aorta sample, from which we derived the ${\mu }_t$ from the OCT measurements and the (adimensional, normalized) amount of elastin ($c_e$) as a clinical marker from the HSI-measured reflectances, with the methodology explained in Section 2. We obtained boxplots of $c_e$ and ${\mu }_t$ for both samples (Figure 1a) to visualize their distributions and overlap. All distributions were different between the healthy and the pathological sample, including some chromophores not considered for representation (collagen and water). Note that the HSV space still needs a third independent magnitude to be completely defined. Initially, we took the profilometry p of the samples as obtained with OCT imaging. The final magnitude-to-HSV encoding was $c_e\to H$, $p\to S$ and ${\mu }_t\to V$.

The selected optical properties were calculated for all ROIs of our samples, yielding the images shown in Figure 1b–d. The elastin maps obtained with HSI were up-scaled to the size of the ${\mu }_t$ maps via linear interpolation for the posterior matching of the H, S and V dimensions. The OCT-based maps (c) and (d) were represented as en face maps to match the visualizations given by our HSI system.

Figure 1e shows the colorimetric combination of the optical properties presented in the same figure for the pathological aorta. Given that we are now working with multiple parameters, the legend for the image should have as many dimensions as the parameters. However, we show the S = 1 plane in the portrayed legend (Figure 1f) since, at this scale, the samples are mostly flat and the profiles p have little to no effect on the final images. The difference in data structure between systems is clearly visible, especially between the $c_e$ and ${\mu }_t$ maps.

To explore the opportunities that this visualization method brings, we combined the HSI-derived elastin and OCT-derived ${\mu }_t$ in the maps shown in Figure 2 for both aortas. The color legend is the same one that was introduced in Figure 1. Color differences are immediately recognized, as well as structural variations upon closer inspection of the samples. By assigning the characteristics of the control samples to the most vibrant green (low $c_e$) and the pathological ones to the darkest red (high $c_e$), healthy regions are immediately recognized both in the control sample (Figure 2a,b) and in the pathological one (Figure 2c,d). Specifically, some heavily damaged aortic regions, marked by the red colors of $c_e$, are visible in the rightmost column of Figure 2d, where the darker spots indicate the reduction in ${\mu }_t$. Other parts of the mosaic combine some dark signatures given by ${\mu }_t$ with cyan colors, indicating that there are structural changes that are not yet visible as a significant variation in the elastin content.

3.2. HSV for Multi-Parameter Imaging: Dystrophic Degree of Mice Samples

The application of this method is not restricted to the combination of measurements from different systems. In Figure 3, we show the transformations for the mice muscle dataset through the en face maps. The clinical marker in this case is the $\Delta n$, which indicates the level of internal structure in the samples (higher $\Delta n$ means a stronger and healthier internal structure), and which was assigned to H. Again, this implies the immediate identification of healthy vs. pathological samples through color variance. Upon further exploration of the images, healthier (greener) regions on all samples, and specifically on the control one (Figure 3a), are complemented by ${\mu }_t$, where the strong anisotropy indicated by the stripes defined by ${\mu }_t$ is also reflected by the stronger birefringence. The 1-month-old mouse sample (Figure 3a) is the complete opposite, where no spatial structure of ${\mu }_t$ is visible at all, which in turn yields negligible birefringence values (red), indicating the lack of internal structure and incremented muscular deterioration. Finally, the 3-month-old mouse sample (Figure 3c) lies in between the other two, with some spatial features visible and mild birefringence.

4. Discussion

The proposed encoding of optical properties to components of the HSV color space was applied to two use cases: the combination of parameters from different imaging modalities (healthy and pathological aorta samples) and the combination of parameters from a single system (dystrophic and control mice muscles). These measurements, taken with OCT, HSI and PS-OCT, allowed us to obtain relevant tissular optical properties for the samples, namely ${\mu }_t$, $c_e$ and $\Delta n$, respectively. All distributions were centered on different values; therefore, one could theoretically use any of them for H.

In the case of multiple-modality imaging, the OCT system provides measurements with a resolution of approximately $13\mathsf{\mu }$m/px, while our HSI device yields $0.9$ mm/px at the working distance for the aortas. This means that ${\mu }_t$ (Figure 1d) is the best for encoding V in the HSV color space since it has a much higher resolution and spatial variation than the HSI chromophores (Figure 1b). As discussed in Section 1, luminance (or, in this case, V) is the best dimension for the representation of high-resolution data due to the increased ability to detect patterns in luminance variations.

To represent S, it is necessary to use a magnitude that is independent of the other two so that the HSV space is well defined. We used the profilometry p of the sample for this task (Figure 1c), calculated as the depth of maximum intensity in the OCT images. This means that higher-resolution and better-focused regions will generate vibrant and saturated colors, while lower-resolution and out-of-focus positions will provide less saturated and, therefore, less distracting colors. Since the aortic tissue was flat, the S values were mostly constant across both samples.

For the aortas, to create the mapping of ${\mu }_t$ to V, we used a transformation that took the maximum and minimum of all ${\mu }_t$ values across the aorta dataset and re-scaled them to the 0–1 range of V. Then, this transformation was applied to each ${\mu }_t$ value individually. By doing so, we kept the color scale consistent across all ROIs and samples (Figure 2). The same was done for H and S, but the re-scaled, equalized range of H was 0.3–1, to avoid the cyclic nature of H, as mentioned in Section 2.

Figure 1b–d use the same perceptually uniform colormap (magma) [36], which essentially means that the selected colors have a lightness that grows linearly from 0 to 1. It is instantly clear how high-resolution data (Figure 1c,d) benefit from this type of representation, with hotspots being clearly highlighted. Discerning the elastin content variations (Figure 1b) is more difficult, due to having to distinguish between multiple shades of what is essentially the same color. On the other hand, encoding $c_e$ as H makes it easier to pinpoint subtle variations, without losing the spatial information given by ${\mu }_t$ (Figure 1e).

Using the combinative visualization method allows for immediate discrimination between the two types of samples, both through the color difference between samples and also through the color uniformity (Figure 2). The co-registration with the ${\mu }_t$ maps helps to pinpoint the structural differences in areas with different elastin content, especially when zooming in on one of the ROIs (Figure 1e). For example, some red/magenta areas (high $c_e$) coincide with darker colors (low ${\mu }_t$), but, when looking at all the images, it is clear that there is no apparent visual correlation between the attenuation and elastin content. This could mean that structural changes caused by the damaging pathologies do not coincide with an increase in elastin, or that the change in the internal structure of the aortas is visible before such composition changes take place. The fact that no color is unsaturated means that all samples are flat and well focused.

In the case of multiple-parameter imaging, the process of assigning properties to HSV for the muscles (Figure 3) is identical to the case of the multiple-system imaging, with the dynamic range of $\Delta n$ requiring histogram equalization as described in Section 2. We kept the S and V as the same components as for the multi-system case. For this dataset, the ${\mu }_t$ distributions overlapped between samples; however, structural differences are still perfectly visible after assigning them to V variations. Using high-spatial-resolution data for all three components of the HSV color space presents no issue since it allows us to still see structural and clinical differences, and the dimension and shape of the samples’ profiles are comparable since we see little to no change in saturation.

For the muscles, the amount of texture provided by ${\mu }_t$ seems to correlate at first glance with the birefringence $\Delta n$, which is expected from healthy, anisotropic, muscular tissue. Nonetheless, additional information can still be retrieved from the combination of parameters, given that the ${\mu }_t$-$\Delta n$ effect is less apparent for heavily damaged samples (Figure 3b). Specifically, for the 1-month-old mouse sample, the spatial variations of ${\mu }_t$ are negligible; however, changes in hue reveal that the center of the muscle is more damaged (red) than its surroundings (magenta), which indicates that $\Delta n$ might be a more sensitive parameter to structural changes, while ${\mu }_t$ is better for the enhancement of superficial features.

The aortas showed damage observable without aid that was revealed in strikingly different colors between the healthy and damaged samples (Figure 2b,d). Nevertheless, similar trends were visible in the color distribution portrayed in the muscles, where the deterioration was not visible to the naked eye (Figure 3). This shows that as long as the color marker (H, a clinically relevant parameter) has sufficiently different distributions for control and pathological samples, the proposed method is suitable to combine diagnostic and non-diagnostic information in images with an enhanced context.

Ideally, with sufficiently large datasets, this method could consider the maximum and minimum values of the studied optical parameters, as well as their probability distributions, to create a universal framework for visualization that allows for a comparison between multiple samples, imaging modalities or devices. Similar studies of this type of color encoding to different areas have already been implemented [24,25,26], which indicates that this approach might be applicable to any form of dataset. However, any HSV or color encoding, including the one presented in this work, has the evident drawback of not being colorblind-friendly. In such cases, an attempt to optimize S and V while keeping H constant (or varying in a small range) might compensate for the lack of color resolution without compromising the pipeline of the method.

5. Conclusions

In this study, we have presented a color-space-based approach to combine optical properties in a single, information-rich visualization. The method is built upon making use of the distribution separation between two classes of clinically relevant parameters to obtain a single image that has all the advantages of its individual components at once.

We applied it to the combination of multi-modality (OCT and HSI) and multi-parametric (PS-OCT and OCT) optical properties. When most optical properties had different distributions for the different clinical cases, the choice of each parameter as a component of the HSV color space was based on the spatial frequency of the data. By doing so, we created image maps that colored the highly structured data through the low-structured information. When the resolution of the data is the same for all parameters, the selection of the transformation to HSV is only based on the differences in data distribution between clinical categories. Since the objective is for color to be the primary label for the pathological state of the sample, we always assign to H the optical property that has distributions with the least amount of overlap between the types of samples, with the green values corresponding to those of control samples.

Finally, the proposed combination of optical properties leads to maps with enhanced information that can be useful to clinicians to detect inter- and intra-sample differences at a single glance, but they also allow for an in-depth analysis without having to re-adapt between uses of different colormaps, thus allowing for faster decision making in environments where this might be crucial.