Nondestructive Evaluation of Heritage Object Coatings with Four Hyperspectral Imaging Systems

Abstract

Advanced imaging techniques can noninvasively characterise, monitor, and evaluate how conservation treatments affect cultural heritage objects. In this specific field, hyperspectral imaging allows nondestructive characterisation of materials by identifying and characterising colouring agents, binders, and protective coatings as components of an object’s original construction or later historic additions. Furthermore, hyperspectral imaging can be used to monitor deterioration or changes caused by environmental conditions. This paper examines the potential of hyperspectral imaging (HSI) for the evaluation of heritage objects. Four cameras operating in different spectral ranges were used to nondestructively scan a beehive panel painting that originated from the Slovene Ethnographic Museum collection. The specific objective of this research was to identify pigments and binders present in the samples and to spatially map the presence of these across the surface of the art piece. Merging the results with databases created in parallel using other reference methods allows for the identification of materials originally used by the artist on the panel. Later interventions to the original paintings can also be traced as part of past conservation campaigns.

1. Introduction

Imaging techniques used in the conservation of cultural heritage objects play an important role in authentication, documentation, assessment of material composition, detection of past conservation, evaluation of object condition, and programming of further restoration measures. In the past, all these activities relied solely on the personal experience of well-trained experts [1]. However, with the development of alternative diagnostic techniques, modern “high-tech” instruments are increasingly used in the field of art analysis. Advances in high-resolution silicon charge-coupled device (CCD) technology have greatly improved the availability and performance of digital imaging systems [2] in various applications [3,4]. The optoelectronics revolution provided new solutions for in situ measurements, the wide application of miniaturised portable devices [5,6,7] or smartphones [8,9], offering optimal performance at relatively low cost. Even though different methods can be used to study the authenticity, composition, and state of preservation of art objects, implementing nondestructive and highly reliable techniques is of particular interest. For qualitative analyses of pigments and binders, methods have often been chosen that do not require sampling from the art object and allow in situ investigations, such as infrared (IR), Raman spectroscopy, or X-ray fluorescence [10]. However, the analysis can assess only a very narrow fragment of the artwork under investigation. The measurement is usually repeated several times to increase the signal-to-noise ratio or to obtain spatially resolved information. The accurate assessment of the composition of pigments, binders, and protective coatings, as well as their deterioration processes, is relevant for the development of novel protective systems compatible with the cultural heritage (CH) objects of interest [11]. In particular, the research potential of merging various classical analytical techniques that provide additional information of different types (spectral responses, topography, chemical composition, etc.) with digital imaging is enormous [12,13].

Hyperspectral imaging (HSI) cameras operating in bands of the visible–near-infrared and short-wave infrared (VNIR-SWIR) regions are used to determine the characteristics and properties of various materials such as soils, minerals, rocks, water, and vegetation. Lately, HSI has been successfully applied in the field of cultural heritage to identify and classify paint pigments, monitor object deterioration and colour changes, evaluate the effectiveness of conservation/restoration interventions, and accurately digitise artworks [14,15,16,17]. The main advantage of HSI over VNIR spectroscopy is that imaging allows a detailed analysis of the spatial distribution of particular pigments or binders. However, both techniques allow measurements to be performed in a noninvasive way, eliminating any possible physical contact with the surface of the CH objects [18,19]. It is important to mention that the values of light absorption coefficients of pigments for X-rays and infrared (IR) are often inverse. Lead white, for example, strongly absorbs X-rays but is largely transparent to IR. Conversely, bone black strongly absorbs IR radiation but is nearly transparent to X-rays. It has been reported that HSI spectroscopy is able to distinguish pigments with similar elemental composition that could not be differentiated by other spectroscopic methods [13]. According to Roselli and Testa [20], HSI is an extension of conventional conservation photography and does not require collecting samples. Alternatively, HSI can enable the screening and evaluation of archaeological artefacts, as each HSI pixel contains spatially resolved chemical information [21]. However, an important technical challenge is to ensure imaging with an adequate signal-to-noise ratio in low-light conditions, such as those encountered in most restoration laboratories or museums. Long exposure time, even if beneficial for spectral resolution, can be problematic in the evaluation of art objects due to their sensitivity to deterioration [22].

HSI is a powerful combination of spectroscopy and imaging and allows the simultaneous acquisition of spectral and spatial information. HSI data are usually arranged in a three-dimensional cube called a hypercube, which has one spectral and two spatial dimensions. Each pixel of the image is associated with a corresponding spectrum captured in a particular spectral range. Such information may refer to a detailed colour mapping assessed in the visible range (VIS), as well as to chemical absorption or physical scattering. In the latter case, different radiation ranges are used (depending on the application) covering short (SWIR), mid (MWIR), and long (LWIR) waves of infrared. The acquired signal (hypercube) can be transformed into reflection, absorption, or transmission and correlated with various chemical and physical properties of the sample. The most common approaches to reduce the data complexity obtained by HSI scanning are multivariate data compression tools such as principal component analysis (PCA). Other methods such as partial least squares (PLS) regression or partial least squares discriminant analysis (PLSDA) establish relationships between the spectral data and the constituents of the sample [23]. Alternatively, hyperspectral images can be analysed using artificial intelligence as proposed by Kleynhans et al. [24]. However, large training datasets are needed to successfully explore convolutional neural networks (CNNs). New methods for acquiring high-resolution hyperspectral images [25,26,27] or new algorithms for post-processing spectral data [22,28,29,30,31,32,33] are currently of interest to engineers and researchers working in this field. Similarly, multisensor systems that allow automatic characterisation routines of art objects with high accuracy and precision are desired [34].

Image processing methods allow the grouping and mapping of artist materials and alteration products according to their spectral similarities, highlighting aspects that are not detectable by visual inspection. However, given recent developments in optics and electronics, some limitations in detector sensitivity, as well as data storage and mining, might be problematic. Often, after preliminary research with hyperspectral imaging, multispectral cameras equipped with a set of narrow-band optical filters are developed to reduce hardware costs and increase computational speed while still providing fully usable information [35,36].

Analysis of CH objects in the short-wave infrared (SWIR) and mid-wave infrared (MWIR) regions offers the possibility of visualising hidden details in the inner layers of paintings and highlighting the underdrawings [37]. Stratigraphic analysis is a highly interesting approach to studying the inner layers of complex surfaces. Most of the applicable techniques, including microscopy, radiography, fluorescence, diffractometry, and spectroscopy, are considered nondestructive [13,38,39,40,41,42,43,44] and are consequently suitable for the characterisation of cultural heritage objects. However, the effective penetration depth of the corresponding radiation depends on its nature, energy, and wavelength. The operating depth of X-ray diffractometers (XRD) varies from a few to approximately 100 µm, depending on the optical density of the material and its X-ray attenuation. Scanning electron microscopy in combination with energy-dispersive X-ray spectroscopy (SEM/EDX) provides information on the subsurface to approximately 2 µm thick. The excitation wavelength of light in Raman spectroscopy determines the light penetration depth. It corresponds to 0.7 µm with a 532 nm source and increases to 12 µm when a 785 nm laser source irradiates the surface. Attenuated total reflectance (ATR) Fourier transform infrared spectroscopy (FTIR) is widely used to evaluate the physiochemical properties of materials. The use of a particular crystal, such as germanium (Ge), zinc selenide (ZnSe), thallium-bromoiodide (KRS-5), silicon (Si), or diamond (C), affects the refractive index of the light and the depth of penetration of the infrared into the surface of the object. In this case, it varies between approximately 0.5 and 2 µm. A much higher penetration depth is found for near-infrared radiation, varying between 0.5 and 5 mm, again depending on the optical density of the tested material and the wavelength of the irradiating light [45].

The aim of this study was to investigate the suitability for assessing coated wood panels using four hyperspectral imaging systems operating in different spectral ranges. The result of the scanning was then used to produce a detailed colour and chemical mapping of the investigated cultural heritage objects. This work is part of a wider research campaign looking at the evaluation of painted wood panels using a variety of non- or semi-destructive methods. An overall objective is to provide a scientific basis for a routine methodology that allows for the automatic identification of pigments and binders. This is indispensable for the reliable preparation of mock-ups or to support the conservation of CH objects.

2. Materials and Methods

2.1. Experimental Sample

Painted beehive panels are paintings on rectangular wooden supports of smaller dimensions that were attached to the frontal planes of outdoor beehives. They were produced only in certain Slovene folk regions (mainly in Carniola and Carinthia), predominantly between the mid-18th and early 20th centuries [46]. As such, they are considered unique in the European folk art cultural heritage, with no known preceding examples outside Slovenia. It is believed that the beehive panels were traditionally coated using oil painting techniques and locally available colouring agents, although no specific material-analytical studies of the beehive panels have yet been published. The depictions in the paintings represent numerous motifs, ranging from religious, secular, satirical, and imaginary to ornamental. Stylistically, they are almost always subject to a limited support size, often painted somewhat naively, without elements of fine art painting, and executed in vivid colours. Artists of these paintings usually belonged to the rural craftsmen class and made the panels for local beehive keepers and farmers. The case study presented here is an 1836 beehive panel painting titled “Unidentified scene of military violence” (sln. “Neugotovljiv prizor vaškega nasilja”), measuring 254 × 128 × 14 mm³ (Figure 1), which was kindly made available for research by the Slovene Ethnographic Museum (sln. Slovenski etnografski muzej).

2.2. Characterisation Methods

Four hyperspectral cameras, described in

Table 1. Spectroscopic characteristics of the hyperspectral cameras used for the study.

Hyperspectral Camera	FX10	FX17	SWIR	MWIR
Spectral region	VNIR	NIR	SWIR	MWIR
Spectral range (nm)	400−1000	900−1700	1000−2500	1550−5950
Number of bands (pixels)	224	224	288	154
Spatial resolution (pixels)	1024	640	384	640
Field-of-view width (mm)	165	190	195	256

, manufactured by SPECIM (Oulu, Finland), were used to record the beehive panel. All cameras were operated in push-broom mode, which allowed line-by-line spectral measurements. White (spectralon) and black (detector background noise) references were measured before each scan of the panel. Halogen lamps were used as the light source for VNIR, NIR, and SWIR cameras, while thermal radiation was used as the source for the MWIR system. Evince software from Prediktera (Umea, Sweden) was used for analysis and exploration of the hyperspectral images. Standard normal variate (SNV) and mean centring were applied as pre-processing before PCA modelling.

The representative spectra for selected points of interest on the panel were calculated as the mean of a small surface area around that point, assuming that only pixels appearing the same were captured. The set of selected spectra was normalised by an extended multiplicative scatter correction before calculating the average spectrum.

3. Results and Discussion

Images acquired via hyperspectral imaging contain spectra in which both the light absorbed by the functional groups of the chemical components and the scattered light are recorded. While light absorbance/reflectance/transmittance can be related relatively straightforwardly to the chemical composition of materials, the interpretation of light scattering is more complex. The scatter is highly dependent on surface properties, such as surface roughness, the refractive index of the pigments used, and the suspension media used for painting and conservation [47].

Each hyperspectral cube (image) contains thousands of spatially resolved spectra. Since many of them are similar, it is very useful to reduce the size of the dataset as the first step of the analysis. Image processing tools based on multivariate techniques, such as PCA, are often used for this purpose in exploratory analysis of data [48]. The PCA algorithm exploits the fact that adjacent bands of hyperspectral images are highly correlated and often convey almost the same information about the object [49]. PCA is therefore used to decorrelate and reduce the amount of spectral information in a hyperspectral image. It aims to find the best representation of the hyperspectral data in terms of its variance. First, the calculation of the eigenvalue decomposition of the covariance matrix is performed. Then, only the eigenvectors that correspond to the largest eigenvalues are selected, thus achieving a reduction in the dimensionality of the original data while preserving the variance of the data. It is a routine procedure to mean-centre the data before PCA calculation [50].

The significance of principal components (PCs) can be expressed as a linear combination of scores and loadings that form a model that mimics the original variables. The difference between the model and the original dataset is expressed as residuals. It is desired that the sum of the residuals should be as small as possible and contain no signal features other than noise. The importance of each original variable is defined by a given PC and recorded as the model loading. This can be represented on bidimensional plots (loading plots) to ease visualisation of intercorrelations between the original variables. Conversely, the score plot contains information about similarity, groupings, and trend patterns that characterise the object and is represented as spatially resolved pixels [51].

Figure 2 shows the results of PCA analysis of the image acquired by the FX10 HSI camera in the visible/NIR range. Score plots are shown for the three most relevant principal components, as well as the corresponding loadings. Analysis of images in the visible range provides precise spectral information about the colour distribution at a single-pixel resolution. Unlike typical colour images, where the entire spectra are represented only by red, green, and blue (RGB), the information provided by the hyperspectral camera is much more accurate and contains 224 bands. PC1 and PC2 highlight the figures depicted and the right part of the panel, while PC3 highlights the background, showing the dark soldier’s headgear and woman’s dress. Analysis of the loadings is challenging in this case, as the available literature refers to the visible spectra of certain pigments, rather than to their combination, as in the case of investigated panels. Paint mixing and paint overlaying are very common practices, although understanding the contribution of the different components to the final spectrum is not trivial [52,53]. The shape of the first loading is similar to the reflectance spectrum of lead white that seems to be present in the analysed sample [54]. The first loading has a positive value in the wavelength range between 450 and 650 nm, corresponding to blue, green, and yellow. The portion corresponding to the red colour, as well as the near-infrared region (NIR), is negative. Loading 2 is negative in the visible range (except for the violet/blue part) and in the narrow part of the NIR range (up to 800 nm). The third loading exhibits a similar trend to the second loading in the blue range, but an opposite trend at higher wavelengths.

Figure 3 shows the results of PCA analysis of the hyperspectral cubes taken with the FX17 camera in the 900−1700 nm range. PC1 highlights all parts of the panel that were damaged, clearly showing the uncoated wood. Both PC2 and PC3 highlight the contours of the paintings. The first loading has negative values between 950 and 1400 nm, corresponding to the third and second overtone ranges. The detected positive values for 1400−1700 nm are partially covered by the first and second overtone regions. The colour determining property contained in the visible spectrum is often sufficient for the identification of different pigments. However, some paint components (including pigments and binders) have spectral features that become visible only at longer wavelengths [55]. Images acquired in the SWIR and MWIR regions therefore provide information about the chemical composition rather than the colour appearance on the CH object surface. In this range, the spatial resolution is reduced from 1024 (FX10 camera) to 384 and 640 pixels for SWIR and MWIR, respectively. Nevertheless, the analysis of the scores and loadings highlights certain similarities of some parts of the image.

Reflectance spectroscopy in SWIR provides information on vibrational transitions, which are mostly overtones and combination bands of fundamental transitions occurring in the mid-IR. This spectral range could be extremely useful for the conservation field, as it allows the identification of a large number of inorganic and organic materials that do not exhibit discriminating features in the VNIR part of the electromagnetic spectrum [11,56,57]. The absorption features of organic materials such as the paint binding media, including the amide groups of proteins and lipid bands of oils and waxes, can be observed [58,59,60].

The first principal component of the SWIR spectra (Figure 4) highlights the headgear of the soldier, the woman’s dress, and the external border of the panel. The loading has positive values at 1400−2550 nm, corresponding to the first overtone and the combination band region. Spectral features detected at 1940 nm and 2300 nm might be related to the first overtone of the ester carbonyl stretching mode related to the antisymmetric/symmetric stretching and bending of the methylene CH₂ groups in the drying oils, respectively [61]. According to Gabrieli et al. [19], the wavelength position of this lipid band can be used to discriminate between a drying oil (2304 nm), egg yolk (2309 nm), or wax (2312 nm). Close bands at 1910 nm and 2330 nm were also identified as kaolin and calcite by Brocchieri et al. [62], who performed SWIR mapping in parallel with XRF measurement. Analogous bands are also evident in Loadings 2 and 3. PC2 highlights the soldier’s hat, sword, and shoes, and the frame of the mirror on the wall. It also shows the border on the painting, which appears to have a similar composition to the objects mentioned. The dark green colour used for the above-mentioned part was identified as emerald green by Raman spectroscopy [63]. PC3 highlights a similar picture feature appearance as PC2.

Mid-infrared spectroscopy in reflection mode is widely used to characterise inorganic and organic coating materials. Similarly, MWIR hyperspectral imaging has been reported to have great potential for the characterisation and spatial mapping of paints [64]. However, the proper implementation of such a hyperspectral imaging system is challenging in terms of both instrumental set-up and illumination. Infrared radiation was provided in the experimental system by two heating bars, each located in a quarter cylindrical mirror to concentrate the radiation as a line covering the field of view of the HSI camera. Scanning was performed in a few seconds to minimise potential damage to the sample due to extensive heat exposure. The results of PCA of the MWIR spectra are shown in Figure 5. The first principal component shows the damage to the soldier’s jacket as well as some other parts of the panel (e.g., its borders). PC1 has a strong negative value in the spectral range of 3500−5500 nm and a positive value in the range of 1500−3500 nm. PC2 highlights the soldier’s headgear and boots, which most likely contain similar pigments. It is well evident that the woman’s dress (left part of the panel) differs in chemical composition, even though visually it appears to be a similar colour to the soldier’s headgear. The dress is composed of Prussian blue (iron (II, III) hexacyanoferrate (II, III)) [63]. The loading of PC2 contains relevant information in the range of 2300–2700 nm, where its value is negative. In the other part of the spectrum, the loading value oscillates around 0, which describes a negligible association with the spectral variables. The analysis of PC3 proves that the compositions of the colours used for painting the human figures and part of the wall are relatively similar. The central part of the loading (2700−4200 nm) contains several negative and positive peaks. The remaining parts oscillate relatively close to the neutral level.

As stated previously by Delaney et al., the identification of pigments from reflectance spectroscopy alone is often difficult [60]. Hyperspectral imaging was therefore performed in parallel with other spectroscopic methods, such as Raman and FTIR spectroscopy, to identify the pigments and binders of the investigated objects [63]. Both techniques were used to measure a limited number of panel locations with the intention of providing qualitative information about their composition. Noninvasive Raman spectroscopy was used to detect different pigments, such as carbon-based black (Location 1), lead white (Location 1 and Location 2), and iron oxides (Location 1). Lipids and triterpenoid resin were identified at all three beehive panel locations by total reflection FTIR spectroscopy. The lipidic medium was also identified in the SWIR spectra of all three locations of the beehive panel, based on the typical doublet at 2347 and 2311 nm [65]. Conversely, HSI allows scanning of the entire panel surface, enabling the monitoring of artworks and possible restoration interventions over time at relatively low cost compared to other analytical methods [57,66]. An additional advantage of the HSI system used in this research was that four complementary cameras provided complete spectral information from visible to mid-wavelength infrared. Detailed analysis of the spectra can therefore be supported using the abovementioned reference methods. An example of the combined hyperspectral imaging spectra is presented in Figure 6 for three specific regions of interest (ROIs) with different appearances. The most noticeable differences are observed in the visible range of the spectrum, expressing a variation in colour appearance in each ROI. However, deviation of the spectrum shape is also evident in near-infrared bands, especially in the MWIR. As expected, the spectra provided by the FX17 and SWIR cameras seem to overlap. This confirms the high repeatability and reliability of all spectroscopic systems investigated, resulting in a similar spectroscopic representation regardless of the camera type.

Detailed analysis of the hyperspectral images, including both raw spectrum studies and visualisation of higher-degree principal components, led to the discovery of several features that were not apparent by simple panel observations, such as underdrawings, retouching, and/or pentimenti. An example is presented in Figure 7, where hidden underdrawings can be detected in the inner layer of the studied beehive panel painting. The specific spectral marking was detected at 1500 nm on images taken with the FX17 and SWIR hyperspectral cameras. In addition to the handwritten text, several other pencil-like elements of the sketch are noticeable.

NIR photography was used to detect underdrawings in the late 1930s when Ian Rawlins used an NIR camera to improve the visual assessment of paintings [20]. Due to the ability of NIR to penetrate some pigments and the selective absorption by some pigments in the NIR range, it is possible to differentiate carbon-based from organic-based compounds. Moreover, NIR photography might also help in distinguishing paintings by different artists, as some of them are known to make preparatory sketches [13,14]. Furthermore, such information is very valuable to discovering the intentions of artist alterations and the way they work, and, thus, to better understanding the creation process [2].

The reported results show a high potential of multirange hyperspectral imaging for the assessment of cultural heritage objects, especially wooden panels covered with organic coatings. However, it is not a simple task to interpret hyperspectral images directly without an extensive database of reference measurements. Such reference data are indispensable for the proper calibration of chemometric models and automated interpretation of hyperspectral images. In parallel with the laboratory work, an extensive literature review was performed with a focus on summarising the state of the art in spectral band assessment as reported by other authors. Unfortunately, due to the very complex nature of the coating systems used in studied cultural heritage objects, a straightforward and complete interpretation of the spectra was limited. For this reason, an extensive experimental campaign is currently underway to form reference datasets using IR microscopy and Raman spectroscopy. The analysis is specifically focused on the beehive panels as Slovenian heritage and will be integrated to further improve the results reported in this manuscript. An innovative algorithm for multisensor data fusion is also being developed. It will enable combination of the hyperspectral information, both spectrally and spatially resolved, acquired in all the investigated spectral ranges.

4. Conclusions

Nondestructive techniques are commonly used for rapid screening or detailed examination of artworks. They are particularly suitable when sampling of the artwork is not permitted and other destructive or semidestructive methods are ruled out. X-ray techniques penetrate most materials with little interaction and are well suited for taking absorption distribution images of paintings in a nondestructive manner. Hyperspectral imaging, however, is much faster in acquiring data and is therefore better suited for screening a large collection of works. Both techniques are highly complementary as they are sensitive to different characteristics of the pigments, either chemical structure or elemental composition. Hyperspectral imaging combines spectroscopy and imaging, which allows the simultaneous acquisition of spectral and spatial information. The extensive characterisation of a panel painting in different spectral ranges performed in this study provided detailed information on colour tonality distribution and enabled chemical mapping of the investigated sample. Traces of lead white, emerald green, Prussian blue, and carbon-based black were identified as pigments used by the artist. In addition, lipid and triterpenoid resins were detected in the spectra as binders. The analysis of the results allowed the identification of damaged areas of the studied beehive panel painting, as well as hidden underdrawings in the inner layer.

In the case of art objects, analytical identification is difficult due to the inhomogeneity of their surface and the complex composition of the source materials. Nevertheless, precise and fast methods such as hyperspectral imaging not only provide information on specific pigments and binders used, but also allow their mapping to the entire object. Optimally, hyperspectral data should be combined with other nondestructive analyses to provide a more complete evaluation of art objects. Merging the results obtained with databases created by other reference methods allows the identification of the materials originally used by the artist. These can be contrasted with other additions that were later introduced into the painting as part of past conservation campaigns.