Next Article in Journal
Three-Dimensional Microwave Imaging: Fast and Accurate Computations with Block Resolution Algorithms
Previous Article in Journal
Evaluation of Strain Transition Properties between Cast-In Fibre Bragg Gratings and Cast Aluminium during Uniaxial Straining
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Sensors
Volume 20
Issue 21
10.3390/s20216281
sensors-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

An Automated Pipeline for Image Processing and Data Treatment to Track Activity Rhythms of Paragorgia arborea in Relation to Hydrographic Conditions

by
Ander Zuazo
1,
Jordi Grinyó
2,
Vanesa López-Vázquez
3,
Erik Rodríguez
3,
Corrado Costa
4,
Luciano Ortenzi
4,
Sascha Flögel
5,
Javier Valencia
3,
Simone Marini
6,7,
Guosong Zhang
8,
Henning Wehde
8 and
Jacopo Aguzzi
2,7,*
1
Deusto Sistemas, 01015 Vitoria-Gasteiz, Spain
2
Instituto de Cièncias del Mar (ICM-CSIC), E-08003 Barcelona, Spain
3
DS Labs, 01015 Vitoria-Gasteiz, Spain
4
Consiglio per la ricerca in agricoltura e l’analisi dell’economia agraria (CREA)-Centro di ricerca Ingegneria e Trasformazioni agroalimentari, Via della Pascolare 16, Monterotondo, 00015 Rome, Italy
5
GEOMAR, Helmholtz Centre for Ocean Research Kiel, 24148 Kiel, Germany
6
Institute of Marine Sciences, CNR, 19032 La Spezia, Italy
7
Stazione Zoologica Anton Dohrn, 80122 Naples, Italy
8
Institute of Marine Research, N-5817 Bergen, Norway
*
Author to whom correspondence should be addressed.
Sensors 2020, 20(21), 6281; https://0-doi-org.brum.beds.ac.uk/10.3390/s20216281
Submission received: 22 September 2020 / Revised: 30 October 2020 / Accepted: 2 November 2020 / Published: 4 November 2020
(This article belongs to the Section Intelligent Sensors)
Browse Figures
Versions Notes

Abstract

:
Imaging technologies are being deployed on cabled observatory networks worldwide. They allow for the monitoring of the biological activity of deep-sea organisms on temporal scales that were never attained before. In this paper, we customized Convolutional Neural Network image processing to track behavioral activities in an iconic conservation deep-sea species—the bubblegum coral Paragorgia arborea—in response to ambient oceanographic conditions at the Lofoten-Vesterålen observatory. Images and concomitant oceanographic data were taken hourly from February to June 2018. We considered coral activity in terms of bloated, semi-bloated and non-bloated surfaces, as proxy for polyp filtering, retraction and transient activity, respectively. A test accuracy of 90.47% was obtained. Chronobiology-oriented statistics and advanced Artificial Neural Network (ANN) multivariate regression modeling proved that a daily coral filtering rhythm occurs within one major dusk phase, being independent from tides. Polyp activity, in particular extrusion, increased from March to June, and was able to cope with an increase in chlorophyll concentration, indicating the existence of seasonality. Our study shows that it is possible to establish a model for the development of automated pipelines that are able to extract biological information from times series of images. These are helpful to obtain multidisciplinary information from cabled observatory infrastructures.

1. Introduction

Capturing deep ocean environmental features, habitat heterogeneity and associated biological components is increasingly required, especially in the light of growing impacts by human activities [1]. Despite the important role of the oceans within the earth system, marine ecosystems are significantly under-surveyed in time and space [2]. Our knowledge of species behavior, interspecific relationships and resulting community composition and biodiversity is incomplete in the deep-sea, due to difficulties in sampling [3]. High costs and logistic constrains in vessel-assisted surveys (e.g., by trawling and Remotely Operated Vehicle-ROV) prevent data acquisition at a high-frequency over continuous and prolonged time intervals [4].
Quantifying species behaviors and their variability at different spatiotemporal scales and how that variability affects the functioning of ecosystems and their productivity (e.g., the rate of transference of carbon matter and energy), requires the development of advanced customized hardware and software monitoring technologies [1,5,6]. One focal point for the establishment of those technologically advanced monitoring strategies is to increase our capability to track the organisms’ responses to changing oceanographic conditions in a remote, high-frequency and prolonged fashion [1].
Massive rhythmic movements by millions of individuals occur throughout the water column and on the seabed, as a result of the response of species to physical and biogeochemical cycles in light intensity and photophase duration as well as tidal currents. Those cycles are produced by the Earth–Moon reciprocal movement in relation to the Sun [7,8,9,10,11,12,13,14,15,16]. Many species have evolved time-keeping mechanisms (i.e., biological clocks) to cope and anticipate those predictable environmental changes [9,10]. For motile megafauna, crawling, walking and swimming rhythms can be characterized in terms of fluctuations in hauled or video-counted animals [6,10,16,17]. Differently, for sessile species such as Cold-Water Corals (CWCs), similar data for comparison are poorer. Hauling is potentially destructive, although some species survive the sampling procedure (increasing that survival chances with ROV operations). When colonies are transferred to laboratories, their activity can be tracked under the simulation of some environmental cycles such as currents, e.g., [18]. Alternatively, imaging should be adjusted in situ to portray polyps’ extrusion and retraction patterns, a technological solution being valuable to couple activity patterns to oceanographic control (e.g., hydrodynamism), substrate changes and time-related modifications of the faunal assemblages within CWC reefs [19,20,21].
Fortunately, imaging technologies are on the verge to be deployed on an increasing number of cabled observatory networks worldwide [4]. Artificial Intelligence (AI) routines such as machine learning can be used to survey the biological status of CWC from large image data sets [22]; for example, in situ high-frequency imaging from cabled observatory platforms such as the Lofoten-Vesterålen Observatory (LoVe; Norway) proved that the reef building CWC Lophelia pertusa displays activity rhythms over the 24-h range, where polyp extrusions are influenced by tidal currents [21]. Similarly, in the Stjernsund between Tromsö and Hammerfest, the gorgonian bubblegum coral Paragorgia arborea showed a similar tidal-oriented filtering activity [23].
To date, it is of strategic relevance to increment data collection capabilities of cabled observatories with autonomous image treatment and multiparametric data processing capabilities [6,24]. Monitoring autonomous functions needs to be developed through a pipeline of consecutive and automated steps for the acquisition, processing, validation and multivariate treatment of multiparametric data [12,25,26,27,28,29]. A major goal would be to pave the route to establish a real-time data processing and interpretation software toolkit, specifically designed for underwater observatories. That toolkit would enable the automated tracking of species behavior (and consequently rhythmic activity), providing the end-users of online statistic tools to evaluate the effects of different oceanographic variables (e.g., salinity, temperature, chlorophyll concentration, turbidity and water column depth as proxy for tides) to the biological response [6,24,27,30,31].
This study proposes an advanced AI-based Convolutional Neural Network (CNN) routine for the tracking of behavioral activity in the conservation-iconic Paragorgia arborea [32,33,34], to be used as baseline for the establishment of monitoring programs at the Lofoten-Vesterålen (LoVe) deep-sea observatory. Taking as inspiration previous computer science experiences and data treatments for this species in the area [21,23], we also acquired high-frequency time-lapse images of one coral colony with a fixed camera. We consequently established a CNN approach to evaluate colony changes in surface coverage in terms of percentages of extruded and retracted polyps as a proxy for the activity rhythm of the species. That approach was developed in a real-world context, where the marked oceanographic variability and operational monitoring challenges (e.g., light flickering and variable imaging angle) occurred [28]. Chronobiology-oriented statistics and advanced Artificial Neural Network (ANN) multivariate regression modeling were also used on all acquired behavioral and oceanographic data sets, to provide examples for further automation steps of multiparametric data treatment to be used to establish a pipeline of real-time treatment of ecological information at cabled observatories.

2. Materials and Methods

2.1. The Lofoten-Vesterålen (LoVe) Ocean Observatory

The Norwegian continental shelf hosts some of the richest CWC reefs in the world [35]. A network of cabled video-observatories endowed with oceanographic multiparametric sensor technology (i.e., the Lofoten-Vesterålen, LoVe) was installed in the Hola Trough ~20 km off the north-west coast of the Lofoten Islands (Norway; Figure 1) [36]. LoVe has been in operation since 2013, to monitor the biological status of CWC mounds and their associated communities [37]. Node 1 (Figure 1A) is located in the south-eastern part of the Hola Trough at a depth of ~260 m, just south of the Vesterålsgrunnen bank, where CWCs are mostly constituted by Lophelia pertusa [38,39]. It has two satellite platforms and the #2 provided the imaging data set for this study (Figure 1B), being surrounded by P. arborea colonies (Figure 1C).

2.2. The Monitored Soft Coral Species

Paragorgia arborea (Linnaeus, 1758) is a semi-cosmopolitan deep-sea gorgonian [40,41,42,43,44], occurring on rocky and on dead coral frameworks, facing prevailing current [45,46,47]. This species occurs in fjords, continental margins, submarine canyons and seamounts [48,49,50]. Its arborescent morphology provides habitat and a niche to a wide variety of invertebrates [51,52] and fishes [53]. Due to slow growth and low recruitment rates, P. arborea is highly vulnerable to anthropogenic impacts such as trawling and long-line fishing [54,55] as well as sediment resuspension from bottom trawling or drill cuttings [56]. Future management plans should pay special attention to polyp behavior and its temporal expression in relation to the oceanographic factors, as an indicator of anthropogenic stress.

2.3. Image and Oceanographic Data Collection

The temporal activity of an individual colony of P. arborea was studied with 1-h time-lapse images acquired by a High Definition camera (Metas A.S.) from 23rd February (11:30) to 11th June (23:30) 2018. Although the camera was stereoscopic, only mono-images at a resolution of 2751 × 2206 pixels were used. A total of 6799 images were acquired. Two flashes on both sides of the camera illuminated the scene for 3 s prior to camera capture.
Oceanographic data such as temperature, salinity, turbidity, chlorophyll-a (hereafter named as chlorophyll) and water column depth (as a proxy for tidal bulges) were collected at 1 s frequency by sensors installed beside the camera (Table 1), in the same satellite #2 frame infrastructure (see Figure 1C). We selected the oceanographic data corresponding to the timing of each image. This simultaneous acquisition of oceanographic data is of relevance to enforce an experimental in-situ approach to the characterization of deep-sea species ecological niches [24]. In particular, one should notice that we used the following algorithm for the calculation of the chlorophyll concentration:
Chl a = 1.050 * Chl fl (day) + 1.0129 for daytime values
Chl a = 0.963 * Chl fl (night) + 0.2159 for nighttime values
where Chl and fl stand for chlorophyll and fluorescence. The LoVe observatory was under establishment at the time of the conducted measurements, so total values may be subject to change. In the present phase, the observatory focus is laid on the quality assurance of data.

2.4. Image Treatment

Different steps were implemented to extract the biological information on coral activity from images, based on extruded or retracted polyps (see below in Section 2.4.2). These steps were presented as a processing pipeline, to provide a model for the future implementation of automated procedures (Figure 2). All pipeline image treatment steps are described along with the terminal steps related to the time-series and multivariate statistical treatment and modeling.

2.4.1. Image Enhancement

The images were usually very dark. Thus, different computer vision approaches were tested for enhancing the foreground/background contrast while simultaneously maintaining the original color, prior to the automated extraction of the morphological index (see next Section 2.4.2.). Several filtering procedures were evaluated as Gamma, Logarithmic, Sigmoid, as well as Histogram Equalization, Re-Scale intensity and Contrast Limited Adaptive Histogram Equalization (CLAHE). A total of 20 images were randomly selected to be filtered with all the mentioned methods. CLAHE was the best and only rendering technique that achieved both objectives.

2.4.2. Supervised Tagging of Coral Areas

Images of the coral were processed, in order to extract an index for polyp extrusion and retraction, as a proxy for colony activity. That activity pattern was indicated here as bloated at full polyp extrusion (e.g., likely during filter-feeding phases) and non-bloated, when all polyps were fully retracted with a transient semi-bloated phase of mixed colony behavior (i.e., with clusters of extruded and retracted polyps) [57,58,59,60,61].The two major statuses, bloated and non-bloated morphology, are presented in Figure 3 for the whole colony. A third transient semi-bloated status is also considered and shown in Figure 4 along with other bloated and non-bloated ones for comparison, within enlarged views.
A training dataset for the automated classification of the three coral states was created. That dataset was established by randomly choosing 1000 images out of the total (i.e., 6799 images). In each image, 3 different rectangular regions of the coral (such as those presented in Figure 4, as an example) were manually tagged, to determine the activity status, resulting in 3000 section images: 1376 bloated, 708 non-bloated, 916 semi-bloated. As the tagging was manually performed, the sizes of each of the rectangular regions was slightly different in size.

2.4.3. Coral Segmentation

A background/foreground image segmentation was implemented in order to improve the classification performance of the coral behavioral status in terms of area coverage by polyps in the extruded or retracted status. A color-based (red) mask segmentation by Hue Saturation Value (HSV) [62] was used for this approach. The images were transformed into HSV color space and the mask color ranges were set to extract the foreground coral region. Next, a bitwise comparison was made between the image and the color mask, producing a binary threshold image. The red-segmented color mask processing allowed for the extraction of the pixels belonging to the coral with a fast and low-intensive computational burden (Figure 5). Then, several filters were applied to empirically compare the segmentation results [63]: Blur, Gaussian Blur, Bilateral, and finally, Median. The different filter effectiveness is evidenced by image treatment outputs in Figure 5.
The Median filter performed the better in terms of reducing white noise and preserving the details of the coral. A 3 × 3 kernel version of that filter was applied to the resulting images. For each of the 3000 rectangular section images, red mask coral segmentation was applied (Figure 6).

2.4.4. Data Preparation

Coral segmentation with a color mask allowed for the pre-processing of each of the manually tagged sections of the coral. For each of these regions, the segmentation techniques were applied (see above). From each segmented image, 120 pixels belonging to the coral (white pixels) and 40 pixels belonging to the background (black pixels) were randomly selected and encoded into a text file together with a pixel label: bloated, semi-bloated, non-bloated and background. Thus, for each image, 160 pixels were stored.

2.4.5. The Convolutional Neural Network (CNN) Model

To characterize the temporal expression of polyp behavioral rhythm (i.e., bloated, semi-bloated and non-bloated), a CNN model [64] was implemented to segment and hence identify the pixels belonging to each behavioral category, as follows. A six-layer CNN was trained with image pixel annotations and the corresponding images from the previous training stage. For each of the annotation pixels, a 30 × 30 crop image was created from its corresponding coral image where the middle pixel was the annotation pixel, following a strategy similar to Patches-GT [65]. This strategy is based on expanding the labeled ground truth pixels into labeled patches around those pixels. It assumes that the surrounding pixels of a labeled one are identical. Each of these cropped images was labelled with its corresponding pixel class. These cropped images and their labels were used as training/test data.
The CNN was implemented with a Keras library [66]. The proposed CNN (Figure 7) was composed of: a convolutional layer with 32 outputs and an input shape of (30,30,3); another convolutional layer with 32 outputs; a dropout layer with a rate of 0.25; a convolutional layer with 64 outputs; another convolutional layer with 64 outputs; a dropout layer with a rate of 0.25; a convolutional layer with 128 outputs; another convolutional layer with 128 outputs; a dropout layer with a rate of 0.25; a dense layer with 512 outputs; and finally a final dropout layer with a rate of 0.5 plus a dense layer with 4 outputs. The last dense layer had a value of 4, being equal to the number of statuses to classify (i.e., bloated, semi-bloated and non-bloated) plus the background. All the convolutional layers had 3 × 3 filters and the Rectified Linear Unit (ReLU) activation functions, except for the last dense layer, which had a Softmax activation function. Even convolutional layers had MaxPooling2d with a pool size of (2,2).
As for the optimization used to train the model, several variations of the algorithm ADAM were compared through trial and error. The best algorithm used a learning rate of 0.0001, beta_1 of 0.9 and beta_2 of 0.999. These parameters allowed the CNN to converge faster. The model was compiled with a loss function of “categorical_crossentropy” [67], since a non-binary class problem was addressed.
Data augmentation was used on the train/test crop images to have a much wider dataset that would yield a larger representation of coral shapes, by horizontal and vertical flips as well as width and height shift ranges of 0.1. A 10-fold cross-validation [68] was used to test the model. Each fold was stratified and randomized, in order to preserve the percentage of samples of each class. Within the 10-fold cross-validation, 9 folds were used for training and 1-fold for validation, repeating the experiment 10 times, by changing the validation fold each time (and as consequence the training folds). The final performance was the average of all the single validation performances. A batch size (number of training samples utilized in each iteration) of 256 was set for the training. The network was trained with 100 epochs (number of times the algorithm works through the training dataset).

2.4.6. The Segmentation of Coral Behavioral Statuses by CNN Modelling

The procedure to segment the images was performed via 30 × 30 window crops and yielded to the classification of coral behavior into the 3 categories plus the background (see Section 2.4.2. above). As we needed to iterate the pixel matrix of high-resolution images, the required computational time would have been too high (3000 s in average). In order to create a lighter computational tool, columns and rows were iterated with a step-width of 10. With this approach, the processing time of each image was reduced to ~75 s. This approach allowed us to create a map over the image in which each classified 30 × 30 window could partially overlapped other windows [69]. This procedure created a more realistic and precise map to interpret the activity status of the coral.

2.5. Behavioral and Environmental Time Series Compilation

The CNN automated approach enabled us to quantify the coral activity status from all considered images. Once the segmentation of each image was concluded, we could classify and count the pixels belonging to bloated, semi-bloated and non-bloated categories. In order to calculate an overall percentage of activity statuses for the whole coral colony, the pixels belonging to the three behavioral categories were summed up. Then, for each status, the sum of its pixels was divided between the total amount of coral pixels considered, obtaining a relative percentage.
Finally, a final dataset was created based on environmental and oceanographic data (as collected at times corresponding to images) as follows: date, time of the day, overall percentage of colony in bloated, semi-bloated and non-bloated status, readings for temperature, salinity, turbidity, chlorophyll, and depth.

2.6. Time Series Analysis

The occurrence of significant periodicities in all temporal data sets was analyzed with the Lomb–Scargle periodogram by the El Temps chronobiology software package (www.el-temps.com). All periodicities were screened in a window of frequencies between 600 and 1620 min (equivalent to 10–27 h, respectively), encompassing ranges for diurnal and semi-diurnal tides plus the day–night cycle [16]. In the periodogram output plot, the peak exceeding a significant threshold (p < 0.05) represented the maximum percentage of total data variance, explained by the inherent dominant periodicity [70].
Waveform analysis for the 3 coral activity statuses and the oceanographic data was performed to identify the phase (i.e., peaks timing) of rhythms and cycles [13]. All time series were subdivided into 24-h segments with a 1-h sampling frequency. All values were averaged at corresponding 1-h time intervals throughout all segments. Resulting averages (plus standard deviations) were displayed in 24-h plots (i.e., the waveform), along with a Midline Estimated Statistic of a Rhythm (MESOR). This is a daily average, obtained by re-averaging all waveform values [70]. In the waveform plot, the phase of a rhythm (i.e., the averaged peak) is identified by all values above the MESOR. Waveform plots for the three coral activity statuses were represented with intermediate dusk and dawn timings at the 23rd of April 2018 (i.e., considering the 5-months’ extension of the image acquisition period), as indicated by the local astronomical web service (https://www.timeanddate.com/sun/norway/bergen): dusk, 21:30 and dawn, 6:00. This plot implementation was done in order to show the linkage of detected rhythms upon the light intensity cycle.

2.7. Multivariate Modeling of Coral Activity

An AI classification approach was used to predict the activity status of the coral, considering a global binary behavior for the colony, with 2 statuses as bloated and non-bloated (semi-bloated was merely transient) [21,71]. We considered 7 independent and conditioning variables (i.e., the input x-block): time of the day (reported as a unit fraction of 24 h), chlorophyll, temperature, depth, turbidity, salinity and the Field Of View (FOV). The latter is a binary dummy variable, while the remaining 6 are continuous. The continuous variables were standardized (i.e., centered with respect to their mean and scaled by the relative standard deviation). The dependent response variable (y-block) was the bloated percentage of each image, discretized into a binary variable (i.e., bloated, not-bloated); when the percentage of the bloated surface was higher than or equal to 50%, it was converted to 1, and, when lower, it was converted to 0.
The x-block was trained several times using shallow neural networks for binary classification with different hidden layer sizes. If no convergence was reached due to underfitting, the number of features was increased by considering the square and cubic terms of the x-block obtained by multiplying its columns in a combinatorial fashion. In order to mitigate the overfitting, the cross-validation Root Mean Square (RMS) error was calculated as a function of the number of features. Thus, 133 features were obtained and used to train a Multi-Layer Feed forward artificial neural Network (MLFN; [72,73]) with a single hidden layer architecture, using sigmoid hidden and SoftMax output neurons. The ANN-based MLFN approach was trained with the Bayesian regularization back propagation algorithm [74,75], as implemented in the deep learning MATLAB toolbox. The dataset was partitioned using 70% of the observations (N = 1150) as training set and the rest as test set (N = 503). The test set was used to validate the model. This partitioning was optimally chosen with the Euclidean distances, selecting parameters without a priori knowledge of a regression model. The cost function was minimized using the normalized RMS error performance function with a 10−10 threshold on the gradient. Precision (P) and Recall (R) were also calculated on training and test sets.
In order to extract the most informative features, among the 133 acquired ones, a feature importance analysis was conducted as follows [76]. The hidden layer matrix (9 nodes x 133 variables) was a posteriori analyzed by considering its elementwise absolute value. From this matrix, the maximum value for each variable (i.e., column) was extracted, obtaining a 1 × 133 row vector. The top 15 most significant variables were chosen. The larger the value, the more relevant the contribution to the MLFN model. The model was developed by using the MATLAB 9.7 R2019b Deep Learning Toolbox.
Having the phenomenon of an activation shape, a comparison between the real percentage of bloated coral surfaces and the sigmoid of the most relevant standardized quantitative variables (i.e., time of the day, chlorophyll, salinity, temperature, depth and turbidity) was made. Pearson linear correlation and their probability were estimated.

3. Results

3.1. CNN Coral Segmentation: Evaluation of Training and Performance

After the CNN was trained, 3000 images were segmented via a 30 × 30 crop window classification, in order to have at least one value of coral activity status per one hour. Bloated pixels of the coral were visualized as the blue color, semi-bloated pixels as purple and non-bloated ones as red (Figure 8). The resulting CNN was used to segment the remaining images from the time series. The CNN model had an average test accuracy of 86.16% and an average Area Under Curve (AUC) score of 0.9732 in the 10-fold cross-validation of the model.
After the 10-fold cross-validation, the CNN model was trained with a 70/30 split in the dataset. This meant that a random 70% was used as training data, while the 30% left was used as test data. The CNN gave a final test accuracy of 78.08% and a validation AUC of 93.47%. The obtained Precision Accuracy (PA) and User Accuracy (UA) are shown in Table 2.

3.2. Time Series Analysis

Time series analysis was performed on coral activity data (percentage of colony surface coverage by extruded and retracted polyps) as a final step in the pipeline of information treatment (see Figure 2). During the study period, polyp activity was registered continuously in terms of the percentage of colony surface, showing a bloated, non-bloated and semi-bloated status (Figure 9). Colony surface coverage values for bloated and non-bloated statuses attained maximum values close to 100% and 90%, respectively (Table 3; Figure 9A,C). Differently, the semi-bloated condition attained lower maximum area coverage values, close to 70% (Figure 9B). The oceanographic data contained large gaps as a product of transient sensor malfunction and pauses in measurement activity during platform maintenance (Figure 9D).
Periodogram analysis outputs for time series of those percentages of colony coverage in bloated, non-bloated and semi-bloated statuses are presented in Table 3. Significant 24-h patterns were reported for all three activity variables. No analysis was conducted on time series of oceanographic data, since those data were non-continuous.
Waveform analysis of the three different coral activity statuses are presented in superposition to show their phases relationship (Figure 10). One should notice that the temporal pattern depicted by this graphic output sustains periodogram results (i.e., a single large peak per 24-h cycle; see Table 3). That combined and overlapped plotting evidenced the occurrence of a bloated peak towards dusk (and, consequently, a drop in non-bloated percentages at that time). Waveform values above the MESOR, showed for the bloated time series a significant increase from 14:30 to 00:30 (i.e., the peaks duration corresponds to all values above the MESOR). Conversely, the non-bloated status peaked from 2:30 to 14:30. Similarly, the semi-bloated condition showed a significant increase from 1:30 to 10:30 (with an isolated significant increase also at 13:30).
The same waveform analysis was also performed for the oceanographic variables (Figure 11), to show the of phases relationships among their cycles. Considering MESOR values (not plotted but reported in Table 3), the marked bimodal water column depth variations (as proxy for tidal current regimes) are evident (two peaks: 11:30–15:30 and 0:30–5:30). This hydrodynamic pattern affected turbidity, which also showed two major sharp increases over 24 h, slightly delayed in comparison to water column peaks (16:30 and 07:30). Conversely, chlorophyll showed a high variability (wide SD bar ranges) over the 24-h cycle. Salinity and temperature were also very variable over the 24 h and peaked as follows: salinity between 11:30 and 20:30 and temperature between 10:30 and 18:30.
Some clues about the occurrence of seasonal trends in polyp extrusion activity were found during our monitoring period (Figure 12). This trend was evidenced by the computing of monthly averages in polyps’ extrusion activity in comparison with the oceanographic variables. A significant increase (considering MESORs values in Table 3) in overall activity occurred in April, May and June, matching a similar pattern for chlorophyll and turbidity. Monthly values of chlorophyll for April, May and June were significantly higher than those registered in February and March (PERMANOVA, PSEUDOF: 474, p = 0.001). For monthly turbidity values, a significant increase was found from April to May (PERMANOVA, PSEUDOF: 3177, p = 0.02).

3.3. Multivariate Modeling of Coral Activity

Multivariate statistic treatment and modelling were performed as a final step for our automated data treatment pipeline (see Figure 2), to evaluate the response of biological data with respect to ambient environmental conditions. Our choice for the ANN class modelling was based on its feasibility for automation when the phenomenon to be analyzed has a non-linear temporal dynamic. In Figure 13, the RMS error, as a function of the number of features, is reported for training, test and validation sets. Since the cross-validation curve has a minimum at 133 features, we have chosen this value to perform the following calculations. The results were obtained averaging over 10 random trials.
Only 1653 observations (from 2nd February to 8th June 2018) were processed out of 3000. The remaining behavioral data were eliminated because they occurred during a gap in oceanographic data acquisition (see Figure 9D). The algorithm converged after 500 iterations with 0.26% of false predictions during training of the network and 31% of wrong predictions during tests. The output of the “softmax” function is a real number 0 < p ≤ 1, which represents the modelled probability of the coral to be bloated. Table 4 reports the principal results of the MLFN model used to predict bloated vs. non-bloated corals on the base of 133 variables (x-block). The confusion matrixes of the training and test sets are reported in Table 5. In the training set, only three out of 1150 (0.26%) were wrongly classified. During testing, 156 out of 503 (31%) samples were misclassified. Among these 156 + 3 (159) misclassified observations, 30 (18.9%) showed a percentage of bloated coral surface ranging from 20% to 80%. In this large interval (20–80%), there were only 197 observations (11.9%) of which 30 were misclassified (15.2%). In the remaining intervals (0–20% and 80–100%) the error was much lower (8.9%). This confirmed the dominant binary character of coral behavior, where polyps are globally open (bloated) or retracted within the calyx (non-bloated) in an almost synchronous mode. An intermediate state (semi-bloated) is only reached for a very limited time, thus confirming the appropriateness of the classification approach. The precision (P) values of the training and test sets were, respectively: 0.999 and 0.721. The recall (R) values of the training and test sets were, respectively: 0.997 and 0.761.
Considering the feature importance analysis, conducted to understand which variable or combination of variables had a higher impact on the constructed MLFN model for coral activity response, the top 10 features were the following: temperature2, salinity2, chlorophyll × salinity, hour × depth/pressure × FOV, hour × salinity, hour × temperature x salinity, depth × salinity, chlorophyll2 × depth, depth3, depth × salinity × FOV.
Due to the structure of the x-block, the contribution of the continuous features (time of day, chlorophyll, temperature, depth, turbidity and salinity) was difficult to assess. On the other hand, given the binary attitude of coral behavior, samples were arranged for plotting in a saturation curve with a sigmoidal shape, to show an activation profile, i.e., the progressive percentage of bloated coral surface versus the growing status of the environmental variable. Figure 14 shows the comparison between the real percentage of bloated coral surface and the sigmoid of each standardized quantitative variable. One should notice that, if sorted accordingly, the standardized variables’ time of the day and chlorophyll showed an increasing linear relationship. As can be seen, the sigmoid of the time of the day (A) and chlorophyll (B) significantly correlate with the bloated status of the coral, r = 0.87 (p < 0.001) and r = 0.99 (p < 0.001), respectively. On the other hand, the sigmoid of the depth (E) and turbidity (F) are significantly anti-correlated with the bloated status of the coral; r = −0.78 (p < 0.001) and r = −0.99 (p < 0.001), respectively. The sigmoid of the temperature (D) is significantly correlated with the bloated status of the coral derivative: r = 0.17 (p < 0.001). The sigmoid of the salinity (C) is not correlated.

4. Discussion

In this study, we proposed a new pipeline for the extraction of biological information referring to the activity status of a sessile organism, the iconic bubblegum deep-sea coral species P. arborea, based on automated segmentation and CNN processing. That pipeline was also implemented in two relevant steps for time series statistic treatment for cabled observatories multidisciplinary data, based on chronobiology and multivariate statistics. This study indicates a new route for the further development of online tools for real-time processing of complexly interrelated multiparametric bio- and oceanographic data sets, which is still missing in most of the data management structures of cabled observatories today. Such an effort is at the core of our future capability to efficiently establish monitoring and surveillance programs with those marine infrastructural assets [24,26].
With the proposed pipeline for information extraction and data processing, we evidenced a tidal-independent and crepuscular activity pattern. A previous work on P. arborea image treatment showed tidal rhythms in polyp activity through the manual analysis of images for activity status classification [23]. Moreover, previous authors used logistic regressions (combined with the Recursive Feature Elimination), to decide which oceanographic variables were significantly linked with coral activity. Here, we used chronobiology-oriented statistical approaches (i.e., periodogram and waveform analyses), combined with an advanced Artificial Neural Network (ANN) modelling statistic. With this data treatment pipeline, we showed that the diurnal coral extrusion rhythm is phased at dusk and it is therefore strongly influenced by the time of the day rather than the tide status. Apparently, the difference in results between both studies can be attributed to increased analytic precision obtained with automated CNN routines in comparison to the manual processing of images.

4.1. Implications of a CNN-Based Pipeline for Ecological Information Treatment at Cabled Observatories

Cabled observatory cameras are still used for temporally scattered studies with no ambition for establishing institutionalized monitoring programs. This is because very few imaging devices are capable of hosting intelligence on board for extracting time series of biological data in a fully autonomous fashion [27,30]. This study indicates a new way of using optical data to link biological processes to the environmental forcing, by using a CNN-based automation procedure in the extraction of biological information from temporally extensive sets of acquired images. The tailored CNN procedure was a simple segmentation tool designed for this local experiment based on a 6-layer classification CNN. For the moment, no comparison studies were made with other segmentation approaches, such as CNN-based or Fully Convolutional based Network (FCN-based) approaches. However, CNN efficiency was tested, showing to be reliable for the objectives described in [77]. The 78.08% test accuracy and 93.47% AUC value are promising results even though user precision and product precision results could be improved with more balance between all classes.
Future studies should include a comparison with state-of-art approaches, such as Mask R-CNN [78] and other FCN architectures [79]. Furthermore, if more public datasets of deep-sea Paragorgia arborea become available, one should try using different colonies in different locations to achieve a more complete and adaptable segmentation tool.
Here, we efficiently used that image filtering and reconstruction procedures prior to CNN treatment. The artificial background illumination within the FOV varied in its intensity over consecutive photos for the flickering of the lighting system and also for the absorption and scattering effects caused by suspended particulate (i.e., turbidity). Many approaches were proposed in the scientific literature for improving the quality of underwater images, e.g., [80,81,82,83]. Restoration methodologies are based on the physical model of the light propagation and are used for correction of the effects of light absorption and scattering [84,85,86]. Contrastingly, enhancement methodologies do not assume any physical model and are based only on computer vision approaches for improving the light and color distribution and for reducing the hazing effects due to suspended particulate load [87,88,89,90].
The variation of the underwater acquisition conditions (e.g., light distribution and water turbidity) and the huge variety of marine fauna with respect to shape and size are the most relevant factors affecting the performance of the automated recognition and classification of underwater visual data [91,92]. All this differentiated information can be successfully managed by the supervised machine learning approach, where a set of examples representing all the information needed for discriminating the relevant specimens, can be used for training of the automated algorithms for recognition and classification [93,94]. The relevant information in visual data (i.e., polyp extrusion and retraction statuses as proxy for activity rhythms) was recognized and classified by combining computer vision and artificial intelligence methodologies after supervised training [95]. That training was efficiently carried out on a set of manually classified quadrants with only a reduced number of images, resulting in a performance that is comparable to that of a visual inspection operated by expert users.
The lack of automation in image treatment is strongly constraining the extraction of information from marine imaging devices [4,92] and, unfortunately, a universal customization is not feasible [27,30]. This situation is impairing the use of remote monitoring by cabled observatories to track changes in the status of ecosystems at high ecological levels of complexity (i.e., animals and the services they provide) [12]. Fortunately, the scientific literature is becoming rich in consolidated and emerging methodologies on computer vision and pattern recognition, e.g., [96,97,98,99]. CNN methodologies are becoming diverse, offering several customized examples of innovation for specific applications and environmental contexts (reviewed in [28]). A more innovative step forward is the evaluation of CNN procedures within the context of entirely automated pipelines for data treatment at cabled observatories, as the one we have drawn here (see Figure 2; [29]). Those pipelines should be considered and debated in relation to the feasibility to create future cyber-infrastructures including advanced digital libraries for storing, accessing and processing the data [24].

4.2. Functional Explanation of the Detected P. arborea Polyps’ Extrusion and Retraction Rhythms

Here, we report P. arborea polyps’ extrusion and retraction rhythms associated to the day-night rather than the tidal cycle, with activity peaking form the second half of the day toward a maximum at dusk. Anthozoan polyp extrusion has been positively correlated with current velocity [100], food availability or water temperature [101]. In deep-sea aphotic realms, activity rhythms relationship with current regimes are due to organic carbon transport [102]. In the LoVe deployment area, strong tidal cycles were supposed to drive the polyp extrusion of the CWC Lophelia pertusa [21]. It has been known for decades that the Lofoten area is a region of high mesoscale activity and associated to the amplification of eddy kinetic energy currents, i.e., [103]. The tidal forces mainly impacting the area are the semidiurnal lunar and solar tides and, although a factor smaller, by the diurnal lunar and solar ones (i.e., [102]). Despite the high variability in the circulation pattern within the area, the impact of hydrodynamics on the short-term behavior of the bubblegum coral seems to be limited, and, because of that, no deeper analysis was conducted on time series of oceanographic data.
Our results suggest a synchronization of Paragorgia polyps’ activity upon solar background illumination rather than tides. High-energy monochromatic blue radiation is still present even at depths greater than 250 m (depending on local turbidity conditions [103]). The benthic community at the LoVe deployment area may therefore be influenced by solar light-driven intensity variations. Our reported polyps’ activity could be related to the 24-h intermittent presence and absence of predators (e.g., pelagic amphipods and euphausiids; [104]) that could be visually lurking on the coral surface at a specific moment of the day-night cycle. Nudibranchs, snails and sea-stars are also predators of deep-sea gorgonian corals [105]. Their presence activity may progressively increase over the seabed at daytime in our study site, affecting Paragorgia’s behavior.
In this scenario, the applied multivariate statistic ANN modelling approach demonstrated a non-linear dependence of Paragorgia polyp extrusion in relation to some combination of oceanographic variables. Bloating was correlated with chlorophyll concentrations over consecutive months as a proxy for seasonality (see Figure 13), which cannot be confirmed here for the lack of longer time series of biological data. Sherwood et al. [106] identified that that a main food source for CWC is fresh phytodetritus. Fresh chlorophyll inputs can be transported to deep-sea canyon areas by tidal pulses on a 24-h basis and high concentrations can be detected below photic and despotic areas [107]. Furthermore, during the study period, polyp extrusion activity (i.e., the bloated status likely associated to filtering) increased from March to June being associated with an incremental increase in chlorophyll concentration. This would agree with previous observations on shallow water octocorals, where polyp extrusion rates seasonally fluctuated depending on overall food inputs [108,109,110].

5. Conclusions

Our results reaffirm that unexplored underwater environments can be analyzed with the help of traditional computer vision approaches combined with deep learning approaches. In doing so, we established a model for the development of highly integrated pipelines for the extraction of biological information from times series images and consequent online treatment of multidisciplinary information at cabled observatory infrastructures. The computer vision approaches for the improvement and enhancement of underwater image quality plays an important role that, when combined with techniques aimed at the classification of relevant subjects, can produce relevant operational advances in the ecological monitoring capability of cabled observatories.

Author Contributions

Conceptualization, J.A., E.R., J.V., S.M.; Data curation, E.R., A.Z., V.L.-V., G.Z., H.W.; Formal analysis, J.A., A.Z., V.L.-V., C.C., L.O., J.G.; Funding acquisition, E.R., H.W.; S.F.; Methodology, J.A., A.Z., V.L.-V., C.C., L.O., S.M.; Project administration, E.R., G.Z., H.W.; Software, A.Z., V.L.-V.; Writing—review and editing. All authors equally contributed. All authors have read and agreed to the published version of the manuscript.

Funding

This project is funded by The Norwegian Research Council, Federal Ministry for Economic Affairs and Energy of Germany (03SX464C) and the Helmholtz Gemeinschaft Deutscher Forschungszentren (HGF) project Modular Observation Solutions for Earth Systems (MOSES), Spanish Centre for the Development of Industrial Technology (EXP 00108707/SERA-20181020), and co-funded by European Union’s Horizon 2020 research and innovation program under the framework of European Research Area Network (ERA-NET) Cofund Maritime and Marine Technologies for a new Era (MarTERA).

Acknowledgments

We are grateful to our funding sources for supporting our consortium and our efforts. J.A. is a member of the Tecnoterra Associated Research Unit established between the Consejo Superior de Investigaciones Cientificas (CSIC) and the Universidad Politécnica de Cataluña (UPC).

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Danovaro, R.; Aguzzi, J.; Fanelli, E.; Billet, D.; Gjerde, K.; Jamieson, A.; Ramirez-Llodra, E.; Smith, C.R.; Snelgrove, P.V.R.; Thomsen, L.; et al. A new international ecosystem-based strategy for the global deep ocean. Science 2017, 355, 452–454.3. [Google Scholar] [CrossRef] [PubMed]
  2. Thurber, A.R.; Sweetman, A.K.; Narayanaswamy, B.E.; Jones, D.O.; Ingels, J.; Hansman, R. Ecosystem function and services provided by the deep sea. Biogeosciences 2014, 11, 3941–3963. [Google Scholar] [CrossRef] [Green Version]
  3. Danovaro, R.; Fanelli, E.; Aguzzi, J.; Billett, D.; Carugati, L.; Corinaldesi, C.; Dell’Anno, A.; Gjerde, K.; Jamieson, A.J.; Kark, S.; et al. Ecological indicators for an integrated global deep-ocean strategy. Nat. Ecol. Evol. 2020, 4, 181–192. [Google Scholar] [CrossRef] [PubMed]
  4. Aguzzi, J.; Chatzievangelou, D.; Marini, S.; Fanelli, E.; Danovaro, R.; Flögel, S.; Lebris, N.; Juanes, F.; De Leo, F.C.; Del Rio, J.; et al. New high-tech flexible networks for the monitoring of deep-sea ecosystems. Environ. Sci. Technol. 2019, 53, 6616–6631. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  5. Aguzzi, J.; Albiez, J.; Flögel, S.; Godø, O.R.; Grimsbø, E.; Marini, S.; Pfannkuche, O.; Rodriguez, E.; Thomsen, L.; Torkelsen, T.; et al. A Flexible autonomous robotic observatory infrastructure for bentho-pelagic monitoring. Sensors 2020, 20, 1614. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  6. Rountree, R.A.; Aguzzi, J.; Marini, S.; Fanelli, E.; De Leo, F.C.; del Río Fernandez, J.; Juanes, F. Towards an optimal design for ecosystem-level ocean observatories. Oceanog. Mar. Biol. Ann. Rev. 2020, 58, 79–106. [Google Scholar]
  7. Seibel, B.A.; Drazen, J.C. The rate of metabolism in marine animals: Environmental constraints, ecological demands and energetic opportunities. Philos. Trans. R. Soc. B 2007, 362, 2061–2078. [Google Scholar] [CrossRef] [Green Version]
  8. Wagner, H.-J.; Kemp, K.; Mattheus, U.; Priede, I. Rhythms at the bottom of the deep sea: Cyclic current flow changes and melatonin patterns in two species of demersal fish. Deep Sea Res. I 2007, 54, 1944–1956. [Google Scholar] [CrossRef]
  9. Naylor, E. Chronobiology of Marine Organisms; Cambridge University Press: Cambridge, UK, 2010. [Google Scholar]
  10. Aguzzi, J.; Company, J.B. Chronobiology of deep-water decapod crustaceans on continental margins. Adv. Mar. Biol. 2010, 58, 155–225. [Google Scholar]
  11. Aguzzi, J.; Costa, C.; Menesatti, P.; García, J.A.; Bahamon, N.; Puig, P.; Sarda, F. Activity rhythms in the deep-sea: A chronobiological approach. Front. Biosci. Landmark 2011, 16, 131–150. [Google Scholar] [CrossRef] [Green Version]
  12. Aguzzi, J.; Costa, C.; Matabos, M.; Azzurro, E.; Lázaro, A.; Menesatti, P.; Sardà, F.; Canals, M.; Delory, E.; Cline, D.; et al. Challenges to the assessment of benthic populations and biodiversity as a result of rhythmic behaviour: Video solutions from cabled observatories. Oceanog. Mar. Biol. Ann. Rev. 2012, 50, 235–286. [Google Scholar]
  13. Aguzzi, J.; Fanelli, E.; Ciuffardi, T.; Schirone, A.; Craig, G.; the KM3NeT Consortium. Inertial bioluminescence rhythms at the Central Mediterranean KM3NeT deep-sea neutrino telescope. Sci. Rep. 2017, 7, 44938. [Google Scholar] [CrossRef] [Green Version]
  14. Hays, G.C.; Ferreira, L.C.; Sequeira, A.M.; Meekan, M.G.; Duarte, C.M.; Bailey, H.; Bailleul, F.; Bowen, W.D.; Caley, M.J.; Costa, D.P.; et al. Key questions in marine megafauna movement ecology. Trends Ecol. Evol. 2016, 31, 463–475. [Google Scholar] [CrossRef] [Green Version]
  15. Milligan, R.; Scott, E.; Jones, D.; Bett, B.; Jamieson, A.; O’Brien, R.; Costa, S.P.; Rowe, G.; Ruhl, H.; Smith, K.; et al. Evidence for seasonal cycles in deep-sea fish abundances: A great migration in the deep SE Atlantic? J. Anim. Ecol. 2020, in press. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  16. Aguzzi, J.; Fanelli, E.; Ciuffardi, T.; Schirone, A.; De Leo, F.C.; Doya, C.; Kawato, M.; Miyazaki, M.; Furushima, Y.; Costa, C.; et al. Faunal activity rhythms influencing early community succession of an implanted whale carcass offshore in Sagami Bay, Japan. Sci. Rep. 2018, 8, 11163. [Google Scholar] [CrossRef] [Green Version]
  17. De Leo, F.C.; Ogata, B.; Sastri, A.R.; Heesemann, M.; Mihály, S.; Galbraith, M.; Morley, M.G. High-frequency observations from a deep-sea cabled observatory reveal seasonal overwintering of Neocalanus spp. in Barkley Canyon, NE Pacific: Insights into particulate organic carbon flux. Prog. Oceanogr. 2018, 169, 120–137. [Google Scholar] [CrossRef] [Green Version]
  18. Tsounis, G.; Orejas, C.; Reynaud, S.; Gili, J.M.; Allemand, D.; Ferrier-Pagès, C. Prey-capture rates in four Mediterranean cold water corals. Mar. Ecol. Prog. Ser. 2010, 398, 149–155. [Google Scholar] [CrossRef]
  19. Boolukos, C.M.; Lim, A.; O’Riordan, R.M.; Wheeler, A.J. Cold-water corals in decline: A temporal (4 year) species abundance and biodiversity appraisal of complete photomosaiced cold-water coral reef on the Irish Margin. Deep Sea Res. I 2019, 146, 44–54. [Google Scholar] [CrossRef]
  20. Bøe, R.; Bellec, V.K.; Dolan, M.F.J.; Buhl-Mortensen, P.; Buhl-Mortensen, L.; Slagstad, D.; Rise, L. Giant sandwaves in the Hola glacial trough off Vesterålen, North Norway. Mar. Geol. 2009, 267, 36–54. [Google Scholar] [CrossRef]
  21. Osterloff, J.; Nilssen, I.; Järnegren, J.; Van Engeland, T.; Buhl-Mortensen, P.; Nattkemper, T.W. Computer vision enables short-and long-term analysis of Lophelia pertusa polyp behaviour and colour from an underwater observatory. Sci. Rep. 2020, 9, 1–12. [Google Scholar] [CrossRef] [PubMed]
  22. Nunes, J.A.C.C.; Cruz, I.C.S.; Nunes, A.; Pinheiro, T.H. Speeding up coral reef conservation with AI-aided automated image analysis. Nat. Mach. Intell. 2020, 2, 292. [Google Scholar] [CrossRef]
  23. Johanson, A.N.; Flögel, S.; Dullo, W.-C.; Linke, P.; Hasselbring, W. Modeling polyp activity of Paragorgia arborea using supervised learning. Ecol. Informat. 2017, 39, 109–118. [Google Scholar] [CrossRef]
  24. Aguzzi, J.; Chatzievangelou, D.; Francescangeli, M.; Marini, S.; Bonofiglio, F.; del Río, J.; Danovaro, R. The hierarchic treatment of marine ecological information from spatial networks of benthic platforms. Sensors 2020, 20, 1751. [Google Scholar] [CrossRef] [Green Version]
  25. Aguzzi, J.; Doya, C.; Tecchio, S.; De Leo, F.L.; Azzurro, E.; Costa, C.; Sbragaglia, V.; Del Río, J.; Navarro, J.; Ruhl, H.A.; et al. Coastal observatories for monitoring of fish behaviour and their responses to environmental changes. Rev. Fish Biol. Fisher. 2015, 25, 463–483. [Google Scholar] [CrossRef] [Green Version]
  26. Cuvelier, D.; de Busserolles, F.; Lavaud, R.; Floc’h, E.; Fabri, M.-C.; Sarradin, P.-M.; Sarrazin, J. Biological data extraction from imagery: How far can we go? A case study from the Mid-Atlantic Ridge. Mar. Environ. Res. 2012, 82, 15–27. [Google Scholar] [CrossRef] [Green Version]
  27. Corgnati, L.; Marini, S.; Mazzei, L.; Ottaviani, E.; Aliani, S.; Conversi, A.; Griffa, A. Looking inside the ocean: Toward an autonomous imaging system for monitoring gelatinous zooplankton. Sensors 2016, 16, 2124. [Google Scholar] [CrossRef] [Green Version]
  28. Marini, S.; Fanelli, E.; Sbragaglia, V.; Azzurro, E.; Fernandez, J.D.R.; Aguzzi, J. Tracking fish abundance by underwater image recognition. Sci. Rep. 2018, 8, 1–12. [Google Scholar] [CrossRef] [Green Version]
  29. Lopez-Vazquez, V.; Lopez-Guede, J.M.; Marini, S.; Fanelli, E.; Johnsen, E.; Aguzzi, J. Video image enhancement and machine learning pipeline for underwater animal detection and classification at cabled observatories. Sensors 2020, 20, 726. [Google Scholar] [CrossRef] [Green Version]
  30. Marini, S.; Corgnati, L.; Mantovani, C.; Bastianini, M.; Ottaviani, E.; Fanelli, E.; Aguzzi, J.; Griffa, A.; Poulain, P.-M. Automated estimate of fish abundance through the autonomous imaging device GUARD1. Measurement 2018, 126, 72–75. [Google Scholar] [CrossRef]
  31. Papworth, D.J.; Marini, S.; Conversi, A. A novel, unbiased analysis approach for investigating population dynamics: A case study on Calanus finmarchicus and its Decline in the North Sea. PLoS ONE 2020, 11, e0158230. [Google Scholar] [CrossRef]
  32. Auster, P.J. Are deep-water corals important habitats for fishes? In Cold-Water Corals and Ecosystems; Springer: Berlin/Heidelberg, Germany, 2005; pp. 747–760. [Google Scholar]
  33. Fosså, J.H.; Skjoldal, H.R. Conservation of Cold-Water Coral Reefs in Norway. In Handbook of Marine Fisheries Conservation and Management; Oxford University Press: New York, NY, USA, 2010. [Google Scholar]
  34. Bright-Diaz, L.M.; Strychar, K.B.; Shirley, T.C. Compounds from deep-sea bubblegum corals, Paragorgia arborea, elicit anti-predation behavior in fish. Open Mar. Biol. J. 2011, 5, 58–67. [Google Scholar]
  35. Roberts, J.M.; Wheeler, A.J.; Freiwald, A. Reefs of the deep: The biology and geology of cold-water coral ecosystems. Science 2006, 312, 543–547. [Google Scholar] [CrossRef] [Green Version]
  36. The Lofoten Vesterålen Ocean Observatory Website. Available online: https://love.equinor.com/ (accessed on 8 July 2020).
  37. Godø, O.R.; Johnsen, S.; Torkelsen, T. The LoVe Ocean Observatory is in Operation. Mar. Technol. Soc. J. 2014, 48, 24–30. [Google Scholar] [CrossRef]
  38. Osterloff, J.; Nilssen, I.; Nattkemper, T.W. A computer vision approach for monitoring the spatial and temporal shrimp distribution at the LoVe observatory. Methods Oceanog. 2016, 15, 114–128. [Google Scholar] [CrossRef]
  39. Purser, A. A time series study of Lophelia pertusa and reef megafauna responses to drill cuttings exposure on the Norwegian margin. PLoS ONE 2015, 10, 1–29. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  40. Herrera, S.; Shank, T.; Sánchez, J. Spatial and temporal patterns of genetic variation in the widespread antitropical deep-sea coral Paragorgia arborea. Mol. Ecol. 2012, 21, 6053–6067. [Google Scholar] [CrossRef] [PubMed]
  41. Sarmiento, J.L.; Gruber, N. Ocean Biogeochemical Dynamics; Princeton University Press: Princeton, NJ, USA, 2006. [Google Scholar]
  42. Tendal, O.S. The north Atlantic distribution of the octocoral Paragorgia arborea (L., 1758) (Cnidaria, Anthozoa). Sarsia 1992, 77, 213–217. [Google Scholar] [CrossRef]
  43. Buhl-Mortensen, L.; Olafsdottir, S.H.; Buhl-Mortensen, P.; Burgos, J.M.; Ragnarsson, S.A. Distribution of nine cold-water coral species (Scleractinia and Gorgonacea) in the cold temperate North Atlantic: Effects of bathymetry and hydrography. Hydrobiologia 2015, 759, 39–61. [Google Scholar] [CrossRef]
  44. Watanabe, S.; Metaxas, A.; Sameoto, J.; Lawton, P. Patterns in abundance and size of two deep-water gorgonian octocorals, in relation to depth and substrate features off Nova Scotia. Deep Sea Res. I 2009, 56, 2235–2248. [Google Scholar] [CrossRef]
  45. Mortensen, P.B.; Buhl-Mortensen, L. Morphology and growth of the deep-water gorgonians Primnoa resedaeformis and Paragorgia arborea. Mar. Biol. 2005, 147, 775–788. [Google Scholar] [CrossRef]
  46. Tong, R.; Purser, A.; Unnithan, V.; Guinan, J. Multivariate statistical analysis of distribution of deep-water gorgonian corals in relation to seabed topography on the Norwegian margin. PLoS ONE 2012, 7, e43534. [Google Scholar] [CrossRef]
  47. Bennecke, S.; Metaxas, A. Is substrate composition a suitable predictor for deep-water coral occurrence on fine scales? Deep Sea Res. I 2017, 124, 55–65. [Google Scholar] [CrossRef]
  48. Lundsten, L.; Barry, J.P.; Cailliet, G.M.; Clague, D.A.; DeVogelaere, A.P.; Geller, J.B. Benthic invertebrate communities on three seamounts off southern and central California, USA. Mar. Ecol. Prog. Ser. 2009, 374, 23–32. [Google Scholar] [CrossRef] [Green Version]
  49. Brooke, S.; Watts, M.; Heil, A.; Rhode, M.; Mienis, F.; Duineveld, G.; Davies, A.; Ross, S. Distributions and habitat associations of deep-water corals in Norfolk and Baltimore Canyons, Mid-Atlantic Bight, USA. Deep Sea Res. II 2017, 137, 131–147. [Google Scholar] [CrossRef] [Green Version]
  50. Sánchez, J.A. Systematics of the bubblegum corals (Cnidaria: Octocorallia: Paragorgiidae) with description of new species from New Zealand and the Eastern Pacific. Zootaxa 2005, 1014, 1–72. [Google Scholar] [CrossRef]
  51. Buhl-Mortensen, L.; Mortensen, P.B. Distribution and diversity of species associated with deep-sea gorgonian corals off Atlantic Canada. In Cold-water Corals and Ecosystems; Springer: Berlin/Heidelberg, Germany, 2005; pp. 849–879. [Google Scholar]
  52. De Clippele, L.; Buhl-Mortensen, P.; Buhl-Mortensen, L. Fauna associated with cold water gorgonians and sea pens. Cont. Shelf Res. 2015, 105, 67–78. [Google Scholar] [CrossRef]
  53. Krieger, K. Distribution and abundance of rockfish determined from a submersible and by bottom trawling. Fish. Bull. 1993, 91, 87–96. [Google Scholar]
  54. Sherwood, O.A.; Edinger, E.N. Ages and growth rates of some deep-sea gorgonian and antipatharian corals of Newfoundland and Labrador. Can. J. Fish. Aquat. Sci. 2009, 66, 142–152. [Google Scholar] [CrossRef]
  55. Bennecke, S.; Kwasnitschka, T.; Metaxas, A.; Dullo, W.-C. In situ growth rates of deep-water octocorals determined from 3D photogrammetric reconstructions. Coral Reefs 2016, 35, 1227–1239. [Google Scholar] [CrossRef]
  56. Purser, A.; Orejas, C.; Moje, A.; Thomsen, L. The influence of flow velocity and suspended particulate concentration on net prey capture rates by the scleractinian coral Balanophyllia europaea (Scleractinia: Dendrophylliidae). J. Mar. Biol. Assoc. UK 2014, 94, 687–696. [Google Scholar] [CrossRef]
  57. Bradbury, R.R.; Reichelt, R.R. Fractal dimension of a coral reef at ecological scales. Mar. Ecol. Prog. Ser. 1983, 10, 169–171. [Google Scholar] [CrossRef]
  58. Basillais, É. Coral surfaces and fractal dimensions: A new method. Comptes Rendus Académie Sci. Ser. III (Sci. Vie) 1997, 320, 653–657. [Google Scholar] [CrossRef]
  59. Martin-Garin, B.; Lathuilière, B.; Verrecchia, E.P.; Geister, J. Use of fractal dimensions to quantify coral shape. Coral Reefs 2007, 26, 541–550. [Google Scholar] [CrossRef] [Green Version]
  60. Reichert, J.; Backes, A.R.; Schubert, P.; Wilke, T. The power of 3D fractal dimensions for comparative shape and structural complexity analyses of irregularly shaped organisms. Methods Ecol. Evol. 2017, 8, 1650–1658. [Google Scholar] [CrossRef]
  61. George, E.E.; Mullinix, J.; Meng, F.; Bailey, B.; Edwards, C.; Felts, B.; Haas, A.; Hartmann, A.C.; Mueller, B.; Nulton, J.; et al. Relevance of coral geometry in the outcomes of the coral-algal benthic war. BioRxiv 2018, 327031. [Google Scholar] [CrossRef]
  62. Chen, T.-W.; Chen, Y.-L.; Chien, S.-Y. Fast image segmentation based on K-Means clustering with histograms in HSV color space. In Proceedings of the 2008 IEEE 10th Workshop on Multimedia Signal Processing, Cairns, Australia, 8–10 October 2008; pp. 322–325. [Google Scholar]
  63. Huang, T.; Yang, G.; Tang, G. A fast two-dimensional median filtering algorithm. IEEE Trans. Acoust. Speech Sig. Proc. 1979, 27, 13–18. [Google Scholar] [CrossRef] [Green Version]
  64. Xi, Y. Available online: https://github.com/yuxi120407/Coral-reef-image-segmentation-and-classification (accessed on 8 July 2020).
  65. Alonso, I.; Cambra, A.; Muñoz, A.; Treibitz, T.; Murillo, A.C. Coral-segmentation: Training dense labeling models with sparse ground truth. In Proceedings of the IEEE International Conference on Computer Vision Workshops, Venice, Italy, 22–29 October 2017; pp. 2874–2882. [Google Scholar]
  66. Cholltet, F. Available online: https://github.com/fchollet/keras (accessed on 8 July 2020).
  67. De Boer, P.T.; Kroese, D.P.; Mannor, S.; Rubinstein, R.Y. A tutorial on the cross-entropy method. Ann. Oper. Res. 2015, 134, 19–67. [Google Scholar] [CrossRef]
  68. Kohavi, R. A study of cross-validation and bootstrap for accuracy estimation and model selection. Internat. Joint Conf. Artificial Intel. (IJCAI) 1995, 14, 1137–1145. [Google Scholar]
  69. Wang, S.; Chen, W.; Xie, S.M.; Azzari, G.; Lobell, D.B. Weakly supervised deep learning for segmentation of remote sensing imagery. Remote Sens. 2020, 12, 207. [Google Scholar] [CrossRef] [Green Version]
  70. Refinetti, R. Circadian Physiology; CRC Press: Boca Raton, FL, USA, 2019. [Google Scholar]
  71. Tsounis, G.; Rossi, S.; Laudien, J.; Bramanti, L.; Fernandez, N.; Gili, J.M.; Arntz, W. Diet and seasonal prey capture rates in the Mediterranean red coral (Corallium rubrum L.). Mar. Biol. 2006, 149, 313–325. [Google Scholar] [CrossRef]
  72. Mossalam, A.; Arafa, M. Using Artificial Neural Networks (ANN) in projects monitoring dashboards’ formulation. HBRC J. 2017, 14, 385–392. [Google Scholar] [CrossRef]
  73. Proto, A.R.; Sperandio, G.; Costa, C.; Maesano, M.; Antonucci, F.; Macrì, G.; Scarascia, S.; Mugnozza, G.; Zimbalatti, G. A three-step neural network artificial intelligence modelling approach for time, productivity and costs prediction: A case study in Italian forestry. Croat. J. Forest Eng. 2020, 41, 35–47. [Google Scholar] [CrossRef]
  74. Mackay, D.J.C. Bayesian interpolation. Neural Computat. 1992, 4, 415–447. [Google Scholar] [CrossRef]
  75. Foresee, F.D.; Martin, T.H. Gauss-Newton approximation to Bayesian learning. In Proceedings of the International Joint Conference on Neural Networks (ICNN’97), Houston, TX, USA, 12 June 1997; Volume 3, pp. 1930–1935. [Google Scholar]
  76. Violino, S.; Ortenzi, L.; Antonucci, F.; Pallottino, F.; Benincasa, C.; Figorilli, S.; Costa, C. An open source IoT spectrometer for Italian EVOO traceability and origin assessment: An artificial intelligence approach. Foods 2020, in press. [Google Scholar] [CrossRef]
  77. Fawcett, T. An introduction to ROC analysis. Pattern Recogn. Lett. 2006, 27, 861–874. [Google Scholar] [CrossRef]
  78. He, K.; Gkioxari, G.; Dollár, P.; Girshick, R. Mask r-cnn. In Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy, 22–29 October 2017; pp. 2961–2969. [Google Scholar]
  79. Long, J.; Shelhamer, E.; Darrell, T. Fully convolutional networks for semantic segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA, 7–12 June 2015; pp. 3431–3440. [Google Scholar]
  80. Schettini, R.; Corchs, S. Underwater image processing: State of the art of restoration and image enhancement methods. EURASIP J. Adv. Signal Process. 2010, 1–14. [Google Scholar] [CrossRef] [Green Version]
  81. Bonin, F.; Burguera, A.; Oliver, G. Imaging systems for advanced underwater vehicles. J. Marit. Res. 2011, 8, 65–86. [Google Scholar]
  82. Lu, H.; Li, Y.; Zhang, Y.; Chen, M.; Serikawa, S.; Kim, H. Underwater optical image processing: A comprehensive review. Mob. Netw. Appl. 2017, 22, 1204–1211. [Google Scholar] [CrossRef]
  83. Lu, H.; Li, Y.; Serikawa, S. Computer vision for ocean observing. In Artificial Intelligence and Computer Vision; Springer: Berlin/Heidelberg, Germany, 2017; pp. 1–16. [Google Scholar]
  84. Chiang, J.Y.; Chen, Y.-C. Underwater image enhancement by wavelength compensation and dehazing. IEEE Trans. Image Process. 2011, 21, 1756–1769. [Google Scholar] [CrossRef]
  85. Treibitz, T.; Schechner, Y.Y. Turbid scene enhancement using multi-directional illumination fusion. IEEE Trans. Image Process. 2012, 21, 4662–4667. [Google Scholar] [CrossRef]
  86. Peng, Y.-T.; Cosman, P.C. Underwater image restoration based on image blurriness and light absorption. IEEE Trans. Image Process. 2017, 26, 1579–1594. [Google Scholar] [CrossRef] [PubMed]
  87. Andono, P.N.; Purnama, I.; Hariadi, M. Underwater image enhancement using adaptive filtering for enhanced sift-based image matching. J. Theor. Appl. Inf. Technol. 2013, 51, 392–399. [Google Scholar]
  88. Lu, H.; Li, Y.; Zhang, L.; Serikawa, S. Contrast enhancement for images in turbid water. J. Opt. Soc. Am. 2015, 32, 886–893. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  89. Guo, Y.; Liu, Y.; Oerlemans, A.; Lao, S.; Wu, S.; Lew, M.S. Deep learning for visual understanding: A review. Neurocomputing 2016, 187, 27–48. [Google Scholar] [CrossRef]
  90. Ancuti, C.O.; Ancuti, C.; De Vleeschouwer, C.; Bekaert, P. Color balance and fusion for underwater image enhancement. IEEE Trans. Image Proc. 2018, 27, 379–393. [Google Scholar] [CrossRef] [Green Version]
  91. Schoening, T.; Bergmann, M.; Ontrup, J.; Taylor, J.; Dannheim, J.; Gutt, J.; Purser, A.; Nattkemper, T.W. Semi-automated image analysis for the assessment of megafaunal densities at the Arctic deep-sea observatory HAUSGARTEN. PLoS ONE 2012, 7, e38179. [Google Scholar] [CrossRef] [Green Version]
  92. Bicknell, A.W.; Godley, B.J.; Sheehan, E.V.; Votier, S.C.; Witt, M.J. Camera technology for monitoring marine biodiversity and human impact. Front. Ecol. Environ. 2016, 14, 424–432. [Google Scholar] [CrossRef]
  93. Dai, J.; Wang, R.; Zheng, H.; Ji, G.; Qiao, X. ZooplanktoNet: Deep convolutional network for zooplankton classification. In Proceedings of the OCEANS 2016-Shanghai; IEEE: Piscataway, NJ, USA, 2016; pp. 1–6. [Google Scholar]
  94. Qin, H.; Li, X.; Liang, J.; Peng, Y.; Zhang, C. DeepFish: Accurate underwater live fish recognition with a deep architecture. Neurocomputing 2016, 187, 49–58. [Google Scholar] [CrossRef]
  95. Jordan, M.I.; Mitchell, T.M. Machine learning: Trends, perspectives, and prospects. Science 2015, 349, 255–260. [Google Scholar] [CrossRef]
  96. Bishop, C.M. Pattern recognition and machine learning. In Information Science and Statistics; Springer: Secaucus, NJ, USA, 2006. [Google Scholar]
  97. Galar, M.; Fernández, A.; Barrenechea, E.; Bustince, H.; Herrera, F. An overview of ensemble methods for binary classifiers in multi-class problems: Experimental study on one-vs-one and one-vs-all schemes. Pattern Recog. 2011, 44, 1761–1776. [Google Scholar] [CrossRef]
  98. Dougherty, G. Pattern Recognition and Classification: An Introduction; Springer Science & Business Media: Berlin/Heidelberg, Germany, 2012. [Google Scholar]
  99. Cruz, R.M.; Sabourin, R.; Cavalcanti, G.D. Dynamic classifier selection: Recent advances and perspectives. Inf. Fusion 2018, 41, 195–216. [Google Scholar] [CrossRef]
  100. Bell, J.J.; Shaw, C.; Turner, J.R. Factors controlling the tentacle and polyp expansion behaviour of selected temperate Anthozoa. J. Mar. Biol. Assoc. UK 2006, 86, 977. [Google Scholar] [CrossRef]
  101. Rossi, S.; Rizzo, L.; Duchêne, J.C. Polyp expansion of passive suspension feeders: A red coral case study. PeerJ 2019, 7, e7076. [Google Scholar] [CrossRef]
  102. Davies, A.J.; Duineveld, G.C.; Lavaleye, M.S.S.; Bergman, M.J.N.; van Haren, H.; Roberts, J.M. Downwelling and deep-water bottom currents as food supply mechanisms to the cold-water coral Lophelia pertusa (Scleractinia) at the Mingulay Reef Complex. Limnol. Oceanog. 2009, 54, 620–629. [Google Scholar] [CrossRef] [Green Version]
  103. Herring, P. The Biology of the Deep Ocean; Oxford University Press: Oxford, UK, 2002. [Google Scholar]
  104. Belonenko, T.V.; Volkov, D.L.; Norden, Y.E.; Ozhigin, V.K. Water circulation in the Lofoten Basin of the Norwegian Sea. Vestn. St.-Peterbg. Univ. Ser. Geol. Geogr. 2014, 2, 108–112. [Google Scholar]
  105. Krieger, K.J.; Wing, B.L. Megafauna associations with deep-water corals (Primnoa spp.) in the Gulf of Alaska. Hydrobiologia 2002, 471, 83–90. [Google Scholar] [CrossRef]
  106. Sherwood, O.A.; Jamieson, R.E.; Edinger, E.N.; Wareham, V.E. Stable C and N isotopic composition of cold-water corals from the Newfoundland and Labrador continental slope: Examination of trophic, depth and spatial effects. Deep Sea Res. I 2008, 55, 1392–1402. [Google Scholar] [CrossRef]
  107. Thomsen, L.; Aguzzi, J.; Costa, C.; De Leo, F.; Ogston, A.; Purser, A. The oceanic biological pump: Rapid carbon transfer to depth at continental margins during winter. Sci. Rep. 2017, 7, 1–10. [Google Scholar] [CrossRef] [Green Version]
  108. Garrabou, J. Life-history traits of Alcyonium acaule and Parazoanthus axinellae (Cnidaria, Anthozoa), with emphasis on growth. Mar. Ecol. Prog. Ser. 1999, 178, 193–204. [Google Scholar] [CrossRef]
  109. Rossi, S. Environmental factors affecting the trophic ecology of benthic suspension feeders. Ph.D. Thesis, Department of Ecology, University of Barcelona, Barcelona, Spain, 2002. [Google Scholar]
  110. Rossi, S.; Coppari, M.; Viladrich, N. Benthic-Pelagic coupling: New perspectives in the animal forests. In Marine Animal Forests; Rossi, S., Bramanti, L., Gori, A., Orejas Saco del Valle, C., Eds.; Springer: Cham, Switzerland, 2016; pp. 855–885. [Google Scholar]
Sensors 20 06281 g001 550
Figure 1. Overview of the Lofoten-Vesterålen (LoVe) canyon area (from Aguzzi et al. [6]). (A) The bathymetric map with the extended cabled transect with nodes (black dots) numbered 1–7 for shallower (light blue) to deeper realms (dark blue) and connected by a marine cable (continuous line). Node 1 is located in the shallowest part of the trough (orange areas represent the cold water coral (CWC) reefs). (B) A 3D detailed representation of the area around Node 1, where satellite no. 2 with camera appears on top of a small mound (encircled in red). (C) Enlarged view of the areas surrounding the satellite #2, where Paragorgia colonies are schematized.
Figure 1. Overview of the Lofoten-Vesterålen (LoVe) canyon area (from Aguzzi et al. [6]). (A) The bathymetric map with the extended cabled transect with nodes (black dots) numbered 1–7 for shallower (light blue) to deeper realms (dark blue) and connected by a marine cable (continuous line). Node 1 is located in the shallowest part of the trough (orange areas represent the cold water coral (CWC) reefs). (B) A 3D detailed representation of the area around Node 1, where satellite no. 2 with camera appears on top of a small mound (encircled in red). (C) Enlarged view of the areas surrounding the satellite #2, where Paragorgia colonies are schematized.
Sensors 20 06281 g001
Sensors 20 06281 g002 550
Figure 2. Pipeline for biological data extraction (via automated video-imaging procedures, based on Convolutional Neural Network (CNN) groundthruted routines) and analysis (via multivariate time-series treatment) of high relevance for advanced ecological monitoring functionalities by cabled observatories. Numbers on the upper right corner of each box (i.e., the treatment pipeline step) are referring to the numerals for the respective Materials and Methods section of this paper.
Figure 2. Pipeline for biological data extraction (via automated video-imaging procedures, based on Convolutional Neural Network (CNN) groundthruted routines) and analysis (via multivariate time-series treatment) of high relevance for advanced ecological monitoring functionalities by cabled observatories. Numbers on the upper right corner of each box (i.e., the treatment pipeline step) are referring to the numerals for the respective Materials and Methods section of this paper.
Sensors 20 06281 g002
Sensors 20 06281 g003 550
Figure 3. Different activity statuses for the coral as bloated (left; for polyp extrusion) and non-bloated (right; polyp retraction), as revealed after Contrast Limited Adaptive Histogram Equalization (CLAHE) image enhancement.
Figure 3. Different activity statuses for the coral as bloated (left; for polyp extrusion) and non-bloated (right; polyp retraction), as revealed after Contrast Limited Adaptive Histogram Equalization (CLAHE) image enhancement.
Sensors 20 06281 g003
Sensors 20 06281 g004 550
Figure 4. Different activity statuses of the coral as bloated (left), semi-bloated (middle) and non-bloated (right) for comparison. Those types of rectangular enlargements were selected in each image for the manual tagging of the behavioral status when establishing the training set.
Figure 4. Different activity statuses of the coral as bloated (left), semi-bloated (middle) and non-bloated (right) for comparison. Those types of rectangular enlargements were selected in each image for the manual tagging of the behavioral status when establishing the training set.
Sensors 20 06281 g004
Sensors 20 06281 g005 550
Figure 5. Red-segmented coral output and the image binarization according to the used filters.
Figure 5. Red-segmented coral output and the image binarization according to the used filters.
Sensors 20 06281 g005
Sensors 20 06281 g006 550
Figure 6. Output of coral section segmentation (A) and binarization with median filter (B).
Figure 6. Output of coral section segmentation (A) and binarization with median filter (B).
Sensors 20 06281 g006
Sensors 20 06281 g007 550
Figure 7. Structure of the proposed CNN.
Figure 7. Structure of the proposed CNN.
Sensors 20 06281 g007
Sensors 20 06281 g008 550
Figure 8. Examples of coral segmentation indicting activity status (above) and original images (below). Non-bloated areas are in red and bloated ones are in blue. Purple areas are in the transient semi-bloated condition.
Figure 8. Examples of coral segmentation indicting activity status (above) and original images (below). Non-bloated areas are in red and bloated ones are in blue. Purple areas are in the transient semi-bloated condition.
Sensors 20 06281 g008
Sensors 20 06281 g009 550
Figure 9. Time series of automated percentages of colony surface with polyps in full extrusion (bloated; (A)) or full retraction (non-bloated; (C)) status as well as the intermediate condition with patches of both (semi-bloated; (B)), plus the concomitantly monitored oceanographic variables for comparison (D).
Figure 9. Time series of automated percentages of colony surface with polyps in full extrusion (bloated; (A)) or full retraction (non-bloated; (C)) status as well as the intermediate condition with patches of both (semi-bloated; (B)), plus the concomitantly monitored oceanographic variables for comparison (D).
Sensors 20 06281 g009
Sensors 20 06281 g010 550
Figure 10. Waveform analysis outputs as averaged percentages for colony surface with polyps in full extrusion (bloated) or full retraction (non-bloated) as well as the intermediate condition (semi-bloated) are reported with the standard deviation. A marked unimodal increase in bloated status (i.e., possibly corresponding to filter feeding activity) is evident in antiphase with non-bloated and semi-bloated statuses (see the MESOR threshold values in Table 3). On should notice that waveforms Y-axes have different scales to make time series pattern as visually comparable. The gray shaded area represents the hours of darkness.
Figure 10. Waveform analysis outputs as averaged percentages for colony surface with polyps in full extrusion (bloated) or full retraction (non-bloated) as well as the intermediate condition (semi-bloated) are reported with the standard deviation. A marked unimodal increase in bloated status (i.e., possibly corresponding to filter feeding activity) is evident in antiphase with non-bloated and semi-bloated statuses (see the MESOR threshold values in Table 3). On should notice that waveforms Y-axes have different scales to make time series pattern as visually comparable. The gray shaded area represents the hours of darkness.
Sensors 20 06281 g010
Sensors 20 06281 g011 550
Figure 11. Waveform analysis outputs of oceanographic data, where all hourly averaged values are presented with their standard deviation.
Figure 11. Waveform analysis outputs of oceanographic data, where all hourly averaged values are presented with their standard deviation.
Sensors 20 06281 g011
Sensors 20 06281 g012 550
Figure 12. Seasonal pattern of variation in P. arborea polyps’ activity and concomitantly measured oceanographic variables. (A) Monthly average percentages of different polyp activity statuses. (B) Monthly averaged oceanographic variables. All data are presented with the standard deviation.
Figure 12. Seasonal pattern of variation in P. arborea polyps’ activity and concomitantly measured oceanographic variables. (A) Monthly average percentages of different polyp activity statuses. (B) Monthly averaged oceanographic variables. All data are presented with the standard deviation.
Sensors 20 06281 g012
Sensors 20 06281 g013 550
Figure 13. Root Mean Square (RMS) error, as a function of the number of variables for training, test and validation sets. The results were obtained averaging over 10 random trials.
Figure 13. Root Mean Square (RMS) error, as a function of the number of variables for training, test and validation sets. The results were obtained averaging over 10 random trials.
Sensors 20 06281 g013
Sensors 20 06281 g014 550
Figure 14. Samples sorted by increasing percentage of bloated coral status (saturation blue lines curves). The sigmoidal function of the centered and standardized oceanographic variables (red lines) are: (A) time of the day, (B) chlorophyll, (C) salinity, (D) temperature, (E) depth, and (F) turbidity. Before processing the oceanographic standardized variables with the sigmoidal function, a moving average was applied in order to smooth their signal.
Figure 14. Samples sorted by increasing percentage of bloated coral status (saturation blue lines curves). The sigmoidal function of the centered and standardized oceanographic variables (red lines) are: (A) time of the day, (B) chlorophyll, (C) salinity, (D) temperature, (E) depth, and (F) turbidity. Before processing the oceanographic standardized variables with the sigmoidal function, a moving average was applied in order to smooth their signal.
Sensors 20 06281 g014
Table
Table 1. Oceanographic variables and sensors used at the LoVe monitoring site for statistic comparison with P. arborea polyp extrusion/activity rhythms.
Table 1. Oceanographic variables and sensors used at the LoVe monitoring site for statistic comparison with P. arborea polyp extrusion/activity rhythms.
SensorModel
Pressure and temperatureAanderaa 4117D
SalinityAanderaa 4319A
TurbidityAanderaa 4112A
ChlorophyllWetLabs BFL2W
Table
Table 2. Precision Accuracy (PA) and User Accuracy (UA) of each of the 4 state categories.
Table 2. Precision Accuracy (PA) and User Accuracy (UA) of each of the 4 state categories.
StatesPAUA
Bloated82.85%80.97%
Non-Bloated61.57%61.57%
Semi-Bloated65.04%65.04%
Background87.36%87.36%
Table
Table 3. Time series parameters as absolute maximum and minimum as well as significant (p > 0.05) periods (both in total minutes (Min) and hours (h)) for all behavioral and oceanographic variables measured at the LoVe observatory in 2018. The Midline Estimated Statistic of a Rhythm (MESOR) is also reported as a result from waveform analysis for all the time series.
Table 3. Time series parameters as absolute maximum and minimum as well as significant (p > 0.05) periods (both in total minutes (Min) and hours (h)) for all behavioral and oceanographic variables measured at the LoVe observatory in 2018. The Midline Estimated Statistic of a Rhythm (MESOR) is also reported as a result from waveform analysis for all the time series.
Period
MaxMinMESORMinh
Bloated99.974.4360.56143523.9
Non-Bloated87.070.0021.42143523.9
Semi-Bloated67.860.0018.02143523.9
Temperature7.315.616.52Not Done
Salinity35.0034.2334.74Not Done
Chlorophyll26.240.327.05Not Done
Turbidity124.285.566.58Not Done
Depth251.15248.88250.10Not Done
Table
Table 4. Characteristics and principal results of the Multi-Layer Feed forward artificial neural Network (MLFN) model (training and internal test) in predicting the coral activity status (i.e., bloated vs. non-bloated) for the training and the test sets.
Table 4. Characteristics and principal results of the Multi-Layer Feed forward artificial neural Network (MLFN) model (training and internal test) in predicting the coral activity status (i.e., bloated vs. non-bloated) for the training and the test sets.
Training (70%)
Number of cases1150
Number of hidden layers1
Number of nodes9
Training time0:7:02
Number of trials500
% bad predictions0.26 (3)
Testing (30%)
Number of cases503
% bad predictions (N)31 (156)
Table
Table 5. Confusion matrix of the test set of the MLFN model used in predicting the classification bloated versus non-bloated activity statuses. Targets are reported on rows. The correctly classified samples are reported on the main diagonal of the matrix.
Table 5. Confusion matrix of the test set of the MLFN model used in predicting the classification bloated versus non-bloated activity statuses. Targets are reported on rows. The correctly classified samples are reported on the main diagonal of the matrix.
BloatedNon-BloatedTotal
Bloated22370293
Non-Bloated86124210
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Zuazo, A.; Grinyó, J.; López-Vázquez, V.; Rodríguez, E.; Costa, C.; Ortenzi, L.; Flögel, S.; Valencia, J.; Marini, S.; Zhang, G.; et al. An Automated Pipeline for Image Processing and Data Treatment to Track Activity Rhythms of Paragorgia arborea in Relation to Hydrographic Conditions. Sensors 2020, 20, 6281. https://0-doi-org.brum.beds.ac.uk/10.3390/s20216281

AMA Style

Zuazo A, Grinyó J, López-Vázquez V, Rodríguez E, Costa C, Ortenzi L, Flögel S, Valencia J, Marini S, Zhang G, et al. An Automated Pipeline for Image Processing and Data Treatment to Track Activity Rhythms of Paragorgia arborea in Relation to Hydrographic Conditions. Sensors. 2020; 20(21):6281. https://0-doi-org.brum.beds.ac.uk/10.3390/s20216281

Chicago/Turabian Style

Zuazo, Ander, Jordi Grinyó, Vanesa López-Vázquez, Erik Rodríguez, Corrado Costa, Luciano Ortenzi, Sascha Flögel, Javier Valencia, Simone Marini, Guosong Zhang, and et al. 2020. "An Automated Pipeline for Image Processing and Data Treatment to Track Activity Rhythms of Paragorgia arborea in Relation to Hydrographic Conditions" Sensors 20, no. 21: 6281. https://0-doi-org.brum.beds.ac.uk/10.3390/s20216281

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop