Next Article in Journal
Removal of Antimony(V) from Drinking Water Using nZVI/AC: Optimization of Batch and Fix Bed Conditions
Next Article in Special Issue
Mercury Accumulation in Commercial Varieties of Oryza sativa L. Cultivated in Soils of La Mojana Region, Colombia
Previous Article in Journal
Insights into Elemental Composition and Sources of Fine and Coarse Particulate Matter in Dense Traffic Areas in Toronto and Vancouver, Canada
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Potential for Interspecies Toxicity Estimation in Soil Invertebrates

Office of Research & Development, U.S. EPA, Gulf Breeze, FL 32561, USA
*
Author to whom correspondence should be addressed.
Submission received: 30 July 2021 / Revised: 5 October 2021 / Accepted: 11 October 2021 / Published: 14 October 2021
(This article belongs to the Special Issue Current Developments in Soil Ecotoxicology)

Abstract

:
Interspecies correlation estimation (ICE) models are linear regressions that predict toxicity to a species with few data using a known toxicity value in a surrogate species. ICE models are well established for estimating toxicity to fish and aquatic invertebrates but have not been generally developed or applied to soil organisms. To facilitate the development of ICE models for soil invertebrates, a database of single chemical toxicity values was compiled from knowledgebases and reports that included 853 records encompassing 192 chemicals and 12 species. Most toxicity data for single chemicals tested in soil media were for species of earthworms, with only limited data for other species and taxa. ICE models were developed for eleven separate species pairs as least squares log-linear regressions of acute toxicity values of the same chemicals tested in both the surrogate and predicted species of soil organisms. Model uncertainty was assessed using leave one out cross-validation as the fold difference between a predicted and measured toxicity value. ICE models showed high accuracy within order (e.g., earthworm to earthworm), but less prediction accuracy in the two across-taxa models (Arthropoda to Annelida and the inverse). This study provides a proof-of-concept demonstration that ICE models can be developed for soil invertebrates.

1. Introduction

Interspecies correlation estimation (ICE) models predict toxicity to a species with limited data using a known toxicity value in a surrogate species [1]. The need to evaluate a rapidly growing number of chemicals and efforts to reduce or replace animal testing have increased the need and use of computational toxicity estimation methods, such as ICE models [2]. Most ICE models have been developed as simple linear regressions of acute toxicity values of the same chemicals tested in both the surrogate and predicted species [3]. An extensive technical basis exists for aquatic ICE models including database curation, model validation, comparison to water quality standards, and use in supplementing the taxa diversity of species sensitivity distributions (e.g., Dyer et al. [4]; Awkerman et al. [5]). Suites of ICE models are currently available for fish and aquatic invertebrates, algae, and terrestrial wildlife on the internet platform Web-ICE (www3.epa.gov/webice/), as well as in the literature. Aquatic ICE models are increasingly being used or considered for ecological risk assessment and regulatory applications [2,6,7].
ICE models for soil invertebrates and plants have not been previously available, despite the ecological significance of soil dwelling organisms, and the increasing knowledge base on the toxicity of chemicals to soil organisms. Common soil invertebrate test species include earthworms (e.g., Eisenia andrei) and springtails (Collembola; e.g., Folsomia candida) [8]. The availability of standardized test methods for some soil invertebrates, such as earthworms (e.g., OECD [9]), could allow many toxicity studies to be performed in a consistent manner. However, the variety of options for acute testing conditions that are recommended within and across species has resulted in inconsistent testing that makes combining data from multiple studies challenging. For example, acute toxicity tests with earthworms can be performed using a 2-day contact toxicity test or a 7–14 day soil exposure.
The objective of the current study was to assess the feasibility of developing ICE models for soil invertebrates to allow for extrapolation between different species and taxa of soil dwelling organisms. Single chemical toxicity data were compiled from the US EPA and simple least squares log-linear regression ICE models were developed as a proof of concept demonstration that ICE models can be developed for soil invertebrates.

2. Materials and Methods

2.1. Data Compilation

Acute toxicity data for soil invertebrates were obtained from the ECOTOX knowledgebase (https://cfpub.epa.gov/ecotox/; downloaded May 2021), Standartox (http://standartox.uni-landau.de/; downloaded May 2021) [10], and compilations of screening values [11]. The search parameters used to obtain data from ECOTOX and Standartox are included in the Supplementary Materials (Table S1). The ECOTOX knowledgebase, in particular, is a comprehensive database of toxicity values taken primarily from peer-reviewed literature that is reviewed and systematically updated quarterly. Both lethal concentration 50% (LC50) values and lowest observed effect concentrations (LOECs) for mortality were compiled and analyzed separately. Toxicity values were filtered to include only data from experiments that used standard acute soil toxicity test durations (1 day for C. elegans, 14 days for earthworms and springtails), endpoints expressed in or convertible to mg/kg soil, and species that had data for at least 3 chemicals. Due to limited data availability, toxicity values for species at any life stage were used and data were not standardized for soil type or environmental conditions (e.g., pH, moisture). All compounds were analyzed as the chemical form originally reported (e.g., metal salts were not converted to toxicity of the elemental form). The resulting toxicity database included 853 records encompassing 197 chemicals and 12 species (Table 1). Prior to model development, species mean acute values (SMAVs) were calculated as the geometric mean of toxicity values (LC50 or LOEC) for each endpoint, species, and chemical. SMAVs were then used in ICE models.

2.2. Model Development

ICE models were developed using least squares log-linear regression, as described by Raimondo et al. [3], by pairing toxicity values by chemical between the surrogate species and predicted species [3]. ICE models describe the relationship of chemical sensitivity between two species by the equation Log10(Predicted Toxicity) = a × Log10(Surrogate Toxicity) + b [3]. For the soil invertebrate ICE models, Predicted toxicity was the SMAV of the species with unknown chemical toxicity, Surrogate Toxicity was the SMAV of the surrogate species, a was the slope of the regression line, and b was the intercept of the regression.
Models were developed separately for SMAVs derived from LOEC and LC50 data. A separate model was developed for each potential pair of species that had data for the same endpoint for at least three common chemicals. The R2 and mean square error (MSE) of each model were calculated as a measure of model fit and model error, respectively. All models were developed using the stats package in R.

2.3. Model Prediction Accuracy

Significant models (p < 0.05) that had at least four data points were evaluated using leave-one-out cross validation (LOOCV), which has previously been employed for evaluating model accuracy [3,12]. In LOOCV, one data point is removed from the dataset. Then, a new model is built with the remaining values and used to predict the removed value. This was systematically reiterated for each Predicted toxicity value in each model. LOOCV analysis was conducted using the caret package in R [13]. The fold differences of removed values were calculated by dividing the non-transformed estimated value by the actual value or vice versa, if the latter was larger. The percentage of estimates that fell within 5-fold of the actual value was calculated for each model and used as a prediction success rate. As mentioned in Willming et al. [12], values within 5-fold of each other are within the range of interlaboratory variation.

3. Results

3.1. Within-Taxa Models

Available data allowed for the development of within-taxa models between earthworm species (same Order: Opisthopora). Using least-squares log-linear regression, a total of 3 significant (p < 0.05) model pairs were developed using LC50 data (out of 7 pairs total) (Table 2) and 6 significant (p < 0.05) models were developed using LOEC data (out of 14 total) (Table 3). LOOCV analysis determined that within-species models predicted removed values within 5-fold of the actual toxicity value 99% of the time and 97% of the time when using LC50 values and LOEC values, respectively.

3.2. Across-Taxa Models

Only two across-taxa models (different Phylums) were significant (p < 0.05). Models were between Folsomia candida (Phylum: Arthropoda, Class: Collemobola) and Eisenia fetida (Phylum: Annelida, Class: Clitellata) (Table 4). LOOCV analysis determined that across-taxa models predicted removed LOEC values within 5-fold of the actual toxicity value 62.5% of the time, on average.

4. Discussion

The current study was a proof-of-concept demonstration that ICE models can be developed for soil invertebrates. ICE models showed high prediction accuracy within the Order Opisthopora (i.e., earthworm to earthworm), but lower accuracy in the two across-taxa models (Arthropoda to Annelida and the inverse). These initial results are similar to observations of ICE model extrapolation in aquatic species, where increasing taxonomic distance results in less prediction accuracy and generally greater uncertainty [12,14]. The soil invertebrate ICE models were developed with compounds from multiple chemical classes, which can reduce model accuracy for pairs of species with limited taxa relatedness. Research with aquatic toxicity ICE models has shown that across-taxa predictions can be improved by developing models using only compounds with the same mode of action (MOA) and same structural class [3]. Additional data would be needed to develop a suite of single chemical class-specific models and to more comprehensively explore whether these models could improve cross-taxa extrapolation in soil invertebrates, but were not available in the explored databases.
Soil invertebrate toxicity data used in this study were not curated beyond acute toxicity endpoint and were not standardized for test conditions, which can result in greater uncertainty in ICE models and lower prediction accuracy [15]. Although toxicity values from knowledgebases such as ECOTOX are typically used without further curation in environmental toxicology applications, it can lead to greater uncertainty and less prediction accuracy in model development [10,15,16]. For example, Hrovat et al. [17] reported orders of magnitude variability in fish acute toxicity values and incomplete records in nearly 70% of the 4654 ECOTOX records assessed. Toxicity data in the current study were not standardized for soil type, which can substantially influence species sensitivity of soil organisms, particularly for ionizable chemicals, such as metals, and phenolics affected by pH and clay content [18]. However, the availability of standardized test methods should provide for datasets of more consistently collected toxicity data (e.g., OECD [9]). Future research should determine if standardization for soil chemistry and physical properties would reduce uncertainty in interspecies extrapolation modeling, particularly for species pairs with large taxonomic distance.
One surprising aspect of this study was the generally limited toxicity data for soil invertebrates from single chemicals tests in soil media. Additional data for single chemical tests are available as short-term contact toxicity tests, but these were excluded from the current study because of apparent lack of ecological relevance to soil exposures. Bioassays of field collected samples are more routinely collected as part of contaminated site assessments (e.g., Reinecke et al. [19]; Fründ et al. [20]), but single chemical test data appear to be rarely reported based on current knowledgebase searches. Earthworm tests dominate the soil toxicity data from the comprehensive ecotoxicity databases used for source data, with few data for other taxa. Given their ecological importance and prevalence of soil contamination globally, the availability of additional toxicity data for soil invertebrates is critically needed and would allow for the expanded development of ICE models and application in ecological assessments. In particular, toxicity data for a wider diversity of species may benefit the development of benchmark values, such as ecological soil screening levels (EcoSSL), by better representing soil ecosystem biodiversity.

Supplementary Materials

The following are available online at https://0-www-mdpi-com.brum.beds.ac.uk/article/10.3390/toxics9100265/s1, Table S1: Summary of search parameters used in the ECOTOX knowledgebase and Standartox databases to obtain toxicity values for use in this study. Spreadsheet S1: Soil ICE database which includes all toxicity data used in this study.

Author Contributions

Conceptualization, M.G.B.; methodology, F.N.L.; formal analysis, F.N.L.; data curation, F.N.L.; writing—original draft preparation, M.G.B.; writing—review and editing, F.N.L. and M.G.B.; supervision, M.G.B.; project administration, M.G.B.; funding acquisition, M.G.B. All authors have read and agreed to the published version of the manuscript.

Funding

This project was supported in part by an appointment to the Research Participation Program at the Gulf Ecosystem Modeling and Management Division, U.S. Environmental Protection Agency, administered by the Oak Ridge Institute for Science and Education through an interagency agreement between the U.S. Department of Energy and EPA. This research received no external funding.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Data supporting reported results is included in the Supplementary Materials.

Acknowledgments

We thank Glenn Suter and Jill Awkerman for reviewing a draft of the manuscript.

Conflicts of Interest

The authors declare no conflict of interest. The views expressed in this article are those of the authors and do not necessarily represent the views or the policies of the U.S. Environmental Protection Agency. Any mention of trade names, products, or services does not imply an endorsement by the U.S. Government or the EPA.

References

  1. Suter, G.W. Organism-level extrapolation models. In Ecological Risk Assessment; Suter, G.W., Ed.; CRC Press: Boca Raton, FL, USA, 2007. [Google Scholar]
  2. Bejarano, A.C.; Wheeler, J.R. Scientific Basis for Expanding the Use of Interspecies Correlation Estimation Models. Integr. Environ. Assess. Manag. 2020, 16, 528–530. [Google Scholar] [CrossRef] [PubMed]
  3. Raimondo, S.; Jackson, C.R.; Barron, M.G. Influence of taxonomic relatedness and chemical mode of action in acute interspecies estimation models for aquatic species. Environ. Sci. Technol. 2010, 44, 7711–7716. [Google Scholar] [CrossRef] [PubMed]
  4. Dyer, S.D.; Versteeg, D.J.; Belanger, S.E.; Chaney, J.G.; Raimondo, S.; Barron, M.G. Comparison of species sensitivity distributions derived from interspecies correlation models to distributions used to derive water quality criteria. Environ. Sci. Technol. 2008, 42, 3076–3083. [Google Scholar] [CrossRef] [PubMed]
  5. Awkerman, J.A.; Raimondo, S.; Jackson, C.R.; Barron, M.G. Augmenting aquatic species sensitivity distributions with interspecies toxicity estimation models. Environ. Toxicol. Chem. 2014, 33, 688–695. [Google Scholar] [CrossRef] [PubMed]
  6. Feng, C.; Wu, F.; Mu, Y.; Meng, W.; Dyer, S.D.; Fan, M.; Raimondo, S.; Barron, M.G. Interspecies correlation estimation-applications in water quality criteria and ecological risk assessment. Environ. Sci. Technol. 2013, 47, 11382–11383. [Google Scholar] [CrossRef] [PubMed]
  7. Wang, X.; Fan, B.; Fan, M.; Belanger, S.; Li, J.; Chen, J.; Gao, X.; Liu, Z. Development and use of interspecies correlation estimation models in China for potential application in water quality criteria. Chemosphere 2020, 240, 124848. [Google Scholar] [CrossRef] [PubMed]
  8. Fairbrother, A.; Barron, M.; Johnson, M. Chapter 42: Methods in Environmental Toxicology. In Hayes’ Principles and Methods of Toxicology, 6th ed.; Taylor & Frances: Boca Raton, FL, USA, 2014; pp. 2024–2065. [Google Scholar]
  9. OECD. Earthworm, Acute Toxicity Tests. In OECD Guideline for Testing of Chemicals 207; OECD Publishing: Paris, France, 1984. [Google Scholar]
  10. Scharmüller, A.; Schreiner, V.C.; Schäfer, R.B. Standartox: Standardizing toxicity data. Data 2020, 5, 46. [Google Scholar] [CrossRef]
  11. Efroymson, R.A.; Will, M.E.; Suter, G.W. Toxicological Benchmarks for Contaminants of Potential Concern for Effects on Soil and Litter Invertebrates and Heterotrophic Process: 1997 Revision; Prepared for the Oak Ridge Laboratory: Oak Ridge, TN, USA, 1997. [Google Scholar]
  12. Willming, M.M.; Lilavois, C.R.; Barron, M.G.; Raimondo, S. Acute Toxicity Prediction to Threatened and Endangered Species Using Interspecies Correlation Estimation (ICE) Models. Environ. Sci. Technol. 2016, 50, 10700–10707. [Google Scholar] [CrossRef] [PubMed]
  13. Kuhn, M. Building Predictive Models in R Using the caret Package. J. Stat. Softw. 2008, 28, 1–26. [Google Scholar] [CrossRef] [Green Version]
  14. Raimondo, S.; Barron, M.G. Application of Interspecies Correlation Estimation (ICE) models and QSAR in estimating species sensitivity to pesticides. SAR QSAR Environ. Res. 2020, 31, 1–18. [Google Scholar] [CrossRef]
  15. Raimondo, S.; Vivian, D.N.; Delos, C.; Barron, M.G. Protectiveness of species sensitivity distribution hazard concentrations for acute toxicity used in endangered species risk assessment. Environ. Toxicol. Chem. 2008, 27, 2599–2607. [Google Scholar] [CrossRef]
  16. Connors, K.A.; Beasley, A.; Barron, M.G.; Belanger, S.E.; Bonnell, M.; Brill, J.L.; de Zwart, D.; Kienzler, A.; Krailler, J.; Otter, R.; et al. Creation of a Curated Aquatic Toxicology Database: EnviroTox. Environ. Toxicol. Chem. 2019, 38, 1062–1073. [Google Scholar] [CrossRef] [Green Version]
  17. Hrovat, M.; Segner, H.; Jeram, S. Variability of in vivo fish acute toxicity data. Regul. Toxicol. Pharmacol. 2009, 54, 294–300. [Google Scholar] [CrossRef] [PubMed]
  18. Van Gestel, C.A.; Borgman, E.; Verweij, R.A.; Ortiz, M.D. The influence of soil properties on the toxicity of molybdenum to three species of soil invertebrates. Ecotoxicol. Environ. Saf. 2011, 74, 1–9. [Google Scholar] [CrossRef] [PubMed]
  19. Reinecke, A.J.; Reinecke, S.A. Earthworms as test organisms in ecotoxicological assessment of toxicant impacts on ecosystems. In Earthworm Ecology; Edwards, C.A., Ed.; CRC Press: Boca Raton, FL, USA, 2004; pp. 299–320. [Google Scholar]
  20. Fründ, H.-C.; Graefe, U.; Tischer, S. Earthworms as Bioindicators of Soil Quality. In Biology of Earthworms; Karaca, A., Ed.; Spinger: Berlin/Heidelberg, Germany, 2011; pp. 261–278. [Google Scholar]
Table 1. Summary of compiled soil invertebrate toxicity values from single chemicals tested in soil medium.
Table 1. Summary of compiled soil invertebrate toxicity values from single chemicals tested in soil medium.
EndpointCommon NameTaxaSpeciesNumber of Chemicals
LC50EarthwormAnnelidEisenia andrei35
EarthwormAnnelidEisenia fetida162
EarthwormAnnelidLumbricus rubellus15
EarthwormAnnelidLumbricus terrestris26
NematodeNematodeCaenorhabditis elegans10
Pot wormAnnelidEnchytraeus albidus3
SpringtailArthropodHeteromurus nitidus5
WoodlouseArthropodPorcellionides pruinosus3
LOECEarthwormAnnelidAllolobophora tuberculata9
EarthwormAnnelidEisenia andrei24
EarthwormAnnelidEisenia fetida26
African EarthwormAnnelidEudrilus eugeniae9
EarthwormAnnelidLumbricus rubellus14
India Blue EarthwormAnnelidPerionyx excavatus9
SpringtailArthropodFolsomia candida9
SpringtailArthropodHeteromurus nitidus4
Table 2. Significant within-taxa models (same Order: Opisthopora) using single chemical median lethal concentration (LC50) data for soil invertebrates. Model parameters included intercept, slope, and R2, the model p-value, mean square error (MSE), and the total number of data points (n). The cross-validation within 5-fold describes the prediction success rate, or percentage of estimated toxicity values from leave one out cross validation that were within 5-fold of the measured LC50 value. Numbers in the left-most column indicate model pairs which use the same two species, alternating which is used as the surrogate.
Table 2. Significant within-taxa models (same Order: Opisthopora) using single chemical median lethal concentration (LC50) data for soil invertebrates. Model parameters included intercept, slope, and R2, the model p-value, mean square error (MSE), and the total number of data points (n). The cross-validation within 5-fold describes the prediction success rate, or percentage of estimated toxicity values from leave one out cross validation that were within 5-fold of the measured LC50 value. Numbers in the left-most column indicate model pairs which use the same two species, alternating which is used as the surrogate.
Model PairPredicted
Species
(y-axis)
Surrogate
Species
(x-axis)
InterceptSlopeR2p-ValueMSEnCross-
Validation within 5-Fold
1Eisenia andreiEisenia fetida0.370.720.634.50 × 10−50.2019100%
Eisenia fetidaEisenia andrei0.340.880.634.50 × 10−50.2419100%
2Eisenia fetidaLumbricus terrestris0.970.600.363.07 × 10−30.2822100%
Lumbricus terrestrisEisenia fetida0.960.600.363.07 × 10−30.2822100%
3Eisenia andreiLumbricus terrestris0.340.720.393.03 × 10−20.4212100%
Lumbricus terrestrisEisenia andrei1.090.540.393.03 × 10−20.321291.67%
Table 3. Significant within-taxa models (same Order: Opisthopora) using single chemical lowest observed effect concentration (LOEC) data. Model parameters included intercept, slope, and R2, the model p-value, mean square error (MSE), and the total number of data points (n). The cross-validation within 5-fold describes the prediction success rate, or percentage of estimated toxicity values from leave one out cross validation that were within 5-fold of the measured LOEC value. Numbers in the left-most column indicate model pairs which use the same two species, alternating which is used as the surrogate.
Table 3. Significant within-taxa models (same Order: Opisthopora) using single chemical lowest observed effect concentration (LOEC) data. Model parameters included intercept, slope, and R2, the model p-value, mean square error (MSE), and the total number of data points (n). The cross-validation within 5-fold describes the prediction success rate, or percentage of estimated toxicity values from leave one out cross validation that were within 5-fold of the measured LOEC value. Numbers in the left-most column indicate model pairs which use the same two species, alternating which is used as the surrogate.
Model PairPredicted Species
(y-axis)
Surrogate Species
(x-axis)
InterceptSlopeR2p-ValueMSEnCross-
Validation within 5-Fold
1Eudrilus euganiaeEisenia fetida−0.071.090.608.77 × 10−30.261090%
Eisenia fetidaEudrilus euganiae1.000.550.608.77 × 10−30.1310100%
2Eudrilus euganiaeAllolobophora tuberculata–0.271.040.964.48 × 10−60.029100%
Allolobophora tuberculataEudrilus euganiae0.350.920.964.48 × 10−60.029100%
3Perionyx excavatusEisenia fetida0.270.940.581.03 × 10−20.211090%
Eisenia fetidaPerionyx excavatus0.830.620.581.03 × 10−20.1410100%
4Eisenia fetidaAllolobophora tuberculata0.590.670.741.50 × 10−30.091090%
Allolobophora tuberculataEisenia fetida0.061.090.741.50 × 10−30.141090%
5Perionyx excavatusAllolobophora tuberculata0.090.900.957.88 × 10−60.029100%
Allolobophora tuberculataPerionyx excavatus0.031.050.957.88 × 10−60.029100%
6Eudrilus euganiaePerionyx excavatus–0.311.130.962.84 × 10−60.029100%
Perionyx excavatusEudrilus euganiae0.360.860.962.84 × 10−60.019100%
Table 4. Significant across-taxa models (different Phylum) using lowest observed effect concentration (LOEC) data. Model parameters included intercept, slope, and R2, model p-value, mean square error (MSE), and the total number of data points (n). The cross-validation within 5-fold describes the prediction success rate, or percentage of estimated toxicity values from leave one out cross validation that were within 5-fold of the measured LOEC value. Numbers in the left-most column indicate model pairs which use the same two species, alternating which is used as the surrogate.
Table 4. Significant across-taxa models (different Phylum) using lowest observed effect concentration (LOEC) data. Model parameters included intercept, slope, and R2, model p-value, mean square error (MSE), and the total number of data points (n). The cross-validation within 5-fold describes the prediction success rate, or percentage of estimated toxicity values from leave one out cross validation that were within 5-fold of the measured LOEC value. Numbers in the left-most column indicate model pairs which use the same two species, alternating which is used as the surrogate.
Model PairPredicted Species
(y-axis)
Surrogate Species
(x-axis)
InterceptSlopeR2p-ValueMSEnCross-Validation within 5-Fold
1Folsomia candidaEisenia fetida0.850.470.933.37 × 10−20.04450%
Eisenia fetidaFolsomia candida–1.552.000.933.37 × 10−20.15475%
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Barron, M.G.; Lambert, F.N. Potential for Interspecies Toxicity Estimation in Soil Invertebrates. Toxics 2021, 9, 265. https://0-doi-org.brum.beds.ac.uk/10.3390/toxics9100265

AMA Style

Barron MG, Lambert FN. Potential for Interspecies Toxicity Estimation in Soil Invertebrates. Toxics. 2021; 9(10):265. https://0-doi-org.brum.beds.ac.uk/10.3390/toxics9100265

Chicago/Turabian Style

Barron, Mace G., and Faith N. Lambert. 2021. "Potential for Interspecies Toxicity Estimation in Soil Invertebrates" Toxics 9, no. 10: 265. https://0-doi-org.brum.beds.ac.uk/10.3390/toxics9100265

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop