Physics-Informed Neural Networks-Based Salinity Modeling in the Sacramento–San Joaquin Delta of California

Abstract

Salinity in estuarine environments has been traditionally simulated using process-based models. More recently, data-driven models including artificial neural networks (ANNs) have been developed for simulating salinity. Compared to process-based models, ANNs yield faster salinity simulations with comparable accuracy. However, ANNs are often purely data-driven and not constrained by physical laws, making it difficult to interpret the causality between input and output data. Physics-informed neural networks (PINNs) are emerging machine-learning models to integrate the benefits of both process-based models and data-driven ANNs. PINNs can embed the knowledge of physical laws in terms of the partial differential equations (PDE) that govern the dynamics of salinity transport into the training of the neural networks. This study explores the application of PINNs in salinity modeling by incorporating the one-dimensional advection–dispersion salinity transport equation into the neural networks. Two PINN models are explored in this study, namely PINNs and FoNets. PINNs are multilayer perceptrons (MLPs) that incorporate the advection–dispersion equation, while FoNets are an extension of PINNs with an additional encoding layer. The exploration is exemplified at four study locations in the Sacramento–San Joaquin Delta of California: Pittsburg, Chipps Island, Port Chicago, and Martinez. Both PINN models and benchmark ANNs are trained and tested using simulated daily salinity from 1991 to 2015 at study locations. Results indicate that PINNs and FoNets outperform the benchmark ANNs in simulating salinity at the study locations. Specifically, PINNs and FoNets have lower absolute biases and higher correlation coefficients and Nash–Sutcliffe efficiency values than ANNs. In addition, PINN models overcome some limitations of purely data-driven ANNs (e.g., neuron saturation) and generate more realistic salinity simulations. Overall, this study demonstrates the potential of PINNs to supplement existing process-based and ANN models in providing accurate and timely salinity estimation.

1. Introduction

Salinity is a critical variable in estuarine environments as it impacts the quality of freshwater withdraws and affects fish migration patterns, spawning habitat, and survivability, among others [1,2,3,4,5]. Salinity management in estuarine environments is thus important to maintain desirable water quality, protect aquatic habitats, support economic (e.g., agriculture and industry) development, and mitigate climate change-induced impacts (e.g., sea water intrusion).

Salinity modeling reflects global causes such as the United Nations Sustainable Development Goals (SDGs) and the pursuit of net-zero emissions. By employing advanced modeling techniques, researchers and policymakers can better understand and manage salinity levels, contributing to achieving net-zero emissions goals (SDG 13 [6]) and ensuring clean water and sanitation (SDG 6 [7]). These modeling efforts are particularly important for estuarine environments with enormous environmental and economic significance, including the Sacramento–San Joaquin Delta (Delta) of California, United States (U.S.).

The Delta is the hub of the complex water system of California, a top-five economy in the world. It is bounded by the two largest river systems in the State: the Sacramento River on the North and the San Joaquin River on the South which collectively contribute freshwater to the Delta. Tides from the Pacific Ocean on the west bring salty sea water into the Delta. Freshwater is pumped from the Delta to support over 25 million people and 15,000 km² of farmlands [8]. Managing water resources in the Delta (e.g., by the Ghyben–Herzberg relationship [9,10] for the saltwater intrusion behavior) is crucial to maintaining the balance between freshwater and saltwater and ensuring the availability of safe drinking water and irrigation for agriculture. The Delta is also an important biodiversity hotspot that provides a habitat for over 750 species of plants and animals [11,12]. State and federal regulatory requirements on maximum allowable salinity levels have been imposed at compliance locations across the Delta to ensure the quality of water (suitable for drinking and agricultural use) and protect endangered species [13,14]. Understanding the spatial and temporal variations in the Delta is foremost to comply with these regulations. Models have been traditionally developed and applied to gain that understanding.

Salinity simulation models applied in the Delta can be categorized into three types: empirical models, process-based models, and data-driven models. Empirical models were among the earliest methods used for salinity simulation in the Delta. The Minimum Delta Outflow (MDO) procedure [15] translates the salinity values in the Delta to determine the Delta outflow required to meet the requirements of water management practices and to quickly test proposed new standards. The G-model [16,17] is another empirical model that captures salinity transport in the Delta. It incorporates the antecedent Delta outflow and the Delta salinity relationship, which is described by the 1-dimensional advection–dispersion equation. This equation is a partial differential equation that models the spatial and temporal variations of salinity with respect to the outflow. By incorporating antecedent outflow information, the G-model has shown improvements in Delta salinity estimation and improved the accuracy of water supply estimates to MDO.

Although similar to empirical models, process-based models are more comprehensive and simulate Delta salinity with detailed physical processes. A popular one-dimensional process-based model is the Delta Simulation Model II (DSM2) [18] which can calculate flow, stages, flow velocities, and various mass transport processes [19]. In particular, DSM2 has made a significant contribution to the salinity transport process by explicitly solving the advection–dispersion equation in its modeling procedures [20,21,22,23,24]. DSM2 solves the advection–dispersion equation to simulate the salinity transport process in the Delta. Although there exist multi-dimensional models (e.g., TRIM2D [25], RMA10 [26], UnTrim [27], and SCHISM [28]), DSM2 is arguably the most commonly used process-based model to simulate Delta salinity and inform water quality operations and decisions within the Delta due to its fewer input data preparation and computing resources requirements [29] and its best-understood performance over decades [30].

Data-driven machine-learning models are prevalent in numerous scientific domains, including forecasting the performance and energy yield of solar farms [31] and in predicting the remaining usable life of lithium-ion batteries [32]. In Delta salinity simulation, machine-learning models are increasingly used as well due to recent advancements in computing power, which enable superior computational efficiency compared to empirical and process-based models. These models rely mainly on data, with little to no physical process information used. In the Delta, data-driven models are often used to either implement empirical models or to emulate process-based models. For example, Multilayer perceptron (MLP) ANNs were introduced in [19] to emulate DSM2 within CalSim [33,34], an operational water resources planning and simulation model, to estimate salinity at 12 locations in the Delta. In [35], the ANN model in [19] was further enhanced using a multitask-designed approach, with a salinity output for each location. Various deep learning architectures were considered in [29] to emulate DSM2 in salinity modeling, estimating salinity at 28 locations using MLP, Long-Short-Term Memory (LSTM), Gated Recurrent Unit (GRU), and Residual Network (ResNet). Additionally, two novel deep learning models, Res-LSTM and Res-GRU, were developed in [36] to model salinity in the Delta. All these ANN-based machine-learning models showed improvements in salinity estimation compared to DSM2, while achieving significant reductions in training and inference time.

However, despite their strong predictive skills and efficiency in training, ANN-based machine-learning models rely solely on data and do not take into account the underlying physical processes of the data. With enough input variables, these models can capture correlations and produce correct results, but the entire learning process is a black-box, making it difficult to interpret their results and determine the causality between input and output data [37]. Moreover, the quality of these models depends heavily on the quantity and quality of the training data, and inference or prediction of untrained variables is not possible due to the limitation of these models to only output the training targets. All these limitations hinder the ability of data-driven models to make hypotheses and interpretations of the physical processes, which are critical for water management practices and planning.

Different approaches to integrating knowledge of physical processes into machine-learning models have been studied to overcome these limitations in the field of water resources engineering. One approach is the use of physics-based models, where the physical constraints are enforced in the models’ architecture. For example, depth-temperature-density physical constraints were hard-coded in the neural network architecture for quantifying uncertainty in lake temperature modeling [38]. Mass-conserving LSTM architecture has also been explored in predicting peak flows in rainfall-runoff modeling [39] and in predicting the salinity of navigable waterways in Belgium [40]. In [41], differentiable, learnable, process-based models with embedded ANN that respect mass balances were proposed to predict untrained hydrologic variables. One popular approach to enforcing the physical constraints is the physics-informed neural network (PINN), which embeds the governing equations of the underlying physical processes into the loss function of a machine-learning model. Specifically, PINNs are artificial neural networks that are designed to solve ordinary and partial differential equations [42,43]. In recent years, PINNs have made significant impacts in many scientific applications, with the number of PINN papers quintupling between 2019 and 2020 and doubling in 2021 compared to 2020 [44]. Particularly in water engineering, PINNs have been studied to predict water surface profiles in a river [45] and to analyze soil-water infiltration processes for different soil types [46].

The current study aims to address the above-mentioned limitations of ANN-based machine-learning models using PINN models. In our proposed PINN models, we impose the underlying physical law governing flow–salinity relationships, namely the advection–dispersion equation, into artificial neural networks. By enforcing the physical law via the advection–dispersion equation, our proposed PINN models respect the flow–salinity relationships and further enhance salinity estimation accuracy. Our PINN-based machine-learning models can be viewed as a pathway between process-based modeling and machine-learning modeling in the Delta salinity modeling, as they maintain predictive accuracy and training efficiency due to their neural network architecture while respecting underlying physical laws by incorporating the advection–dispersion equation. To the best of our knowledge, no applications of PINN for salinity modeling in the Delta have been explored in the literature.

The rest of the paper is organized as follows. In Section 2, we present the methodology of our study, including locations, dataset, machine-learning model architectures, and evaluation metrics. In Section 3, we present the performance results of the machine-learning models of our study, demonstrating salinity estimation improvements of PINN models to benchmark ANNs. Then, we discuss the scientific and practical implications of our study and future research directions in Section 4 and conclude our paper in Section 5.

2. Materials and Methods

2.1. Study Locations and Dataset

The performance of the proposed machine-learning models in simulating salinity is exemplified at locations in the western part of the Sacramento–San Joaquin Delta (Delta) of California (Figure 1). Spanning through hundreds of kilometers of waterways in Northern California, the Delta is formed by the confluence of the Sacramento River from the north and San Joaquin River from the south and serves as a transition zone between the freshwater of the rivers and saltwater of the Pacific Ocean. In particular, freshwater inflows from the rivers travel westward through the Suisun Bay and exit through San Francisco Bay. Due to their proximity to the Pacific Ocean on the west, the four study locations of interest—Martinez, Port Chicago, Chipps Island, and Pittsburg—are more influenced by seawater than freshwater, unlike the locations in the northern or interior parts of the Delta.

Historical daily outflow values and DSM2-simulated daily salinity values at four locations from 1 January 1991 to 31 December 2015 are used as input and output, respectively, to train and test the machine-learning models in this study. Outflow is measured in cubic feet per second, while salinity is measured in electrical conductivity (EC) and is represented by micro-Siemens per cm (μS/cm), which reflects the amount of salt dissolved in water.

2.2. Data Preprocessing

2.2.1. Normalization

We denote $Q_i^{\left(j\right)}$ as the i-th daily outflow value at the j-th location,

$$j\in \left\{\mathrm{Martinez}, \mathrm{Port} \mathrm{Chicago}, \mathrm{Chipps} \mathrm{Island}, \mathrm{Pittsburg}\right\},$$

and $S_i^{\left(j\right)}$ as the target salinity value on day i at location j. We normalize the outflow inputs and the salinity outputs to the range $[0,1]$ by min-max normalization [47]. Specifically, denoting $N_j$ as the total number of daily samples at location j, we normalize $Q_i^{\left(j\right)}$ to

$$Q_i^{\left(j\right)}⟵\frac{Q_i^{\left(j\right)}-\left(\underset{j}{min}\underset{k=1,\dots ,N_j}{min}{Q}_k^{\left(j\right)}\right)}{\left(\underset{j}{max}\underset{k=1,\dots ,N_j}{max}{Q}_k^{\left(j\right)}\right)-\left(\underset{j}{min}\underset{k=1,\dots ,N_j}{min}{Q}_k^{\left(j\right)}\right)}$$

(1)

where we keep the notation $Q_i^{\left(j\right)}$ as the normalized output value of the i-th daily outflow value at the j-th location for simplicity. We apply a similar normalization procedure to the salinity output values as we do to the outflow input values in (1).

2.2.2. Input Memory

Previous study [48] on the DSM2 model showed that the daily salinity depends on the long-term memory of contributing parameters. Similarly, the G-model’s [16,17] success in salinity estimation is attributed to its ability to account for the effects of antecedent outflow on salinity. To account for long-term input memory, recent studies on salinity modeling in the Delta using artificial neural networks [19,29,35,36] have followed the practice of aggregating 118 antecedent daily values into 18 values for the current day input variables. We adopt this same practice in our study for preprocessing the input outflow to account for antecedent outflows. Specifically, for each normalized daily outflow $Q_i^{\left(j\right)}$ we form an outflow data vector ${\mathbf{Q}}_i^{\left(j\right)}\in {\mathbb{R}}^{18}$ by keeping the outflow values of the current day plus the most recent 7 antecedent days as-is, along with 10 successive 11-day averages of the prior 110 days. In other words, we define ${\mathbf{Q}}_i^{\left(j\right)}$ as

$${\mathbf{Q}}_i^{\left(j\right)}={[Q_i^{\left(j\right)},\dots ,Q_{i-7}^{\left(j\right)},\overline{Q_{(i-8)\to (i-18)}^{\left(j\right)}},\dots ,\overline{Q_{(i-107)\to (i-117)}^{\left(j\right)}}]}^T,$$

(2)

where $\overline{Q_{i_1\to i_2}^{\left(j\right)}}$ denotes the average of the normalized outflow values from day $i_1$ to $i_2$. By reducing the 118 antecedent daily outflow values into 18 values, we avoid unnecessary increases in the complexity of the machine-learning models proposed in our study while accounting for antecedent outflow memory.

2.3. Neural Network Architectures

In this work, we present/study three machine-learning models: a conventional multilayer perceptrons (MLP) network, referred to as ‘ANN’, which takes preprocessed outflow data vectors as inputs and outputs target salinity values; a PINN that incorporates the location variable x and the time variable t as additional inputs and embeds the advection–dispersion equation into the loss function; and an extension of the PINN, referred to as ‘FoNet’, which includes an additional input encoding layer to transform inputs to a higher-dimensional feature space via high-frequency functions. To demonstrate the improvement in salinity estimation by the proposed PINN and FoNet models, we keep the three models simple using fully connected networks. This way, we can directly demonstrate the impact of the inclusion of the causality information between input and output data, specifically the advection–dispersion equation for flow–salinity relationships, into the machine-learning models for salinity modeling in the Delta.

2.3.1. Artificial Neural Networks (ANN)

We consider a conventional MLP network, which has been extensively studied in Delta salinity modeling with varying input and output sizes and parameter choices [2,19,24,29,35,36,49]. In this study, we adopt an MLP network with four hidden layers. Its input variables are the daily outflow vector, represented as $\mathbf{Q}={[{\mathbf{Q}}_1,{\mathbf{Q}}_2,\dots ,{\mathbf{Q}}_{18}]}^T$, and its output is the estimated salinity value $\widehat{S}$. Concisely, the network estimated salinity value on the i-th day at the j-th location is $\widehat{S}({\mathbf{Q}}_i^{\left(j\right)};\theta )$. The notation $\theta$ denotes the trainable parameters, weights, and biases of the hidden layers, in the neural network. The number of neurons in the hidden layers and the choice of activation functions are selected by random hyperparameter search, which will be discussed in Section 2.4.1. The architecture of the ANN is shown in Figure 2. The loss function of the ANN is defined as

$$L\left(\theta \right)=\sum _{i,j}{(\widehat{S}({\mathbf{Q}}_i^{\left(j\right)};\theta )-S_i^{\left(j\right)})}^2,$$

(3)

i.e., the widely used mean squared error (MSE) loss function [47]. The ANN is trained by computing an optimal ${\theta }^{*}$ that minimizes $L\left(\theta \right)$.

2.3.2. Physics-Informed Neural Networks (PINN)

We propose a novel machine-learning model in salinity estimation in the Delta, using PINN. Our PINN embeds the advection–dispersion equation into its loss function, therefore incorporating the flow–salinity relationships into the machine-learning model. The one-dimensional advection–dispersion equation for salinity transport is [16,17]:

$$A\frac{\partial S}{\partial t}-Q\frac{\partial S}{\partial x}=\frac{\partial }{\partial x}\left(KA\frac{\partial S}{\partial x}\right),$$

(4)

where $A\left(x\right)$ is the estuary cross-sectional area, $S(x,t)$ is the concentration of salt (i.e., salinity in this study), $Q(x,t)$ is the volumetric flowrate (i.e., outflow in this study), $K(x,t)$ is the longitudinal dispersion coefficient, x is the longitudinal distance (increasing in the upstream direction), and t is the time. Under various assumptions on the initial and boundary conditions and the spatial and temporal dependencies of the flowrate and the dispersion coefficient, different studies have developed techniques for deriving an analytical solution of the advection–dispersion Equation (4). We refer the readers to [50] and references there-within for these various techniques. In this study, the flowrate is spatiotemporally dependent, driven by the data, and the cross-sectional area A and the dispersion coefficient K are assumed to be constants for simplicity.

The PINN model, similar to the ANN, is an MLP network, but with two hidden layers instead of four. Two hidden layers are sufficient for the PINN to obtain adequate results, whereas the ANN requires a deeper network structure. The hyperparameters are selected by random hyperparameter search, which will be discussed in Section 2.4.1. The architecture of the PINN is shown in Figure 3. Besides the number of hidden layers, there are two major differences between the ANN and the PINN models. First, in addition to the outflow data vector, the PINN has two additional input variables: longitudinal distance x and time t. For each day i and each location j, data values $x_i^{\left(j\right)}$ and $t_i^{\left(j\right)}$ are created for the longitudinal distance and time variables, respectively. Both variables are normalized: $x_i^{\left(j\right)}$ with respect to the distance between the westernmost location, Martinez, and the easternmost location, Pittsburg, and $t_i^{\left(j\right)}$ with respect to the number of days between 1 January 1991 and 31 December 2015. The estimated salinity on the i-th day at the j-th location by the PINN is denoted as $\widehat{S}(x_i^{\left(j\right)},t_i^{\left(j\right)},{\mathbf{Q}}_i^{\left(j\right)};\theta )$. The other major difference is in the loss function. In addition to the MSE loss function, the PINN seeks to minimize the advection–dispersion loss function, which ensures that the data satisfy the advection–dispersion Equation (4) and consequently the flow–salinity relationships, i.e., the loss function of the PINN is defined as the sum of the two loss functions:

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill L\left(\theta \right)& =\sum _{i,j}{(\widehat{S}(x_i^{\left(j\right)},t_i^{\left(j\right)},{\mathbf{Q}}_i^{\left(j\right)};\theta )-S_i^{\left(j\right)})}^2\hfill \\ \hfill & +\sum _{i,j}{\left(A\frac{\partial \widehat{S}}{\partial t}\right|}_{(x_i^{\left(j\right)},t_i^{\left(j\right)},{\mathbf{Q}}_i^{\left(j\right)};\theta )}-Q_i^{\left(j\right)}\frac{\partial \widehat{S}}{\partial x}{|}_{(x_i^{\left(j\right)},t_i^{\left(j\right)},{\mathbf{Q}}_i^{\left(j\right)};\theta )}-KA\frac{{\partial }^2\widehat{S}}{\partial x^2}{|}_{(x_i^{\left(j\right)},t_i^{\left(j\right)},{\mathbf{Q}}_i^{\left(j\right)};\theta )}{)}^2.\hfill \end{array}\end{array}$$

(5)

In (5), $Q_i^{\left(j\right)}$ is the normalized daily outflow on the i-th day at the j-th location and corresponds to the first component of the outflow data vector ${\mathbf{Q}}_i^{\left(j\right)}$. The derivatives of the network outputs with respect to the network inputs can be computed efficiently using automatic differentiation with backpropagation [51].

2.3.3. Physics-Informed Fourier Networks (FoNet)

Neural networks are known to favor low-frequency solutions, a phenomenon known as spectral bias [52]. One way to alleviate this issue is to add an encoding layer that transforms the inputs to a higher-dimensional feature space via high-frequency functions [52,53,54]. Our FoNet model includes an encoding layer in our PINN model, where the input encoding layer is trainable. Specifically, denoting the input variables concisely as $\mathbf{x}={[x,t,\mathbf{Q}]}^T\in {\mathbb{R}}^{20}$, the encoding layer is a Fourier feature mapping [54] that encodes $\mathbf{x}$ as

$$\left[\begin{array}{c}sin\left(2\pi W_f\mathbf{x}\right)\\ cos\left(2\pi W_f\mathbf{x}\right)\end{array}\right]\in {\mathbb{R}}^{2n_1},$$

(6)

where $W_f\in {\mathbb{R}}^{n_1\times 20}$ is a trainable frequency matrix. The output of the encoding layer (6) then passes through the rest of the network. The hyperparameters, including the frequency matrix’s projection dimension size $n_1$ as well as the number of neurons and the choice of the activation functions, are selected by hyperparameter search that will be further discussed in Section 2.4.1. The architecture of the FoNet is summarized in Figure 4. The trainable parameters $\theta$ include the frequency matrix $W_f$ and weights and biases of the hidden layers. The FoNet is trained by minimizing the same loss function (5) as the PINN.

It should be pointed out that we considered an ANN model with two hidden layers, the same number of layers as PINN and FoNet models, as well as an ANN model with three hidden layers. However, these ANN models did not perform well in our study. As a result, we opted for a deeper ANN model with four hidden layers, which exhibited improved performance. In

Table A1. ANN performance for 2, 3, 4 layers on Fold 1 test dataset at Port Chicago. The best performing values are highlighted in bold fonts.

	ANN
Evaluation Metrics	2 Layers	3 Layers	4 Layers
$r^2$	0.958	0.965	0.968
Bias	−6.491	−6.242	−5.302
RSR	0.338	0.313	0.307
NSE	0.886	0.902	0.906

in Appendix A.1, we provide a performance comparison between the two-layered, three-layered, and four-layered ANN models on the Fold 1 test dataset at the untrained location Port Chicago as an example.

2.4. Hyperparameter Search and Data Split

2.4.1. Hyperparameter Search

Proper selection of hyperparameters (i.e., the number of neurons and the activation functions) is crucial to achieving optimal model performance. We perform random hyperparameter searches for the machine-learning models to obtain the optimal hyperparameters. The search space for the number of neurons in each hidden layer for each model is set as follows:

• ANN—the number of neurons in hidden layers 1 and 2, denoted as $n_1$ and $n_2$, respectively, lie in the range of $[4,8,12,\dots ,32]$, and the number of neurons in hidden layer 3 and 4, denoted as $n_3$ and $n_4$, lie in the range of $[2,4,6,\dots ,16]$.
• PINN—$n_1$ lies in the range of $[4,8,12,\dots ,32]$ and $n_2$ lies in the range of $[2,4,6,\dots ,16]$.
• FoNet—$n_1$, which represents the projection dimension size of the frequency matrix, lies in the range of $[4,8,12,\dots ,32]$, and $n_2$ and $n_3$, the number of neurons in hidden layer 1 and 2, respectively, lie in the ranges of $[4,8,12,\dots ,32]$ and $[2,4,6,\dots ,16]$, respectively.

The possible activation functions for all three models include ReLu, Tanh, ELU, and Sigmoid functions. Each model is subject to 50 random hyperparameter searches, and the combination of hyperparameters with the smallest loss function value is selected as the optimal hyperparameter. We list these optimal hyperparameter combinations in Appendix A.2.

2.4.2. Data Split

To measure the salinity estimation capabilities of the machine-learning models, we use the Blocked Cross-Validation [55] procedure. Similar to k-fold cross-validation, the Blocked Cross-Validation partitions the dataset into k blocks of equal size, and leaves one block for testing and the other $k-1$ blocks for training. The difference is that there is no initial random shuffling of the data, so the natural temporal ordering is preserved within each block. We consider the Blocked Cross-Validation with 5 blocks for the DSM2-simulated 25-year salinity dataset in the range from 1 January 1991 to 31 December 2015, resulting in 80% of the dataset for training and 20% for testing. However, we do not use any of the data at the Port Chicago location for training. Instead, the salinity value at Port Chicago will only be used for testing to evaluate the models’ performance at an untrained location. The procedure of the Blocked Cross-Validation is illustrated in Figure 5 where we numbered the folds to reference them later.

2.5. Evaluation Metrics

We use the following four statistical metrics to evaluate the performance of the machine-learning models: the square of the correlation coefficient $r^2$ [56], percentage bias [57], root mean standard deviation ratio (RSR) [58], and the Nash–Sutcliffe efficiency coefficient (NSE) [59]. Their formulas are shown in

Table 1. Evaluation metrics.

Name	Definition	Formula
$r^2$	Squared Correlation Coefficient	${\left(\frac{{\sum }_t^T\left|(S_{ref}^t-\overline{S_{ref}})(S_{ML}^t-\overline{S_{ML}})\right|}{T{\sigma }_{ref}{\sigma }_{ML}}\right)}^2$
Bias	Percent Bias	$\frac{{\sum }_t^T(S_{ML}^t-S_{ref}^t)}{{\sum }_t^T{S}_{ref}^t}\times 100\%$
RSR	RMSE-observations standard deviation ratio	$\frac{\sqrt{{\sum }_t^T{(S_{ref}^t-S_{ML}^t)}^2}}{\sqrt{{\sum }_t^T{(S_{ref}^t-\overline{S_{ref}})}^2}}$
NSE	Nash–Sutcliffe Efficiency coefficient	$1-\frac{{\sum }_t^T{(S_{ref}^t-S_{ML}^t)}^2}{{\sum }_t^T{(S_{ref}^t-\overline{S_{ref}})}^2}$

, where S stands for the salinity value and $\overline{S}$ stands for the average salinity value of the dataset. t represents an arbitrary day in the dataset, T is the total number of samples in the dataset, $\sigma$ is the standard deviation, and the subscripts ‘ref’ and ‘ML’ designate the target values (DSM2-simulated) and the machine-learning model-estimated values, respectively. $r^2$ measures the linear-relation strength between the model-estimated salinity and the target salinity, percent bias quantifies how much the model under- or overestimates the salinity, RSR is a standardized representation of the root mean squared error (RMSE) between model outputs and targets, and NSE quantifies the predictive capacity of the models with the global mean of target sequences. For $r^2$ and NSE, a value closer to 1 indicates better performance, while for bias and RSR, a value close to 0 indicates better performance.

2.6. Implementation Details

The experiments are conducted using Python 3.9.16 and the machine-learning models are built and trained using the TensorFlow 2.10.1 library. The programming codes are executed in Google Colaboratory, a hosted Jupyter notebook service that provides access to GPUs. For all three machine-learning models, Adam optimizer [60] with a constant learning rate of 0.01 is used for training. To prevent overfitting of the models, the batch size is set to 128 and the training process is limited to 5000 epochs. Furthermore, if the loss function on the test set does not decrease for consecutive 50 epochs, the training process stops.

3. Results

In this section, we present the performance results of our three machine-learning models, namely the ANN, PINN, and FoNet models. We evaluate the performance of these models quantitatively using four statistical evaluation metrics described in Section 2.5, and qualitatively by visually inspecting the time series plots and the scatter plots. In the first subsection, we present the results on the three trained locations Martinez, Chipps Island, and Pittsburg, and in the second subsection, we present the results on the untrained (independent) location Port Chicago.

3.1. Performance Results on Trained Locations

To compare the performance of our three machine-learning models (ANN, PINN, and FoNet) and assess their generalizability, we use a 5-fold Blocked Cross-Validation and evaluate four statistical metrics ($r^2$, percentage bias, RSR, and NSE) for each fold and location. These metrics help us quantify the accuracy, precision, and goodness of fit of the models to the data. In Figure 6, we present the box and whisker plots of these metrics of the folds for each model on the training data (left column) and test data (right column) at the three locations (Martinez, Chipps Island, and Pittsburg) used for model training. The box and whisker plots show the distribution of the metric values for each model and provide a visual comparison of their performance.

The box and whisker plots in Figure 6 demonstrate a clear improvement in performance for the physics-informed models (PINN and FoNet) over the standard model (ANN). For all four considered metrics and at all three locations, the PINN and FoNet models outperform the ANN model by a significant margin. The PINN and FoNet models attain smaller percent bias and RSR values and larger correlation coefficient $r^2$ and NSE values than the ANN models. In particular, at Pittsburg, the improvement is prominent as the PINN and FoNet models achieve NSE values of around 0.93 on test data, compared to around 0.55 for the ANN model. The PINN and FoNet models perform similarly at locations Martinez and Chipps Island, but at Pittsburg, FoNet has a slight edge over PINN in estimating salinity, with slightly lower variance in its metric values. The exact values of the metrics for each fold are shown in the tables in Appendix A.3. Overall, there is no evidence of overfitting for any of the models, as they perform similarly on the training and test data.

We visually inspect the time series plots of the estimated salinity values of our three machine-learning models in comparison to the target salinity values (DSM2-simulated). For better visibility we consider two models at a time, along with DSM2-simulated salinity values, first comparing the estimated salinity values of ANN and PINN models and then those of PINN and FoNet models. Furthermore, we show the time series plots for test data only to more closely examine the performance of the models for a shorter time span of five years rather than 20 years for training data. Here, in this section, we display the time series plots for Fold 2, i.e., the test data being 1 January 1996 to 31 December 2000, as an exemplary time series plots of the Blocked Cross-Validation procedure. The rest of the time series plots of the folds from the procedure are available in Appendix A.4.

Figure 7 displays the daily time series plots of DSM2-simulated salinity values in blue along with the daily time series plots of estimated salinity values by the ANN and PINN models in green and orange, respectively, at the three locations (Martinez, Chipps Island, and Pittsburg) used for model training. Both machine-learning models perform well in estimating the overall seasonal salinity patterns throughout the five years, but PINN estimates the target salinity values more accurately than ANN overall. PINN outperforms ANN in estimating high peak salinity values. Specifically, at Martinez, PINN estimates the high salinity values more closely to target salinity than ANN, which tends to underestimate them. Similarly, at Chipps Island, PINN follows the temporal pattern of high salinity more closely than ANN, which tends to overestimate these peak values. At Pittsburg, ANN produces estimates that are quite noisy and overshoot the target salinity frequently, while PINN can estimate salinity more closely to the target in its estimations. Similar to the time series plots of Figure 7, Figure 8 illustrates the daily time series plots of PINN and FoNet models in orange and black, respectively, along with DSM2-simulated salinity values in blue. FoNet model performs quite similarly to PINN model but achieves slightly better evaluation metrics. Visual inspection of the time series plots in Figure 8 indicates that FoNet estimates low salinity values more accurately than PINN.

Figure 9 shows scatter plots comparing the estimated salinity values of the three models to the target salinity values (DSM2-simulated). The first, second, and third rows correspond to locations Martinez, Chipps Island, and Pittsburg, respectively, and the first, second, and third columns correspond to the ANN, PINN, and FoNet models, respectively. Each dot on a scatter plot represents a salinity data point in the testing dataset of one of the five folds, with its x-coordinate value being the model’s estimated salinity value and its y-coordinate value being the target salinity value. The scatter plots reveal that the ANN models tend to either underestimate or overestimate the salinity values. Specifically, the ANN model underestimates salinity values at Martinez, overestimates salinity values at Chipps Island, and overestimates salinity values more significantly at Pittsburg. This suggests that the ANN model shifts from underestimation to overestimation from the westernmost to the easternmost location. In contrast, the physics-informed models, PINN and FoNet, do not exhibit such location-dependent underestimation/overestimation behaviors. Both models seem to perform similarly at all three locations. At Pittsburg, FoNet estimates the target salinity slightly more accurately than PINN; the latter overestimates some of the salinity values in the mid-range while the former does not. The averaged statistical metrics of the test datasets of the 5 folds, displayed in each scatter plot, confirm these observations.

3.2. Performance Results on Independent Untrained Location

In this subsection, we evaluate the three machine-learning models at an untrained location, Port Chicago. By validating a model at an untrained location we can measure its predictive capacity at an unknown location for the testing time period at hand.

Figure 10 and Figure 11 illustrate the comparison of the four evaluation metrics ($r^2$, percentage bias, RSR, NSE) for the three models on the Port Chicago test datasets of the five folds in scatter plots. Figure 10 compares the performance of ANN and PINN, while Figure 11 compares the performance of PINN and FoNet. Each dot in a scatter plot corresponds to one of the five folds. The scatter plots in Figure 10 show that PINN outperforms ANN, as it achieves higher $r^2$ and NSE values, and lower RSR values for all five folds. Although PINN obtains a slightly higher percent bias than ANN for a couple of folds, the percentage biases are smaller for PINN in the other three folds. The scatter plots in Figure 11 indicate that PINN performs slightly better than FoNet at Port Chicago, as PINN achieves smaller percent bias and RSR, and larger NSE than FoNet for most of the five folds. Appendix A.3 contains tables with the exact values of the metrics for each fold.

The time series plots in Figure 12 compare the estimated salinity values of our three models to the target salinity values (DSM2-simulated). Once again, for better visibility, we consider two models at a time: the top time series plots show DSM2-simulated salinity values (in blue) with estimated salinity values of ANN (in green) and PINN (in orange), and the bottom time series plots show DSM2-simulated salinity values (in blue) with estimated salinity values of PINN (in orange) and FoNet (in black). These time series correspond to the test dataset of Fold 5, spanning from 1 January 2011 to 31 December 2015, at Port Chicago. Visual inspection indicates that ANN is less accurate than PINN and FoNet in estimating salinity at Port Chicago. Particularly for high salinity values in 2015, the performance of ANN drops as it noticeably underestimates the salinity. In contrast, both PINN and FoNet outperform ANN, and show similar levels of accuracy in salinity estimation. Appendix A.5 provides additional time series plots for the other folds.

Figure 13 displays a comparison of the estimated salinity values of the three models (ANN, PINN, FoNet) to the target salinity values (DSM2-simulated) at Port Chicago in scatter plots. Each dot on a scatter plot represents a salinity data point in the testing dataset of one of the five folds, with its x-coordinate value being the model’s estimated salinity value and its y-coordinate value being the target salinity value. All three models show no clear bias towards underestimation or overestimation at Port Chicago. However, ANN deviates from the target salinity more frequently than PINN and FoNet, especially for salinity values in the mid-to-high range. The salinity estimations of PINN and FoNet are quite similar. The averaged statistical metrics of the test datasets of the 5 folds are displayed in each scatter plot and confirm these observations. In particular, smaller NSE and larger RSR values for ANN indicate its under-performance in comparison to PINN and FoNet. PINN and FoNet have similar values in their four evaluation metrics.

4. Discussion

4.1. Implications

Our study has significant scientific implications. For the first time, we introduce flow–salinity modeling in the Delta using physics-informed neural network (PINN) models and demonstrate their potential to bridge the gap between process-based models and data-driven models. As machine-learning models with neural network structures, PINN models can efficiently simulate salinity, such as data-driven artificial neural network (ANN) models, while incorporating the advection–dispersion equation that governs the underlying physical laws of the flow–salinity relationship, such as process-based models. Furthermore, the PINN models we develop can be applied to other estuarine environments where flow–salinity modeling is of interest.

Our study also demonstrates the practical advantages of using PINN models, which offer comparable efficiency to data-driven ANN models while improving the accuracy of salinity simulations in the Delta. In terms of statistical metrics $r^2$, bias, RSR, and NSE, the PINN models outperform the ANN model across the board. This is particularly appealing to real-time operations and long-term planning which both require desirable estimates on the salinity levels.

4.2. Limitations and Future Work

Although this study demonstrates the improved salinity estimation capabilities of physics-informed neural networks compared to conventional neural networks, it is limited to only four locations which all lie within the flat estuary waterway in the western part of the Delta. There are other important locations of ecological significance in the interior parts of the Delta, where salinity values are less significant due to less impact of seawater and waterways between locations are irregularly shaped, making it difficult to apply the 1-dimensional advection–dispersion equation. In future studies, we plan to explore different methodologies of PINN models to model salinity at these locations. We acknowledge that our PINN models are simplified by assuming constant values for the estuary cross-sectional area A and longitudinal dispersion coefficient K in the advection–dispersion equation. In reality, these parameters can vary both spatially and temporally. However, we emphasize that they were treated as constants in our PINN model for the sake of simplicity. Our main focus was to conduct a proof-of-concept study of the PINN model and compare it with a simpler ANN model. Moreover, we currently lack advection–dispersion coefficient data which are not available from field observations, as it varies with the cross-sectional area. Obtaining the advection–dispersion coefficient data from modeling is an ongoing research effort for us. In future studies, we plan to incorporate space-variant cross-sectional area and advection–dispersion coefficients as additional variables in our PINN models. Furthermore, while the DSM2-simulated salinity values are suitable for validating our proof-of-concept models, they are generally less noisy than historical observed data, which can make it easier for the models to train on. To further strengthen the salinity estimation capabilities of our PINN models, we plan to evaluate them on historically observed salinity data.

In this study, we explore a machine-learning model called FoNet, a variation of a simple PINN model with an MLP architecture. However, there are other variations of PINN models with different neural network architectures. For instance, the long short-term memory (LSTM) architecture has been explored in predicting salinity in navigable waterways in Belgium [40]. There are extensions of FoNet, such as the Spatio-temporal Fourier Feature Network [61], which aims to tackle differential equations exhibiting multi-scale behaviors by applying multiple Fourier feature encodings initialized with different frequencies to input variables. We plan to explore these and other variations of PINN models in the future.

5. Conclusions

Salinity modeling in the Sacramento–San Joaquin Delta of California has traditionally relied on two separate approaches: process-based models and data-driven models. In this study, novel machine-learning models based on the framework of physics-informed neural networks are developed and applied in estimating salinity in the study locations in the Delta. These models integrate the flow–salinity relationships of process-based models with the computational efficiency of data-driven artificial neural network models. Specifically, the advection–dispersion equation, which describes the flow–salinity relationship, is embedded into the loss function of a multilayer perceptron. The findings of the study show that these new models outperform a benchmark artificial neural network in accurately estimating salinity levels in the study locations in the Delta. The efficiency and improved accuracy in this proof-of-concept study indicate the promising potential of these proposed models for salinity estimation in the Delta.