Decision Theory versus Conventional Statistics for Personalized Therapy of Breast Cancer

Abstract

Estrogen and progesterone receptors being present or not represents one of the most important biomarkers for therapy selection in breast cancer patients. Conventional measurement by immunohistochemistry (IHC) involves errors, and numerous attempts have been made to increase precision by additional information from gene expression. This raises the question of how to fuse information, in particular, if there is disagreement. It is the primary domain of Dempster–Shafer decision theory (DST) to deal with contradicting evidence on the same item (here: receptor status), obtained through different techniques. DST is widely used in technical settings, such as self-driving cars and aviation, and is also promising to deliver significant advantages in medicine. Using data from breast cancer patients already presented in previous work, we focus on comparing DST with classical statistics in this work, to pave the way for its application in medicine. First, we explain how DST not only considers probabilities (a single number per sample), but also incorporates uncertainty in a concept of ‘evidence’ (two numbers per sample). This allows for very powerful displays of patient data in so-called ternary plots, a novel and crucial advantage for medical interpretation. Results are obtained according to conventional statistics (ODDS) and, in parallel, according to DST. Agreement and differences are evaluated, and the particular merits of DST discussed. The presented application demonstrates how decision theory introduces new levels of confidence in diagnoses derived from medical data.

1. Introduction

1.1. Biomarkers: A Cornerstone of Personalized Medicine for Breast Cancer

Biomarkers gain importance in selecting treatments optimized and personalized to the individual needs of patients, as envisaged in personalized medicine [1,2,3,4]. Breast cancer treatment has also benefited from personalized medicine, via molecular subtyping [5,6,7,8], the Gene expression Grade Index [9], pathway analysis and networks [10,11,12,13,14] and a plethora of expression signatures [15,16,17,18,19,20,21,22], dedicated to special questions and issues. Well-known indicators supporting therapy selection for breast cancer are PAM50 [23,24], PREDICT [25], and the Genomic Grade Index [26].

For breast cancer, the HER2-status (human epidermal growth factor receptor 2) of a patient is one of the most important prognostic factors [27,28]. The majority of patients (75–85%) are HER2-negative and, therefore, have a much better prognosis. In this work, we focus on these and disregard HER2-positive ones, in order to increase the homogeneity of data and precision of predictions. HER2 is routinely determined by immunohistochemistry (IHC). Numerous studies covered the significance and accuracy of estimates [29,30,31,32,33,34,35,36]. In a previous paper [37], we described, in detail, how to select patients who are HER2 negative to a high degree of confidence by using the ODDS method. We draw on the very same database in the current work.

Among HER2-negative patients, the hormone receptor status of estrogen (ER) and progesterone (PGR) are of focal importance. Clinically, patients are considered receptor-positive if at least one of both receptors (ER or PGR) is found positive. Since hormone receptors play a role in promoting cancer, hormone therapy has to be part of an effective treatment. In patients without metastases, hormone therapy may even render chemotherapy unnecessary.

However, if the receptor status is accidentally estimated as false positive, hormone treatment will not work and the patient may be deprived of life-saving chemotherapy. Therefore, numerous studies have evaluated the quality of receptor assessment [38,39] and revealed a possible rate of misclassification of 10% to 20% [40,41,42]. Although standard operation procedures have been implemented [40,43,44], the improvement of precision is still desirable [45].

One possibility is adding information from gene expression. Some approaches merely used visual inspection to set cut-points between positive and negative [46], some used frequency distributions of expression values [42,47], random sampling [48] or fuzzy rules [49] and other methodological advances in gene expression analysis [50,51,52,53,54,55,56,57].

We have elaborated and improved the above approaches [58,59] by introducing Dempster–Shafer decision Theory (DST) [60] into personalized medicine [37]. This is promising, since decision theory has demonstrated its benefit in many technical settings, such as self-driving cars [61,62,63], observing a driver’s vigilance [62], aviation security [64,65] and also in some medical applications, such as image-based decisions [66], diagnosis of prostate [67] and breast cancer [68]. The specific merit of DST is the capability to handle unclear or contradicting information obtained from different sources about the same issue in question (e.g., receptor status). DST is able to combine multifactor and even diverging evidence, according to exact algorithms, with the potential to increase the precision of medical decisions.

In the present work, we draw on data from our previous paper [37] and elaborate on the key differences between classical statistics and DST. Ternary plots are introduced for the interpretation of probabilities, in case of contradicting evidence—a potent concept from technics is tailored to the needs of personalized medicine.

1.2. Basic Concepts of Decision Theory for Hormone Receptor Status Assessment

DST is a general theory for reasoning with uncertainty [69]. It starts with the outcome of measuring processes, rather than from ‘true values’ present in reality, as conventional statistics does. We present DST in a mode with only two statuses, ‘+’ and ‘−’. This simplifies the formalism significantly.

Suppose some continuous variable, d, is being measured (e.g., d ≙ deepness of IHC-staining). Conventional statistics would derive a single number from d, the probability p for the receptor being positive, given the measured value of d. Consequently, 1 − p would be the probability for being negative.

DST provides two numbers to characterize possible predictions based on measuring d:

• The belief α(d), gives the probability (weight) that, upon measuring this particular value of d, the prediction ‘positive’ can be made based on the quality of measurement (classification ‘with full right’).
• The uncertainty θ(d), characterizing the probability (weight), that the prediction ‘positive’ could root in chance and not in quality of measurement. Belief and uncertainty taken together yield the total probability (termed ‘plausibility’) to obtain the prediction ‘positive’, given the measured value of d (α + θ = pl).
• Finally, a third number can be computed from belief and uncertainty, the probability β(d) for yielding the prediction ‘negative’ by quality of measurement, given d. We always have: α + θ + β = 1; hence, β can be computed from α and θ.

In the special case of only two statuses, as considered here, the triple (α, β, θ) is equivalent to a piece of ‘evidence’. DST not only yields probabilities for a positive versus negative outcome, but, additionally, incorporates the uncertainty of prediction [70]. This represents a significant surplus and motivates its introduction into personalized medicine.

A second advantage of DST is its capability of merging evidence from different sources (see also Figure 1 and the graphical abstract). In our case these will be:

• For estrogen (ER)

  ○

  Receptor status predicted from expression of the receptor gene

  ○

  Receptor status predicted from expression of a co-gene

  ▪

  Combining above evidence by Dempster evidence combination rule ⊕_D

  ○

  Receptor status predicted from IHC

  ▪

  Combining evidence from gene expression and IHC by Yager evidence combination rule ⊕_Y

• For progesterone (PGR)

  ○

  Receptor status predicted from expression of the receptor gene

  ○

  Receptor status predicted from expression of a co-gene

  ▪

  Combining above evidence by Dempster evidence combination rule ⊕_D

  ○

  Receptor status predicted from IHC

  ▪

  Combining evidence from gene expression and IHC by Yager evidence combination rule ⊕_Y

• Hormone receptor status is finally obtained by combining the statuses of estrogen and progesterone using a multiplicative combination rule ⊗

Details and references regarding the above workflow will be given in the following and are illustrated in Figure 1.

1.3. Ternary Plots: A Novel View on Evidence in Personalized Medicine

Another important point of the current work is to introduce so-called ‘ternary plots’, as a tailored tool to display not only probabilities, but also the uncertainties involved. We will evolve the framework step by step, to contrast conventional statistics against DST, thereby featuring the surpluses of DST.

2. Materials and Methods

2.1. Preliminaries on the Structure of the Methods’ Section

For the sake of readability and to present this paper self-contained, data cleansing and the concept of responsibility functions are not expanded in the methods sections but recapitulated in the Appendix A.1, Appendix A.2 and Appendix A.3. These computational procedures equal those detailed in our previous work [37]. Moreover, the methods’ Section 2.2.1, Section 2.2.2, Section 2.2.3 and Section 2.2.4 as well as Appendix A.6 are restricted to a single gene for didactical reasons (see the ‘receptor gene sub-model’ in Figure 1). In a thorough primary introduction, it seems important to demonstrate in detail how evidence from gene expression and IHC intermingle and eventually produce remarkable patterns in the data. These patterns demonstrate the dominant impact of IHC status on final predictions. Figures in Section 2.2.1, Section 2.2.2, Section 2.2.3 and Section 2.2.4 and Appendix A.6 exemplify intricate features on data for estrogen but methods are general and identically apply to all other parts of the ‘full’ model.

In the final sections of methods (Section 2.3), we return to the full model shown in Figure 1, including co-genes. Line-like patterns are smeared out and do not remain visible as clearly as in the single-gene case. This full model was used to obtain the actual results for the patient cohort (Section 3).

2.2. Estrogen Receptor Gene Sub-Model

2.2.1. Logistic Regression as Prerequisite

Receptor status is related to gene expression (x_Expr) as follows: The responsibility function for positive receptor status, $r_{+}$, defines the probability for a positive receptor status, given the expression value x_Expr. Likewise, $r_{-}$ relates to negative receptor status. We used logistic regression

$$\begin{array}{c}{\mathrm{r}}_{+}\left(x_{\mathrm{Expr}}|c_0,c_1\right)=\frac{\exp \left(c_0 + c_1{x}_{\mathrm{Expr}}\right)}{1+\exp \left(c_0 + c_1{x}_{\mathrm{Expr}}\right)}\\ {\mathrm{r}}_{-}\left(x_{{}_{\mathrm{Expr}}}|c_0,c_1\right)=1-{\mathrm{r}}_{+}\left(x_{\mathrm{Expr}}|c_0,c_1\right)\end{array}$$

(1)

and estimated the parameters $c_0$ and $c_1$ against IHC-measurements, separately for each gene and co-gene, for results see

Table A2. Probe sets and logistic regression for receptor genes and co-genes. $c_0$ and $c_1$ are coefficients from logistic regression for IHC values (dependent variable) versus gene expression (independent variable) [37,58]. Probe sets refer to the Affymetrix chip U133A + 2.0. For ‘deviance of fit’, see p. 118 in McCullagh [108]. Upper limits for beliefs ($\widehat{\alpha }$, $\widehat{\beta }$) are explained in Appendix A.4 and Appendix A.5. See Equations (A1) and (A2) for computing the limits, and Equation (4) for their application.

				Logistic Regression Parameters		Logistic Regression Quality		Upper Limits for Beliefs
			Probe Set	${\mathit{c}}_0^{}$	${\mathit{c}}_1^{}$	Deviance of Fit	Number of Samples	$\widehat{\mathit{\alpha }}$	$\widehat{\mathit{\beta }}$
estrogen	gene	ESR1	205225_at	9.905	−1.061	1086.6	2213	0.814	0.887
	co- gene	AGR3	228241_at	5.582	−0.710	1253.1		0.794	0.840
progesterone	gene	PGR	208305_at	7.449	−0.983	1107.4	1700	0.753	0.702
	co- gene	ESR1	205225_at	8.617	−0.834	1249.8		0.618	0.817

. Figure 2 shows the responsibility function r₊ for positive estrogen (red dashed curve). r₋, for negative estrogen (blue dashed curve), is based on the same regression coefficients, see Equation (1). A similar analysis was performed for progesterone, see Figure A1 for graphics and Table A2 for numerical values.

2.2.2. Evidence of Receptor Status Based on Expression of Receptor Gene

Based on logistic regression, gene expression measurements lend themselves to derive evidence of receptor status according to Dempster–Shafer decision theory [70]. In the following, we formulate rules and principles in general terms of ‘gene expression’, x_Expr, to keep notation general (later on, the first example with real data will specifically refer to estrogen receptor diagnostics. Even later, the very same procedure will be applied to progesterone).

Assume the variable gene expression, x_Expr, is prognostic for receptor status. Given a measurement of x_Expr, DST attributes two independent numbers, as outlined below:

• ${\alpha }_{\mathrm{Expr}}\left(x_{\mathrm{Expr}}\right)$: the belief (sometimes also called ‘degree of belief’ or ‘credibility’ [74]) for receptor status being positive on good grounds or by quality of the measuring method that has yielded x_Expr;
• β_Expr: the belief (probability) for receptor status being non-positive (i.e., negative) on good grounds or by quality of the measuring method;
• θ_Expr is a third quantity considered: the probability that the receptor status is uncertain.

α, β and θ are also called ‘masses’ of the respective outcomes. They are by definition larger than or equal to zero (${\alpha }_{\mathrm{Expr}}\ge 0, {\beta }_{\mathrm{Expr}}\ge 0$, ${\theta }_{\mathrm{Expr}}\ge 0$), and if a mass equals zero, in our setting zero corresponds to the ‘empty set’, i.e., an outcome that will never be found [75]. Masses always add up to unity, and hence we talk about normalized mass functions [76]:

$${\alpha }_{\mathrm{Expr}}+{\beta }_{\mathrm{Expr}}+{\theta }_{\mathrm{Expr}}=1$$

(2)

Hence a third number is in fact redundant (may always be computed from the other two). Decision theory even considers a fourth quantity, called plausibility; it is also redundant but intuitive and useful:

$$p{l}_{\mathrm{Expr}}={\alpha }_{\mathrm{Expr}}+{\theta }_{\mathrm{Expr}}=1-{\beta }_{\mathrm{Expr}}$$

(3)

$p{l}_{\mathrm{Expr}}$ indicates the probability of a positive status being plausible, given the measurement x_Expr as is. The plausibility of a given outcome sums up everything either supportive or neutral, but excludes everything advocating the opposite outcome. The exactly opposite outcome is represented by β_Expr.

The output of above procedure is the evidence $\left({\alpha }_{\mathrm{Expr}},{\beta }_{\mathrm{Expr}}\right)$ for receptor status, based on the expression (x_Expr,) of a gene (in general); data in Figure 2 were shown for the receptor gene of estrogen (ESR1). Note that finally, in the ‘full’ model, 4 such pieces of evidence (4 pairs of numbers) will be obtained: (1) for the estrogen receptor gene and (2) its co-gene; (3) for the progesterone receptor gene and (4) its co-gene.

The beliefs in receptor positive, α_Expr, and negative, β_Expr, may be obtained from gene expression alone, x_Expr, as demonstrated above. Doing so, maximum expression corresponds to a responsibility function r₊(x_Expr) close to 1, see Figure 2. However, not even a gene expression that large can guarantee that the receptor is truly positive. Hence, the belief in positivity, α_Expr, actually must be less than 1.

We chose to model this fact by a factor, $\widehat{\alpha }$, called ‘upper limit for belief’ in Table A2. For details of calculation see Appendix A.4 and Appendix A.5.

All in all we obtain the belief in receptor positivity after measuring $x_{\mathrm{Expr}}$:

$$\begin{gathered} {\alpha }_{\mathrm{Expr}}\left(x_{\mathrm{Expr}}\right)={\widehat{\alpha }}_{\mathrm{Expr}}\cdot r_{+}\left(x_{\mathrm{Expr}}|c_0,c_1\right) \\ {\beta }_{\mathrm{Expr}}\left(x_{\mathrm{Expr}}\right)={\widehat{\beta }}_{\mathrm{Expr}}\cdot r_{-}\left(x_{\mathrm{Expr}}|c_0,c_1\right) \end{gathered}$$

(4)

α_Expr is represented by the increasing solid red curve in Figure 2, β_Expr by the declining blue one. The remaining uncertainty, θ_Expr, is easily computed from reformulating Equation (2)

$${\theta }_{\mathrm{Expr}} =1-{\alpha }_{\mathrm{Expr}}-{\beta }_{\mathrm{Expr}}$$

(5)

and is shown as ochre curve in Figure 2. The two numbers (α_Expr, β_Expr) are collectively called ‘evidence’ of receptor status, given a measurement of the continuous variable ‘gene expression’, x_Expr. They enrich the information given by a single number, the probability p, known from conventional statistics, quantifying the chances of receptor statuses, a similar procedure applies to the receptor gene of progesterone, see Figure A1.

2.2.3. Combining Evidence from Receptor Gene Expression and IHC

To further increase precision of receptor status diagnostics, evidence from gene expression (α_Expr, β_Expr) and evidence from IHC (α_IHC, β_IHC) are combined by so-called ‘evidence combination rules’ (ECR). DST offers several such rules [69,74], out of which we consider two, the ‘Dempster–Shafer’ ECR, ⊕_D, and the Yager ECR, ⊕_Y [77,78]. We chose the Yager rule, as it more easily accommodates contradicting items of evidence, see also Section 4.4 in the discussion. Performing some algebra, as detailed in our previous work [37], one finally obtains:

$$\begin{gathered} {\alpha }_{\mathrm{Rez}}={\alpha }_{\mathrm{Expr}}{\alpha }_{\mathrm{IHC}}+{\theta }_{\mathrm{Expr}}{\alpha }_{\mathrm{IHC}}+{\alpha }_{\mathrm{Expr}}{\theta }_{\mathrm{IHC}} \\ {\beta }_{\mathrm{Rez}}={\beta }_{\mathrm{Expr}}{\beta }_{\mathrm{IHC}}+{\theta }_{\mathrm{Expr}}{\beta }_{\mathrm{IHC}}+{\beta }_{\mathrm{Expr}}{\theta }_{\mathrm{IHC}} \\ {\theta }_{\mathrm{Rez}}={\theta }_{\mathrm{Expr}}{\theta }_{\mathrm{IHC}}+{\alpha }_{\mathrm{Expr}}{\beta }_{\mathrm{IHC}}+{\beta }_{\mathrm{Expr}}{\alpha }_{\mathrm{IHC}} \end{gathered}$$

(6)

As IHC-evidence is made up of two sets of constants, combination with gene expression yields two sets of curves, one for IHC⁻ and one for IHC⁺, see Figure 3.

A definite decision for positive receptor status is obtained if the combined evidence exceeds 0.5 (α > 0.5). In that case the belief in positive surmounts the sum of both other beliefs (β + θ ≤ 0.5) and dominates. Hence, the dotted line α = 0.5 represents a decision border and will be analogously outlined in the following figures.

2.2.4. Ternary Plots of Evidence for Personalized Medicine: A Primer

Note that belief, plausibility and uncertainty are not independent but always sum up to unity for a given sample, see Equation (2). This mathematical property allows for a special graphic display, called ‘ternary plot’, as follows. When plotting these data in an ordinary 3-dimensional scatter plot with coordinates (α, β, θ), points of all samples lie within a single plane (of evidence), see Figure 4a. This is due to Equation (2), which—in mathematical terms—is nothing else than the equation of a plane in three dimensions [79]. This ‘plane of evidence’ may be viewed in orthogonal projection (https://en.wikipedia.org/wiki/Orthographic_projection (accessed on 26 March 2022)) which still contains all information but fits into two dimensions and is called ‘ternary plot’, see Figure 4b.

Ternary plots are widely used in technology and science but have only marginally entered the medical sciences [80]. They might also gain importance in personalized medicine but deserve some skillful understanding. Hence we provide a short primer.

A ternary plot is powerful whenever three quantities (hence the name ‘ternary’) add up to a constant, for each individual considered. For example, a biological fluid (say milk) may be composed of water, protein and fat (three components) and nothing else. Clearly, the percentages of water, protein and fat must then add up to 100%. For a set of milk samples, these percentages may be visualized by points in a 3D scatter plot, such as Figure 4a. If we consider mixtures of different composition (e.g., skimmed, normal and fat milk and many other possible kinds) and plot their corresponding 3D points, we will be surprised to realize that all these points lie within a single flat plane in 3-dimensional space. The reason is a mathematical one: if coordinates always add up to a given constant, this is the very representation of a plane in mathematical terms [79]. This may be fruitfully exploited for personalized medicine as follows:

What is true for three components of a substance (milk) is also true for evidence composed of three numbers, α, β and θ, since they also add up to unity due to Equation (2). Hence, points of evidence (α, β, θ) for any single patient, lie within the same (2-dimensional) plane in 3D. This plane always lies in the same, specific position and orientation, for the following reason: the point (α = 1, β = 0, θ = 0), represents a valid point of evidence (adding up to unity) and must be part of the plane. Therefore, the plane cuts the α–axis at α = 1, see Figure 4a. Likewise, the plane also cuts both other axes at β = 1 and θ = 1, respectively. This uniquely defines an equilateral triangle in the 3D coordinate system, see Figure 4a.

Even though the plane of evidence lies embedded in a 3D coordinate system, it is by itself just a 2-dimensional object, as every flat plane is. Therefore, without any loss of information regarding the location of points (representing evidence) we may perform an orthogonal projection along the heavy arrow shown in Figure 4a. This yields a so-called ‘isometric view’. The triangle, viewed face-on, appears equilateral, now in two dimensions, see Figure 4b.

A ternary plot does not have its axes at right angles, as ordinary plots do. To read off the coordinates of a point from such a ternary plot, several methods are available, out of which we propose the following (altitude method), illustrated in Figure 5:

Each of the three components of evidence, e.g., α, the ‘belief in positive’, has its own scale, see the dashed heavy line in orange; it starts on the left with α = 0 at a right angle from a triangle’s left side and runs towards the opposing corner, where α = 1 (indicating ‘surely positive’). See also the scale with numerical values aside. The two other scales, for β and θ, are defined analogously (not shown for simplicity).

Given some point within the ternary plot (see the heavy black dot in Figure 5), corresponding evidence components (α, β, θ) can be read off as follows. Note the lines being drawn perpendicular to each side of the triangle (light dashed lines in red, blue and beige)—they represent the axes for quantification. Values α, β, θ (red, blue, and beige, respectively) can be read off from the corresponding axis’ scale. In this example (Figure 5), the plotted point of the evidence produces a reading of α ≅ 0.22.

Note also the following intriguing features of this ternary plot:

• Parallel lines at right angles with one axis represent constant values for the respective variable (as with ordinary right-angle axes). In particular, the line crossing the α–axis at α = 0.5 (dotted red) discriminates points with α ≤ 0.5 (left upper) from those with α > 0.5 (towards lower right corner), and hence represents a decision border; points right of this border are predicted ‘positive’, since their evidence for positive is greater than for all other options (‘negative’ and ‘uncertain’) taken together.
• Decision borders segregate subsets of samples. For example, all samples within the triangle in the lower right of α = 0.5 (shaded light red) comprise samples predicted positive. Similarly, the subsets of negative and uncertain samples may be defined, see Figure 4b.
• In each corner one piece of evidence totally dominates, assuming a value of unity (α = 1: ‘surely positive’; β = 1: ‘surely negative’ and θ = 1: ‘totally uncertain’).
• Conversely, the footing point of each axis (e.g., α = 0) means that there is no indication whatsoever for the prediction at opposing corner. For example, α = 0 along the left side of the triangle, means that there is no indication whatsoever for a ‘positive’ prediction. All evidence is shared between ‘negative’ and ‘uncertain’ (β and θ). In this case β + θ = 1.
• A special role is played by the triangle’s bottom edge, running from β = 1 (left) towards α = 1 (right): for each sample along this line uncertainty θ equals zero, and all evidence is shared between belief in positive (α) and belief in negative (β), e.g., α = 0.6 and β = 0.4, while θ = 0. One may legitimately ask: “Does this mean that the prediction was made for sure?”. Since α > 0.5 and dominates both other options, we consider this prediction clearly positive. However, α = 0.6 is no more than a probability and not that much larger than the probability of the opposite outcome, β = 0.4. In reality, the outcome may well result in a negative prediction. If θ = 0, evidence masses revert back to ordinary probabilities: p⁺ = 0.6 for positive and hence p⁻ = 0.4 for negative, without indicating any uncertainty about the estimates of these two numbers. Thus, for θ = 0, decision theory’s evidence coincides with ordinary probabilities. In DST terminology the evidence is said to turn ‘Bayesian’ [74].
• In general, for θ > 0, decision theory not only gives estimates for probabilities (α, β) but additionally indicates the uncertainty of those (θ). It hence offers a wider scope of evidence, valuable in particular for personalized medicine.

Ternary plots allow for a highly transparent comparison of our two classification methods (ODDS versus DST) for each single sample:

• The location of the point indicates the prediction according to DST shown by the respective area: red triangular area for positive (+), blue for negative (-) and the white, kite-shaped area for inconclusive (inc).
• At the same time, coloring of points indicates prediction according to ODDS. For most samples, both predictions match. For some samples however, they differ, thus perfectly outlining the contrast between the two prediction methods.

Although ternary plots may seem somewhat unusual for medical application, they offer the unique capability to display three variables in two dimensions, provided their sum is constant, which is true for evidence and many other variables. We think it worth the effort to introduce ternary plots into the field of personalized medicine. They are the most adequate tool for quantitatively presenting evidence, and may in the future represent a cornerstone of personalized medicine.

2.3. Full Model: Evidence, Based on IHC, Genes, Co-Genes

In Section 2.2.1, Section 2.2.2, Section 2.2.3 and Section 2.2.4 and Appendix A.6 description was restricted to the receptor gene (no co-gene considered) in order to explain more transparent details. Now we revert to the whole model, including co-genes, see the flow chart of evidence in Figure 1.

First, we supplement estrogen expression evidence $\left({\alpha }_{\mathrm{Gen}},{\beta }_{\mathrm{Gen}}\right)$ by evidence $\left({\alpha }_{\mathrm{Co}},{\beta }_{\mathrm{Co}}\right)$ from its co-gene, AGR3; the very same procedure outlined in Section 2.2.1 and Section 2.2.2 is carried out to obtain these results, see Table A2.

2.3.1. Progesterone Evidence

Numerical results of the logistic regression for progesterone are shown in Table A2, for responsibility functions, see Figure A1. The co-gene of progesterone, incidentally, was estrogen, see Table A2.

2.3.2. Combining Evidence Form Genes and Co-Genes

Next, evidence from genes and co-genes are combined by the Dempster Evidence Combination Rule (${\oplus }_{\mathrm{D}}$) to obtain the joint evidence from gene expression:

$$\left({\alpha }_{\mathrm{Expr}},{\beta }_{\mathrm{Expr}}\right)=\left({\alpha }_{\mathrm{Gen}},{\beta }_{\mathrm{Gen}}\right){\oplus }_{\mathrm{D}}\left({\alpha }_{\mathrm{Co}},{\beta }_{\mathrm{Co}}\right)$$

(7)

In detail, the Dempster rule [77] reads:

$$\begin{array}{l}{\alpha }_{\mathrm{Expr}}=\frac{{\alpha }_{\mathrm{Gen}}{\alpha }_{\mathrm{Co}} + {\theta }_{\mathrm{Gen}}{\alpha }_{\mathrm{Co}} + {\alpha }_{\mathrm{Gen}}{\theta }_{\mathrm{Co}}}{1 - {\alpha }_{\mathrm{Gen}}{\beta }_{\mathrm{Co}} - {\beta }_{\mathrm{Gen}}{\alpha }_{\mathrm{Co}}}\\ \begin{array}{l}{\beta }_{\mathrm{Expr}}=\frac{{\beta }_{\mathrm{Gen}}{\beta }_{\mathrm{Co}} + {\theta }_{\mathrm{Gen}}{\beta }_{\mathrm{Co}} + {\beta }_{\mathrm{Gen}}{\theta }_{\mathrm{Co}}}{1 - {\alpha }_{\mathrm{Gen}}{\beta }_{\mathrm{Co}} - {\beta }_{\mathrm{Gen}}{\alpha }_{\mathrm{Co}}}\\ {\theta }_{\mathrm{Expr}}=1-{\alpha }_{\mathrm{Expr}}-{\beta }_{\mathrm{Expr}}=\\ =\frac{{\theta }_{\mathrm{Gen}}{\theta }_{\mathrm{Co}}}{1-{\alpha }_{\mathrm{Gen}}{\beta }_{\mathrm{Co}}-{\beta }_{\mathrm{Gen}}{\alpha }_{\mathrm{Co}}}\end{array}\end{array}$$

(8)

Combination of gene and co-gene is carried out along the same lines for estrogen and progesterone.

2.3.3. Combining Evidence from Gene Expression and IHC

As outlined in Section 2.2.3 for single gene case, we now combine the full gene evidence for estrogen with its IHC counterpart according to the Yager rule, see Equation (6), to obtain (α_ER, β_ER, θ_ER). The very same is done for progesterone, yielding (α_PGR, β_PGR, θ_PGR).

2.3.4. Combining Estrogen and Progesterone Receptor Status

In the step to follow, evidence for different targets—estrogen and progesterone—will be combined. Clinically, a breast cancer patient is considered receptor positive, if either the estrogen ‘OR’ the progesterone receptor (or both) is/are positive, and treatment is assigned accordingly. Corresponding decision borders will be shown below (Figure 6). While clinical SOP (Standard Operating Procedure) draws on a crisp logical ‘OR’, as implemented in the ODDS-method, DST offers a wider scope of possibilities. Evidence for estrogen (α_ESR, β_ESR, θ_ESR) and progesterone (α_PGR, β_PGR, θ_PGR) may be combined to obtain evidence for the overall hormone status (α_H, β_H) as follows [37]:

$$\begin{array}{l}{\alpha }_{\mathrm{H}}={\alpha }_{\mathrm{ESR}}+{\alpha }_{\mathrm{PGR}}-{\alpha }_{\mathrm{ESR}}{\alpha }_{\mathrm{PGR}}\\ {\beta }_{\mathrm{H}}={\beta }_{\mathrm{ESR}}\cdot {\beta }_{\mathrm{PGR}}\end{array}$$

(9)

3. Results

3.1. Contrasting Predictions by ODDS versus DST

Predictions via conventional statistics (ODDS) and decision theory (DST) are directly compared for the whole patient cohort in Figure 6. To address clinical relevance, we highlight patients for which DST adds information (see legend), as well as those for which DST increases safety (see legend). For compactness, we abbreviate notation of the IHC receptor status, e.g., ${\mathrm{ER}}_{\mathrm{IHC}}^{-},{\mathrm{PGR}}_{\mathrm{IHC}}^{+}\stackrel{\wedge}{=}\left(-,+\right)$ or ${\mathrm{ER}}_{\mathrm{IHC}}^{+},{\mathrm{PGR}}_{\mathrm{IHC}}^{\mathrm{u}}\stackrel{\wedge}{=}\left(+,0\right)$, with ‘0’ representing ‘undefined’. Likewise, we denote predictions (via ODDS or DST) as ‘neg‘, ‘pos’ and ‘inc’, with ‘inc’ representing ‘inconclusive’.

Note the following features in Figure 6:

• In the left panels, samples are geometrically located according to ODDS scores, but color-coded according to DST prediction.
• Decision borders in ODDS can be directly displayed in an orthogonal, 2-dimensional plot of ‘scores’, see Figure 6, left panels. Decision borders are defined by specific values for each receptor score (ER score, PGR score), see our previous paper [37], and, hence, appear as vertical lines for estrogen and as horizontal lines for progesterone, respectively. The rectangular region (in faint blue) denotes receptor status predicted definitely negative, the L-shaped stripe (no color) denotes inconclusive status, and the L-shaped stripe (in faint red) definitely positive predictions.
• ODDS scores incorporate IHC evidence in an additive fashion. Each of the nine possible IHC statuses (+ +, − −, + −, − +, + 0, 0 +, − 0, 0 −, 0 0) merely differ in shifts along the respective ODDS coordinate (ER score, PGR score). ODDS decision borders are, hence, valid for any combination of IHC statuses.
• In the right panels, samples are geometrically located according to DST evidence, but color-coded according to ODDS.
• Decision borders in DST are most appropriately displayed in ternary plots of evidence, see Figure 6, right panels. Decision borders run along evidence α = 0.5 and β = 0.5, respectively, which appear as straight lines in a ternary plot. DST evidence also incorporates IHC information, and decision lines, hence, also represent unique borders in the ternary plot, valid for any combination of IHC statuses (+ +, − −, + −, − +, + 0, 0 +, etc.).
• In the ternary plot, DST evidence for subsets of patient samples appear in polygonal areas. In fact, these areas root in respective combinations of IHC statuses for estrogen and progesterone (+ +, − −, + −, etc.), as will be scrutinized in the appendix, for those interested in mathematical details. Indeed, these polygonal areas are generalizations of those simple straight lines already seen with single gene expression data (Figure 4). Since each receptor may assume three values (+, −, 0), there are 3² = 9 possible IHC status combinations for two receptors. Some IHC statuses give rise to very distinct arrangements of samples, such as ‘lines’. Other IHC combinations give rise to more polygonal-shaped areas. Details will be discussed below. Data samples along these lines or polygons are seen to cross DST decision borders (dashed lines at α = 0.5 and β = 0.5, respectively). For example, if such a subset of samples crosses from inconclusive to decided, this indicates that IHC on its own was inconclusive, but adding evidence from (increasing) gene expression finally rendered a decision:

  ○

  A stripe of red points originates within the DST-inconclusive, kite-shaped area and protrudes into the positive triangle.

  ○

  The stripe of blue points originates in the DST-inconclusive, kite-shaped area and protrudes into the negative triangle.

Crossing decision borders for given IHCs underpins the importance of information from gene expression being added.

3.2. Clinical Relevance of DST versus ODDS

Agreement and divergence between ODDS and DST are summarized in

Table 1. Prediction by DST (Dempster–Shafer decision Theory) versus ODDS (conventional statistics). Predictions negative (neg), inconclusive (inc) and positive (pos). No samples are classified as fully contradicting (positive versus negative). Differences arise from samples predicted ‘inconclusive’: 144 (5.7%) via DST compared to 69 (2.7%) via ODDS. DST predicts more conservatively than ODDS. Cells with numbers in bold represent patients with ‘gain of information (10)’ and ‘gain of safety (40)’, respectively. As an overall measure of agreement we computed Cohen’s kappa: κ = 0.9287 [81].

Number of Samples		DST
		neg	inc	pos	sum
ODDS	neg	999	45	0	1044
	inc	0	59	10	69
	pos	0	40	1366	1406
	sum	999	144	1376	2519
percentage		DST
		neg	inc	pos	sum
ODDS	neg	39.7%	1.8%	0.0%	41.5%
	inc	0.0%	2.3%	0.4%	2.7%
	pos	0.0%	1.6%	54.2%	55.8%
	sum	39.7%	5.7%	54.6%	100.0%

. Note that both methods never definitely contradict each another (positive versus negative predictions for a given sample); see the zero counts in the corners off diagonal. Differences only occur for samples predicted as inconclusive. In 59 cases, both methods agree in yielding ‘inconclusive’. However, DST reports almost equal numbers of samples from ODDS-negative (45) and ODDS-positive (40) as DST-inconclusive, ending up with 144 inconclusive samples. Conversely, ODDS declares none from DST negative and only 10 from DST-positive as ODDS-inconclusive, ending up with just 69 samples rendered as inconclusive. In general, agreement between ODDS and DST is fine, with 999 + 59 + 1366 = 2424 out of 2519 samples (96.2%), as reflected by the high inter-rater agreement coefficient, Cohen’s kappa: κ = 0.9287 [81].

Besides good overall agreement, possible advantages of DST may be seen twofold, cf. the cells outlined with bold face in Table 1. The very same groups of patients are highlighted with legends in Figure 6:

• For 10 patients, DST predicted a positive receptor status, whereas ODDS had predicted ‘undecided’. Based on the additional information provided by DST, these patients may, upon careful reassessment, be candidates for milder therapies, possibly without chemotherapy (chemo). We, therefore, labelled this group with ‘adding information’ in Figure 6, panel (c).
• For 40 patients, DST predicted ‘undecided’, whereas ODDS had predicted ‘positive’. ‘Undecided’ severely questions abstaining from chemo and calls for a re-assessment at least. We, therefore, labelled this group with ‘increasing safety’ in Figure 6, panel (d).

Hormone receptor diagnostics—in comparison with ODDS and DST—was evaluated regarding its impact on survival. Figure 7 shows survival, free from recurrence, for several relevant subgroups listed in Table 1. Acronyms in the legend of Figure 7 correspond to those in Table 1, and figures in the legend give the numbers of patients with survival data available and number of events (i.e., recurrences) in parenthesis. Naturally, the two largest groups are those that ODDS and DST found in agreement (neg/neg, pos/pos)—they exhibit rich survival curves, with many patients and numerous events. Subgroups with disagreement between ODDS and DST (fortunately) contain only few patients, reflecting the fact that, already, ODDS was an advanced, accurate prediction method. The point of largest possible merit is the subgroup pos/inc: 40 patients considered positive by ODDS could have been deprived of chemotherapy, although being eventually negative. Within this group, survival data were available only for seven, relegating statistical testing meaningless.

For comparison, the IHC+ group was also evaluated, incorporating patients receptor positive either for estrogen OR progesterone, see Figure 7. Such patients are considered receptor positive and treated accordingly by ‘conventional’ clinical therapy allocation. Compared to these, our pos/pos group enjoyed definitely superior survival (log-rank p = 0.03). Since all patients considered in our study were actually treated according to conventional, clinical ‘IHC+’, we might speculate as follows: this actual, former treatment as ‘IHC+’ was confirmed post hoc in our study (by pos/pos) as correct and, hence, these patients experienced much better survival.

Over the years, hormone receptor status has become the most important predictive parameter, which allows for an identification of endocrine-sensitive invasive tumors. The use of hormone-receptor-targeted treatment strategies is associated with an approximately 50% reduction in recurrences and a reduction in breast-cancer-attributed deaths by approximately 30%, and receptor status assessment has, therefore, become the single most important biomarker in early and advanced breast cancer. A correct classification of endocrine sensitivity by receptor measurement is, therefore, critical for individualized treatment, since falsepositive results lead to overtreatment and therapy-associated side effects, which range from menopausal symptoms, infertility and depression, to bone loss and an increase in fractures, and other significant side effects. False negative results, on the other side, subject patients to under-treatment and a profound worsening of the long-term outcome. These profound clinical consequences are contrasted by a number of technical uncertainties: the hormone receptor status is presently assessed by immunohistochemistry, and different standards in tissue fixation, varying protocols, the myriad of commercially available antibodies, inter-observer variability and other technical issues compromise an objective assessment. Moreover, while some labs use a cut-off of 10% of hormone receptor positive cells, others prefer a cutoff of 1%, thus, limiting the value of the current gold standard in receptor assessment. Within this context, prediction models, such as DST and ODDS, can add to further ascertainment of the receptor status. The decision of which model to use could be factored into the decision tree and allow for a more personalized treatment, in the sense that the more conservative DST could be applied in older and frail patients, in whom the significant side effects of endocrine therapy need to be balanced against competing mortalities and might lead to an omission of endocrine therapy, and an additional IHC, performed by an independent laboratory could be helpful in decision making and in potentially sparing patients from therapy-associated side effects. By contrast, ODDS with 0.4% inconclusive rates might be more appropriate in mainstream assessment, since the need for independent reassessment can be reduced.

3.3. Specific Differences in Prediction between ODDS and DST

As noted above, definite predictions were never seen contradicting between ODDS and DST. However, decisions deemed definite in ODDS were rendered inconclusive by DST and vice versa. This becomes evident by contrasting predictions coded by location versus predictions coded by color in Figure 6:

• Within the plane of ODDS scores (left panel), the L-shaped area (colored faint red) denotes samples definitely predicted positive by ODDS (according to location). However, some of them are inconclusive according to DST (colored beige); in fact, 40 DST-inconclusive samples invade the positive, and the other 45, the negative domain of ODDS scores, see Table 1.
• Conversely, the uncolored L-shaped area accommodates samples predicted inconclusive according to ODDS (according to location). However, 10 are colored red, i.e., according to DST, decided positive. In fact, these samples, definitely predicted positive by DST, invade the inconclusive region of ODDS scores and are labelled ‘adding information’, see Figure 6, panel (c) and Table 1.
• Within the ternary plot of DST evidence (right panel), the triangular shaped areas denote samples predicted negative (faint blue) and positive (faint red), respectively, according to DST (by location). However, some samples are color-coded beige, i.e., they were rendered inconclusive by ODDS. Note that the very same samples appear in dual roles along ODDS scores and ternary evidence, respectively (left and right panel).
• Conversely, the uncolored kite-shaped area denotes samples predicted inconclusive according to DST (by location). However, some of them are color-coded red or blue, i.e., definitely predicted as positive or negative according to ODDS. In fact, 40 samples definitely classified positive through ODDS intrude into the ‘inconclusive’ region of DST and have been labelled as ‘increasing safety’, see panel (d). Another 45 definitely predicted negative through ODDS intrude into the ‘inconclusive’ region of DST.

All in all, differences in prediction only occur with samples on the brink of predictability. While one method yields positive or negative, the other may yield ‘inconclusive’. These differences turn up in the off-diagonal elements of Table 1, which are small; see also the percentages. Even if differences are small, they are important for the single patient and seen at the core of personalized medicine.

Moreover, visual inspection of the ternary plot in Figure 6 reveals samples not being evenly distributed over the triangular plane of evidence. Samples, rather, appear in groups, arranged in lines or lengthy polygons. The mechanisms behind the scenes, giving rise to these effects, are scrutinized in Appendix A.6 and Appendix A.7.

4. Discussion

Dempster–Shafer Decision Theory (DST) has been made available for the personalized therapy of breast cancer in a previous paper [37], in particular, to increase the precision of receptor status assessment. Unfortunately, we could not map with ground truth in our papers, since ground truth is not available for the data used. However, we were able to provide a sound comparison between ODDS and DST and pinpoint particular differences in performance. To underpin the usefulness of DST, we have scrutinized the survival of patients with status corrected from positive or negative predictions by ODDS towards ‘inconclusive’ by DST, see Figure 7. Since only a small fraction of patients was to be ‘corrected’ (see Table 1), survival curves degenerate and were included only for completeness. Even if this percentage is small, it seems mandatory, considering the large number of breast cancer patients. In practice, patents rendered inconclusive should receive lab reassessment, in order to reduce false estimates and increase precision.

In addition, we compared patients considered receptor positive according to up-to-date clinical standards (IHC+, red curve) with those considered positive (pos/pos, light blue curve) according to both of our proposed methods, ODDS and DST. Patients positive according to the new methods experienced significantly better survival (log-rank p = 0.03) than those conventionally diagnosed positive, see the red versus the light blue curves in Figure 7.

Comparing ODDS and DST, DST was found to be somewhat more conservative than ODDS. Vice versa, patients considered ‘positive’ by DST, while being considered ‘undecided’ by ODDS, may benefit from this additional information inferred by DST. However, this gain of information has two sides: the ‘positive’ prediction might not really hold in the end, and relying on it may cause harm. Hence, re-evaluation remains the only safe advice in these cases.

4.1. Advantages of Evidence Compared to Probabilities in Conventional Statistics

In addition to our previous work, the implementation of DST is, here, unfolded in three steps:

First, we demonstrate the simplest case, starting with a single gene (the receptor gene) and demonstrate how to:

• Obtain DST evidence from gene expression.
• Obtain DST evidence from IHC.
• Fuse both items of evidence above, via the Yager evidence combination rule [78].
• Display results in a ‘ternary’ plot, a genuine format for presenting evidence.
• Show subgroups of patients with given IHC status, giving rise to specific patterns of samples in evidence space.

In a second step, we demonstrate how to create evidence from co-genes and join them with evidence of receptor genes and IHC (by Dempster and Yager Evidence Combination rule, respectively).

In the third step we demonstrate how to join evidence from estrogen with that from progesterone, using a formula imitating the clinical criterion ‘positive ER or positive PGR’ for ‘receptor positivity’, in terms of Dempster–Shafer mathematics.

This stepwise approach allows for a detailed introduction into ternary plots, demonstrating their applicability to clinical decision making, based on evidence. It becomes clear that evidence not only provides more information about the outcome of a measurement than conventional probability does, but that probabilities are supplemented by uncertainty. Evidence also has the property of three numbers summing up to unity for each single sample considered, and may be advantageously displayed in ternaries. Groups of patients (different IHC statuses) are segregated by the method itself (being either positive or negative).

Data quality is a crucial aspect of personalized medicine. In this work, we have never let gene expression overrule IHC. Technically, this was achieved by selecting the constants $\widehat{\alpha }$ in our model very conservatively. As a consequence, positive IHC estimates were never converted into negative, not even a positive progesterone when estrogen was negative IHC = (−, +). Such IHC estimates occurred in 15 samples, and gene expression by itself would turn them into (−, −), if we had modeled less weight into IHC and more into gene expression.

4.2. How Uncertainty May Help Increase Correctness (Precision)

At first glance, this statement may seem paradoxical. However, DST—in comparison to ODDS—supports this concept, as can be seen from a vivid comparison:

Suppose we have a ballot between two options (pro, contra). If the voter turnout was 100%, we might obtain 75% for pro and 25% for contra (3:1), and with full right, consider this a clear decision. The option ‘pro’ would clearly be implemented, having the majority of voters on its side, see the top bar in Figure 8. Exactly this scenario corresponds to classical statistics, considering a probability p and the probability 1 − p for its opposite.

Now suppose the voter turnout was only 80%, with exactly the same distribution between pro and contra, i.e., 60%:20% = 3:1, see the second bar in Figure 8. In this case also, we would consider it a valid decision, despite 20% non-voters, representing what is termed ‘uncertainty’ in DST. However, the result would not be considered as ‘robust’ as in the first case.

Finally, suppose a voter turnout of just 40%, again with the same ratio between pro and contra of 30%:10% = 3:1, see the third bar in Figure 8. Such a result would not be considered sound enough to draw conclusions from. An uncertainty of 60%—in terms of DST—may render the result ‘un-trustable’, even with a large ratio of probabilities $p:\left(1-p\right)=3:1$. After all, the relative ‘majority’ of 30% is far from absolute (50%). In such a case, a wise politician would not be confident to implement option ‘pro’, since opposition might emerge that is too strong to overcome.

Analog concepts hold for medical diagnostics. As DST introduces uncertainty as the third part of evidence [82,83], borderline or questionable results obtained by classical statistics may be relegated ‘uncertain’, suggesting further assessment and, thereby, increasing final correctness. In addition, significantly different risks may be inferred by falsepositive as compared to falsenegative decisions. For example, a falsepositive receptor status may lead to the avoidance of chemotherapy, in this case, the lifesaving therapy. Accordingly, one might request very low uncertainty, in order to ‘take a positive status serious’, regarding therapeutic consequences. Conversely, a false-negative status might ‘just’ entail unnecessary chemo, a comparatively lower risk. All in all, it is but a clinical decision how much uncertainty seems acceptable.

To allow for evidence-based decisions, the explicit quantification of uncertainty seems utmost desirable.

4.3. Extensions of Decision Rules

The approach presented here may be expanded by considering more than one co-gene, since DST allows us to combine more than two items of evidence. Considerable increases in stability can be expected if such expanded markers are applied to new incoming data.

Another possible extension refers to combination rules.

One basic concept for combining evidence from different sources was introduced by Dubois [84], hence, termed “evidence combination rule (ECR) after Dubois and Prade”. In the case of just two outcomes, this boils down to the Yager rule [78]. Smarandache [85] further generalized combination rules and defined the PCR5 combination rule, relevant for three (or more) outcomes. Fontani [66] proposed fusing the spaces of events in image processing and Denœux introduced weighted combination [69,74,75]. Chen defined distances between evidence [86]. Yang reviewed a framework of evidence combination rules and evidence weighting and discounting [65] and Sentz compiled all rules, in a comprehensive overview [87].

In the present work, we only used the Dempster Evidence Combination Rule (ECR) and the Yager rule [78]. However, this is not mandatory. In fact, a variety of ECRs exist, which differ in behavior in certain situations.

4.4. Modelling Sharp and Soft Clinical Decisions

The Dempster Evidence Combination rule advocates fierce decisions—leaving little uncertainty in the conclusions—even if both pieces of input evidence concede considerable uncertainty. As opposed, given the same input evidence, the Yager rule follows a much softer strategy, transmitting larger uncertainty into its conclusion. We illustrate this by a specific example.

Suppose we have two items of evidence for receptor status: The first piece of evidence, from gene expression (α = 0.8, β = 0.1, θ = 0.1), strongly favors ‘positive’, via large α and small β. Moreover, it claims to be quite ‘sure’ in terms of small θ. The second piece of evidence, from IHC (α = 0, β = 0.7, θ = 0.3), favors ‘negative’, with some larger uncertainty θ = 0.3. Obviously, these pieces of evidence contradict each other quite strongly, and one may legitimately ask ‘what should be the synthesis of these two?’ The answer can be precisely modelled by decision combination rules, according to Dempster (⊕_D) or Yager (⊕_Y), which also exemplifies their difference in approach.

For the current example, the Dempster rule (Equations (7) and (8)) yields as combined evidence E_D = (α = 0.54, β = 0.38, θ = 0.068), expressing a quite ‘sharp’ contradiction (large α, large β), without admitting much uncertainty (small θ). On the contrary, the Yager rule, Equation (6), yields combined evidence E_Y = (α = 0.24, β = 0.17, θ = 0.59), expressing only ‘soft’ contradiction (small α, small β), with quite a lot of uncertainty (large θ).

How can these features be exploited for personalized medicine?

Clinical experts have always been looking for the most beneficial balance in decision making, based on SOPs, their personal experience, and also skill, or even educated guessing, in particularly difficult cases. It has always been the strength and fame of top clinicians to decide correctly in a percentage of cases far above average. However, it may not be fully transparent how such an outstanding clinical performance comes about and could be transferred to young doctors in training. Decision theory tries to bring such ‘clinical expert competence’ down to more formally applicable rules. Of course, it will remain the task of top clinicians to help define and select those rules, based on sound statistical evaluations of clinical studies. Such decision rules, once established, may be incorporated in SOPs and will improve their performance significantly.

While this work exemplifies the use of DST in personalized medicine, related to the very specific field of breast cancer receptor diagnostics, the methods described are universal. Decision theory, in particular, the fusion of diverging evidence (sometimes also called ‘sensor-fusion’), as well as the professional incorporation of uncertainty into biomarker research, seem valuable for all fields of personalized medicine and medicine in general.