Order-of-Addition Orthogonal Arrays with High Strength

Abstract

In order-of-addition experiments, the full order-of-addition designs are often unaffordable due to their large run sizes. The problem of finding efficient fractional OofA designs arises. The order-of-addition orthogonal arrays are a class of optimal fractional order-of-addition designs for the prevalent pair-wise ordering model, under a variety of widely used design criteria. In the literature, the studies on order-of-addition orthogonal arrays focused on strength 2 while the order-of-addition orthogonal arrays of higher strength have not been investigated yet. In this paper, we focus on order-of-addition orthogonal arrays of strength 3. First, the method of constructing order-of-addition orthogonal arrays of strength 3 is proposed. Second, a theoretical result that states that the order-of-addition orthogonal arrays of strength 3 have better balance properties than those of strength 2 is developed. Third, we provide thorough simulation studies which show that the constructed order-of-addition orthogonal arrays of strength 3 have desirable performance for estimating optimal orders of addition.

1. Introduction

In many experiments, the response is definitely affected by the order of processing of the materials or components. We call this class of experiments the order-of-addition (OofA) experiments. An example is the famous experiment of a lady testing tea in which two different orders, “tea preceding milk” and “milk preceding tea”, were tested [1]. To illustrate the characteristics of OofA experiments, we introduce one more example from [2]. In [2], three anti-tumor drugs (coded as $c_1$, $c_2$, and $c_3$, respectively) were added into tumor cells either sequentially (following the six orders $c_1\to c_2\to c_3,c_1\to c_3\to c_2,c_2\to c_1\to c_3,c_2\to c_3\to c_1,c_3\to c_1\to c_2$, and $c_3\to c_2\to c_1$) or simultaneously. The percentage of tumor inhibitions, a larger-the-better response, was measured at 12 h after the last drug was administrated. The largest response was yielded when the three anti-tumor drugs were administrated following the order $c_2\to c_3\to c_1$ rather than simultaneously. The OofA effect also matters in many other scientific disciplines including chemical science [3], bio-chemistry [4], food science [5], and manufacturing [6]. More applications of the order of addition can be found in [7,8] and the references therein.

Three prevalent models for the OofA problem have been proposed. Ref. [9] proposed the pair-wise ordering (PWO) model which will be detailed in Section 2. Ref. [2] proposed the component-position (CP) model which assumes that a component has different OofA effects when it is processed at different positions in an order. Ref. [8] proposed using the mapping-based universal Kriging model for OofA experiments with blocking. In this paper, we consider the OofA experiments without blocking effects which are not suitable for the the mapping-based universal Kriging model. Compared to the CP model, the PWO model has stronger interpretability and fewer parameters to be estimated, indicating less experimental cost. With this in mind, this paper carries out studies under the PWO model.

Suppose m components, denoted as $c_1,c_2,\dots ,c_m$, are considered in an OofA experiment, there are $m!$ different orders. We call a design which consists of these $m!$ different orders the full OofA design. It is often unaffordable to perform OofA experiments by using full OofA designs especially when m is large. For example, when $m=6$, the full OofA design contains 6!(=720) different orders. Thus, the study on efficient fractional OofA designs becomes important. Under the PWO model, [10] proposed a class of fractional OofA designs called the OofA orthogonal arrays (OofA-OAs) which will be defined in Section 2. Ref. [11] proved that OofA-OAs are optimal for the PWO model under some widely used design criteria including D-criterion, where the D-criterion is defined as follows. Suppose $\mathit{X}$ is a model matrix of a design under a certain model, N is the run size of $\mathit{X}$ and m is the number of columns in $\mathit{X}$, then D-efficiency is defined as ${(\mathrm{d}\mathrm{e}\mathrm{t}({\mathit{X}}^T\mathit{X}))}^{1/m}/N,$ where the superscript T denotes transpose. A design with a larger D-efficiency is better. Ref. [11] provided a closed-form construction method for OofA-OAs of strength 2 which have quite large run sizes. Ref. [12] provided smaller OofA-OAs of strength 2 compared to those in [11] via block designs. Ref. [13] proposed a systematical construction method of OofA-OAs of strength 2 which further reduced the run sizes compared to the work in [11,12]. Ref. [2] proposed the component orthogonal arrays which are D-optimal for the CP model. Refs. [2,14,15] respectively proposed different methods of constructing the component orthogonal arrays. Some other work under the PWO model can be found in [16] which extended the PWO model by entertaining interactions of PWO factors, and [17] which proposed a class of minimal-point OofA designs that have good D-efficiencies for the PWO model.

Throughout the literature on efficient fractional OofA designs for the PWO model, there is no study on the OofA-OAs of strength 3 which are D-optimal for the PWO model while saving a considerable amount of experimental costs compared to the full OofA designs, and have better balance properties than those of strength 2, as will be proved in this paper. The contributions of this paper are threefold: (1) we first propose a method of constructing OofA-OAs of strength 3 which is capable of finding non-isomorphic OofA-OAs of strength 3; (2) some balance properties of OofA-OAs of strength 3 are developed; (3) thorough simulation studies are conducted which show that the constructed OofA-OAs of strength 3 have desirable performance on estimating the optimal orders of addition.

The rest of the paper is organized as follows. In Section 2, we introduce the formulation of PWO model and the definition of OofA-OAs. The isomorphism of OofA designs is also defined in this section. Section 3 gives a construction method of OofA-OAs of strength 3. Section 4 explores the balance properties of OofA-OAs of strength 3. The thorough simulation studies, which show that the constructed OofA-OAs of strength 3 have desirable performance on estimating optimal orders of addition, are included in Section 5. Section 6 includes results and discussions. The conclusions are given in Section 7. Some proofs and useful design tables are deferred to Appendix A and Appendix B, respectively.

2. Preliminaries

Denote ${\mathit{O}}_m$ as the full OofA design of m components, where the orders in ${\mathit{O}}_m$ are arranged in reversed lexicographical order. For example, the orders of ${\mathit{O}}_3$ are displayed in

Table 1. Full OofA design ${\mathit{O}}_3$ and full PWO design ${\mathit{P}}_3$.

	PWO Factors (${\mathit{P}}_3$)
${\mathit{O}}_{\mathbf{3}}$	${\mathit{z}}_{12}$	${\mathit{z}}_{\mathbf{13}}$	${\mathit{z}}_{\mathbf{23}}$
$c_3{c}_2{c}_1$	$-1$	$-1$	$-1$
$c_3{c}_1{c}_2$	1	$-1$	$-1$
$c_2{c}_3{c}_1$	$-1$	$-1$	1
$c_2{c}_1{c}_3$	$-1$	1	1
$c_1{c}_3{c}_2$	1	1	$-1$
$c_1{c}_2{c}_3$	1	1	1

.

Suppose ${\mathit{o}}_k$, $k=1,2,\dots ,m!$, is the k-th order in ${\mathit{O}}_m$. Denote $\tau ({\mathit{o}}_k)$ as the observation arising from ${\mathit{o}}_k$. The PWO model is established as

$$\tau ({\mathit{o}}_k)={\beta }_0+\sum _{i=1}^{m-1}\sum _{j=i+1}^m{\beta }_{ij}{\lambda }_{ij}({\mathit{o}}_k)+\epsilon ({\mathit{o}}_k),$$

(1)

where, for $i<j$, ${\lambda }_{ij}({\mathit{o}}_k)=1$ if component $c_i$ precedes $c_j$ in ${\mathit{o}}_k$, otherwise ${\lambda }_{ij}({\mathit{o}}_k)=-1$, $\epsilon ({\mathit{o}}_k)\sim N(0,{\sigma }^2)$ for any ${\mathit{o}}_k$, $\epsilon ({\mathit{o}}_k)$ is independent of $\epsilon ({\mathit{o}}_l)$ for $k\ne l$, and ${\beta }_0$, ${\beta }_{ij}$s are unknown parameters to be estimated. For example when $m=3$, ${\lambda }_{12}(c_1{c}_2{c}_3)=1$ as $c_1$ precedes $c_2$ and ${\lambda }_{12}(c_2{c}_3{c}_1)=-1$ as $c_2$ precedes $c_1$. Let ${\mathit{z}}_{ij}={({\lambda }_{ij}({\mathit{o}}_1),{\lambda }_{ij}({\mathit{o}}_2),\dots ,{\lambda }_{ij}({\mathit{o}}_{m!}))}^T$. We call ${\mathit{z}}_{ij}$ the PWO factor related to components $c_i$ and $c_j$. Column juxtaposing ${\mathit{z}}_{ij}$s, we call ${\mathit{P}}_m=({\mathit{z}}_{12},{\mathit{z}}_{13},\dots ,{\mathit{z}}_{(m-1)m})$ the full PWO design, where ${\mathit{z}}_{ij}$ is ahead of ${\mathit{z}}_{kl}$ if $i<k$; or if $i=k$ and $j<l$. For example, the PWO factors for $m=3$ and full PWO design ${\mathit{P}}_3$ are displayed in Table 1. Denoting $\mathit{D}$ as a fractional OofA design and ${\mathit{P}}_D$ as the fractional PWO design determined by the orders in $\mathit{D}$, we give the definition of the OofA-OA.

Definition 1.

An N-run fractional OofA design$\mathit{D}$is called an OofA-OA of strength t, denoted as OofA-OA$\mathit{(}\mathit{N}\mathit{,}\mathit{m}\mathit{,}\mathit{t}\mathit{)}$, if the ratios among the frequencies of all t-tuples in any t-column subarray of${\mathit{P}}_{\mathit{D}}$equal to the ratios among the frequencies of all t-tuples in the corresponding t-column subarray of${\mathit{P}}_{\mathit{m}}$.

Definition 2 defines isomorphic OofA designs.

Definition 2.

Two OofA designs are said to be isomorphic if one can be obtained from the other by relabeling components or permuting rows.

In [13], the authors showed that non-isomorphic OofA-OAs may have different performances in some situations. For example, under the CP model, the non-isomorphic OofA-OAs may have different D-efficiencies. This is not true for the isomorphic OofA-OAs. In this paper, the construction method we propose is capable of finding the non-isomorphic OofA-OAs of strength 3. For a detailed definition of the CP model, one is referred to [2].

3. Constructions of OofA-OAs of Strength 3

From Definition 1, in order to construct OofA-OAs of strength 3, we need to investigate the frequencies of the three-tuple $(a,b,c)$s, with $a=\pm 1,b=\pm 1$ and $c=\pm 1$, in each of the three-column subarrays $({\mathit{z}}_{ij},{\mathit{z}}_{kl},{\mathit{z}}_{vw})$s of ${\mathit{P}}_m$, where ${\mathit{z}}_{ij}$ is ahead of ${\mathit{z}}_{kl}$ in ${\mathit{P}}_m$, and ${\mathit{z}}_{kl}$ is ahead of ${\mathit{z}}_{vw}$ in ${\mathit{P}}_m$. Note that $m=6$ is the smallest m such that ${\mathit{P}}_m$ has the three-column subarrays $({\mathit{z}}_{ij},{\mathit{z}}_{kl},{\mathit{z}}_{vw})$s with $i,j,k,l,v$, and w being mutually different. As shown in

Table 2. Classifications of three-column subarrays of ${\mathit{P}}_m$.

Type	${\mathit{T}}_1$	${\mathit{T}}_2$	${\mathit{T}}_3$	${\mathit{T}}_4$	${\mathit{T}}_5$	${\mathit{T}}_6$	${\mathit{T}}_7$	${\mathit{T}}_8$	$({\mathit{z}}_{\mathit{i}\mathit{j}},{\mathit{z}}_{\mathit{k}\mathit{l}},{\mathit{z}}_{\mathit{v}\mathit{w}})$	Examples
$t_1$	$\frac{m!}{6}$	$\frac{m!}{6}$	0	$\frac{m!}{6}$	$\frac{m!}{6}$	0	$\frac{m!}{6}$	$\frac{m!}{6}$	$i=k,j=v,l=w$	12 13 23
$t_2$	$\frac{5m!}{24}$	$\frac{m!}{24}$	$\frac{m!}{8}$	$\frac{m!}{8}$	$\frac{m!}{8}$	$\frac{m!}{8}$	$\frac{m!}{24}$	$\frac{5m!}{24}$	$j=w,k=v$	14 23 24
$t_3$	$\frac{5m!}{24}$	$\frac{m!}{8}$	$\frac{m!}{8}$	$\frac{m!}{24}$	$\frac{m!}{24}$	$\frac{m!}{8}$	$\frac{m!}{8}$	$\frac{5m!}{24}$	$i=k,j=w$	13 14 23
$t_4$	$\frac{m!}{24}$	$\frac{m!}{8}$	$\frac{5m!}{24}$	$\frac{m!}{8}$	$\frac{m!}{8}$	$\frac{5m!}{24}$	$\frac{m!}{8}$	$\frac{m!}{24}$	$j=k,l=v$	12 23 34
$t_5$	$\frac{m!}{12}$	$\frac{m!}{4}$	$\frac{m!}{12}$	$\frac{m!}{12}$	$\frac{m!}{12}$	$\frac{m!}{12}$	$\frac{m!}{4}$	$\frac{m!}{12}$	$j=l=v$	13 23 34
$t_6$	$\frac{m!}{12}$	$\frac{m!}{12}$	$\frac{m!}{12}$	$\frac{m!}{4}$	$\frac{m!}{4}$	$\frac{m!}{12}$	$\frac{m!}{12}$	$\frac{m!}{12}$	$j=k=v$	12 23 24
$t_7$	$\frac{m!}{8}$	$\frac{5m!}{24}$	$\frac{m!}{24}$	$\frac{m!}{8}$	$\frac{m!}{8}$	$\frac{m!}{24}$	$\frac{5m!}{24}$	$\frac{m!}{8}$	$i=k,j=v$	12 13 24
$t_8$	$\frac{m!}{8}$	$\frac{5m!}{24}$	$\frac{m!}{8}$	$\frac{m!}{24}$	$\frac{m!}{24}$	$\frac{m!}{8}$	$\frac{5m!}{24}$	$\frac{m!}{8}$	$i=k,l=v$	12 13 34
$t_9$	$\frac{m!}{8}$	$\frac{m!}{24}$	$\frac{m!}{8}$	$\frac{5m!}{24}$	$\frac{5m!}{24}$	$\frac{m!}{8}$	$\frac{m!}{24}$	$\frac{m!}{8}$	$j=k,l=w$	12 24 34
$t_{10}$	$\frac{m!}{8}$	$\frac{m!}{8}$	$\frac{5m!}{24}$	$\frac{m!}{24}$	$\frac{m!}{24}$	$\frac{5m!}{24}$	$\frac{m!}{8}$	$\frac{m!}{8}$	$j=w,l=v$	14 23 34
$t_{11}$	$\frac{m!}{8}$	$\frac{m!}{8}$	$\frac{m!}{24}$	$\frac{5m!}{24}$	$\frac{5m!}{24}$	$\frac{m!}{24}$	$\frac{m!}{8}$	$\frac{m!}{8}$	$j=v,l=w$	13 24 34
$t_{12}$	$\frac{5m!}{24}$	$\frac{m!}{8}$	$\frac{m!}{24}$	$\frac{m!}{8}$	$\frac{m!}{8}$	$\frac{m!}{24}$	$\frac{m!}{8}$	$\frac{5m!}{24}$	$i=k,l=w$ or $j=l,k=v$	12 14 34 13 23 25
$t_{13}$	$\frac{m!}{4}$	$\frac{m!}{12}$	$\frac{m!}{12}$	$\frac{m!}{12}$	$\frac{m!}{12}$	$\frac{m!}{12}$	$\frac{m!}{12}$	$\frac{m!}{4}$	$i=k=v$ or $j=l=w$	12 13 14 14 24 34
$t_{14}$	$\frac{m!}{6}$	$\frac{m!}{6}$	$\frac{m!}{12}$	$\frac{m!}{12}$	$\frac{m!}{12}$	$\frac{m!}{12}$	$\frac{m!}{6}$	$\frac{m!}{6}$	$i=k$ or $j=l$	12 13 45 13 23 45
$t_{15}$	$\frac{m!}{6}$	$\frac{m!}{12}$	$\frac{m!}{12}$	$\frac{m!}{6}$	$\frac{m!}{6}$	$\frac{m!}{12}$	$\frac{m!}{12}$	$\frac{m!}{6}$	$k=v$ or $l=w$	12 34 35 12 35 45
$t_{16}$	$\frac{m!}{6}$	$\frac{m!}{12}$	$\frac{m!}{6}$	$\frac{m!}{12}$	$\frac{m!}{12}$	$\frac{m!}{6}$	$\frac{m!}{12}$	$\frac{m!}{6}$	$j=w$	14 25 34
$t_{17}$	$\frac{m!}{12}$	$\frac{m!}{6}$	$\frac{m!}{6}$	$\frac{m!}{12}$	$\frac{m!}{12}$	$\frac{m!}{6}$	$\frac{m!}{6}$	$\frac{m!}{12}$	$l=v$	12 34 45
$t_{18}$	$\frac{m!}{12}$	$\frac{m!}{6}$	$\frac{m!}{12}$	$\frac{m!}{6}$	$\frac{m!}{6}$	$\frac{m!}{12}$	$\frac{m!}{6}$	$\frac{m!}{12}$	$j=v$	13 24 35
$t_{19}$	$\frac{m!}{12}$	$\frac{m!}{12}$	$\frac{m!}{6}$	$\frac{m!}{6}$	$\frac{m!}{6}$	$\frac{m!}{6}$	$\frac{m!}{12}$	$\frac{m!}{12}$	$j=k$	12 23 45
$t_{20}$	$\frac{m!}{8}$	$\frac{m!}{8}$	$\frac{m!}{8}$	$\frac{m!}{8}$	$\frac{m!}{8}$	$\frac{m!}{8}$	$\frac{m!}{8}$	$\frac{m!}{8}$	*	12 34 56

Note: T₁, T₂, T₃, T₄, T₅, T₆, T₇, T₈ represent the three-tuples (−, −, −), (−, −, +), (−, +, −), (−, +, +), (+, −, −), (+, −, +), (+, +, −), and (+, +, +), respectively; for each t_i, all the equal numbers among i, j, k, l, v, w are displayed in column “z_ij, z_kl, z_vw”, and * means that i, j, k, l, v, w are mutually different.

, for $m\ge 6$, there are 20 different types of ratios among the frequencies of the eight three-tuple $(a,b,c)$s, and the run size of an OofA-OA of strength 3 should be a multiple of 24. Lemma 1 formally summarizes these findings.

Lemma 1.

For $m\ge 6$, ${\mathit{P}}_m$ has 20 different types of ratios among the frequencies of the eight three-tuple $(a,b,c)$s, with $a=\pm 1,b=\pm 1$ and $c=\pm 1$, as shown in Table 2, and the run size of an OofA-OA of strength 3 should be a multiple of 24.

Remark 1 below shows the frequencies of the eight three-tuple $(a,b,c)$s for ${\mathit{P}}_4$ and ${\mathit{P}}_5$.

Remark 1.

In Table 2, the types $t_1$-$t_{13}$ apply to ${\mathit{P}}_4$ and the types $t_1$-$t_{19}$ apply to ${\mathit{P}}_5$. The run sizes of OofA-OAs of strength 3 for $m=4$ and $m=5$ should also be multiples of 24.

In the following, we introduce the method of constructing OofA-OAs of strength 3. Denote ${\mathbf{\varphi }}_{ij,kl,vw}(-,-,-)$ as an $m!$-dimensional vector in which the uth entry is 1 if the three-tuple $(-,-,-)$ appears in the uth row of the three-column subarray $({\mathit{z}}_{ij},{\mathit{z}}_{kl},{\mathit{z}}_{vw})$ of ${\mathit{P}}_m$, and is 0 otherwise. The $m!$-dimensional vectors ${\mathbf{\varphi }}_{ij,kl,vw}(-,-,+),$ ${\mathbf{\varphi }}_{ij,kl,vw}(-,+,-),$${\mathbf{\varphi }}_{ij,kl,vw}(-,+,+),$ ${\mathbf{\varphi }}_{ij,kl,vw}(+,-,-),$${\mathbf{\varphi }}_{ij,kl,vw}(+,-,+),$ ${\mathbf{\varphi }}_{ij,kl,vw}(+,+,-),$${\mathbf{\varphi }}_{ij,kl,vw}(+,+,+)$ are similarly defined. Let

$$\begin{array}{ccc}\hfill {\mathbf{\Phi }}_{ij,kl,vw}^T& =& ({\mathbf{\varphi }}_{ij,kl,vw}(-,-,-),{\mathbf{\varphi }}_{ij,kl,vw}(-,-,+),{\mathbf{\varphi }}_{ij,kl,vw}(-,+,-),{\mathbf{\varphi }}_{ij,kl,vw}(-,+,+),\hfill \\ \hfill & =& {\mathbf{\varphi }}_{ij,kl,vw}(+,-,-),{\mathbf{\varphi }}_{ij,kl,vw}(+,-,+),{\mathbf{\varphi }}_{ij,kl,vw}(+,+,-),{\mathbf{\varphi }}_{ij,kl,vw}(+,+,+)).\hfill \end{array}$$

(2)

For m, there are $\left(\genfrac{}{}{0pt}{}{\left(\genfrac{}{}{0pt}{}{m}{2}\right)}{3}\right)$ such ${\mathbf{\Phi }}_{ij,kl,vw}$s. Let $\mathbf{\Phi }$ be the matrix generated by row-juxtaposing all the $\left(\genfrac{}{}{0pt}{}{\left(\genfrac{}{}{0pt}{}{m}{2}\right)}{3}\right)$${\mathbf{\Phi }}_{ij,kl,vw}$s. For a fractional OofA design $\mathit{D}$ of m components, let $y_k(\mathit{D})=1$ if the order ${\mathit{o}}_k$ is in $\mathit{D}$ and $y_k(\mathit{D})=0$ otherwise, where $k=1,2,\dots ,m!$. Let ${\mathit{Y}}_D={(y_1(\mathit{D}),y_2(\mathit{D}),\dots ,y_{m!}(\mathit{D}))}^T$. We establish a sufficient and necessary condition for $\mathit{D}$ to be an OofA-OA of strength 3.

Theorem 1.

A fractional OofA design $\mathit{D}$ is an OofA-OA $(N,m,3)$ if and only if ${\mathit{Y}}_D$ is a solution of

$$\mathbf{\Phi }{\mathit{Y}}_D=(N/m!)\mathrm{vdiag}(\mathbf{\Phi }{\mathbf{\Phi }}^T),$$

(3)

where $\mathrm{vdiag}(·)$ is a column vector consisting of the diagonal elements of a matrix.

Theorem 1 shows that once we have a solution to (3), we can construct an OofA-OA of strength 3 according to this solution. Example 1 shows this point.

Example 1.

Let ${\mathit{Y}}_D$ be a 5!(=120)-dimensional vector which has entries 1s in its 1, 6, 16, 22, 26, 28, 40, 46, 51, 53, 57, 59, 66, 71, 75, 77, 81, 83, 95, 99, 101, 105, 107, and 120-th rows and 0s in the remainder of its rows. It can be verified that ${\mathit{Y}}_D$ is a solution of (3) with $m=5$ and $N=24$. Then, an OofA-OA $(24,5,3)$ can be constructed according to ${\mathit{Y}}_D$. The resulting design is ${\mathit{A}}_{5.1}^{24}$ displayed in

Table A1. Row numbers of an OofA-OA(24,5,3).

	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15	16	17	18	19	20	21	22	23	24	${\mathit{D}}_{\mathbf{CP}}$
${\mathit{A}}_{5.1}^{24}$	1	6	16	22	26	28	40	46	51	53	57	59	66	71	75	77	81	83	95	99	101	105	107	120	0

D_CP: the relative D-efficiency under the CP model.

of Appendix B.

It is an infeasible task to directly solve Equation (3), we employ 0–1 linear programming to find solutions. Corollary 1 below states this approach.

Corollary 1.

For a given $\mathit{c}\in R^{m!}$, an $m!$-dimensional vector, if ${\mathit{Y}}_D$ is a solution of the 0–1 linear optimization problem,

$$\begin{array}{c}min{\mathit{c}}^T{\mathit{Y}}_{D}subject to:\hfill \\ \mathbf{\Phi }{\mathit{Y}}_D=(N/m!)\mathrm{vdiag}(\mathbf{\Phi }{\mathbf{\Phi }}^T),and {\mathit{Y}}_D\in {\{0,1\}}^{m!},\hfill \end{array}$$

(4)

then the rows chosen according to ${\mathit{Y}}_D$ compose an OofA-OA $(N,m,3)$.

Remark 2.

For given m and N, if Equation (3) has solution(s), then the optimization problem (4) has solution(s) for any $\mathit{c}$. Note that our interest is to find the solution ${\mathit{Y}}_D$ instead of minimizing ${\mathit{c}}^T{\mathit{Y}}_D$. Any programming solver can be employed to find ${\mathit{Y}}_D$s. Here, we use intlinprog “from Matlab. Given m, N and any$\mathit{c}$, intlinprog” reports one ${\mathit{Y}}_D$ of (4) unless (3) has no solution for the given m and N.

As an illustration, we apply Theorem 1 and Corollary 1 to finding OofA-OA $(N,m,3)$s with $m=5,6$ and $N=24,48,72$. For given N and m, we use 2000 random $\mathit{c}$s to find different solutions of (3). With these different solutions, we display some non-isomorphic OofA-OAs of strength 3 which have larger relative D-efficiencies under the CP model, where the relative D-efficiency of a fractional OofA design is the ratio between the D-efficiency of this fractional OofA design and that of its corresponding full OofA design. By doing so, it shows that non-isomorphic OofA-OAs may have different D-efficiencies under the CP model as pointed out in [13].

With 2000 random $\mathit{c}$s, we found only one OofA-OA $(24,5,3)$, up to isomorphism, whose row numbers are displayed in Table A1 of Appendix B. The OofA-OA $(24,5,3)$${\mathit{A}}_{5,1}^{24}$ provides relative D-efficiency 0 under the CP model. In order to find more non-isomorphic OofA-OA $(24,5,3)$s, another 8000 random $\mathit{c}$s are used to find solutions of (3). All the resulting OofA-OA $(24,5,3)$s are isomorphic to ${\mathit{A}}_{5,1}^{24}$. We conjecture that there is only one OofA-OA $(24,5,3)$ up to isomorphism. With 2000 random $\mathit{c}$s, in

Table A2. Row numbers of ten non-isomorphic OofA-OA(48,5,3)s.

	${\mathit{A}}_{5.1}^{48}$	${\mathit{A}}_{5.2}^{48}$	${\mathit{A}}_{5.3}^{48}$	${\mathit{A}}_{5.4}^{48}$	${\mathit{A}}_{5.5}^{48}$	${\mathit{A}}_{5.6}^{48}$	${\mathit{A}}_{5.7}^{48}$	${\mathit{A}}_{5.8}^{48}$	${\mathit{A}}_{5.9}^{48}$	${\mathit{A}}_{5.10}^{48}$
1	1	1	2	1	1	1	1	1	3	3
2	5	4	4	4	5	6	4	2	5	6
3	8	5	5	8	6	8	8	4	8	7
4	10	8	7	9	9	11	9	10	10	9
5	13	10	10	14	11	13	13	11	11	11
6	15	13	11	16	12	15	16	15	13	13
7	18	16	15	19	13	18	18	17	18	17
8	20	20	17	20	18	19	19	20	19	18
9	22	22	22	22	21	22	21	22	23	19
10	23	23	24	26	23	23	23	24	25	24
11	26	26	27	27	27	25	26	27	28	26
12	28	28	30	30	28	28	30	29	30	29
13	31	33	31	33	32	32	31	32	33	31
14	36	35	32	35	33	36	36	34	35	34
15	37	37	34	40	36	38	37	38	36	36
16	38	40	38	41	37	39	39	40	39	38
17	40	44	40	45	42	40	41	43	41	39
18	44	46	43	46	43	43	44	44	43	41
19	45	48	48	47	47	45	45	48	48	44
20	48	51	51	50	49	47	46	50	49	45
21	49	53	53	52	52	49	50	51	51	50
22	53	55	57	54	56	52	54	55	54	54
23	56	56	59	55	58	56	55	57	56	55
24	58	58	60	57	59	58	58	60	58	60
25	62	62	63	64	63	61	62	62	63	61
26	63	64	65	65	66	63	63	64	65	62
27	64	67	68	67	67	66	65	68	69	66
28	68	71	72	70	71	68	67	69	71	68
29	69	72	73	72	75	69	69	72	74	69
30	71	75	77	73	77	71	72	74	75	74
31	74	78	79	76	80	74	74	75	78	76
32	76	80	83	78	82	76	76	79	79	79
33	81	81	84	80	84	79	80	81	84	84
34	83	83	85	81	85	84	82	84	86	87
35	85	85	86	85	86	86	85	86	88	88
36	90	90	90	88	90	88	88	89	90	89
37	91	91	92	93	91	91	93	92	91	93
38	94	94	96	94	92	92	94	93	94	95
39	95	95	98	95	98	94	95	96	98	97
40	98	99	101	99	102	98	98	99	100	100
41	99	102	103	101	103	100	100	101	102	104
42	103	103	106	102	105	104	103	105	105	108
43	106	105	107	103	108	106	108	106	107	109
44	110	107	109	108	110	109	110	107	109	112
45	112	110	111	111	112	112	112	109	112	114
46	116	112	113	113	113	117	115	113	114	115
47	117	115	116	115	117	119	116	115	117	116
48	120	120	118	120	120	120	120	120	119	118
$D_{\mathrm{CP}}$	0.94	0.93	0.90	0.90	0.89	0.89	0.89	0.89	0.89	0.88

D_CP: the relative D-efficiency under the CP model.

and

Table A4. Row numbers of ten non-isomorphic OofA-OA(48,6,3)s.

	${\mathit{A}}_{6.1}^{48}$	${\mathit{A}}_{6.2}^{48}$	${\mathit{A}}_{6.3}^{48}$	${\mathit{A}}_{6.4}^{48}$	${\mathit{A}}_{6.5}^{48}$	${\mathit{A}}_{6.6}^{48}$	${\mathit{A}}_{6.7}^{48}$	${\mathit{A}}_{6.8}^{48}$	${\mathit{A}}_{6.9}^{48}$	${\mathit{A}}_{6.10}^{48}$
1	3	20	5	15	4	7	28	1	5	1
2	12	38	15	29	22	18	46	18	15	18
3	30	55	22	40	50	20	52	63	34	42
4	36	62	35	57	64	26	70	68	47	56
5	80	69	41	72	84	42	73	73	53	66
6	90	82	67	74	105	45	78	106	63	73
7	91	93	81	110	116	70	82	116	77	100
8	119	102	117	138	122	87	98	126	131	106
9	129	113	130	140	136	115	110	131	137	128
10	139	122	140	146	169	133	127	177	155	143
11	166	147	148	162	190	142	153	191	161	158
12	176	175	155	177	201	145	163	194	175	168
13	198	182	187	188	212	167	174	208	193	192
14	201	188	201	193	228	192	195	225	233	208
15	216	194	211	234	241	204	204	237	240	223
16	224	213	249	247	255	279	210	245	256	229
17	247	222	273	263	270	285	232	258	261	263
18	261	232	287	271	285	291	235	267	270	267
19	274	269	313	287	294	312	242	293	273	273
20	308	280	330	311	298	320	270	298	299	290
21	317	290	343	318	317	329	295	312	323	312
22	319	302	359	322	330	341	310	315	325	328
23	336	304	367	351	347	378	315	324	367	339
24	344	318	377	365	353	392	322	343	380	345
25	381	337	394	371	374	398	332	355	384	379
26	385	342	408	377	381	403	358	368	402	387
27	404	348	414	395	399	413	362	370	430	401
28	411	363	416	401	404	418	403	388	431	411
29	420	372	439	439	423	433	412	395	441	425
30	434	396	449	449	427	450	427	441	451	441
31	436	450	476	472	451	458	435	455	500	462
32	454	468	504	484	480	497	448	470	504	465
33	466	470	507	501	487	517	451	496	521	488
34	493	483	528	525	497	522	478	501	529	494
35	515	516	534	541	509	523	509	524	550	518
36	521	529	540	549	515	531	533	537	551	532
37	543	560	554	567	520	533	558	551	559	539
38	569	567	576	574	555	579	561	554	576	556
39	580	571	577	596	592	598	591	573	608	584
40	603	584	616	601	599	617	597	585	623	586
41	616	599	623	622	605	634	607	608	628	593
42	643	601	640	627	611	637	624	613	636	608
43	657	663	647	646	616	644	632	622	649	623
44	671	669	650	650	631	651	638	636	666	643
45	674	679	661	666	641	656	662	638	679	667
46	696	692	666	686	681	689	688	688	686	688
47	707	700	680	701	710	714	698	691	700	691
48	709	706	705	707	717	716	713	720	705	720
$D_{\mathrm{CP}}$	0.77	0.77	0.75	0.74	0.72	0.70	0.70	0.69	0.68	0.67

D_CP: the relative D-efficiency under the CP model.

, 10 non-isomorphic OofA-OA $(N,5,3)$s with $N=48,72$ alone with their relative D-efficiencies under the CP model are displayed, respectively. The OofA-OA $(24,6,3)$ does not exist. When using intlinprog” to find solutions to (3) with $m=6$ and $N=24$, it is reported that no solution can be found. With 2000 random $\mathit{c}$s, in Table A4 and

Table A5. Row numbers of ten non-isomorphic OofA-OA(72,6,3)s.

	$\mathit{A}{72}_{6.1}$	${\mathit{A}}_{6.2}^{72}$	${\mathit{A}}_{6.3}^{72}$	${\mathit{A}}_{6.4}^{72}$	${\mathit{A}}_{6.5}^{72}$	${\mathit{A}}_{6.6}^{72}$	${\mathit{A}}_{6.7}^{72}$	${\mathit{A}}_{6.8}^{72}$	${\mathit{A}}_{6.9}^{72}$	${\mathit{A}}_{6.10}^{72}$
1	3	13	7	4	6	5	7	8	3	4
2	12	20	17	10	25	10	13	14	10	8
3	24	26	22	26	42	20	18	27	23	10
4	33	41	35	37	51	23	19	29	38	27
5	44	48	57	46	53	36	44	36	45	36
6	60	55	68	49	66	37	54	42	50	45
7	74	56	75	69	75	61	57	53	63	54
8	89	62	81	88	77	67	64	63	82	64
9	96	90	95	91	101	87	81	77	95	74
10	105	100	107	98	108	93	93	85	99	89
11	107	110	114	112	113	111	105	100	111	103
12	109	134	124	116	122	114	116	109	115	115
13	127	139	125	122	131	116	123	120	121	120
14	132	144	131	135	133	134	131	130	136	133
15	134	147	152	142	153	148	150	143	161	138
16	149	166	160	167	161	160	155	154	165	141
17	156	176	172	177	168	169	174	158	170	154
18	172	177	185	181	184	183	183	165	200	157
19	177	186	194	189	192	189	209	169	207	163
20	193	188	207	197	198	193	213	173	215	183
21	212	203	227	202	206	196	220	192	220	188
22	214	206	236	210	229	216	228	199	232	216
23	227	215	246	221	230	219	251	216	235	228
24	229	227	247	226	237	233	262	223	242	235
25	247	252	264	229	253	247	268	239	252	246
26	253	254	267	246	259	256	273	243	254	249
27	279	265	276	249	268	266	285	250	270	272
28	284	285	285	268	269	272	287	263	273	286
29	286	293	292	293	280	287	291	287	276	295
30	294	297	301	295	299	298	298	293	295	311
31	296	305	317	298	301	301	312	296	315	318
32	306	318	324	321	308	317	313	301	326	319
33	320	322	330	329	316	320	326	320	344	334
34	335	328	337	333	340	337	332	328	353	344
35	336	340	356	337	357	338	334	343	360	360
36	337	347	365	345	364	350	337	359	363	369
37	356	348	371	360	370	360	368	377	384	379
38	362	361	376	366	371	367	372	395	389	392
39	376	374	395	373	384	371	377	397	405	397
40	400	376	400	378	391	377	391	401	412	403
41	403	382	404	392	410	392	396	415	418	412
42	409	395	417	420	424	396	397	425	426	418
43	429	408	432	422	430	404	444	430	427	427
44	435	439	441	428	439	426	452	445	434	434
45	437	450	453	448	456	440	469	458	441	453
46	444	463	470	453	465	450	471	462	451	472
47	458	467	479	463	475	458	475	474	460	480
48	472	473	481	475	488	473	482	498	467	495
49	473	481	497	480	495	476	497	501	468	509
50	485	497	501	492	500	483	504	505	491	517
51	495	501	525	494	519	492	506	519	498	522
52	520	507	528	511	525	514	520	524	507	533
53	532	524	535	516	532	525	527	538	509	539
54	533	530	549	524	543	540	535	548	514	541
55	538	543	551	530	550	542	551	550	522	545
56	560	565	555	532	551	560	560	554	544	558
57	561	569	560	555	563	565	565	556	547	562
58	572	580	575	560	571	576	570	574	555	579
59	578	586	582	579	583	579	585	578	590	581
60	591	598	608	598	599	581	612	594	592	594
61	607	603	610	599	603	595	623	609	597	612
62	617	620	619	614	609	612	625	626	608	627
63	622	622	630	622	624	625	640	630	610	633
64	632	625	631	631	630	630	644	636	623	645
65	633	650	640	647	633	634	646	653	631	647
66	648	661	646	654	650	660	653	658	636	651
67	683	687	664	659	664	663	673	676	638	653
68	686	690	673	665	670	684	680	685	661	674
69	696	692	687	675	683	686	690	693	669	678
70	703	706	691	684	689	705	695	694	696	689
71	707	711	707	698	699	707	699	697	698	694
72	713	715	714	712	718	710	714	710	719	698
$D_{\mathrm{CP}}$	0.87	0.87	0.86	0.86	0.86	0.85	0.84	0.84	0.84	0.84

D_CP: the relative D-efficiency under the CP model.

, 10 non-isomorphic OofA-OA $(N,6,3)$s with $N=48,72$ alone with their relative D-efficiencies under the CP model are displayed, respectively.

4. Some Properties of OofA-OAs of Strength 3

In [13], it is pointed out that when projecting an OofA-OA $(N,m,2)$ onto any two components, the resulting design is an $N/2$-replication of ${\mathit{O}}_2$, and when projecting an OofA-OA $(N,m,2)$ onto its any three components, the resulting design is an $N/6$-replication of ${\mathit{O}}_3$. As will be seen in Theorem 2 below, an OofA-OA $(N,m,3)$ poses additional balance properties when it is projected onto its any four components.

Theorem 2.

For any OofA-OA$\mathit{(}\mathit{N}\mathit{,}\mathit{m}\mathit{,}\mathit{3}\mathit{)}$${D}$,

(i) 

when$\mathit{D}$is projected onto its any two components${\mathit{c}}_{\mathit{i}}$and${\mathit{c}}_{\mathit{j}}$, the resulting design is an$\mathit{N}/2$-replication of${\mathit{O}}_2$;

(ii) 

when$\mathit{D}$is projected onto its any three components${\mathit{c}}_{\mathit{i}}\mathit{,}{\mathit{c}}_{\mathit{j}}$, and${\mathit{c}}_{\mathit{k}}$, the resulting design is an$\mathit{N}/6$-replication of${\mathit{O}}_3$;

(iii) 

when$\mathit{D}$is projected onto its any four components${\mathit{c}}_{\mathit{i}},{\mathit{c}}_{\mathit{j}},{\mathit{c}}_{\mathit{k}}$, and${\mathit{c}}_l$, the resulting design is an$\mathit{N}/24$-replication of${\mathit{O}}_4$.

Example 2 below illustrates the balance properties stated in Theorem 2.

Example 2.

The fractional OofA design

$${\mathit{A}}_{5.1}^{24}={\left(\begin{array}{cccccccccccccccccccccccc}5& 5& 5& 5& 4& 4& 4& 4& 3& 3& 3& 3& 3& 3& 2& 2& 2& 2& 2& 1& 1& 1& 1& 1\\ 4& 4& 2& 1& 5& 5& 2& 1& 5& 5& 4& 4& 2& 1& 5& 5& 4& 4& 1& 5& 5& 4& 4& 2\\ 3& 1& 3& 3& 3& 2& 3& 3& 2& 1& 2& 1& 1& 2& 3& 1& 3& 1& 3& 3& 2& 3& 2& 3\\ 2& 2& 1& 2& 1& 1& 1& 2& 4& 4& 5& 5& 4& 5& 4& 4& 5& 5& 5& 4& 4& 5& 5& 4\\ 1& 3& 4& 4& 2& 3& 5& 5& 1& 2& 1& 2& 5& 4& 1& 3& 1& 3& 4& 2& 3& 2& 3& 5\end{array}\right)}^T$$

in Table A1 is an OofA-OA $(24,5,3)$, where we use the Arabic numbers $1,2,3,4,5$ instead of $c_1,c_2,c_3,c_4,c_5$ to denote the components, respectively, to save space. Projecting the design ${\mathit{A}}_{5.1}^{24}$ onto the components 1 and 2, we obtain design

$${\mathit{H}}_1={\left(\begin{array}{cccccccccccccccccccccccc}2& 1& 2& 1& 1& 2& 2& 1& 2& 1& 2& 1& 2& 1& 2& 2& 2& 2& 2& 1& 1& 1& 1& 1\\ 1& 2& 1& 2& 2& 1& 1& 2& 1& 2& 1& 2& 1& 2& 1& 1& 1& 1& 1& 2& 2& 2& 2& 2\end{array}\right)}^T.$$

In ${\mathit{H}}_1$, each order of the components 1 and 2 appears 12 times. Similar balance properties can be obtained when projecting ${\mathit{A}}_{5.1}^{24}$ onto other two components. Projecting the design ${\mathit{A}}_{5.1}^{24}$ onto the components 1, 2, and 3, we obtain design

$${\mathit{H}}_2={\left(\begin{array}{cccccccccccccccccccccccc}3& 1& 2& 1& 3& 2& 2& 1& 3& 3& 3& 3& 3& 3& 2& 2& 2& 2& 2& 1& 1& 1& 1& 1\\ 2& 2& 3& 3& 1& 1& 3& 3& 2& 1& 2& 1& 2& 1& 3& 1& 3& 1& 1& 3& 2& 3& 2& 2\\ 1& 3& 1& 2& 2& 3& 1& 2& 1& 2& 1& 2& 1& 2& 1& 3& 1& 3& 3& 2& 3& 2& 3& 3\end{array}\right)}^T.$$

In ${\mathit{H}}_2$, each order of the components 1, 2, and 3 appears 4 times. Similar balance properties can be obtained when projecting ${\mathit{A}}_{5.1}^{24}$ onto other three components. Projecting the design ${\mathit{A}}_{5.1}^{24}$ onto the components 1, 2, 3, and 4, we obtain design

$${\mathit{H}}_3={\left(\begin{array}{cccccccccccccccccccccccc}4& 4& 2& 1& 4& 4& 4& 4& 3& 3& 3& 3& 3& 3& 2& 2& 2& 2& 2& 1& 1& 1& 1& 1\\ 3& 1& 3& 3& 3& 2& 2& 1& 2& 1& 4& 4& 2& 1& 3& 1& 4& 4& 1& 3& 2& 4& 4& 2\\ 2& 2& 1& 2& 1& 1& 3& 3& 4& 4& 2& 1& 1& 2& 4& 4& 3& 1& 3& 4& 4& 3& 2& 3\\ 1& 3& 4& 4& 2& 3& 1& 2& 1& 2& 1& 2& 4& 4& 1& 3& 1& 3& 4& 2& 3& 2& 3& 4\end{array}\right)}^T.$$

In ${\mathit{H}}_3$, each order of the components 1, 2, 3, and 4 appears once. Similar balance properties can be obtained when projecting ${\mathit{A}}_{5.1}^{24}$ onto other four components.

The design $\mathit{A}$ in Example 3 is an OofA-OA $(24,5,2)$ but not an OofA-OA $(24,5,3)$. As will be seen, when projecting $\mathit{A}$ onto the components 1, 2, 3, and 4, the resulting design does not have the balance properties stated in Theorem 2 (iii).

Example 3.

Projecting the design

$$\mathit{A}={\left(\begin{array}{cccccccccccccccccccccccc}5& 5& 5& 5& 5& 4& 4& 4& 4& 4& 3& 3& 3& 3& 3& 2& 2& 2& 2& 1& 1& 1& 1& 1\\ 4& 3& 3& 2& 1& 5& 3& 2& 2& 1& 5& 5& 4& 2& 1& 5& 4& 3& 1& 5& 4& 3& 2& 2\\ 2& 4& 2& 1& 2& 1& 2& 5& 1& 3& 2& 1& 1& 4& 4& 4& 3& 1& 5& 4& 5& 2& 4& 3\\ 3& 1& 4& 3& 4& 2& 5& 1& 3& 5& 1& 4& 5& 5& 2& 3& 1& 4& 3& 3& 3& 5& 5& 5\\ 1& 2& 1& 4& 3& 3& 1& 3& 5& 2& 4& 2& 2& 1& 5& 1& 5& 5& 4& 2& 2& 4& 3& 4\end{array}\right)}^T$$

onto the components 1, 2, 3, and 4 obtains

$${\mathit{H}}_4={\left(\begin{array}{cccccccccccccccccccccccc}4& 3& 3& 2& 1& 4& 4& 4& 4& 4& 3& 3& 3& 3& 3& 2& 2& 2& 2& 1& 1& 1& 1& 1\\ 2& 4& 2& 1& 2& 1& 3& 2& 2& 1& 2& 1& 4& 2& 1& 4& 4& 3& 1& 4& 4& 3& 2& 2\\ 3& 1& 4& 3& 4& 2& 2& 1& 1& 3& 1& 4& 1& 4& 4& 3& 3& 1& 3& 3& 3& 2& 4& 3\\ 1& 2& 1& 4& 3& 3& 1& 3& 3& 2& 4& 2& 2& 1& 2& 1& 1& 4& 4& 2& 2& 4& 3& 4\end{array}\right)}^T.$$

In ${\mathit{H}}_4$, the order 1432 appears two times and the order 4321 appears once. Clearly, ${\mathit{H}}_4$ is not an ${\mathit{O}}_4$ and thus does not have the balance properties stated in Theorem 2 (iii).

Remark 3.

The balance properties in Theorem 2 make OofA-OAs of strength 3 useful in the situation where $m-4$ or more components are found inactive after experimentations. The OofA-OAs of strength 3 may have larger run sizes compared to OofA-OAs of strength 2. For example, when $m=5$, the smallest run size of the OofA-OAs of strength 2 is 12 while the smallest run size of the OofA-OAs of strength 3 is 24. Note that a larger design run size implies more observations. People may choose to use OofA-OAs of strength 2 or 3 according to their practical needs. When $m-4$ components are found inactive after experimentations, the OofA-OAs of strength 3 would be better choices.

5. Simulation Studies

We conducted thorough simulation studies to investigate the performances of the constructed OofA-OAs presented in Table A1, Table A2,

Table A3. Row numbers of ten non-isomorphic OofA-OA(72,5,3)s.

	${\mathit{A}}_{5.1}^{72}$	${\mathit{A}}_{5.2}^{72}$	${\mathit{A}}_{5.3}^{72}$	${\mathit{A}}_{5.4}^{72}$	${\mathit{A}}_{5.5}^{72}$	${\mathit{A}}_{5.6}^{72}$	${\mathit{A}}_{5.7}^{72}$	${\mathit{A}}_{5.8}^{72}$	${\mathit{A}}_{5.9}^{72}$	${\mathit{A}}_{5.10}^{72}$
1	1	2	1	2	2	1	1	1	1	1
2	3	3	2	3	3	3	3	3	3	2
3	5	4	4	5	4	5	4	5	5	4
4	6	5	6	6	5	8	5	6	6	6
5	8	8	9	7	7	9	8	8	7	9
6	9	9	11	9	8	10	10	9	11	11
7	10	10	12	11	10	11	12	12	12	13
8	11	12	13	12	12	14	14	14	14	15
9	13	13	15	14	13	15	15	16	15	16
10	15	17	17	15	15	18	16	17	16	17
11	17	18	18	16	17	19	18	18	17	19
12	18	20	21	17	18	20	20	20	19	21
13	19	21	22	20	19	23	21	21	20	22
14	22	24	23	21	20	24	23	22	22	23
15	23	25	25	22	24	26	25	25	24	25
16	26	27	27	25	25	27	26	26	25	26
17	28	29	29	28	29	28	29	27	26	28
18	30	30	30	30	30	29	30	30	28	30
19	31	32	32	31	31	31	31	32	32	33
20	33	33	33	32	34	34	35	33	34	35
21	35	34	36	33	35	35	36	36	35	37
22	36	36	37	35	36	36	37	38	38	39
23	37	37	38	37	37	38	39	40	39	40
24	38	41	40	40	39	39	41	41	40	41
25	40	42	42	41	41	41	42	42	42	43
26	41	44	43	42	42	44	44	44	43	45
27	43	45	45	43	44	45	46	45	44	46
28	45	48	47	46	45	47	48	46	46	47
29	48	49	49	47	48	48	50	49	47	49
30	49	51	50	49	51	49	51	51	50	50
31	53	53	52	52	53	50	53	53	51	52
32	54	54	53	54	54	52	54	54	52	54
33	55	55	55	56	55	54	55	55	55	55
34	56	57	58	57	56	56	57	57	56	56
35	58	59	59	59	57	57	58	59	57	58
36	60	60	60	60	60	58	59	60	60	60
37	61	62	61	61	61	62	61	62	61	61
38	63	64	63	64	62	63	62	64	62	63
39	65	65	65	65	64	64	64	65	64	65
40	66	67	66	66	65	68	65	66	66	66
41	68	69	68	67	67	69	67	67	67	67
42	69	72	69	70	69	70	70	68	68	69
43	71	74	72	71	72	71	72	70	70	71
44	74	75	74	73	74	73	73	73	71	72
45	76	76	75	77	76	74	75	75	73	74
46	78	77	76	78	78	76	77	76	75	75
47	79	80	77	79	79	78	78	77	78	76
48	81	81	81	80	80	79	80	80	79	78
49	83	82	82	82	81	81	81	81	80	80
50	84	83	83	84	83	84	82	82	81	81
51	85	85	84	85	85	85	83	83	84	82
52	86	87	86	86	88	88	87	85	86	84
53	88	89	87	87	89	89	89	87	88	86
54	90	90	89	90	90	90	90	90	90	88
55	91	92	92	91	91	91	93	92	91	91
56	94	94	93	92	94	93	94	94	92	93
57	95	96	96	96	96	94	95	95	93	95
58	98	97	97	97	99	95	97	97	95	96
59	99	99	98	101	100	98	99	99	99	98
60	102	101	99	102	101	99	100	101	100	99
61	103	102	102	103	103	100	101	102	101	100
62	104	103	104	104	104	101	104	103	104	102
63	106	105	105	106	106	103	105	105	105	104
64	108	107	106	108	107	106	106	107	106	105
65	109	108	108	109	109	108	107	108	107	106
66	110	110	110	110	110	109	110	111	109	108
67	112	112	112	111	112	111	111	113	111	110
68	114	113	113	114	114	113	114	114	114	112
69	115	114	115	116	115	114	115	115	115	115
70	116	115	117	117	116	116	116	117	118	117
71	117	117	119	118	118	117	119	119	119	119
72	120	119	120	119	119	120	120	120	120	120
$D_{\mathrm{CP}}$	0.97	0.97	0.97	0.96	0.96	0.96	0.96	0.95	0.95	0.95

D_CP: the relative D-efficiency under the CP model.

, Table A4, Table A5. It was shown that the constructed OofA-OAs of strength 3 have the desirable capability of estimating the optimal orders of addition. For saving space, we only use the OofA-OA $(24,5,3)$$A_{5.1}^{24}$ (in Table A1) and OofA-OA $(48,6,3)$$A_{6.1}^{48}$ to illustrate the simulation studies we have conducted. The other OofA-OAs of strength 3 presented in Table A2, Table A3, Table A4, Table A5 have either close or better performance of estimating optimal orders of addition than $A_{5.1}^{24}$ and $A_{6.1}^{48}$.

Without loss of generality, suppose the underling true optimal orders for $m=5$ and $m=6$ are

$$1\to 2\to 3\to 4\to 5 \mathrm{a}\mathrm{n}\mathrm{d}$$

(5)

$$1\to 2\to 3\to 4\to 5\to 6,$$

(6)

respectively. Establish the PWO models for $m=5$ and $m=6$ as

$$\tau ({\mathit{o}}_k)={\beta }_0+\sum _{i=1}^4\sum _{j=i+1}^5{\beta }_{ij}{\lambda }_{ij}({\mathit{o}}_k)++\epsilon ({\mathit{o}}_k)\mathrm{a}\mathrm{n}\mathrm{d}$$

(7)

$$\tau ({\mathit{o}}_k)={\beta }_0+\sum _{i=1}^5\sum _{j=i+1}^6{\beta }_{ij}{\lambda }_{ij}({\mathit{o}}_k)+\epsilon ({\mathit{o}}_k),$$

(8)

respectively, where ${\mathit{o}}_k$ is an order in $A_{5.1}^{24}$ and $A_{6.1}^{48}$, $\epsilon ({\mathit{o}}_k)\sim N(0,{\sigma }^2)$, and $\epsilon ({\mathit{o}}_k)$ is independent of $\epsilon ({\mathit{o}}_l)$ for $k\ne l$. For both models (7) and (8), the values of ${\beta }_0$ are set to be 1 and the values of ${\beta }_{ij}$s are set following the four scenarios

(S1)

all of the ${\beta }_{ij}$s in models (7) and (8) equal to $0.5$;

(S2)

all of the ${\beta }_{ij}$s in models (7) and (8) equal to 1;

(S3)

all of the ${\beta }_{ij}$s in models (7) and (8) equal to 3;

(S4)

all of the ${\beta }_{ij}$s with $1\le i\le 2$ and $1\le j\le 4$ in models (7) and (8) equal to $0.5$, and the other ${\beta }_{ij}$s in models (7) and (8) equal to 5.

For each scenario $S1$, $S2$, $S3$, and $S4$, the simulation procedure is designed as follows.

• For each order ${\mathit{o}}_k$ of $A_{5.1}^{24}$ (or $A_{6.1}^{48}$ for $m=6$), randomly draw $\epsilon ({\mathit{o}}_k)$ from $N(0,\sigma )$ with a given $\sigma$ ($=1,3,5$).
• Compute $\tau ({\mathit{o}}_k)$ by model (7) (or model (8) for $m=6$), where $\epsilon ({\mathit{o}}_k)$ is the one obtained in Step 1, ${\beta }_0=1$ and ${\beta }_{ij}$s are referred to each scenario of $S1$, $S2$, $S3$, and $S4$.
• Fit model (7) (or model (8) for $m=6$) to obatin a ${\beta }_{ij}$s’ least squre estimations ${\widehat{\beta }}_{ij}$s by $\mathit{Y}=(\tau ({\mathit{o}}_1),\tau {({\mathit{o}}_2,\dots ,\tau ({\mathit{o}}_N))}^T$ and $\mathit{X}=({\mathbf{1}}_N,\mathit{P})$, where $N=24$ (or $N=48$ for $A_{6.1}^{48}$), ${\mathit{o}}_k$s are from $A_{5.1}^{24}$ (or $A_{5.1}^{24}$), and $\mathit{P}$ is the PWO design corresponding to $A_{5.1}^{24}$ (or $A_{6.1}^{48}$).
• Test the significance of ${\beta }_{ij}$s. For the two-sided alternative $H_1:{\beta }_{ij}\ne 0$, the p-value is evaluated by $2\times \mathrm{P}\mathrm{r}\mathrm{o}\mathrm{b}(t>|\frac{{\widehat{\beta }}_{ij}}{\sqrt{c_{ij}}\widehat{\sigma }}|)$, where t follows the t-distribution with 13 (or 32) degrees of freedom for $A_{5.1}^{24}$ (or $A_{6.1}^{48}$), $c_{ij}$ is the diagonal entry of ${({\mathit{X}}^T\mathit{X})}^{-1}$ corresponding to ${\widehat{\beta }}_{ij}$, and the significance level is set to be $0.05$.
• Let $\mathit{S}=(s_{12},s_{13},\dots ,s_{(m-1)m})$, where $s_{ij}$ is the sign of ${\widehat{\beta }}_{ij}$ if ${\beta }_{ij}$ is significant, and otherwise $s_{ij}=0$.
• The underlying order is correctly estimated if there is no element $-1$ in $\mathit{S}$. Repeat this simulation procedure 10,000 times, and summarize the frequency (out of 10,000) of correct estimations of the underlying optimal order.

The simulation results are displayed in

Table 3. The frequencies of correct estimations for the true underling orders.

Designs	$\left(\left(\genfrac{}{}{0pt}{}{\mathit{m}}{2}\right)+1\right)/\mathit{m}!$	$\mathit{\sigma }$	$\mathit{S}1$	$\mathit{S}2$	$\mathit{S}3$	$\mathit{S}4$
		1	0.999	1	1	0.997
$A_{5.1}^{24}$	0.2 $(24/120)$	3	0.993	1	1	0.986
		5	0.997	0.996	0.994	0.990
		1	1	0.996	1	0.993
$A_{6.1}^{48}$	0.07 $(48/720)$	3	0.961	0.994	1	0.985
		5	0.955	0.990	0.995	0.973

with different values of $\sigma$ (=$1,3$, and 5).

From Table 3, the OofA-OAs(24,5,3) $A_{5.1}^{24}$ and OofA-OAs(48,6,3) $A_{6.1}^{48}$ have quite high frequecies (out 10,000) of correct estimations of the true underlying optimal orders while saving a significant experimental cost compared to the full OofA designs as shown in the second column of Table 3.

6. Results and Discussions

The OofA-OAs are a class of D-optimal fractional OofA designs under the prevalent PWO model. The OofA-OAs of strength 2 have been studied in a few studies, we defer to [11,12,13]. However, there is no study on the OofA-OAs of strength 3. This paper studies the OofA-OAs of strength 3 for the first time in the literature.

In Theorem 2, it is shown that the OofA-OAs of strength 3 have better balance properties than those of OofA-OAs of strength 2. These balance properties make OofA-OAs of strength 3 more useful when $m-4$ or more components are found inactive after experimentations. For such a motivation, we propose a systematical construction method for OofA-OAs of strength 3 in Theorem 1. The proposed construction method is capable of finding non-isomorphic OofA-OAs of strength 3, noting that non-isomorphic OofA-OAs may have different performances for other OofA models such as the CP model. In Table A1, Table A2, Table A3, Table A4, Table A5, non-isomorphic OofA-OAs of strength 3 are provided which provides quite high D-efficiencies under the CP model. When models are not prespecified, the OofA-OAs of strength 3 which can provide higher D-efficiencies for both PWO and CP models are desirable.

To further show the efficiencies of the constructed OofA-OAs of strength 3, thorough simulation studies are provided in Section 5. From Table 3, the constructed OofA-OAs of strength 3 can provide quite high frequencies (out of 10,000) of correct estimations of the true underlying orders. This indicates that the OofA-OAs of strength 3 are capable of estimating the optimal order of addition.

7. Conclusions

As a class of efficient fractional OofA designs, OofA-OAs are optimal for the PWO model under a variety of widely used design criteria [11]. In the literature, the studies on OofA-OAs were focused on strength 2 while OofA-OAs of strength 3 have not been studied yet. The high strength results in two major challenges of this work. The first one is the classification of three-column subarrays of ${\mathit{P}}_m$ (as shown in Table 2). As previously stated, ${\mathit{P}}_6$ is the smallest full PWO design to investigate the classification of the three-column subarrays of ${\mathit{P}}_m$ with respect to the ratios among the frequencies of the three-tuple $(a,b,c)$s with $a=\pm 1$, $b=\pm 1$ and $c=\pm 1$. The PWO design ${\mathit{P}}_6$ has $\left(\genfrac{}{}{0pt}{}{\left(\genfrac{}{}{0pt}{}{6}{2}\right)}{3}\right)=455$ three-column subarrays to be classified which is not so easy as the counterpart problem in the case of strength 2. One is referred to [10,11] for the classification of the two-column subarrays of ${\mathit{P}}_m$. The second one is the derivation of Theorem 2. The derivation of Theorem 2 concerns analyses of an equation system consisting of 3640 equations, i.e., the equation system (3) for $m=6$. These large number of equations make the derivation of Theorem 2 more challengeable as indicated by the proof of Theorem 2.

Despite the challenges stated above, this paper provides a threefold contribution. First, this paper provides a method of constructing OofA-OAs of strength 3. This method is capable of finding non-isomorphic OofA-OAs of strength 3. Second, some balance properties of this class of designs are developed. It is shown that OofA-OAs of strength 3 have better balance properties than OofA-OAs of strength 2. For example, when projecting an OofA-OA of strength 3 onto any four components, all of the 24(=4!) orders in the resulting design appear equally often. This balance property is useful when $m-4$ components are found inactive after experimentations. For practical usage, some non-isomorphic OofA-OAs of strength 3 are also provided. Third, the thorough simulation studies are conducted which show that the constructed OofA-OAs of 3 are quite capable of estimating optimal orders of addition.