Research Questions with PICO: A Universal Mnemonic

Abstract

A well-formulated research question should incorporate the components of a ‘problem’, an ‘intervention’, a ‘control’, and an ‘outcome’—at least according to the PICO mnemonic. The utility of this format, however, has been said to be limited to clinical studies that pose ‘which’ questions demanding correlational study designs. In contrast, its suitability for descriptive approaches outside of clinical investigations has been doubted. This paper disagrees with the alleged limitations of PICO. Instead, it argues that the scheme can be used universally for every scientific endeavour in any discipline with all study designs. This argument draws from four abstract components common to every research, namely, a research object, a theory/method, a (null) hypothesis, and the goal of knowledge generation. Various examples of how highly heterogenous studies from different disciplines can be grounded in the single scheme of PICO are offered. The finding implies that PICO is indeed a universal technique that can be used for teaching academic writing in any discipline, beyond clinical settings, regardless of a preferred study design.

1. Introduction

A prominent heuristic for formulating research questions is the PICO scheme. Arising from clinical research [1], this mnemonic suggests that a good study design should be clear about four elements that constitute the acronym: first, the ‘patient’ or ‘population of interest’, or more generally, the ‘problem’ (P); second, the ‘intervention’ (I) or main treatment that is performed on that P; third, the ‘comparison’ or ‘control’ (C) treatment; and fourth, the ‘outcome’ (O) that is to be observed. A simple example would be to ask whether, among children who are tired after an hour of sports (the patient or P), a cup of ginger tea (the intervention or I) leads to less tiredness (the outcome or O) than a cup of black tea (the control or C).

The scheme has been handy because “(f)ormulating a research question is an intellectually challenging and time-consuming undertaking” [2], a task so difficult that, according to one study, even two thirds of scientific publications express their research questions inadequately [3]. As imprecise wordings of study aims could negatively impact the further development of research [4]; explaining research goals clearly is “an issue of highest concern” [4]. It is because of this that PICO has become an often-used technique in the teaching of academic writing [5,6,7], aiding students with articulating research objectives coherently without missing key components of their intended study.

Despite its parochial origin in a clinical context, some have conjectured that the PICO model might be useful for other disciplines as well [8]. However, this remains in doubt; most discussions of PICO still remain centred on clinical–medical questions [4,9,10], sometimes even doubting its suitability for that narrow setting [11]. The core weakness reported is that PICO depends purely on ‘which’ questions, requiring an arbitrarily preselected alternative between two medical treatments, without being able to tackle interrogations of ‘what’, ‘when’, ‘how’, or ‘why’ [12]. It was thus claimed that with a PICO-based research question, one can only generate study designs that detect correlations between a treatment and a desired outcome [13]. In other words, the literature tends to regard PICO as a mnemonic suitable for quantitative investigations and experimental setups, but not necessarily for qualitative research and theory-prone treatises [14,15]. Or, to use common textbook terminology [16], PICO seemed apt for correlational study designs in clinical settings, but not for descriptive ones and nonclinical research.

The present paper argues otherwise. It will demonstrate that PICO offers a universal mnemonic that can be used for every study design in every discipline and every knowledge-building endeavour. To do so, the following section will modify the usual PICO components (“the clinical PICO scheme”) via two steps of abstraction. The first abstraction will lift PICO from purely clinical settings into a general scheme of correlational study designs (“the correlational PICO scheme”). The second abstraction will then put PICO into relation with four components inherent to every scientific research endeavour so as to allow for its use even for descriptive research (“the universal PICO scheme”). The presumed universality of PICO will then be illustrated with the help of highly heterogeneous research questions; various papers from various journals in various disciplines will serve as examples that are read through the single lens of PICO. The discussion section, finally, points to blind spots of the heuristic (e.g., it cannot, for instance, guarantee the formulation of ‘interesting’ or ‘solid’ questions) and offers practical implications (e.g., for using PICO in teaching academic writing) and further venues for related research (e.g., the integration of PICO into scholarly metadata).

2. Abstraction I: The Correlational PICO Scheme

One way to classify study designs is to distinguish descriptive investigations from correlational ones [16]. Descriptive research explores a given object without analysing a precise, quantifiable relationship among its components, while correlational study designs aim to find robust associations among measurable variables (Note that both types may be either qualitative or quantitative, albeit descriptive research tends to be associated with qualitative methods and correlational studies with quantitative ones. Certainly, a clear-cut distinction between qualitative and quantitative approaches is not possible, and one may leave it to the reader to decide whether the correlational claim in the last sentence was more quantitative or qualitative in nature—just to hint at the arbitrariness of this distinction. Despite the imperfection of this classification, one may stick with it for illustrative purposes due to its textbook popularity; this paper will later transcend it anyway by focussing on their underlying commonality).

The PICO scheme bears an obvious affinity with correlational study designs. To use an example from the Cold War literature on international relations, a theory may argue that in a conflictual, bipolar setting of the international system, summit meetings between two superpowers will likely take place in neutral countries [17]. There is thus a variable—the likeliness of diplomatic meetings hosted by neutral states—that is dependent on another parameter, namely the level of conflictual tensions within world politics. The dependent variable (DV) and the independent variable (IV) are assumed to correlate: the greater the conflict, the more important neutral countries are for diplomatic summit meetings. Or, to cite again the simple example from above, tiredness (DV) is dependent on the (non)consumption of ginger tea (IV).

Such study designs can involve control groups. A research based on the tea-and-tiredness-puzzle could treat one group of randomly assigned children with ginger tea, while treating another group of children with black tea and then analyse the degree of vigilance after the respective treatment [18]. If there is no significant difference in the groups’ respective outcomes, then the null hypothesis holds, i.e., the default assumption that there is no association between the IV and the DV.

While controls are necessary for estimating the impact of a treatment (of the IV), it is also possible to fully dispense with them. Even if controls are lacking, studies can still contribute valuable observational evidence, and they may still aim at rendering a null hypothesis less plausible.

These aspects of the correlational study design—DV, IV, and null hypothesis/control group—can be plausibly squeezed into the PICO scheme. The problem or P is the research object under study or the units that are exposed to an experimental manipulation. The intervention or I is the IV, or the treatment that the researcher undertakes on the units. The control or C is either the control group or, in its absence, simply the null hypothesis. The outcome or O is the DV, or the potential outcomes of a given treatment.

Table 1. The correlational PICO scheme.

Components of PICO	Components of Correlational Study Designs	Example I	Example II
Problem	Units of observation	Children (after exercising sports)	Superpower diplomacy
Intervention	Independent variable	Ginger tea	Conflicts in world politics
Control	Control variable (or null hypothesis)	Black tea	(Null hypothesis)
Outcome	Dependent variable	Degree of tiredness	Importance of neutral countries

juxtaposes the PICO model with these components of correlational study designs. One may label it the correlational PICO scheme.

The terminology of this correlational PICO scheme deviates slightly from the classical clinical one. It thus liberates PICO from mere clinical ‘which’ questions. It does not presuppose two alternative medical treatments anymore, but can rather encompass any observation as long as it conforms to a correlational study design. Even nonclinical questions, such as those about Cold War superpower diplomacy, can now find a plausible translation into the PICO format, at least as long as they aim at finding associations among two measurable phenomena.

3. Abstraction II: The Universal PICO Scheme

The previous section moved PICO away from a clinical setting to a broader and more-encompassing correlational one. However, can PICO also be used for descriptive research designs?

The answer does not seem straightforward. Descriptive research often lacks a clear-cut identification of an IV and a DV. The question of whether society poses an autopoietic system [19] cannot be easily illustrated with variables that exhibit linear, unidirectional effects. No experiment can put society into a laboratory and expose it to an experimentally controlled manipulation of autopoiesis, and this is even less the case in a randomised controlled trial. The research question is nevertheless a valid one that has been pursued in a rich strand of sociological literature. However, if such scholarly endeavours dispense with the variables known from correlational study designs, how, then, can one make sense of the PICO scheme?

In other words, do both descriptive and correlational study designs allow for the use of PICO so as to formulate plausible research questions? Can one modify PICO with the help of universals inherent to every research so as to find a novel and abstract scheme for formulating study aims in a way that it is more generalisable than the traditional PICO model?

What are these universal components inherent to every scientific undertaking? One may assume that the following four aspects are always present:

(a)

First, every scientific undertaking needs to specify a research object. The observer must clearly delineate a phenomenon and approach it with a certain sense of puzzle (In contrast, the outright absence of any unit of observation would not count as research, but rather as a submersion into a noisy chaos lacking perceivable signals).

(b)

Second, the research project needs a methodical approach as nourished by a theory. There should be an idea as to how to proceed towards the research object in a sequence of practical or conceptual steps (In contrast, if there was a focus on a unit of observation, but no theoretical or methodical interest, then one could interpret it as a mere sensuous or cognitive awareness, but not as research).

(c)

Third, the research must be guided by a hypothesis. At the most fundamental level, this hypothesis is at least an implicit, minimal one that assumes that the chosen method or theory will indeed lead to knowledge generation about the research object, for otherwise the research would not be undertaken (In contrast, approaching a unit of observation with a theory or method, but without the minimum hypothetical assumption that the application of this theory or method would lead to knowledge generation might pose a playful, Dadaistic, or disoriented speculation, but not research).

(d)

Finally, the scholarly endeavour must aim at generating new knowledge (In contrast, if that aim is deficient—that is, if there is a focus on a unit of observation with a specified theory/method that is assumed to be apt for generating an explanation about that unit of observation but doing so without pursuing the goal of knowledge generation, then one may count it as a preliminary step of data collection, but not as full-fledged research).

These four aspects—(a) the unit of observation, (b) the theory/method, (c) the hypothesis, and (d) the ultimate aim of knowledge generation—form the conditio sine qua non of all research in the modern understanding. This assumption holds across all study designs, regardless of whether the investigation models its research on the existence of IVs and DVs or not and regardless of whether the scientific project is operationalised empirically or transcendentally, quantitatively or qualitatively, nomothetically or ideographically, or correlationally or descriptively. It is only when all four aspects are present that one can speak of a research endeavour in the contemporary scientific system, regardless of the study design in question.

Juxtaposing these universal components with the PICO scheme allows one to generate an understanding of the mnemonic on a more abstract level. PICO would no longer be limited to ‘which’ questions, correlational approaches, quantitative investigations, and study designs with IVs and DVs. Instead, one should be able to use the PICO scheme for all scientific endeavours with any study design in every discipline. One may find the higher abstraction behind the mnemonic to be best operationalised in the following way:

(a)

The ‘problem’, or P, poses the research object or the unit that is being observed by the research;

(b)

The ‘intervention’, or I, denotes the application of a method or a theory or the sequence of conceptual or practical steps with which knowledge generation about the research object can be achieved;

(c)

The ‘control’, or C, are alternative theories or methods, in the absence of which it is the null hypothesis;

(d)

The ‘outcome’, or O, is the aim of knowledge generation, often the goal of attaining a plausible explanation for an underlying research puzzle (albeit it could also be the collection of a large-scale dataset for further analytical use, or the construction of a taxonomy, etc.).

Due to its high abstraction claiming to encompass every single research endeavour, one could call this the universal PICO scheme (

Table 2. The universal PICO scheme.

Components of PICO	Components of All Research Designs
Problem	Research object
Intervention	Application of a theory or method
Comparison	Alternative theories or methods (or, in their absence, the null hypothesis)
Outcome	Knowledge generation

) as opposed to the clinical and the correlational ones.

This understanding of the PICO mnemonic might indeed be applicable universally for all research. It does not invalidate the clinical or the correlational versions of PICO; instead, it adds a further possibility as to how to phrase a research question, one that goes beyond clinical ‘which’ questions or correlational study designs.

A typical application of this universal PICO model could look like the following: Would approaching P with the help of I allow one to generate a better O than with C? If C is implicit, it could be as follows: Would approaching P via I aid in attaining O? (The C is the null hypothesis, namely the assumption that the use of I on P would not aid in attaining O.)

For example, the first paragraph of this section asked: “Can PICO (P) be modified with the help of universals inherent to every research (I) so as to find a novel and abstract scheme for formulating study aims (O) in a way that it is more generalizable than the traditional PICO model (C)?” Or one could understand the Luhmannian question such that it asked whether society (P) can be better understood (O) via the theory of autopoietic systems (I). In cases where the intervention is not a theory, but rather a method, the scheme remains similar: Does webscraping (I) allow one to generate a large dataset (O) of journal editors (P), a dataset that is much bigger and has better replicability than all other previous attempts (C)? This may sound more elaborate than a mere “Can we collect data about journal editors on a grand scale?” [20]. Or, when it comes to the academic activities of post-Soviet countries (P), does a bibliometric analysis (I) generate robust insights into the research productivity and scientific impact of their publications (O)? This might be a more elaborate alternative to a vague “What can we say about the publication activities of the post-Soviet countries?” [21].

The following section will provide further examples in a more systematic way to ensure that the illustrations stem from various backgrounds.

4. Examples: Unifying Heterogeneous Research Questions

This assumed generalisability of the universal PICO model would be best indicated by using highly heterogenous works with different study designs from distinct disciplines and to exemplify how the one and single lens of the mnemonic can cover their various research goals. For this illustrative purpose, this section looks at each of the eight categories in Google Scholar Metrics [22] (as of mid-January 2022), finds the respective journal with the highest h5-index, and applies the PICO model to the research question of the most-cited paper in each of these eight journals.

Table 3. The sample of papers whose study aims are illustrated based on the universal PICO scheme; the research categories are taken from Google Scholar Metrics.

Category	Journal	Paper
Business, Economics, and Management	American Economic Review	[23]
Chemical and Material Sciences	Advanced Materials	[24]
Engineering and Computer Science	IEEE/CVF Conference on Computer Vision and Pattern Recognition	[25]
Health and Medical Sciences	New England Journal of Medicine	[26]
Humanities, Literature, and Arts	Digital Journalism	[27]
Life Sciences and Earth Sciences	Nature	[28]
Physics and Mathematics	Physical Review Letters	[29]
Social Sciences	Journal of Business Ethics	[30]

provides an overview of the papers in the sample.

The first example, from the American Economic Review, asks whether a regression analysis of tax data (I) can provide robust insights into the long-term impacts (O) experienced by children who were part of the ‘Moving to Opportunity’ experiment (P). (The control or C is an implicit null hypothesis).

The second case pertains to a paper in Advanced Materials, which focused on nonfullerene-based polymer solar cells (P) by trying out a new combination of a conjugated polymer (I) that would lead to a better performance regarding power conversion efficiency (O) than fullerene-based polymer solar cells (C).

Third, a study at the IEEE/CVF Conference on Computer Vision and Pattern Recognition applied a residual learning framework (I) to neural networks (P) so as to generate substantially deeper neural nets despite lower complexity (O) compared to VGG nets (C).

Fourth, a paper in The New England Journal of Medicine sought to understand the coronavirus (P) by extracting data from 1099 patients (I) so as to generate robust preliminary knowledge about aggregate clinical patterns of the then epidemic (O).

In the fifth example, this time from the journal Digital Journalism, a systematic review of the relevant academic literature (I) was applied with regard to ‘fake news’ (P) so as to come up with a useful definition of that phenomenon (O)—a definition that avoids some pitfalls of earlier ones (C).

The sixth publication, one published in Nature, used a full-length genome sequence (I) of the coronavirus (P) to find out which species it belongs to (O).

The seventh paper is from Physical Review Letters. It reported a transient gravitational-wave signal (P), analysed it with the help of general relativity theory (I), leading to the insight that the signals posed an observation of a binary black hole merger (O).

Finally, an article in The Journal of Business Ethics explored academic and policy-related applications (I) of the term ‘circular economy’ (P) with the aim of constructing a new definition (O) that is different from older ones that had lacked a social dimension (C).

While highly heterogeneous, each of the examples allowed one to rewrite their research goals in terms of the PICO scheme. In other words, different kinds of research using descriptive–qualitative designs as well as correlational–quantitative ones were shown to be able to be viewed through the same lens without losing their plausibility.

Note that despite the convention of labelling it as a research question, the study aim does not have to “display the syntactic format of a question” [15]. It does not necessarily carry an interrogative tone signposted with a question mark [31]; this might have been imperative in an oral culture where questions steered conversations of problem solving, such as in Plato’s dialogues, but the focus on syntactic questions becomes less relevant in a written, text-based culture [32]. The research question—or rather research objective or study aim or goal of investigation—can instead be formulated as “declarative statements” [15] or even remain implicit [32,33] as long as the reader can “deriv(e) specific research objectives” [2] from a given scientific manuscript (which applies analogously to closed yes/no questions [34]).

Whether stated in an interrogative manner or in a declarative statement, the most important finding here is that the ability of PICO to unite such heterogeneous research suggests that the mnemonic is not as limited as previous comments have suggested.

5. Discussion

The PICO scheme, a mnemonic for formulating research aims, is so generalisable that it can cover all kinds of study designs. It is not limited to correlational studies in clinical settings, as previously thought, but instead can be used for any research in every discipline. With correlational study designs, PICO may harbour the four components of a unit of observation, an independent variable, a dependent variable, and a control group. Beyond correlations, and on a more abstract level, all scientific endeavours inherently contain four general components that can likewise be juxtaposed to PICO: first, a unit of observation; second, a theory or method; third, a (null) hypothesis; and fourth, the goal of obtaining a plausible explanation of the object under study. By equalling P (or ‘problem’) with the research object, I (or ‘intervention’) with the theory or method, C (or ‘comparison’) with the null hypothesis or with previous theories and methods, and O (or ‘outcome’) with the specific research aim (like the goal of knowledge generation or data collection), this paper argued that PICO is useful for all research questions and for all study designs in all disciplines.

If PICO is not parochially limited to specific ‘which’ questions in clinical studies anymore, then everyone who engages in science education and the teaching of academic writing could adopt this technique so as to convey a useful mnemonic to all students and future researchers—regardless of their actual research field or preferred methodologies. It is indeed already “by far the most widely used model for formulating clinical questions” [35], but the present finding could suggest that PICO be used more widely, even in nonclinical settings and all kinds of disciplines. The pedagogical effect, however, remains to be empirically tested. It is yet unknown whether this version of PICO helps students to be more efficient with focused research projects (albeit there are promising results with regards the use of PICO in literature searches [35,36]), and it still remains hypothetical whether compliance with PICO is really “associated with a higher quality of research reporting” [37]. What this paper did was merely suggest that PICO can be useful for noncorrelational, nonclinical studies, with possible implications for the teaching of academic writing.

Despite the claimed universality of PICO, users should also note what PICO does not do. It does not, for instance, offer minimal criteria of what makes a study aim worthwhile, significant, interesting, or examinable [38]. For instance, the question whether one can generate knowledge (O) about the time of departure of a specific Saturday flight from Vienna to Tashkent (P) via a nonparticipant observation at the airport (I), with the null hypothesis that one cannot (C), may be squeezed into the PICO format, but it would lack any scholarly significance. PICO offers a technique for formulating research questions, but it does not mean that all questions formulated with the help of that technique are solid ones. There are other criteria at play that judge whether a paper’s stated goal is really interesting or not [4,39,40,41], but these criteria reside within the mnemonic’s blind spot.

In addition, the PICO scheme does not guarantee that the questions sound more natural, more plausible, or more elaborate. There are often more parsimonious, even trivial ways of asking research questions, for instance [42], “What treatments efficiently increase sperm quality?” sounds (perhaps blunt, but is still) more natural than asking, “Does a systematic review of randomised controlled trials (I) regarding men’s semen parameters (P) generate robust knowledge about the possible treatments that efficiently increase sperm quality (O)”? PICO might thus achieve lengthier vocabularies with more elaborate semantics clothing a research goal. However, sometimes, the trivial ways of asking questions may better satisfy demands of simplicity and comprehensibility. Rather than judging a formulation’s elegance, the illustrations of this paper merely served to offer a demonstration of how heterogeneous disciplines and study designs can be translated into a single, common, abstract model of PICO. Both the contents and the aesthetics of the stated research objectives resided outside the paper’s focus.

The issue of elegant phrasings, however, could offer a hint for future research. This paper referred to two new possibilities of how PICO can be operationalised; one was specific to correlational studies containing an IV and a DV (the correlational PICO scheme), while the other was more general (the universal PICO scheme). This was not to say, however, that one of the models was superior to the other one. The possibility to operationalise PICO in two or three distinct ways simply opens up the option to attempt various phrasings of one and the same research aim. An assumption that could be tested systematically is that in correlational studies, the correlational PICO model offers a more ‘natural’ reading of a research question than the abstract PICO model. The question, “Given interstate economic sanctions (P), are democracies (I) better at mitigating industrial risks (O) than nondemocracies (C)?” with the IV and the DV occupying the I and C of PICO might sound more natural than, “Can the assumption that international economic sanctions are like recessions (I) help us in finding out theoretically (O) whether democracies or nondemocracies are better at minimizing industrial accident damages (P)?”, where the C is implicitly the null hypothesis [43]. Or the question, “Is international aid (P) delivered by international organisations and nongovernmental organisations (I) more effective in enhancing processes of democratisation (O) than aid delivered by corporations (C)?” [44] might sound handier than, “With regards to international aid (P), can we use a specific organisational theory regarding the accountability of actors as well as a dataset from the Varieties of Democracy project (I) so as to find out which kinds of donors are associated with large increases in democratic values (O), rather than if we did not use this theory and dataset (C)?” Again, these examples may hint at the fact that the correlational PICO model is indeed more suitable for correlational study designs than the universal PICO model, but this assumption awaits systematic testing.

Another interesting venue for PICO would be to have a script automatically extracting research questions from a given body of scientific text so as to structure it into the four components of PICO. In other words, “much like a structured abstract, a structured question could be imposed” [3]. Based on such a machine, one could generate an open dataset of thousands or millions of research questions with four columns (one for each PICO component). With such a dataset, one could loop through all these works so as to scientometrically analyse their trends and patterns across disciplines and time [45]. Such a metadata scheme could also help with building better search engines that allow for a more precise discovery of scholarly works, such as based on a given method or theory or research goal, something that already exists for the clinical literature at the PubMed PICO Tool [46,47] but which could be broadened to other disciplines as well. This implementation would thus come close to the original intention of the mnemonic as a tool for research discovery [48] and literature search, rather than just for the formulation of research questions [11,36,49]. In addition, the aggregate patterns of such metadata could also provide insights into the prevalence (or absence) of explicit control groups in specific disciplines. Any additional meta-scientific data, possibly catalysed by PICO, can inform debates about the dynamics of the broader research landscape [50]. Whatever the future venues, PICO seems to await many research questions to be asked.