Different Roles for Multiple Perspectives and Rigorous Testing in Scientific Theories and Models: Towards More Open, Context-Appropriate Verificationism

Abstract

A form of context-appropriate verificationism is proposed that distinguishes between scientific theories as evolving systems of ideas and operationally-specified, testable formal-empirical models. Theories undergo three stages (modes): a formative, exploratory, heuristic phase of theory conception, a developmental phase of theory-pruning and refinement, and a mature, rigorous phase of testing specific, explicit models. The first phase depends on Feyerabendian open possibility, the second on theoretical plausibility and internal coherence, and the third on testability (falsifiability, predictive efficacy). Multiple perspectives produce variety necessary for theory formation, whereas explicit agreement on evaluative criteria is essential for testing. Hertzian observer-mechanics of empirical-deductive scientific models are outlined that use semiotic operations of measurement/evaluation, computation, and physical action/construction. If models can be fully operationalized, then they can be intersubjectively verified (tested) irrespective of metaphysical, theoretical, value-, or culture-based disagreements. Verificationism can be expanded beyond simple predictive efficacy to incorporate testing for pragmatic, functional efficacy in engineering, medicine, and design contexts. Such a more open, pragmatist, operationalist, epistemically-constructivist perspective is suggested in which verification is contingent on the type of assertion (e.g., heuristic, analytic, empirical, pragmatic), its intended purpose, degree and reliability of model-based evidence, and existence of alternate, competing predictive models. Suggestions for epistemological hygiene amidst the world-wide pandemic of misinformation and propaganda are offered.

“True ideas are those that we can validate, corroborate and verify. False ideas are those that we can not. That is the practical difference it makes to us to have true ideas; that, therefore, is the meaning of truth, for it is all that truth is known as. This thesis is what I have to defend. The truth of an idea is not a stagnant property inherent in it. Truth happens to an idea. It becomes true, is made true by events. Its verity is in fact an event, a process, the process, namely, of its verifying itself, its veri-fication. Its validity is the process of its valid-ation.”

—William James, 1907 [1]

1. Introduction

In the midst of our present historical-epistemological crisis of fake news [2] and the debasement of conceptions of truth in all its forms by authoritarian movements and ideologies right and left, practical guidelines for how we can ground our beliefs in empirical evidence (observation, measurement), reason (logic, computation), and rigorous test (demonstrated predictive and pragmatic efficacy) are sorely needed. In this context there is a need to develop sensible, context-appropriate guidelines for evaluating, maintaining, using, and modifying our own beliefs. Like the logical empiricist philosophers and operationalistically-minded scientists of a century ago [3,4,5,6,7,8,9,10,11], I believe that scientific theory and practice can serve as a demonstration of how the search for truth can be conducted in a manner that is radically open-minded, internally coherent, and subject to rigorous test.

Classical verificationism certainly had its faults. In my opinion, many of them were caused by entanglements with logical atomism, vagaries of natural language analysis, equation of verification with meaning, dogmatic adoption of narrow, eliminativist and non-psychological interpretations of meaning, militant falsificationism, and the retreat from operationalism to ungrounded, postulational possible-world semantics.

In particular, earlier verificationist stances involved over-reaching restrictions of “meaning” to either analytically provable/disprovable or empirically verifiable/falsifiable assertions [4,12], the rendering of theoretical concepts not explicitly tied to observables as “meaningless,” and an inability to handle generalizations such as universal laws (e.g., “all flamingos are pink” [12]). These narrow conceptions and definitions tainted verificationism and obscured the more general motivation for verification, which involves the necessity of employing some kind of explicit test. See [13] for an overview of arguments against verificationism and its abandonment by large sections of the philosophy community in the middle of the last century.

To be fair, there were movements within logical positivism and the “unity of science movement” [14] that sought to liberalize empiricism in various ways by including strong vs. weak forms of verification (certainty vs. possibility vs. impossibility of testing [15]), relaxation of testability and falsifiability criteria [12], probabilistic situations [8], and pragmatic considerations [16]. The most widely accepted arguments against verificationism were semantic, i.e., that the verifiability of a statement depends on its semantics (e.g., [17,18], see also [19] for discussion of earlier semantics-based objections).

The semantic objection is a potentially serious problem if verification principles are to be applied to statements expressed in natural language. If the interpretation of a natural language statement is left unconstrained, and unspecified, its meaning can be ambiguous such that then no clear, agreed-upon criterion for verifiability exists. One response to semantic objections of this sort is to specify how a natural language statement is to be interpreted. Hempel’s concepts of cognitive meaning and significance were proposed to constrain how natural language statements should be interpreted and subsequently verified [12,13].

The semantic problem is avoided entirely if one demands that an assertion or question be clarified through provision of accompanying operational definitions. To make an assertion, the nature of the assertion needs to be made clear in terms of how it is to be evaluated (see §8, precept 6). To ask a well-posed question, it must be clear what can count as an answer. As Schlick [20] said, “When in general, are we sure that the meaning of a question is clear to us? Evidently when and only when we are able to state exactly the conditions under which it is to be answered in the affirmative, or as the case may be, the conditions under which it is to be answered in the negative. By stating these conditions, and by this alone, is the meaning of a question defined”.

In contrast with natural language, in which meaning is often not made completely explicit, scientific formal-empirical models accompanied by their operationalized definitions for measurements and computations include descriptions of what exactly needs to be done by others in the community of observers to test them (§5). In the abandonment of verificationism by large sections of the philosophy community, the absolute necessity for explicit, operational definitions and critical testing was discarded in favor of internal coherence.

The context-flexible verification suggested here involves the expansion of the notion of testing to realms beyond empirical prediction, thereby opening verificationism to practical realms where efficacy rather than predictability per se is sought. As Morris [16] put it, “The resulting comprehensive point of view, embracing at once radical empiricism, methodological rationalism, and critical pragmatism, may appropriately be called scientific empiricism.” Here rationalism (logic), empiricism (facts), and pragmatism (values) correspond to Morris’s syntactic, semantic, and pragmatic aspects of signs [21], which in turn map to operations of computation, measurement and action, and goal-related valuations in cybernetic, purposive percept-coordination-action systems (§2.2) and in observers (§5) [22,23,24].

The notion of testing—verification—can also be expanded beyond simple falsification of models to incorporate decision-theoretic degrees of predictive reliability and other kinds of test. In addition to empirical tests, verification can encompass analytical proofs (demonstrations of consistency in finite formal systems [6,25,26]) as well as demonstrations of functional efficacy and sufficiency (e.g., testing in engineering and medicine). What separates “hard” from “soft” disciplines is whether there is a rigorous testing component to what they do.

A more flexible, context-appropriate verificationist perspective can be developed based on the recognition that science involves both informal processes of discovery, theory creation and refinement, and a formal process of hypothesis testing that involves building and testing explicit models (

Table 1. Proposed stages of theories and models in experimental sciences.

Phase	Operations	Goals
CONCEPTION “Anything Goes”	Heuristic, creative, imaginative Problem identification Preliminary observations Preliminary theory construction	Pose basic questions Generate new possibilities Survey possibility space Abduction: propose possible underlying mechanisms
DEVELOPMENT Viability	Eliminate implausible elements Refine theory Find observables Propose concrete models	Define theory’s scope Ensure internal coherence Outline explanatory power Draw possible consequences Propose experiments
PROOF Testing	“Normal science” Refine experimental methods Test: confirm/falsify models Select among alternative models	Show (predictive) efficacy Experimental tests of models Iterative model revisions Explanatory systematicity
USE	Use models to predict Use models to design	Increase efficacy of practical arts (engineering, medicine)

). Although the progression is not always sequential and one-way, scientific theories proceed through different stages of development, from birth, to refined maturity, to rigorous testing, and finally to general acceptance and use. Each stage requires different guiding purposes and procedures in the transition from the heuristic and abductive “contexts of discovery” to operational methods in “contexts of justification” [7,27]. In early stages of theory formation, open-ended creative possibility is paramount, whereas in refinement stages, internal coherence, plausibility, and potential testability become increasingly important. In the final stages, explicit models are constructed that can then be rigorously tested. After testing, models that have been demonstrated to be successful in fulfilling specific purposes, such as analytic consistency, predictive reliability, or pragmatic efficacy, are put to practical use.

One can make clear distinctions between informal, psychological creative processes in early stages, refinements in middle stages, and tests in later stages. Thus, processes and contexts of discovery can be separated from those of justification, even if in practice scientists go back and forth between them as their investigations proceed [27,28]. Similarly, ongoing observations (explorations) can proceed amidst any stage of the process, with new data potentially triggering both incremental and radical revisions of concepts, theories, and explicit models.

Science therefore inevitably spans informal, semi-explicit, and formal domains. The creative, imaginative, psychological process, by its inchoate and ambiguous nature, is extremely difficult to formalize (and may be inherently unformalizable), whereas parts of the theory refinement process are amenable to explicit description. The testing process, in contrast, can be completely operationalized, both in the formal-predictive and non-symbolic aspects of models (§5). The formal-predictive parts are the mathematical, algorithmic computations that make predictions from the outputs of measuring devices (observed initial conditions). The nonsymbolic aspects involve physical action, such as building and calibrating measuring devices and actions taken to prepare experimental systems, which almost inevitably involve trainable manual skills and implicit, procedural knowledge.

Metaphysical, postulational, and speculative ideas are extremely useful as heuristics in formative stages. Abductive reasoning is essential in both formulating theories and in proposing concrete models. Feyerabendian “anything goes” is the appropriate modus operandi in early stages [29], whereas synthesis and explanatory coherence in middle stages require somewhat different attitudes [30,31,32,33]. Model evaluation, including, but certainly not limited to, Popperian falsifiability, requires altogether different, more hard-nosed and dispassionate critical methods and mindsets [34].

The theme of this special issue concerns agreement, disagreement, and multiple perspectives. In situations where mutual trust is lacking, if we can find common means of verifying analytical, empirical, and pragmatic assertions, then at minimum, we can share common facts and demonstrations of what works, and what doesn’t. If we can agree on means of verification, then avenues of cooperation may open, even in the face of incommensurable or antagonistic private beliefs and values.

2. Preliminary Definitions: Truth, Meaning, Knowledge, Purpose, Testing

2.1. Conceptual Context

Some preliminary personal and philosophical orientation may be helpful to readers. By way of introduction, I regard myself as a philosophically-minded theorist and experimentalist in theoretical biology and neuroscience. Here, I draw on my own past experiences reading the literatures and working in conceptual (philosophical), theoretical, and experimental scientific realms. These range from conceptual, explanatory frameworks to general theories to specific theories and finally to specific predictive models. Conceptual frameworks involved systems theory, adaptive and self-organizing purposive cybernetic systems [35], semiotics, and cybernetics [36,37]. General theories involved, temporal coding [38], neural timing nets [39,40], and the neural concomitants of conscious awareness [41]. Specific theories involved the temporal theory of pitch perception [42,43], and specific predictive models involved protein folding [44] and musical pitch perception [45,46].

Working in these domains with widely varying apertures has led to the notion of qualitatively different stages of scientific development and the different concerns and activities they entail. My temporal theory of brain function [40] is a general theory in its very tentative, formative, abductive stages, whereas the theory of temporal coding of sensory qualities [38,42,43,47] is based on surveying existing neurophysiological and psychophysical literature. In contrast, the empirical [48], analytical [45], and computer-based models for pitch and consonance [43,46,49] make specific testable predictions that can be quantified in their predictive efficacy and evaluated in terms of the breadth of perceptual phenomena they predict.

Although I regard science as very far from infallible, its institutions highly imperfect and suboptimal, and its intellectual life far too susceptible to conventional, consensus thinking than it might be, at least science and engineering, unlike many other human arenas such as religion and politics, hold the critical search for truth and efficacy as their core disciplinary values.

Coming from the natural sciences, my general orientation is similar to that of van Fraassen [32] when he says:

“I try to be an empiricist, and as I understand that tradition (what it is, and what it could be in days to come) it involves a common sense realism in which reference to observable phenomena is unproblematic: rocks, seas, stars, persons, bicycles. Empiricism also involves certain philosophical attitudes: to take the empirical sciences as a paradigm of rational inquiry, and to resist the demands for further explanation that lead to metaphysical extensions of the sciences. There is within these constraints a good deal of leeway for different sorts of empiricist positions. For my part, specifically, I add a certain view of science, that the basic aim—equivalently, the base-line criterion of success—is empirical adequacy rather than overall truth, and that acceptance of a scientific theory has a pragmatic dimension (to guide action and research) but need involve no more belief than that the theory is empirically adequate.”

The context-appropriate verificationist approach advocated here is empiricist, internalist, structuralist, and psychologically- and epistemically-constructivist [50,51], and not realist, externalist, social-constructivist or social-determinist.

The perspective is relativist in the sense that theories and models need to be evaluated in terms of specific purposes. Whereas the purposes of a given model may be a free (arbitrary, unspecified) choice of the observer (there is an inherent relativism between motives of different actors), if its purpose is well-specified and sufficiently operationalized in the form of an agreed-upon metric of performance such that success can be clearly evaluated, then models can be directly compared with each other with respect to that particular purpose. Relativism disappears once testing criteria are specified.

The perspective shares some common ground with some varieties of scientific realism, such as fallibilism and critical scientific realism [52,53]. Realism encompasses a broad gamut of ontological and epistemological assumptions concerning the existence, mind-independence, and knowability of the external world (“reality”). Niiniluoto [53] lists six independent types of realist assumptions: ontological (existence, mind-independence), semantical (objective language-world relations), epistemological (access to reality), axiological (value realism), methodological, and ethical (independent existence of moral values).

The agnostic position advocated here could be characterized more as “non-realist” as opposed to “anti-realist” in that it is agreed that a realm external to observers exists, that this realm has (mind independent) states and processes that exist and proceed in the absence of observers. Further, in agreement with many non-platonic and non-theistic forms of realism, the sole access to this realm is through empirical observation (sensing/measurement processes).

Psychological realism is the commonsense working hypothesis that there is a world beyond our senses that has some underlying unobserved structure that influences what we do observe. A non-believer in ontological realism can also reasonably hold that psychological realism can be heuristically productive in the abductive imagination of what unobserved underlying processes might produce the behaviors we can observe. This is similar to the situation of a strict finitist who denies the independent existence of actual infinities on the grounds that they can never be observed [26], but has no objection to heuristic use of ideas of infinity. Notions of omniscient observers and the “realities” they behold can be conceptually useful in theory formation even if none exist [54]. In my view, unverifiable metaphysical beliefs do not necessarily either help or impair the conduct of scientific investigation.

Where flexible verificationism parts company even with fallibilistic forms of scientific realism (e.g., truth as “approximate” [55]) is our neo-Kantian rejection of the belief that exhaustive knowledge of the specific form of the external realm is possible. In this view knowledge is inevitably partial due to the limited set of observables that we have on hand and the finite number of measurements that be made [56]. If “objective reality” is inevitably ill-defined, then any conception of truth based on correspondences with it will also inevitably be ill-defined as well. The phrase “approximate truth” does not capture the fundamental ambiguities inherent in unknown unknowns.

In place of truth-values as correspondences with a partially-known, ill-defined realm of some objective reality, flexible verificationism instead relies on efficacy demonstrated through empirical testing. Efficacy can take many forms, of which predictive (empirical) efficacy is only one.

Keas [57] has pointed out that theories and models in the sciences can be valued for a number of “theoretical virtues”: evidential accuracy (how well theories fit empirical evidence), causal adequacy (how well theories provide causal explanations), explanatory depth (range of phenomena explained), internal consistency and coherence, (explanatory unity and parsimony), coherence with other bodies of knowledge, beauty, simplicity (economy of explanation), unification (breadth of types of phenomena explained), durability (ability to persistently handle new data over time), and fruitfulness (heuristic usefulness in generating additional hypotheses, inquiries, and discoveries). In the framework of Table 1, most of his virtues apply to theory creation and refinement phases, but some, such as simplicity, unification, durability, depth, and accuracy could apply to models in the testing phase as well. To these virtues, one could also add dimensions of intelligibility (intellectual accessibility), which one would expect to be dominant in the refinement phase, and usefulness (practical use value), which would presumably be dominant in the use phase. Although Keas’ dimensions are couched in terms of scientific theories and models, one can envision analogous general, domain-universal dimensions in the practical and fine arts that concern effective design and action (§7): degree of efficacy, range of applicability, systematicity, universality, economy, robustness, adaptability, extendability, intelligibility, and beauty.

Certainly, the contents of minds are highly influenced by social conventions and pressures, such that social psychology and sociology could in principle make predictions as to which scientific problems are addressed and even which kinds of theories are likely to be contemplated, but these externalities, however strong, cannot negate the partial autonomy that scientists do have as free individuals with free minds in choosing what to study and how to study it. Whereas values and culture do play roles in theory creation and formation, they should be absent from testing phases (the testing of the load capacity capability of a bridge should never depend on the values and culture of the engineers who designed it). Although values and culture influence what observers choose to attempt to predict, once formal-empirical models are constructed and their methods operationalized and publicly explicated, values and culture (should) play no role in the measurement process, the formal predictive computational process, or in the final evaluation of predictive efficacy.

2.2. Definitions of Terms

To head off confusion, some terms will be defined. The term truth is used here advisedly, in a non-realist, pragmatist, radical constructivist sense [50,51], to mean an assertion or belief that has been verified through testing. Analytical truths, which we take to be truths-by-conventional-necessity [58], are the outcome of a proof process that demonstrates their consistency, i.e., how things must be so. These are “necessary truths,” given the symbols and conventions (sign-states and rules) that are assumed. Empirical truths are the outcome of testing processes that are contingent on measurements (“contingent truths”). Predictive truths are the outcome of testing via operationalized formal-empirical models. Pragmatic truths are the outcome of testing of the sufficiency and efficacy of specified actions to achieve some specified, desired result. In this view, truth is modal, contingent on the type of verification involved.

Knowledge is an informational distinction that has been proven useful for some purpose [23]. To be more specific, knowledge is a distinction that enables reliable achievement of some specified internal goal within a purposive, cybernetic percept-coordination-action system. Beliefs are justified to the extent that they have been tested and found to reliably enable some function. Knowledge is thus justified belief.

A purposive, cybernetic percept-coordination-action system is a system that is steered by internal goals and feedback mechanisms in its ongoing sensory-motor interactions with its environment (Figure 1). The holding of an informational distinction in a particular representational state X₁ as opposed to X₂ (e.g., the present location of a food item) partially enables the system to achieve goal Y (finding the food item). Knowledge of X enables Y, provided that the system can achieve the goal given the distinction. If the system cannot use X to achieve Y or any other function, then the distinction is irrelevant information, not knowledge.

This naturalized conception of knowledge ties knowledge to a specific situated, embodied, purposive observer-actor context. Knowledge is always knowledge for what purpose in what situation. Knowledge can guide effective action (enactive, procedural knowledge) or it can alter internal mental maps that can be used for orientation and guidance of future behavior (representational knowledge).

Percept-coordination-action systems can be recognized in terms of their functional organization, which consists of functional states and operations. I have proposed a semiotic, functional working ontology that consists of sign-distinctions, signs, and primitive operations (measurements, computations, actions, and non-semiotic physical processes) that mediate between explicate sign-states and implicate physical states [22,23,24,59]. The framework was influenced by the analysis of “modeling relations” in the theoretical biology and biological cybernetics of Pattee [60] and Rosen [61,62], as well as Ashby’s theory of models (“systems”) [63] and Klir’s codified systems science [64]. The primitive operations can be distinguished by an external observer by their characteristic observed state transition structures, provided that an appropriate observational frame (set of observables) is chosen. For example, measurements have contingent state-transitions, whereas computations have determinate ones. For a measurement, one reference state transits to two or more final states, contingent on the interaction of a sensor with its surrounds. For a computation, each predecessor “total machine” state has but one successor state, such that the state transition is determinate. This provides an operational means of distinguishing between sensing (measurement) operations and coordinative percept-action linkage (computation) operations in biological systems [59].

The primitive operations answer different types of queries: analytical questions are answered by computational demonstrations; empirical questions are answered through measurements, and pragmatic questions of efficacy are answered by actions, as evaluated vis-à-vis particular goals. Thus, in the structure of these primitive operations are clear divisions between analytic truths-by-convention, empirical facts, actions, and values (purposes). Despite many efforts by analytic philosophers to muddle these divisions, e.g., [17], clear demarcations are possible.

The semiotic framework is embedded within a larger hylomorphic ontology in which informational functions supervene on particular organizations of material substrates (i.e., no function without matter + organization). Within this perspective, mind is the informational organization of the brain. More accurately, mind is the informational organization of a whole body that includes the nervous system embedded within it.

The high-level, abstract semiotic-functionalist framework is naturalistic in the sense that it describes the structures and functions of animal nervous systems, minds and behavior. Like Rosen’s anticipatory systems theory [61], it can also be applied to artificial systems (computers, robots) and externalized systems (human organizations, scientific models). The semiotic-functionalist framework encompasses adaptive, learning processes and evolutionary processes. Adjustments can involve adaptive re-arrangements of existing functional states (switching, feedback to “software”, Piagetian assimilation) or evolutionary physical construction of new states (feedback to structure, “hardware”). It is adaptive and combinatorially-creative in that it permits internal modification of functionalities contingent on improvements in performance vis-à-vis internal goals (e.g., drives for homeostasis, survival, reproduction). It is evolutionary and emergent-creative in that it permits physical construction of new structures that can subserve qualitatively new functions, analogous to Piagetian accommodation.

The framework is psychologically-constructivist because it assumes the vantage point of limited observer-actors that are nevertheless capable of modifying and augmenting their internal structures and organizations. By altering their sensors, effectors, and coordinative faculties, these observer-actors can change their relations to the worlds around them. In terms of modeling relations, these are changes in observables (measuring devices), formal predictive models (computations), and actions (physical construction of measuring devices and experimental preparations).

An informational distinction is an alternative internal mental, configurational state. Informational distinctions are differences in the internal functional organization of percept-coordination-action systems that make differences in subsequent internal representations or behaviors. Types of knowledge range from fully specifiable and communicable distinctions to ill-defined and ineffable distinctions (e.g., explicit, propositional knowledge, conceptual knowledge, procedural knowledge, implicit skills).

Meaning, as used here, refers to the manifold consequences of a message for its recipient (received meaning) for subsequent internal state and/or behavior. Intended meaning consists of the consequences that the sender of a message desires to be brought about in a receiver. Senders and receivers can be biological systems, in which case internal meanings are psychological (e.g., changes in beliefs, emotions, motivations), or they can be artificial informational systems, in which internal meanings involve functional states (e.g., changes in internal representations or goals). This expanded, supralinguistic, semiotic conception of meaning includes syntactics, semantics, pragmatics [21] and encompasses other dimensions of mental and informational function as well [24].

This internalist, psychological/functional conception of meaning is distinct from both imagined social constructivist meaning and platonic possible world semantics that disconnect meaning from concrete, purposive, observer-actors. Meaning is “in the head,” contra (middle period) Putnam [65], because messages may not be interpreted by all receivers in exactly the same way. Meaning is ultimately private to individual cognizers and ultimately idiolectic. For example, although the words “dog” and “cat” may have approximately similar public meanings in dog and cat lovers, the words will have different private psychological meanings in the form of different emotional responses for individuals in the two groups.

Shared, public meanings can be constructed by a community through iterated, ongoing cooperative coordinations of behavior in shared attentional spaces [66]. Public, conventional meanings are a subset of private meanings that are used for reliable communication and coordination of behavior. We use words and phrases in a conventional manner that we believe will likely be understood by others in the way we intend. Even such public meanings of mundane utterances such as “pass the salt” must all be refracted through (interpreted by) individual minds that hold mental sociolinguistic models of what the speaker likely intends.

Operational definitions [11] involve the construction of shared procedures that are in effect public, operational meanings that no longer depend on private meanings [11]. Operational definitions are an integral part of routine science—in every scientific paper, there is a methods section that spells out exactly what needs to be done to set up experimental conditions and measuring devices such that experimental results can be replicated. There can be tacit skills involved (e.g., recognizing specific details in a microscope, pulling glass microelectrodes, calibrating a sensor), but these are also operationalized through hands-on training of techniques. In the formal predictive parts of empirical models and in computer models and simulations, the computer code used to calculate predictions can be shared such that there is no ambiguity in the procedures used.

Verification is the process by which an assertion is put to test. The test can involve an observation or demonstration that either confirms or denies the asserted outcome of the test. There can be tests of prediction in which a model simply predicts the outcome of a subsequent measurement, or there can be tests of functional efficacy—performance vis-à-vis some specified goal—in which some outcomes are preferred over others. In the semiotic-ontology, simple predictions involve non-valuated measurements, whereas functional tests are evaluative measurements. Fields with different types of purposes use different types of evaluative criteria (§7).

3. Scientific Theories vs. Models

Scientific theories and models as a touchpoint for taking up questions concerning the role of agreement and disagreement of human participants within various modes or life-stages of scientific theories. Science is a reasonable starting point because scientists, their communities, institutions, and practices are deeply concerned with empirical truths, such that issues of justified belief constantly come up in their practices. These concerns are shared by other communities where fact-checking, testing, and accurate reporting are important: engineering, design, medicine, journalism.

A strong distinction is made here between scientific theories and scientific models.

Scientific theories, as discussed here, are conceptual systems. They are individual and collective intellectual constructs that consist of both tacit, privately-held and explicit, publicly-shared ideas about the correlational and causal structure of the world [67]. As systems of ideas, unless they are explicitly codified and formalized, theories are only partially defined, such that their implications cannot be operationalized. Such conceptual systems are not themselves in a form that is amenable to rigorous test. In this view, the purpose of scientific theories is both to generate, refine, and organize ideas and to guide the construction of explicit empirical-predictive models.

Scientific formal-empirical models, in contrast, are explicit constructions for the testing of hypotheses that are designed to be subjected to rigorous, unambiguous test. Here models include not only the formal-predictive part, usually described in terms of mathematical operations (algorithms), but also include all of the procedures and skills needed to set up controlled experimental conditions (“preparing the system”) and to construct and calibrate measuring devices. Formal scientific models lend themselves to explicit descriptions in terms of signs (pointer readings, symbols) and operations (measurement, computation, construction of measuring devices and experimental “preparations of the system”), such that clear, intersubjectively-verifiable agreements can be reached regarding whether a given prediction is successful or not.

4. Scientific Theories as Informal Systems of Ideas

Scientific theories here are systems of related ideas that are used to describe, understand, and predict observed phenomena that are related to some aspect of the world at large. For this discussion, we will sidestep all of the Kuhnian socio-psychological aspects of concrete scientific movements and practices [68,69] and treat theories purely as conceptual paradigms. Here we will treat scientific theories as intellectual frameworks for understanding the world, generating questions about it, and providing general answers (explanations). Theories are intellectual schemas that provide heuristics, working assumptions, conceptual paradigms, root metaphors [70], explanatory modes, and/or methodological approaches. As with other types of idea systems (“ideologies”), theories can range from somewhat loosely defined and articulated sets of ideas to extremely well-defined and delineated assumptions and tenets.

As psychological and social constructs, i.e., as individual and shared conceptual systems, theories are almost never rendered completely explicit by axiomatization. As informal sets of beliefs, their contents and implications may be only incompletely understood by adherents, practitioners, agnostic neutral observers, and opponents.

Although each scientist has their own private conceptual representation of what a given theory entails (what it means to them), it is nevertheless possible to discuss the common beliefs that scientists hold regarding a theory. As with non-scientific ideologies, theories have core tenets that embody their essentials as well as “protective belts” that serve as substantive and rhetorical defenses against alternatives [28]. Theories, as ideologies that guide investigation, hypothesis formulation, and model testing, serve as general types of explanations rather than specific, testable hypotheses.

5. Operational Structure of Scientific Models

In contrast to theories, models, as defined here, are formal, explicit empirical hypothetico-deductive frameworks for making predictions on the basis of observations (measurements) and predictive algorithms (computations). Formal-empirical models are distinct from analogical, physical models, such as Lillie’s iron-wire model of the neuron [71].

They include not only the formal, predictive (logico-mathematical computational) part, but also all of the experimental procedures used to make measurements. Whereas scientific theories may contain tacit, hidden assumptions, interpretational ambiguities, and “protective belts”, formal empirical models, to be good models, should be completely explicit in their structure.

The process of making and testing a predictive model was explicated in the late 19th century by Helmholtz, Hertz [72], Mach, and Einstein [73]. Succeeding early 20th century theorists further refined operationalist descriptions of experimental procedures, measurements, model building and testing [9,74,75]. W. Ross Ashby’s systems theory [63] and George Klir’s systems science [64] were further refinements of the operational structures of formal-empirical models.

Models, as discussed here, are completely operationalized entities. Anyone who can replicate the experimental methods and predictive algorithms should be able to make their own observations that replicate those made by others and to test model’s predictions. Unlike propositions or natural language statements, all of the terms and operations in a formal-empirical model have been operationalized, such that their interpretations and use within the modeling framework is completely explicated and thus able to be reliably reproduced.

In Hertz’s scheme (Figure 2), a set of measurements is made whose outcome states (Eddington’s “pointer readings” [76]) are translated into alternative discrete sign-states that signify the model’s initial conditions. A formal-predictive algorithm maps the initial sign-states to some final state, which is the predicted state. The formal-predictive algorithm is a computation operation in the semiotic framework mentioned above whose sign-states can be put in a 1:1 correspondence with those of a determinist finite-state automaton. A second measurement is made and the output signs of this measurement are compared with the predicted state. If the predicted and final observed state are the same, then the prediction has been confirmed; if not, the prediction has failed. The scheme is called a “commutation diagram” because the two directed paths in the scheme and lead to the same end state.

In many situations a formal-predictive model will successfully predict some phenomenon with less than perfect accuracy. Over many prediction attempts, the observed frequency of correct predictions, in frequentist terms, its probability of success [8], can be compiled, such that the observer has a metric to assess the model’s predictive reliability. Decision-theoretic measures, such those used in psychophysics [77] and systems-theoretic criteria [64], can also serve as measures of the quality of a model. Knowledge can thereby be weighted by its reliability in bringing about a desired result (successful prediction), such that degrees of certainty can be assigned, and more certain knowledge can be distinguished from that which is less certain.

When there is a need or desire to improve predictive efficacy, then the observer needs to revise the model, either by changing the formal predictive algorithm or by changing the observables (Figure 2). The observables here include not only the measuring devices and how they are calibrated, but also all of the physical actions that are taken to “prepare the experimental system” to make a measurement (not shown). The choice of what to predict is a free choice of the observer, i.e., completely dependent on the observer’s goals.

Predictive models are primarily used in the experimental sciences, where the goal is to understand correlative and causal relationships between observable processes. In the experimental sciences, typically experimental variables are controlled so that clear relations can be observed and causal chains identified. Predictive models are less common in the social sciences, where the complexity, variability, and historical nature of human behavior makes control of experimental conditions much more difficult, if not impossible. Where experimental variables cannot be controlled, scientific investigation must rely on (usually statistical) descriptions of individual and collective behaviors that may only permit ranges or statistical distributions of predicted outcomes, making falsifications of models much more difficult and equivocal. As a result, in the social sciences, social theories are used more as explanatory frameworks rather than for generating and testing specific predictive models.

6. Stages of Theory Formulation, Development, and Testing

It should be said at the outset that much of experimental science proceeds in the absence of any generative theoretical framework. This pattern of incremental progress is especially pronounced in the biological sciences, where, unlike physics, general theories are widely disdained. Here, local, hypotheses are produced, experiments are run, preliminary results garnered, papers submitted, grants written, and the cycle proceeds without much theoretical reflection or innovation.

In those experimental fields where theories are also held in esteem, theories serve to germinate ideas that eventually flower to produce testable models. Arguably, as intellectual projects, scientific theories pass through three broadly defined stages: formation (birth), development and refinement (maturation), and testing (proof of efficacy). If they are successful in producing models that reliably predict specific phenomena, then theories have a fourth stage of practical use. Each stage involves a different set of activities and purposes (Table 1), such that the stages can also be regarded as modes of inquiry [78]. Because of their dependencies, the stages typically follow each other in the proposed order. However, this temporal order is not rigid, such that further theoretical refinements and even reconsiderations of basic premises of the whole paradigm, though rare, can go on in parallel during the testing phase.

The phases involve an initial, exploratory, creative stage in which a problem is recognized and outlined, preliminary observations and surveys of alternative explanations are made, and possible theories are envisioned. This is the arena of creative imagination of possibilities, where “anything goes” is an appropriate working, “brainstorming” strategy [29].

The theory conception phase is followed by a more sober, critical refinement stage in which major possible alternatives are triaged, with less plausible ones put aside, internal incoherences resolved, explanations refined, observables are considered, and explicit models formulated.

Abductive psychological processes can operate in theory creation, refinement, and model-building phases. Abduction, as used here in terms of a creative, psychological cognitive process for hypothesis formation [79] rather than a formal procedure for logical inference in explanatory justification (“inference to the best explanation”). The term abduction (“retroduction”) was first proposed by Peirce c. 1858 as a process of hypothetical reasoning (see [80] for a comprehensive treatment of both uses and their histories).

In the sciences, abduction is most commonly regarded in Peirce’s original sense as the process of imagining underlying processes and causal mechanisms that might explain some observed aspects of a system’s behavior. As such, abduction is a psychological process of hypothesis formation. For example, early theories of auditory sound localization [81,82] and pitch perception [83] postulated the existence of time-delay neural networks whose behavior emulates key aspects of those perceptual functions, as had been revealed in overt judgments by listeners in psychophysical experiments. The theories that arose from the early abductive process led to more refined explicit predictive models [84,85] that then could be rigorously tested against observed perceptual capabilities.

A final, more critical, evaluative stage puts specific models to empirical test, where they can be either confirmed to varying degrees or falsified by the operations described above (§5). The specific models either correctly predict some phenomenon at least some of the time, lending plausibility to the model, or they don’t. In this testing stage, assessment of predictive efficacy and Popperian “falsification” are appropriate.

Falsification is a criterion that tells us whether the model can be tested at all. If a model cannot be falsified, i.e., fail in any conceivable way, then either it is tautologous, or it has not been specified clearly enough to be tested. To paraphrase Wolfgang Pauli, in the latter case the model is “not even wrong” in the sense that it produces either correct or incorrect predictions. For example, a proposed model may not have been fully operationalized, i.e., its predictive algorithm is not specified in enough detail such that it could be implemented by others. Or specific sets of observables, including all of their accompanying experimental preparations, may not have been given. For example, in a neurophysiology experiment, this can include surgical procedures, type of anesthesia, which kinds of electrodes to use and how to make them, how to deliver stimuli, how to record spikes and measure their timings. Part of the verification process is the consideration of alternative explanations and model predictions. In practice, models are not discarded if they are not completely effective predictors, especially if they are more reliable predictors than rival models. In the phase of model-testing, the iterative adjustments of §5 may be made, and if no adequate adjustments can be found, then parts of the theoretical framework may need to be considered, and a new generation of models constructed and then tested.

Models that predict the same, or more usually overlapping, sets of phenomena that are generated within the same theoretical framework or by entirely different frameworks can be compared for predictive reliability, scope, and efficacy. Even if a model is “falsified” by producing some incorrect predictions, this does not mean that either the model or its theoretical framework should be discarded entirely [28], again especially in the absence of strong, alternative models and explanations.

Provided that some of the models generated by the theory work to reliably predict some part of the world, the theory then becomes widely accepted, and its successful models can be used in practice.

To summarize, different stages of theory birth, development and testing require different kinds of processes. Theories in early stages of formation require more latitude than more established competitors and should not be subjected to rigorous testing and falsification until they have had the opportunity to refine their ideas and propose experimental tests.

If different stages of scientific development have different sets of aims and criteria that are development-appropriate, then at least some of the polarities within pragmatism can be transcended. One of these tensions exists within pragmatism between pragmatisms of the left and right. Pragmatism of the left is a relativistic, subjectivistic, post-modern perspective, a la Rorty, whereas pragmatism of the right is concerned with pragmatic efficacy and objectivity (I would conceive of right-pragmatist “objectivity” in terms of intersubjective replicability). As Marsonet puts it:

“Usually, those who are interested in pragmatism from an historical point of view tend to forget that, from the beginning, a substantial polarity is present in this tradition of thought. It is a dichotomy between what Rescher calls “pragmatism of the left”, i.e., a flexible type of pragmatism which endorses a greatly enhanced cognitive relativism, and a “pragmatism of the right”, a different position that sees the pragmatist stance as a source of cognitive security. Both positions are eager to assure pluralism in the cognitive enterprise and in the concrete conduct of human affairs, but the meaning they attribute to the term “pluralism” is not the same”

[86]

If different criteria are used for different stages of scientific development that have fundamentally incommensurable intellectual purposes, then the pragmatist umbrella can span both left and right to cover a wide range of disparate, yet related positions as divergent as those of Rorty, Feyerabend, Popper, van Fraassen, Lakatos, and Rescher.

7. Expanding Verification to Include Different Types of Testing

In addition to the sciences, parallels for each of the phases in Table 1 exist in many other areas where ideas, theories, and practices are refined, tested and revised. These fields include philosophy, mathematics, engineering, medicine, the fine arts, and social practices (

Table 2. Areas of intellectual endeavor, goals, and evaluative criteria.

Area	Purpose	Specific Goals	Tests
Philosophy	Analysis of concepts Refinement of concepts	Clarify ideas Produce new ideas	Heuristic efficacy Productive efficacy Internal consistency
Mathematics (ordering)	Formulate theories and descriptions of order	Internal consistency Logical necessity	Mathematical proof by demonstration or argument
Sciences (understanding)	Understand how the world works—describe relations between observables	Predict observable phenomena	Predictive efficacy
Engineering Architecture Design	How to design artefacts & systems that reliably realize some set of goals	Effective design relative to some purpose (goal).	Pragmatic efficacy Functional efficacy
Therapeutic arts (healing)	Find ways to restore or optimize function (medical, therapeutic arts)	Effective interventions	Clinical trials (show efficacy)
Fine arts (engaging)	Produce works that bring about desirable, meaningful psychological states	Induce psychological responses, facilitate meaning	Aesthetic efficacy
Social design Urban planning Political design Law design Economic design	Improve social functions, social stability, cooperation, living standards, health, welfare, security, happiness	Bring about positive social change Minimize negative social change	Social efficacy (however defined)

). These various areas of intellectual endeavor have fundamentally different purposes (goals), and therefore have different types of criteria for their evaluation in the testing phase. For example, in mathematics, the testing phase involves formal proof. In engineering and medicine, it involves demonstration of efficacy, i.e., some artefact or intervention reliably subserves some defined function. In the fine arts, the goal is aesthetic efficacy, i.e., that the process produce works that are engaging and meaningful in some way to audiences and/or to the artist who created them.

Whereas the social sciences are primarily concerned with understanding social relations and dynamics, social design, sometimes derisively called “social engineering,” involves the design or modification of social institutions and practices to realize desired social goals. The goals can involve maintaining order and social stability, improving quality of life (e.g., better education, health, housing, security, social inclusion, standard of living, more desirable psychological states), increasing social participation, and minimizing negative social behaviors (e.g., crime, pollution, domination, stress, alienation). Social design can involve the design, implementation, and modification of political systems, legal structures and practices, and modes of economic organization, production, and exchange.

8. Precepts for Verification of Beliefs

Although I was formally trained in organismic and theoretical biology, cybernetics and systems theory, and auditory neuroscience rather than philosophy, I strongly believe that philosophy matters, that the analysis of concepts has irreplaceable direct practical import for science, engineering, and the humanities. Our ideas should have concrete implications for how we think and act.

In these epistemically-challenging, propaganda-laden times a revived verificationist “show me the evidence” attitude is sorely needed. Such attitudes can provide a useful guide for how we hold, maintain, change, and communicate our beliefs, not only in science but in many other aspects of life as well.

As an exercise in epistemological hygiene, some precepts, maxims, and proverbs are proffered:

• Be self-conscious about what you believe and why you believe it. Be intellectually honest with yourself as much as possible. Be aware of what is the basis for your belief, be it past experience, direct or indirect evidence, reason, social convention, faith, spirituality, emotional comfort, or motivational inspiration.
• “Base your beliefs on evidence, not faith, and consider alternative hypotheses.” [30]. Hitchens’ Razor: That which can be asserted without evidence, can be rejected without evidence [87].
• Test your beliefs for internal consistency.
• Test whether your beliefs are dogmatically held by evaluating whether they are falsifiable by any conceivable turn of events. Recognize the difference between beliefs that cannot be tested and those that can. Assess each belief in terms of whether it can be tested and how it can be tested.
• Demarcate those beliefs that are well-founded from those that are purely provisional, tentative, entirely speculative, or aspirational [88]. Estimate how likely the belief will be proven correct. Evaluate how certain or uncertain is your knowledge, and how dear the consequences of lesser degrees of certainty.
• How is a statement to be taken? Assess what is the appropriate mode to adopt a belief—is it meant as heuristic, poetry, humor, sarcasm, empirical fact, analytical truth, accurate prediction, or a claim of what works?
• Honest communication. Above all, be sincere and, if possible, be clear. Mean what you say and say what you mean. Be clear about what mode you intend your audience to take what you say so there is no misunderstanding about the nature of the message conveyed. Communicate your subjective assessment of the reliability of a belief along with the belief. Think before you propagate a suspicious belief of uncertain origin. If possible, go to original sources. If not, provisionally trust second-hand accounts. Exercise care in communication of a belief commensurate with how much harm (or benefit) it might be expected to cause.
• Evaluate ideas, not people. A good idea should stand or fall on its own merits, not on the basis of who proposed it. Heavily discount arguments from authority, whether personal or institutional. Reject elitism as a mode of appraisal [88]. Likewise, ignore ad hominem dismissals of an idea. Recognize that scientism is antithetical to the critical spirit of science and that scientific consensus is a not a guarantee of truth. Test conventionally held, consensus beliefs from time to time.
• Avoid undue skepticism. Trust, but verify (Russian proverb). Reject unreasonable skepticism as a mode of appraisal [88].
• Learn to recognize lies [89,90]. Never trust liars, especially habitual ones. Ignore what they say, watch what they do (American proverb). Learn to recognize propaganda. Deconstruct ideologies. Analyze what other beliefs a given belief entails as tacit assumptions. Who benefits?
• Focus on essentials. Beware false dichotomies [91]. Beware straw man arguments. Don’t lose track of core issues. Don’t be distracted by diversionary arguments. Lying with statistics is shockingly simple [90].
• Openness. Be willing to change your beliefs if experience, reason, new evidence, or better explanation presents itself that contradicts existing beliefs in a serious way. Don’t be afraid of considering new ideas, but do compare them with existing ones and test them when feasible. Always keep as many alternative explanations on the table as possible, and weigh them for possibility, plausibility, generalizability, internal coherence, predictive and pragmatic efficacy. Try (test) everything, keep what works. Thessalonians 5:21.