Next Article in Journal
Cognitive Network Science Reconstructs How Experts, News Outlets and Social Media Perceived the COVID-19 Pandemic
Previous Article in Journal
Inter-Work and Ethical Vigilance: Two Scenarios for the (Post-)Pandemic Future of Systems Thinking
Previous Article in Special Issue
Leveraging Quasi-Experimental Methods to Estimate Model Structure: Understanding School Funding Changes in Response to Court Orders
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Results of Beer Game Trials Played by Natural Resource Managers Versus Students: Does Age Influence Ordering Decisions? †

1
Department of Agriculture, Agribusiness, and Environmental Science & King Ranch® Institute for Ranch Management, Texas A&M University-Kingsville, Kingsville, TX 78363, USA
2
Innovation Associates Organization Learning, Exeter, NH 03833, USA
3
King Ranch® Institute for Ranch Management, Texas A&M University-Kingsville, Kingsville, TX 78363, USA
4
Department of Animal Sciences, Colorado State University, Fort Collins, CO 80523, USA
5
Office of the President, South Dakota State University, Brookings, SD 57007, USA
*
Author to whom correspondence should be addressed.
Developed from “Turner, B.L.; Goodman, M.; Mathis, C.; Machen, C.; Rhoades, R.; Dunn, B. Lessons from a long-term Beer Game dataset played by natural resource managers: reinforcing systems education across disciplines. In Proceedings of the 35th International Conference of the System Dynamics Society, Cambridge, MA, USA, 16–20 July 2017”.
Submission received: 29 July 2020 / Revised: 5 October 2020 / Accepted: 10 October 2020 / Published: 13 October 2020

Abstract

:
Systems involving agriculture and natural resources (AGNR) management and representing integrations of biologic, geologic, socio-economic, and climatic characteristics are incredibly complex. AGNR managers purport using a systems-oriented mental model while many observed management and policy strategies remain linear or symptom-driven. To improve AGNR professionals’ systems thinking abilities, two programs, the King Ranch® Institute for Ranch Management at Texas A&M University-Kingsville (KRIRM) and the Honors College at South Dakota State University (SDSUHC), implemented the famous Production Distribution Simulation Game (a.k.a. the Beer Game) into their programs beginning in 2003 and 2011. A Beer Game database consisting of 10 years of trials or over 270 individual players was compared to seminal work in the literature as well as to one another. We found that AGNR managers and students performed worse than players in a seminal Beer Game study. More interestingly, we found that younger players adapted more readily to inventory surpluses by reducing the order rates and effective inventories significantly when compared to older players (p < 0.10 for retailer and distributors, and p < 0.05 for wholesales and factories). We substantiated our results to those in more recent studies of age-related decision-making and in the context of common learning disabilities. Lastly, we discuss some implications of such decision-making on 21st century AGNR problems and encourage AGNR disciplines to better integrate system dynamics-based education and collaboration in order to better prepare for such complex issues.

1. Introduction

The nature of agricultural and natural resource (AGNR) systems is inherently complex due to biologic, geologic, economic, social, policy, and climatic characteristics and delays which are powerful and oftentimes longer than in corporate settings. Contemporary AGNR problems have worsened over time, increasingly affecting the livelihoods of producers, local communities, and food systems globally. These 21st century challenges operate at multiple temporal and spatial scales and include problems such as climate variability and change [1,2,3], water resource scarcity [4,5,6,7], soil erosion and land degradation [8,9,10,11], biodiversity loss [12,13,14], and limits to agricultural productivity and food security [15,16,17,18,19,20], among others. These problems feedback on one another making sustainable resource management even more perplexing. Tomek and Robinson [21] introduce such problems in the following:
“Although the agricultural sector is a declining component of most national economies, agricultural product prices remain important both economically and politically. They strongly influence the level of farm incomes, and in many countries the level of food and fiber prices are important determinants of consumer welfare and the amount of export earnings. A decline of only a few cents per pound in the prices of such internationally traded commodities as sugar, coffee, and cocoa can have serious political and economic repercussions in such countries as Mauritius, Colombia, and Ghana. As Deaton [22] pointed out, inaccurate forecasts of commodity prices led to poor policy prescriptions for African nations. Even in the United States, a large drop in the farm price of hogs was reported as likely to drive 24,000 pork producers out of business.”
[23]
They then describe several reasons why such problems persist:
“The characteristics of agricultural product price behavior relate importantly to the biological nature of the production process. Significant time lags exist between a decision to produce and the realization of output, and actual production may exceed or fall short of planned production by a considerable margin. At least a year is required for producers to change hog production, 3 years to change the supply of beef, and 5 to 10 years for growers to increase the output of tree crops such as apples. Yields vary from year to year because of variability in weather conditions and the presence or absence of diseases or insect infestations…”
[21]
Farmers’ mental models are also central to understanding the dynamics of AGNR systems:
“Farmers’ production decisions are based partly on their expectations about future yields and prices (i.e., expected profitability) of the alternative commodities they might produce. Since these expectations are not always realized, price and yield risks exist in farming, and the way expectations are formed and acted on by farmers may affect a cyclical component to supply and prices.”
[21]
Lastly, interactions between economics, decision-making processes, and the constraints of fixed resources converge to create complex dynamics that are difficult to manage due to delayed information feedbacks:
“The nature of resources, like land and equipment, used in farming is such that producers cannot easily make major changes in production plans in response to expected price changes… [typical demand assumptions are] of one-way causation from prices to quantities, but this assumption is the opposite of a common view about the way that prices for agricultural commodities are thought to be determined. That is, current production of agricultural commodities is based on decisions made by farmers many months, even years, in the past. Thus, current supply may be influenced very little by current price. If so, then causality runs essentially from quantity to price; quantity is predetermined by prior events.”
[21]
From its genesis, system dynamics (SD) has taken an interest in such problems (e.g., [24]). Many AGNR systems offer few incentives to adopt systems-oriented mental models that facilitate addressing root causes of issues. Most management efforts have relied on easily accepted (reductionist) methods promoted from within disciplinary silos [25] that often expose learning disabilities in complex systems (described below). As a result, many AGRN problems have gone unaddressed [26]. For example, scientists from a variety of disciplines have criticized reductionist approaches most commonly used in AGNR systems:
“An approach is required which takes account of ecological, economic and social aspects of change and that is able to interpret and synthesize information, generated from a range of sources in a manner which is policy relevant.”
[27]
“I doubt if we stand a good chance of achieving understanding of the components, and of the interactions among them, as long as we insist on maintaining the comfort of our specialist or discipline zone. All indications to me are that we need more integrations of our disciplinary efforts both within and among beef cattle problem areas if we are to make the greatest contribution to developing technology for maximizing the amount of edible beef of a given quality per unit of resource use.”
[28]
To improve and enhance the educational outcomes of AGNR educational programs to better prepare managers for 21st century challenges, in 2003 the King Ranch® Institute for Ranch Management (KRIRM) at Texas A&M University-Kingsville established an innovative curriculum grounded in system dynamics. The KRIRM mission is to educate professionals that will improve resource management across agricultural systems throughout the world. To achieve this, KRIRM offers an intensive two-year graduate program, a less intensive certificate program, and a distance education leadership program for those professionals in industry who cannot attend full-time. The core of each of these programs is a week-long lectureship course in systems thinking that serves as the foundation to each program. Noticing similar needs in undergraduate education, in 2010 the South Dakota State University Honors College (SDSUHC) implemented a systems thinking workshop to enhance students’ learning capabilities prior to completing undergraduate research experiences.
With no “go-to” agricultural educational tool capable of providing as powerful insights as the Beer Game, KRIRM and SDSUHC have used the game as the students’ first exposure to decision making in complex systems. The Beer Game, played for decades in corporate workshops as well as management, business, or systems modeling classes, introduces important lessons about complex, dynamic systems. Designed as a simplified supply chain, the game encourages students to adopt a systems-oriented (i.e., nonlinear system dynamics) perspective. For AGNR managers, possessing a systems-oriented mental model is often purported while the implemented strategies remain linear or symptom-driven. These linear mental models have been reinforced via the fragmented, siloed nature of AGNR education, similarly observed in other fields. Since the inception of the KRIRM and SDSUHC programs, Beer Game performance records have been collected in an integrated database used for debriefing new students each year.
The purpose of this paper is to examine how the decisions among participants with differing domain experience varied from older (KRIRM) to younger (SDSUHC) players. In many industries, experience and wisdom can differentiate effective and successful managers from the pack. It is thought that these managers think more systemically than others since their experiences over time have enhanced their ability to make dynamic inferences less obvious to novices. However, results from dynamic decision-making research has shown that accumulated experiences do not improve performance [29,30]. We aimed to explore this hypothesis further.
First, we review literature on the Beer Game and mental models and dynamic decision-making. We then describe the participants represented in our database and how performance data were screened using a Beer Game simulation model prior to analysis of individual decisions. Results are presented and discussed, including: (a) how well our participants performed compared to seminal Beer Game studies [31,32]; (b) some significant performance gaps observed between the two groups (KRIRM and SDSUHC); and (c) how overcoming the common learning disabilities seen in complex systems will require effective education regarding the dynamics of complex systems. We conclude with a challenge for all practitioners in systems science and AGNR to increase collaboration and support a wider array of educational and research opportunities due to the important role AGNR managers will have in addressing complex 21st century challenges.

2. Beer Game Overview and Decision-Making Research over Time

2.1. Rules of the Beer Game

The Beer Game portrays a production-distribution system characterized by an inventory stock for each of four respective players: factory (or brewer), distributor, wholesaler, and retailer arranged in a linear distribution chain [31,33]. Pennies or plastic chips represent cases of beer. Flows influencing each of the stocks include the information flow of orders from the retailer toward the brewer and the physical transport of goods from one stock to the next through the supply chain, including a two-week delay. Exogenous to the system are consumer orders (delivered through a deck of cards at the retailer end of the chain) and the inputs to the brewer’s inventory production (i.e., supply of chips the brewer has access to). For each week of the game, customers purchase from the retailer, who then orders from the wholesaler, who then orders from the distributor, who then orders from the factory, who then produces additional cases of beer to meet the anticipated demand. The objective of the game is to minimize total costs throughout the supply chain, where holding inventory costs each player $0.50 per case per week or $1.00 per case per week penalty when inventory is in backlog (representing lost revenues and discontent of customers when stockouts persist). Each player manages their respective inventory through forecasting demand, which is informed by the orders of their customer one-step down the supply chain, and placing orders to their suppliers one-step up the supply chain. However, only the retailer can see the actual demand from customer orders week by week. This information limitation is further constrained since players are not allowed to communicate with one another, therefore coordination is impossible [31,33]. Although simplified from real world systems, the game has revealed important characteristics and limitations in human decision-making (described below) and the principles of systems (Section 4).

2.2. Research on Decision-Making in Relation to the Structures of Complex Systems

Research into human decision-making and performance in complex systems has been aided by management flight simulators which have allowed system dynamicists to study an individual’s decision-making within simple system structures, similar to that of the Beer Game (Table 1). Rouwette et al. [34], Aramburo et al. [35], and Mohaghegh and Furlan [36] have summarized the key findings of the major works in this area based on characteristics of the system or task complexity: Delays, strength and types of feedback, exogenous change, use of heuristics or pattern matching mental models, systems thinking skills, and domain experience. In general, the presence of time delays as well as the length of delays tends to reduce performance [32,37], stronger feedback loops tend to reduce performance [32,37,38,39,40,41,42], while exogenous changes have shown either reduced [40,41,42], improved [43], or no effect on performance [44]. Additionally, research has found consistent use of several decision making heuristics, particularly ones that employ pattern matching reasoning [30,36,45,46,47,48,49,50,51,52,53,54], while others have found evidence of poor systems thinking skills to be pervasive throughout most individuals [55,56,57,58].
Whenever the context of the decision situation changes (e.g., changing the decision interval in order to speed up or slow down experience generation; increasing the transparency and therefore knowledge of the model or task being managed; or the nature of the decision information such as interface design and types of feedback provided to the user), participants tend to do somewhat better [61,62,63,64,66,67,70,71], or at least no worse than their prior performance without the added transparency [38,43,46,65,70,72,73,74]. Lastly, personal characteristics that likely impact performance (e.g., stated goals; mental model or cognitive styles; the number of players in the simulator) have also shown mixed results, with improved performance resulting from whole system goals and a higher degree of similarity in participant mental model with the structure of the simulation [74,75,76,77,78]. Other studies have shown no effect on performance due to personal characteristics [42,67,72,73,74].
More recent advances in decision making research using simulations of the Beer Game task have shown that coordination risk (the risk that individuals’ decisions contribute to a collective outcome but the decision rules followed by each individual are not certain) which contributes to the bullwhip effect can be mitigated with coordination stock (holding additional on-hand inventory) but that the behavioral causes of the supply chain instability are robust [68,79,80,81]. Sterman and Dogan [69] show that because of this persistence of instability, individuals are likely to seek larger safety stocks (hoarding) or order more than what is demanded of them (phantom ordering). These irrational responses were shown to be triggered by environmental stressors, which overwhelmed individuals’ rational decision-making abilities or when individuals inappropriately applied decision heuristics incompatible with effective performance in the game. Emotional, psychiatric, and neuroanatomical factors are also discussed in Sterman and Dogan [69,82,83].
Finally, task performance studies that test domain experience, which is often used to understand expertise and where it is assumed that individuals with greater experience employ more powerful heuristics than novices [84,85], have shown that domain experience does not influence task performance [29,42]. This has been found for tests between participant groups (experienced versus inexperienced [29]) or where participants were allowed multiple attempts, given training or an intervention, or with varying feedback protocols [30,49,51,55,85,86,87,88,89].

3. Materials and Methods: Natural Resource Managers and Students Play the Beer Game

Two notes from the literature are important to make before introducing this study. First, Rouwette et al. [34] found that participants in dynamic decision-making task experiments have primarily been sampled from university student populations, with professionals (e.g., university staff) being only a small subset of the participants. This is important because (a) students, however knowledgeable, generally lack the real-world experience and accumulated wisdom that seasoned managers possess and, (b) simulated games could likely represent industries or management situations that differ from the students’ own, making it more difficult for students to comprehend or self-motivate themselves to think systemically about their performance compared to those that begin with more accumulated experience and are therefore able to relate their task performance to real world experiences. Second, previous research has shown that experience does not improve performance on dynamic decision tasks (e.g., [29,30,49,51,55]), but several limitations of those studies warrant reconsideration of that conclusion. For example, Brunstein et al. [29] concluded that domain experience was not a strong indicator for overcoming failures in stock-flow tasks, however, participants only differed by 1.5 years between experienced and inexperienced groups and where each group made decisions under separate environments (one on paper and one online). Additionally, although some studies have utilized both undergraduate students versus graduate students with at least several years of management experience, they did not compare results between groups at the same task (i.e., some experiments only used undergraduates, some experiments only used graduate students [30]).
In this study, we overcome both of these limitations since all participants were subject to the same experimental conditions: All played the traditional Beer Game for a period of about two hours up to 35 weeks, and all trials were facilitated by the same Beer Game instructor. We examined the domain experience hypothesis by analyzing a Beer Game database created from over 10 years of classes at KRIRM and SDSUHC. The process of the study may be summarized as follows (Figure 1) with details in the sections below: Each groups’ decisions were organized into a single database where the team costs were calculated and ranked based on participant entries on the Beer Game record sheets, then the decisions were used as inputs to a Beer Game model to generate model predicted team costs and ranks. Comparison was done between team costs and ranks using participant record sheets and the model predictions to screen for teams that used improper accounting and those teams were removed from final analysis of the participant decisions.

3.1. Participant Profiles

Since the KRIRM programs target agriculture industry leaders, natural resource conservation professionals, and up-and-coming farm and ranch managers, the majority of participants already possess four year degrees (B.S. or B.A.), in some cases graduate degrees (M.S. or Ph.D.), and arrive with at least 15 years (ranging from 10–40 years) of professional experience in production agriculture or resources conservation (mean age = 45.58 years, ranging from 23 to 65). Generally, the systems these participants operate within are laden with time-delays (e.g., crop and livestock production begin with producers making genetic selections on the types of production they wish to market, which take years to decades for payoffs to realize; growing seasons generally allow only one harvest per year and livestock replacement and maintenance efforts requiring two years or longer). Since participants come from agricultural businesses embedded with the agricultural food system, they have intimate knowledge of supply chain dynamics, since in the real world they sit at the producer, distributor, wholesaler, or retailer positions. We were highly interested in analyzing the decisions made in the Beer Game by these groups, since similarities between participant mental models and dynamic decision contexts have been shown to improve performance. Similarly, conservation professionals operate in organizations whose goals are inherently holistic and long-term, since ecosystem restoration or conservation operates at decadal to century time scales. Graduate students in AGNR disciplines but without significant professional experience have also participated, however they have generally been no more than 20% of the participants. Few, if any, undergraduates have ever participated in the KRIRM classes. However, the attendees at the SDSUHC course has been weighted to Honors College undergraduate students (many from family farms or ranches but without managerial experience) that are preparing for undergraduate research experiences (>80%). The SDSUHC class participants (mean age = 23.89 years, ranging from 19 to 55) have also included faculty members throughout the College of Agriculture and Biological Sciences (e.g., animal scientists; wildlife and fisheries scientists; agricultural economists; <20%). In total, there were 55 KRIRM teams and nine SDSUHC teams. Although age does not always equate to professional experience, in this study mean participant age provides the best proxy for experience, given the professionals attending KRIRM would not have done so if not for the experience and managerial responsibility they hold in their organization and the students attending SDSUHC are full-time undergraduates with devoted time to research and were much less dependent on work experience. Although there are a few outliers in each case (graduate students at KRIRM, faculty in SDSUHC), the influence these individuals had on overall performance was controlled for by placing no more than one graduate student or faculty on any individual team.

3.2. Database Description

The data was compiled in Microsoft Excel™ beginning in 2004 (Figure 2), the first year the Beer Game was played at KRIRM. The SDSUHC games were added for the years 2012 through 2014. Orders for retailers, wholesalers, distributors, and factors were kept on one sheet (‘Orders’ tab), with weeks 1 through 35 repeated for each position down the spreadsheet. A similar convention was used to record inventory or backlog (‘Inventory’ tab). The entries for each team stop at 35 weeks to reduce leveling effects (see [31]). In total, the raw data included 64 teams.
Data for orders and inventories were obtained directly from each team’s record sheets. For reference, each team’s total cost was entered at the top of each tab (beginning in cell B5), however, costs at each player position were not recorded. Average costs of each year’s teams were also calculated (e.g., cell C5). The remaining tabs of the database included graphs of trends-over-time of participant performances (e.g., average, best, and worst performances) used as visual aids in the debriefing sessions of participants in the KRIRM and SDSUHC classes (excluding the current year teams, which used the record sheets from their own performances).
The database includes two kinds of uncertainties that must be recognized. The first error common to all Beer Game results is due to human mistakes made playing the game (e.g., getting ahead or behind the weekly schedule; correctly recording inventory or backlog; correctly calculating costs, etc.). However, our database includes another risk of error due to the transfer of information contained on the record sheets into the Excel file. Since the games were played at the beginning of a week-long systems thinking class, results were generally entered by graduate students the following week, or about one week after the completion of the game itself, which precluded any clarification of results, inaccuracies, or illegible entries by participants, as well as the human error involved in the actual transfer of data.
Before proceeding to the analysis, these errors had to be reconciled in the database or teams simply removed from the analysis due to such large errors in effective inventory and costs. Sterman [31] identified that Beer Game teams with the highest costs were the most prone to accounting errors, and therefore, reduced that sample size from 48 to 11 teams, which were generally the best performing. However, successful teams can be just as susceptible to human accounting errors, since mistakes (or variance from optimum decision levels) made around a given average order quantity will not affect the overall rank of teams given lower average orders and therefore inventories. Besides addressing these errors, we also had to reconcile the costs of each player position, since only the total costs per team were entered in the database (which was critical if comparisons were to be made to previous results presented in the literature). Rather than discarding only the poorer performing and analyzing the most successful, but not necessarily lesser flawed teams, we developed a Beer Game model to compare the observed team performances with expected performance given equal accounting standards. The model (described below) aided in identifying the teams with the greatest accounting errors that should be discarded from the final analyses as well as captured costs of each player position, allowing us to compare a more representative range of teams rather than only the most successful to previous research results.

3.3. Beer Game Model

The Beer Game model was developed in Vensim™ (Ventana Systems, Inc., Harvard, MA, USA) in the same table top configuration of the Beer Game, with physical flows for inventories and information flows for orders (Figure 3; player positions are abbreviated to Retailer = r; Wholesaler = w; Distributor = d; Factory = F, and are aggregately described in the italicized names that represent formulation for each of the positions). Equations were developed from previous Beer Game models [31,90] and are provided in Appendix A. Stocks of inventory (inventory [position]) are controlled by flows of cases (in[position] and sold[position]). Backlog stocks account for unmet demand along each position and are used to calculate effective inventory (eff inv [position]). ORDer represented customer orders, beginning at four cases and which steps to eight at the fifth week. Rather than using the decision order algorithms from either [31] or [90] (i.e., a smooth function which provides first order exponential smoothing to represent an averaging process to place orders from each sector), observed orders made by our participants were input into the model (import [position] placed orders). This allowed for the least error prone data in the database, orders placed (where no calculations are required for record keeping), to be used to evaluate teams using equal accounting standards.

3.4. Data Analyses

Initial data analyses consisted of screening the database entries for teams with the greatest errors by comparing observed total team costs and team ranks to the expected total costs and ranks given equal accounting standards. This was achieved via the Beer Game model. Outlier teams were identified and removed from the dataset. For the teams remaining, actual player orders and the modeled team and position costs were used for the analyses. Modeled costs were used to ensure comparison between teams was fair given the likelihood of unknown errors in the dataset and we attempted to minimize these through the screening process used to identify and discard teams. The remaining teams were then used to conduct two different analyses: (1) Due to the similar background and interests of all participants in natural resources management, we examine the performance of all teams. Since no control and treatments were conducted in our Beer Game trials, we simply examine the participant performances in the database with the results of experiments presented in the SD literature (H0: Databasetc = Reportedtc). (2) Due to the unique participant profiles at the two locations (mostly experienced professionals at KRIRM, mostly undergraduates at SDSUHC) we compared team performances between the more experienced and less experienced groups (H0: KRIRMtc = SDSUHCtc).

4. Results and Discussion

4.1. Model Comparison of Team Performances

The database included both KRIRM (n1 = 55) and SDSUHC (n2 = 9) teams since 2004 (total n = 64). Initial screening of the team performances revealed strong fit between total costs (Figure 4a) and team rank (Figure 4b). Overall, the expected costs and ranks of teams fit fairly well, r2 values of 0.90 and 0.89, respectively (Table 2). Despite the overall strong correlations, we identified 20 teams that clearly did not fit the expected costs pattern between the modeled and observed costs of the majority of teams, resulting in a total n of 44 (n1 = 38; n2 = 6). Importantly, the discarded teams were fairly normally distributed throughout the database, with three teams removed from the top quartile, six from the third quartile, eight from the second quartile, and three from the bottom quartile.
Removing these teams significantly improved the match between observed and expected team performances and ranks, with r2 values of 0.97 and 0.98 (Table 2; Figure 4c,d). Average errors (in terms of total $ costs, $/week, or cases/week) decreased $75 and $2, respectively. Removing the teams with inconsistent costs relative to the remaining teams created a significantly improved fit in team ranks (e.g., from six to 11 exact rank matches, or from nine to 25 percent; only five teams with rank discrepancies greater than three positions, down from 55 to 11 percent). Utilizing the Beer Game model in this way allowed us to screen the database for the teams that most likely had the greatest accounting errors and gave added confidence that the remaining teams, although not perfect in their accounting, were accurate enough to allow comparison across the dataset. The proportion of discarded teams due to likely accounting errors (31% of the original database) was therefore much smaller than [31], which discarded 75% of that database due to errors, indicating that players may do a better job of accounting than was previously expected.

4.2. Participants’ Performance across the Database

The team average total costs relative to the benchmark costs (identified in [31]) are shown in Table 3. The average team cost was over 23 times the benchmark and twice the average reported in [31] (Table 3; Figure 5), although that study only reported scores of the best performing teams. The wholesaler, distributor, and factory ratios of actual to benchmark costs were as high as 30 times greater than optimal cost levels, however, the retailers in our group performed similarly to other studies (Table 3). The differences in total costs and costs of each sector to the benchmark costs were all highly significant, and compared to Sterman [31], all sectors were significant except the retailer. To identify how well the best performing teams in our database performed relative to previous studies, total team and individual position costs were summarized into quartiles (Table 3). The top performing teams in [31], whose team average ($2028) and position average costs (retailer $383, wholesaler $635, distributor $630, factory $380) fell most closely between our third and fourth quartile of team performances, indicating similar performance between the above average teams. These results held across positions (Table 3).
Similar oscillations, amplifications, and phase lags were observed between our team performances and common Beer Game results (Table 4; Figure 6). Orders and inventories expressed large fluctuations, with average inventory recovery of 25.5 weeks. Backlogs of inventories migrate from the retailer to the factory similar to typical Beer Game results (Figure 6), with the peak order rate at the factory being over three times the peak order rate of the retailer. Closed loop gains (Δ[factory orders]/Δ[customer orders]) averaged nearly 1400%, or double that reported by Sterman [31]. Maximum backlogs averaged 35 cases and occurred between 34 and 35 weeks (Table 4). As expected, inventories overshoot initial levels, peaking at week 35. Phase lags were more evenly distributed than typical Beer Game runs, however this was likely due to the larger sample size smoothing out the week of peak order rates. Participants’ anticipated minimum inventory (date of minimum inventory minus date of week order rate) were generally delayed by one or two weeks, indicating reactive strategies that did not account for orders in the supply line (orders placed but not yet received) and perpetuated extreme inventory levels later in the game.
Although the overall scores were poorer than team performances reported in the literature, the top 10 teams in our database (or ≈25%) performed better than the top 25% reported in Sterman [31] and held across all game positions except for the factory (Table 5).
Retailer, wholesaler, and distributor costs were all significantly lower (which contributed to an overall significantly lower team total cost), while the factory costs were significantly higher. Periodicity and phase lags were noticeably shorter and amplification lower than the Sterman [31] teams. Of the top 10 teams of our database, the SDSUHC groups were disproportionately represented. Eight of the top 10 teams came from the KRIRM participants (≈21% of the KRIRM sample) while two teams came from the SDSUHC participants (≈33% of the SDSUHC sample).

4.3. Comparison of Performances from More and Less Experienced Participants

We hypothesized that the older, more experienced group (KRIRM) would perform better on the Beer Game task than the less experienced players, primarily undergraduate students (SDSUHC). We found no evidence to support this (Table 6), as neither the team total costs nor any of the player position costs were significantly different. This corroborates previous conclusions that management experience may not mitigate misperceptions of feedback [38]. However, qualitative analyses of the trends in effective inventory and order rates tell a more interesting story (Figure 7). The SDSUHC teams appeared to achieve maximum inventory earlier than the KRIRM groups and by week 35 were reducing their overall inventory levels back toward the ‘anchored’ inventory level of 12. This was achieved through overall lower average order rates (Table 6; Figure 7). Although retailer orders were similar, wholesaler, distributor, and factory average order rates differed from as low as one to as high as six cases per week. After initial inventory recovery, discrepancies in order rates were even larger (up to eight cases at the factory level) and were all statistically significant across positions (Table 6). Based on the change in slope of order rates and effective inventories after week 29 for the SDSUHC teams, it appears the younger players began accounting for cases in delivery much sooner than the KRIRM groups, whose maximum effective inventory levels continued to rise. It is possible that several interesting features are at work that created the divergence in trends of effective inventory between players.
First, the older KRIRM participants could have continued to order more cases after the initial inventory recovery as a way to accumulate “coordination stock” to hedge against the risk that customer orders will significantly change in the future (based on their perception of customer orders as well as experience in the real-world) or in case the other players deviate from the near equilibrium (but-suboptimal) position that the game reaches by week 30 (i.e., compensate for obvious weaknesses in their teammates) [68]. Relying on real-world experience requires participants to determine strategy via comparison of the game to previous experience by analogy, however, decision makers who reason by analogy in complex dynamic situations have not performed as well as those who do not [71].
Second, the older participants were likely less inclined to lower their order rates after inventory recovery, since the initial strategy (increase the order rate to get out of backlog) eventually paid off. In other words, so long as they achieved zero backlog, they were not as heavily anchored to the initial inventory level as the younger players. It has been shown that experience with a particular set of behaviors improves performance, but that as opportunity costs of trying new strategies rises, individuals will experiment with fewer decisions and are less likely to identify superior methods compared to their status quo [91,92]. It is likely that the opportunity costs to change strategies appeared to be too high for the older players.
Third, the younger, less experienced players in the SDSUHC teams significantly lowered their order rates after inventory recovery compared to the KRIRM group (Table 6). Although inventories are affected by the choices of the other players, participants are forced to discretely place new orders based on each new inventory level, and new order rates represent desired change in the stock of the individual player. Therefore, each choice in order is aimed at closing the gap between desired and actual states of inventory (albeit with the necessary receiving and shipping delays).
Our older players increased order rates to get out of backlog, and rather than decreasing order rates once effective inventories recovered, continued to order at relatively high rates (i.e., they were heavily anchored to the choices that worked to get themselves out of backlog), while our younger participants made a more abrupt shift to lower order rates upon inventory recovery and escalation. Younger players in our sample were more heavily anchored to the initial inventory level and were therefore more responsive to escalating inventory levels (and therefore costs) by lowering their order rates significantly (Table 6).
Recent psychology research strengthens these conclusions. For example, research on dynamic decision making choices of younger versus older adults has shown that older adults (age 60–84) perform better on choice-dependent tasks, which require learning how previous choices influence current performance and making a new decision based on that knowledge [93,94]. Older players in our sample were more heavily anchored to their previous strategy that worked (order more cases to get out of backlog), and because of that success continued to do so. Research on younger decision makers (age 18–23) has shown that they perform better on choice-independent tasks (where learning requires exploiting the options that give the highest reward on each new trial [93,94]) and students have best learned dynamic decision making in systems by ‘doing’ and ‘failure’ rather than ‘knowing’ or relying on experience [70]. Older adults have also been shown to base their decisions on changes in states, compared to younger adults, who are more apt to change decisions based on comparison of expected values of new trials [95].
Several cognitive mechanisms or learning impairments may be underlying these patterns. For example, work has shown that age-related impairments in learning may result from declines in phasic dopaminergic signals in older versus younger adults [96], likely contributing to the deficits in feedback-driven reinforcement learning in older adults [97]. In two exploratory choice task experiments to understand how younger and older adults differ in their exploratory choices, Blanco et al. found that strategies by the two groups were qualitatively different (with older adults performing worse), in part due to older adults applying a strategy shaped by their wealth of real-world decision-making experience that may be ill-suited in some decision environments due to increased working memory loads [98]. Worthy et al. suggested that older adults’ departures from state-based decision strategies in favor of immediate reward strategies were due to age-related declines in the neural structures needed for more computationally demanding (e.g., goal oriented) decision making [99]. This cognitive burden on working memory load likely leads participants to focus more on immediate versus delayed consequences of decisions [100]. Lastly, Kurnianingsih et al. found that older adults (aged 61–80) were significantly more uncertainty averse for both risky and ambiguous choices and exhibited strategies with decreased use of maximizing information [101], which likely contributes to learning deficits observed in healthy older adults driven by a diminished capacity to represent and use uncertainty to guide learning [102]. This corroborates others who have shown that younger adults more willingly explore task structures when unexpected rewards or costs indicate a need for a shift in decision strategy, compared to older adults who show preservative behavior and have deficits in updating expected values of alternative decisions [103].
Our results coincide with those observed in age-related studies [93,94,95,96,97,98,99,100,101,102,103] that likely explain the discrepancy in order rates between groups (Table 6; Figure 8). Our results are also strengthened by the conclusions in Rouwette et al. [34]. Rouwette et al. [34] found that: (1) there exist few to no fundamental differences between system dynamics-oriented tasks and performance task games from other social science disciplines, and (2) it was important to note that simulation players have primarily been sampled from university student populations. The psychology literature supports a difference in task performance by age and we have overcome the weakness of relying on university student populations by including a majority of teams composed as working professional in AGNR fields.

4.4. Implications for Agricultural and Natural Resource Management

There are a number of key lessons from the Beer Game in general and from this study in particular that are of interest for AGNR management. The boom–bust nature of the Beer Game occurs due the inherent ordering and shipping delays coupled with the overwhelming tendency of players to ignore their supply line. Natural resource managers embedded in real-world systems with extremely long time-delays (e.g., year to decadal scales) performed just as bad, if not worse, than managers from corporate contexts at identifying and managing the delayed-inventory management task in the Beer Game. Results closely corresponded to typical results of other Beer Game trials, indicating that our participants, despite intimate knowledge of AGNR systems had adopted a similar decision rule identified by Sterman [31], where participants anchor their initial expectations to the starting inventory level that inevitably produces extremely poor results. This is due to the misperception of delayed feedback between placing and receiving orders and not fully accounting for cases in the supply line, both of which lead to over-ordering and instability in even the best performing teams (Figure 5). Even those that recognize and manage systems with many time delays that often vary from months to years in length, they still commit the same errors as ones without such experience with delays.
What are some examples from AGNR systems of failures to account for such delays and supply-on-order and what implications might there be for AGNR management in the 21st century? Unfortunately, numerous AGNR cases can be found. First, it is important to recognize how supply-lines are adjusted in AGNR systems. Producers typically have two leverage mechanisms: adjusting the number of units in production (e.g., total land under cultivation; total animal inventory, etc.) or adjusting the production per unit (e.g., production per unit of cultivated area in cropping systems; yield per head in livestock systems, etc.). Employing either of these options poses interesting trade-offs in the ability to adequately adjust the supply line. Increasing the number of units in production subjects producers to delays on the order of two to four years, while reducing units in production can occur quite rapidly (within a year). On the other hand, increasing the production per unit (through selective plant or animal breeding to enhance production potential) shortens the delay in increasing the supply line, but the genetic enhancement of the overall population makes reduction in per unit productivity extremely difficult if not impossible. To illustrate the importance of these two mechanisms to AGNR systems, consider two recent examples from the United State: corn market boom-and-bust and the contraction of the dairy industry.
The U.S. corn market for decades saw market prices oscillate between $2–4 per bushel and producers’ land use decisions remained relatively stable around 78 million acres (Figure 9). In response to a step change in demand in the mid-2000s arising from renewed energy policies incentivizing ethanol production (similar to the step change observed in the Beer Game), prices rose to a peak of between $6–7 per bushel between 2011 and 2013 due to the inventory shortages resulting from the surge in capacity utilization to fill the increased demand. Producers, aiming to capitalize on the rising prices, began expanding planted area of corn by 20%, much of which onto land that had previously been retired from cultivation. Inevitably, there was a delay in productivity (which continues to increase with investment in crop production potential) as these areas came out of retirement. Failing to account for the supply line (i.e., newly converted land that had not reached its full production potential yet), total production over-shot the increased demand, resulting in a collapse of corn prices back to historical levels by 2014. As of 2020, no significant land use correction has occurred (Figure 9).
The corn market example is a conspicuous case. A more subtle but just as powerful example may be found in the U.S. dairy industry. Dairy production is highly seasonal, peaking in late spring and bottoming in winter. Likewise, dairy production consumption is seasonal, peaking in the late fall during the holiday season. Because of the mismatch in peak supply and demand periods, managing inventory is critical for a stable market environment. As a result, prior to 1960, the U.S. dairy industry experienced cycles of expansion and contradiction similar to many other livestock industries as a result of its commodity cycle (Figure 10). Farm policy interventions in the U.S. began managing these dynamics by purchasing and storing large volumes of milk inventory to buffer seasonal variations in supply and establishing minimum price supports that helped minimize price volatility. Under these conditions, dairy herds were able to consolidate, with 50% fewer head in 1980 compared to 1960. Simultaneously, investments in animal potential yielded a 200% increase in per head productivity. In the late 1980s, U.S. farm policy lowered support prices and government inventories (or coordination stocks) ceased to function as a buffer against seasonal supply and demand imbalances. This increased the price volatility (which has weakened farm business planning, debt repayments, and dairy farm solvency) and the importance of private inventory holdings [104,105].
Why the increasing price volatility (amplitude) despite a stable dairy herd level? In part, seasonality of milk production inevitably creates oscillations in inventory and therefore price. However, the amplitude has significantly increased, with greater gaps between seasonal highs and lows, indicating large shifts in inventory (booms and busts similar to the Beer Game). Booms in supply (which drive price declines) have resulted not from increasing animal units, but increasing production per head (up 400% compared to 1960), and the industry has not counteracted this productivity with reducing total animal units. Instead, inventory corrections have been made through dumping (119 million pounds in 2016, 170 million pounds in 2017, over 145 million pounds in 2018; greater dumping rate is expected in 2020 due to the coronavirus pandemic; [106,107,108]). Clearly, as indicated in farm gate milk prices, this is a low leverage strategy that only temporarily corrects inventory and prices and prolongs the stress to remaining dairy producers as the volatility rises due to the continual rise in incoming inventory (that necessitates increased dumping) that will not soon change due to investments over time in herd productivity (i.e., permanent gains in genetic potential that has raised milk yield per head) that have accrued or have not yet been realized due to delays in the system.
What are the implications for the future of AGNR systems management? Without accounting for the supply line on order in AGNR supply chains, AGNR managers will continue to respond in ways to perpetuate the problems stemming from inherent oscillations and will continue to look for external causes to blame (e.g., environmental variability, government policy change, consumer behavior, etc.) for internal industry dilemmas [104]. System structure can be defined by the basic interrelationships that influence, regulate, or control behavior (including external constraints), but structure more importantly is the endogenous decision-making rules, operating policies, goals, and modus operandi, many of which are unwritten and embedded in the culture of industries and organizations. For example, given the productivity-driven goals and mental models of the dairy industry, order rate (i.e., investment in per head productivity) has not slowed, despite the recognition that the market is over-supplied. Failure to recognize how our decisions interact with the system as a whole hinders our ability to find and effectively apply leverage to systemic problems (leverage often comes from new ways of thinking [109]).
AGNR professionals must overcome the same common learning disabilities that are seen in humans across cultures and contexts [60,95] and the barriers that impede our learning about complex systems [33]. Almost regardless of history or experience, when inserted into a given position within a system or organization, the structure incentivizes that we “become our position.” In AGNR, managers often view their position as “producers” or those who “feed the world,” reinforcing tendencies to view success based on their own productivity rather than how effectively they have met consumer expectations or balanced socio-economic and environmental concerns (e.g., the soil and water externalities cited above). Since many externalities are never felt by those that made the decisions that created the problems and because AGNR delays are particularly lengthy, our ‘knee jerk’ reactions are to assign blame to others around us and we fail to effectively learn from experience and the collective wisdom of others in the system:
“To oscillate, the time delay must be (at least partially) ignored. The manager must continue to initiate corrective actions in response to the perceived gap between the desired and actual state of the system even after sufficient corrections to close the gap are in the pipeline… Learning to recognize and account for time delays goes hand in hand with learning to be patient, to defer gratification, and to trade short-run sacrifice for long-term reward. These abilities do not develop automatically. They are part of a slow process of maturation. The longer the time delays and the greater the uncertainty over how long it will take to see the results of your corrective actions, the harder it is to account for the supply line.”
[110]
Similar learning disabilities and the consequences they exert on decision making have been observed in other natural resource management studies [111,112,113].
To overcome these disabilities and barriers, the SD profession has prioritized and advocated for systems-based education from K-12th grade levels up to university graduate programs (see the Creative Learning Exchange at clexchange.org, as well as works of Forrester [114,115,116,117,118,119] and others [50,55,120,121]). Given the results of our Beer Game database and our experience in the AGNR professions, the need for systems education in these disciplines is as desperately needed as ever if effective change is to be expected and gaps in the 21st century challenge begin to sustainably close.
AGNR professionals with systems education could likely achieve significantly different results compared to professionals without systems-oriented education. For example, thinking in systems forces us to recognize the interconnectedness and dynamic complexity of the problem at hand, the physical stocks and flows central to the issue, and time-delays between decisions and results. Systems thinking and system dynamics modeling also encourages us to maintain an unwavering commitment to the highest standards and rigor of scientific method by recognizing and correcting our hidden biases and documenting and testing our assumptions about the problem. By doing so, we can explore a wider decision space for new or previously unrecognized leverage points to achieve our goals [59]. Achieving the 21st century agriculture challenge requires input and collaboration across disciplines and cultures. System dynamics can provide a common unifying language to facilitate such collaboration.

4.5. Study Limitations

There are a number of limitations of the study as presented. First, for general research purposes, our use of the traditional board, pencil, and paper based Beer Game is unconventional, and it has been noted that this use of the Beer Game is no longer acceptable because of the high rate of clerical and data recording errors. All modern Beer Game studies are expected to use a computer version of the game to prevent such errors and to offer tight controls on information availability among the players. Due to the structure of the lectureships where our participants played the traditional Beer Game and the available computer resources at the time, computer applications have been generally unfeasible. We attempted to limit clerical and data errors by screening the data with an application of a Beer Game model. Second, traditional incentive scheme in the board game version ($1 entry fee and winner-take-all) is not consistent with current standards for experimental studies in economics, in which people are paid in proportion to their performance. Lastly, previous Beer Game studies have estimated an ordering decision rule to test the misperceptions of feedback hypothesis, which we have omitted here but is planned for in the future.

5. Conclusions

System dynamics facilitates investigation of the dynamic consequences of choices made by decision-makers in complex, feedback driven systems. Played by thousands over many decades, the well-known Beer Game has become a fundamental learning tool which reveals critical learning disabilities and illustrates how system structure creates behaviors over time (e.g., oscillatory inventory/backlog and exuberant rising costs). Here, a Beer Game database generated by over 270 AGNR managers (87%) and students (13%) was analyzed. A distinct facet of our study is that the majority of the participants in our database had deep AGNR management experience and least a B.S. degree (many with M.S. or Ph.D. degrees). The performance of these managers was poorer than managers in a seminal Beer Game study. More interestingly, we found evidence that younger players (in this case undergraduate AGNR students) were willing to change their decision strategies sooner and with greater magnitudes in response to pressures of the game compared with their older counterparts. In light of the many 21st century AGNR problems (e.g., food and agriculture production, natural resource capacity and environmental quality, pollution mitigation, etc.), being able to identify and communicate the dynamic complexity of problems and overcome common learning disabilities would greatly benefit AGNR managers. System dynamics provides such a framework and has been integrated to two AGNR programs described here and we encourage other AGNR programs to likewise adopt a system dynamics approach. Future work may explore emotional and psychological factors underpinning dynamic decision-making.

Author Contributions

Conceptualization, B.L.T.; methodology, B.L.T.; formal analysis, B.L.T. and M.G.; data curation, R.M., C.M., R.R., and B.D.; writing—original draft preparation, B.L.T.; writing—review and editing, M.G., R.M., C.M., R.R., and B.D.; project administration, B.D. and C.M.; funding acquisition, C.M. and B.L.T. All authors have read and agreed to the published version of the manuscript.

Funding

This research was partially supported by USDA-NIFA Higher Education Challenge under Grant No. 2018-70003-27664 and NSF-CREST Grant No. 1914745.

Acknowledgments

We would like to thank five anonymous conference paper reviewers for their helpful comments and critiques. All were used to revise and improve specific sections of the paper. Student research assistants Chris Flores-Lopez and Nicholas Cavazos assisted with literature review and verifying bibliographic information.

Conflicts of Interest

The authors declare no conflict of interest.

Appendix A

Table A1. Model equations for the Beer Game model to aid in team performance evaluation in this study.
Table A1. Model equations for the Beer Game model to aid in team performance evaluation in this study.
EquationUnits
1Backlog D[Team] = INTEG (bFlow D[Team],0)cases
2Backlog F[Team] = INTEG (bFlow F[Team],0)cases
3Backlog R[Team] = INTEG (bFlow R[Team],0)cases
4Backlog W[Team] = INTEG (bFlow W[Team],0)cases
5bFlow D[Team] = ordered W[Team]-sold D[Team]cases/Week
6bFlow F[Team] = ordered D[Team]-sold F[Team]cases/Week
7bFlow R[Team] = ORDer-sold R[Team]cases/Week
8bFlow W[Team] = ordered R[Team]-sold W[Team]cases/Week
9coming[Team] = ordered F[Team]cases/Week
10Cost[Team] = INTEG (cost increase[Team],0)$
11cost increase[Team] = cost of backlog[Team] ∗ (Backlog D[Team] + Backlog F[Team] + Backlog R[Team] + Backlog W[Team]) + cost of inventory[Team] ∗ (inventoryD[Team] + inventoryF[Team] + inventoryW[Team] + inventoryR[Team])$/Week
12cost of backlog[Team] = 1$/cases/Week
13cost of inventory[Team] = 0.5$/cases/Week
14d cost increase[Team] = (cost of inventory[Team] ∗ inventoryD[Team]) + (cost of backlog[Team] ∗ Backlog D[Team])$/Week
15d costs[Team] = INTEG (d cost increase[Team],0)$
16eff inv D[Team] = inventoryD[Team]-Backlog D[Team]cases
17eff inv F[Team] = inventoryF[Team]-Backlog F[Team]cases
18eff inv R[Team] = inventoryR[Team]-Backlog R[Team]cases
19eff inv W[Team] = inventoryW[Team]-Backlog W[Team]cases
20f cost increase[Team] = (cost of inventory[Team] ∗ inventoryF[Team]) + (cost of backlog[Team] ∗ Backlog F[Team])$/Week
21f costs[Team] = INTEG (f cost increase[Team],0)$
22import D placed orders[Team]: = GET XLS DATA(‘Book1.xlsx’, ‘orders (D)’,’A’, ‘B2′)cases/Week
23import F placed orders[Team]: = GET XLS DATA(‘Book1.xlsx’, ‘orders (F)’,’A’, ‘B2′)cases/Week
24import R placed orders[Team]: = GET XLS DATA(‘Book1.xlsx’, ‘orders (R)’,’A’, ‘B2′)cases/Week
25import W placed orders[Team]: = GET XLS DATA(‘Book1.xlsx’, ‘orders (W)’,’A’, ‘B2′)cases/Week
26in D[Team] = DELAY FIXED(sold F[Team], 2, 4)cases/Week
27in F[Team] = DELAY FIXED(coming[Team], 2, 4)cases/Week
28in R[Team] = DELAY FIXED(sold W[Team], 2, 4)cases/Week
29in W[Team] = DELAY FIXED(sold D[Team], 2, 4)cases/Week
30inventoryD[Team] = INTEG (in D[Team]-sold D[Team],12)cases
31inventoryF[Team] = INTEG (in F[Team]-sold F[Team],12)cases
32inventoryR[Team] = INTEG (in R[Team]-sold R[Team],12)cases
33inventoryW[Team] = INTEG (in W[Team]-sold W[Team],12)cases
34ORDer = 4 + STEP(4, 5)cases/Week
35ordered D[Team] = DELAY FIXED (import D placed orders[Team], 1, 4)cases/Week
36ordered F[Team] = DELAY FIXED (import F placed orders[Team], 1, 4)cases/Week
37ordered R[Team] = DELAY FIXED (import R placed orders[Team], 1, 4)cases/Week
38ordered W[Team] = DELAY FIXED (import W placed orders[Team], 1, 4)cases/Week
39r cost increase[Team] = (cost of inventory[Team] * inventoryR[Team]) + (cost of backlog[Team] * Backlog R[Team])$/Week
40r costs[Team] = INTEG (r cost increase[Team],0)$
41sold D[Team] = MIN(inventoryD[Team] + in D[Team], ordered W[Team] + Backlog D[Team])cases/Week
42sold F[Team] = MIN(inventoryF[Team] + in F[Team], ordered D[Team] + Backlog F[Team])cases/Week
43sold R[Team] = MIN(inventoryR[Team] + in R[Team], ORDer + Backlog R[Team])cases/Week
44sold W[Team] = MIN(inventoryW[Team] + in W[Team], ordered R[Team] + Backlog W[Team])cases/Week
45TIME STEP = 1week
46w cost increase[Team] = (cost of inventory[Team] ∗ inventoryW[Team]) + (cost of backlog[Team] ∗ Backlog W[Team])$/Week
47w costs[Team] = INTEG (w cost increase[Team],0)$

References

  1. Akerlof, K.; Maiback, E.W.; Fitzgerald, D.; Cedeno, A.Y.; Neuman, A. Do people “personally experience” global warming, and if so how, and does it matter? Glob. Environ. Chang. 2012. [Google Scholar] [CrossRef]
  2. Sheffield, J.; Wood, E.F.; Roderick, M.I. Little change in global drought over the past 60 years. Nature 2012, 491, 435–438. [Google Scholar] [CrossRef]
  3. Trenberth, K.E.; Dai, A.; van der Schrier, G.; Jones, P.D.; Barinchivich, J.; Briffa, K.R.; Sheffield, J. Global warming and changes in drought. Nat. Clim. Chang. 2013, 4, 17–22. [Google Scholar] [CrossRef]
  4. Taylor, R.G.; Scanlan, B.; Doll, P.; Rodell, M.; van Beek, R.; Wada, Y.; Longuevergne, L.; Leblanc, M.; Famiglietti, J.S.; Edmunds, M.; et al. Groundwater and climate change. Nat. Clim. Chang. 2012. [Google Scholar] [CrossRef] [Green Version]
  5. Haddeland, I.; Heinke, J.; Biemans, H.; Eisner, S.; Florke, M.; Hanasaki, N.; Konzmann, M.; Ludwig, F.; Masald, Y.; Schewe, J.; et al. Global water resources affected by human interventions and climate change. Proc. Natl. Acad. Sci. USA 2014, 111, 3251–3256. [Google Scholar] [CrossRef] [Green Version]
  6. Savenige, H.H.G.; Hoekstra, A.Y.; van der Zaag, P. Evolving water science in the Anthropocene. Hydrol. Earth Syst. Sci. 2014, 18, 319–332. [Google Scholar] [CrossRef] [Green Version]
  7. Walsh, C.L.; Blenkinsop, S.; Fowler, H.J.; Burton, A.; Dawson, R.J.; Glenis, V.; Manning, L.J.; Kilsby, C.G. Adaptation of water resource systems to an uncertain future. Hydrol. Earth Syst. Sci. Discuss. 2015, 12, 8853–8889. [Google Scholar] [CrossRef]
  8. Admundson, R.; Berhe, A.A.; Hopmans, J.W.; Olson, C.; Sztein, A.E.; Sparks, D.L. Soil and human security in the 21st century. Science 2005, 348. [Google Scholar] [CrossRef] [Green Version]
  9. Montgomery, D.R. Soil erosion and agricultural sustainability. Proc. Natl. Acad. Sci. USA 2007, 104, 13268–13272. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  10. Pimentel, D. Soil Erosion: A Food and Environmental Threat. Environ. Dev. Sustain. 2006, 8, 119–137. [Google Scholar] [CrossRef]
  11. Mahmood, R.; Pielke, R.A., Sr.; Hubbard, K.G.; Niyogi, D.; Dirmeyer, P.A.; McAlpine, C.; Carleton, A.M.; Hale, R.; Gameda, S.; Beltran-Przekurat, A.; et al. Land cover changes and their biogeophysical effects on climate. Int. J. Climatol. 2014, 34, 929–953. [Google Scholar] [CrossRef]
  12. Seto, K.C.; Gueralp, B.; Hutyra, L.R. Global forecasts of urban expansion to 2030 and direct impacts on biodiversity and carbon pools. Proc. Natl. Acad. Sci. USA 2012, 109, 16083–16088. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  13. Van den Bergh, J.C.J.M.; Grazi, F. Ecological footprint policy? Land use as an environmental indicator. J. Ind. Ecol. 2013. [Google Scholar] [CrossRef]
  14. Bellard, C.; Bertelsmeier, C.; Leadley, P.; Thuiller, W.; Courchamp, F. Impacts of climate change on the future of biodiversity. Ecol. Lett. 2012, 15, 365–377. [Google Scholar] [CrossRef] [Green Version]
  15. Pittelkow, C.M.; Liang, X.; Lingquist, B.A.; van Groenigen, K.J.; Lee, J.; Lundy, M.E.; van Gestel, N.; Six, J.; Venterea, R.T.; van Kessel, C. Productivity limits and potentials of the principles of conservation agriculture. Nature 2014, 517, 365–368. [Google Scholar] [CrossRef]
  16. Brouder, S.M.; Gomez-Macpherson, H. The impact of conservation agriculture on smallholder agricultural yields: A scoping review of the evidence. Agric. Ecosyst. Environ. 2014, 187, 11–32. [Google Scholar] [CrossRef]
  17. Wheeler, T.; von Braun, J. Climate Change Impacts on Global Food Security. Science 2013, 341, 508–513. [Google Scholar] [CrossRef]
  18. Shindell, D.; Kuylenstierna, J.C.I.; Vignati, E.; van Dingene, R.; Amann, M.; Klimont, Z.; Anenberg, S.C.; Muller, N.; Janssens-Maenhout, G.; Raes, F.; et al. Simultaneously mitigating near-term climate change and improving human health and food security. Science 2012, 335. [Google Scholar] [CrossRef] [Green Version]
  19. van Ittersum, M.K.; Cassman, K.G.; Grassini, P.; Wolf, J.; Tittonell, P.; Hochman, Z. Yield gap analysis with local to global relevance—A review. Field Crops Res. 2013, 143, 4–17. [Google Scholar] [CrossRef] [Green Version]
  20. Teixeira, E.I.; Fischer, G.; van Velthuizen, H.; Walter, C.; Ewert, F. Global hot-spots of heat stress on agricultural crops due to climate change. Agric. For. Meteorol. 2013, 170, 206–215. [Google Scholar] [CrossRef]
  21. Tomek, W.G.; Robinson, K.L. Agricultural Product Prices; Cornell University Press: Ithaca, NY, USA, 2003. [Google Scholar]
  22. Deaton, A. Commodity Prices and Growth in Africa. J. Econ. Perspect. 1999, 13, 23–40. [Google Scholar] [CrossRef] [Green Version]
  23. Kilman, S. Hog Maket Collapse on Glut of Animals; Big Blow to Farmers. Wall Street Journal, 27 November 1998; A2. Available online: http://www.wsj.com/articles/SB912119847536847000(accessed on 12 October 2020).
  24. Meadows, D.H.; Meadows, D.L.; Randers, J.; Behrens, W.W., III. The Limits to Growth: A Report for the Club of Rome’s Project on the Predicament of Mankind; Universe Books: New York, NY, USA, 1972; ISBN 0-87663-165-0. [Google Scholar]
  25. Beautement, P.; Broenner, C. Complexity Demystified: A Guide for Practitioners; Triarchy Press: Devon, UK, 2011. [Google Scholar]
  26. Turner, B.L.; Menendez, H.M.; Gates, R.; Tedeschi, L.O.; Atzori, A.S. System Dynamics Modeling for Agricultural and Natural Resource Management Issues: Review of Some Past Cases and Forecasting Future Roles. Resources 2016, 5, 40. [Google Scholar] [CrossRef] [Green Version]
  27. Park, J.; Seaton, R.A.F. Integrative research and sustainable agriculture. Agric. Syst. 1996, 50, 81–100. [Google Scholar] [CrossRef]
  28. Dunn, B.H.; Johnson, J.J. A systems approach to beef improvement. In Proceedings of the 2010 Beef Improvement Federation Research Symposium and Annual Meeting, Columbia, MO, USA, 28 June 28–1 July 2010. [Google Scholar]
  29. Brunstein, A.; Gonzales, C.; Kanter, S. Effects of domain experience in the stock-flow failure. Syst. Dyn. Rev. 2010. [Google Scholar] [CrossRef]
  30. Cronin, M.A.; Gonzales, C.; Sterman, J.D. Why don’t well-educated adults understand accumulation? A challenge to researchers, educators, and citizens. Organ. Behav. Hum. Decis. Process. 2009, 108, 116–130. [Google Scholar] [CrossRef]
  31. Sterman, J.D. Modeling Managerial Behavior: Misperceptions of Feedback in a Dynamic Decision Making Experiment. Manag. Sci. 1989, 35, 321–339. [Google Scholar] [CrossRef] [Green Version]
  32. Sterman, J.D. Misperceptions of feedback in dynamic decision making. Organ. Behav. Hum. Decis. Process. 1989, 43, 301–335. [Google Scholar] [CrossRef] [Green Version]
  33. Sterman, J.D. Learning in and about complex systems. Syst. Dyn. Rev. 1994, 10, 291–330. [Google Scholar] [CrossRef] [Green Version]
  34. Rouwette, E.A.J.A.; Größles, A.; Vennix, J.A.M. Exploring Influencing Factors on Rationality: A Literature Review of Dynamic Decision-Making Studies in System Dynamics. Syst. Res. Behav. Sci. 2004, 21, 351–370. [Google Scholar] [CrossRef]
  35. Aramburo, S.A.; Acevedo, J.A.C.; Morales, Y.O. Laboratory experiments in the system dynamics field. Syst. Dyn. Rev. 2012, 28, 94–106. [Google Scholar] [CrossRef]
  36. Mohaghegh, M.; Furlan, A. Systematic problem-solving and its antecedents: A synthesis of the literature. Manag. Res. Rev. 2020. [Google Scholar] [CrossRef]
  37. Diehl, E.W.; Sterman, J.D. Effects of feedback complexity on dynamic decision making. Organ. Behav. Hum. Decis. Process. 1995, 62, 198–215. [Google Scholar] [CrossRef] [Green Version]
  38. Diehl, E.W. A study on human control in stock-adjustment tasks. In Proceedings of the International Conference of the System Dynamics Society, Stuttgart, Germany, 10–14 July 1989. [Google Scholar]
  39. Paich, M.; Sterman, J.D. Boom, bust, and failures to learn in experimental markets. Manag. Sci. 1993, 39, 1439–1458. [Google Scholar] [CrossRef] [Green Version]
  40. Young, S.H.; Chen, C.P.; Wang, S.-W.; Chen, C.H. The landmine structure and the degrees of freedom of decision-making. In Proceedings of the 15th International System Dynamics Conference, Istanbul, Turkey, 19–22 August 1997. [Google Scholar]
  41. Langley, P.A.; Paich, M.; Sterman, J.D. Explaining capacity overshoot and price war: Misperceptions of feedback in competitive growth markets. In Proceedings of the 16th International Conference of the System Dynamics Society, Québec City, QC, Canada, 20–23 July 1998. [Google Scholar]
  42. Yang, F.; Huang, J.; Feng, X.; Yang, M.M. Decision-making in a dynamic task: Effects of goal orientation on stocks and flows performance. Chin. Manag. Stud. 2020, 14, 695–713. [Google Scholar] [CrossRef]
  43. Sengupta, K.; Abdel-Hamid, T.K. Alternative conceptions of feedback in dynamic decision making environments: An experimental investigation. Manag. Sci. 1993, 39, 411–428. [Google Scholar] [CrossRef]
  44. Schultz, F.; Dutta, P.; Johnson, P. Mental models and decision making in a dynamic health care environment. In Proceedings of the 18th International Conference of the System Dynamics Society, Bergen, Norway, 6–10 August 2000. [Google Scholar]
  45. Bakken, B.E. Frequency and experiential learning in unstable markets. In Proceedings of the 1992 International Conference of the System Dynamics Society, Utrecht, The Netherlands, 14–17 July 1992. [Google Scholar]
  46. Barlas, Y.; Özevin, M.G. Testing the decision rules used in stock market models. In Proceedings of the 19th International Conference of the System Dynamics Society, Atlanta, GA, USA, 23–27 July 2001. [Google Scholar]
  47. Gary, M.S.; Wood, R.E. Mental models, decision rules, and performance heterogeneity. Strateg. Manag. J. 2011, 32, 569–594. [Google Scholar] [CrossRef]
  48. Moxnes, E. Misperceptions of basic dynamics: The case of renewable resource management. Syst. Dyn. Rev. 2004, 20, 139–162. [Google Scholar] [CrossRef]
  49. Sterman, J.D.; Booth Sweeney, L. Cloudy skies: Assessing public understanding of global warming. Syst. Dyn. Rev. 2002, 18, 207–240. [Google Scholar] [CrossRef] [Green Version]
  50. Sterman, J.D.; Booth Sweeney, L. Understanding public complacency about climate change: Adults’ mental models of climate change violate conservation of matter. Clim. Chang. 2007, 80, 213–238. [Google Scholar] [CrossRef]
  51. Moxnes, E.; Saysel, A.K. Misperceptions of global climate change: Information policies. Clim. Chang. 2009, 93, 15–37. [Google Scholar] [CrossRef] [Green Version]
  52. Cronin, M.A.; Gonzales, C. Understanding the building blocks of system dynamics. Syst. Dyn. Rev. 2007, 23, 1–17. [Google Scholar] [CrossRef]
  53. Pala, O.; Vennix, J.A.M. Effect of system dynamics education on systems thinking inventory task performance. Syst. Dyn. Rev. 2005, 21, 147–172. [Google Scholar] [CrossRef]
  54. Sterman, J.D. Does formal system dynamics training improve people’s understanding of accumulation? Syst. Dyn. Rev. 2010, 26, 316–334. [Google Scholar] [CrossRef] [Green Version]
  55. Booth Sweeney, L.; Sterman, J.D. Bathtub dynamics: Initial results of a systems thinking inventory. Syst. Dyn. Rev. 2000, 16, 249–286. [Google Scholar] [CrossRef] [Green Version]
  56. Jensen, E.; Brehmer, B. Understanding and control of a simple dynamic system. Syst. Dyn. Rev. 2003, 19, 119–137. [Google Scholar] [CrossRef]
  57. Mohaghegh, M.; Größler, A. The dynamics of operational problem-solving: A dual-process approach. Syst. Pract. Action Res. 2020, 33, 27–54. [Google Scholar] [CrossRef]
  58. Barnabè, F.; Davidsen, P.I. Exploring the potentials of behavioral system dynamics: Insights from the field. J. Model. Manag. 2019, 15, 339–364. [Google Scholar] [CrossRef]
  59. Sterman, J.D. All models are wrong: Reflections on becoming a systems scientist. Syst. Dyn. Rev. 2002, 18, 501–531. [Google Scholar] [CrossRef]
  60. Rahmandad, H.; Repenning, N.; Sterman, J. Effects of feedback delay on learning. Syst. Dyn. Rev. 2009, 25, 309–338. [Google Scholar] [CrossRef]
  61. Größler, A. Structural transparency as an element of business simulators. In Proceedings of the 16th International Conference of the System Dynamics Society, Québec City, QC, Canada, 20–23 July 1998. [Google Scholar]
  62. Größler, A.; Maier, F.H.; Milling, P.M. Enhancing learning capabilities by providing transparency in business simulators. Simul. Gaming 2000, 31, 197–229. [Google Scholar] [CrossRef] [Green Version]
  63. Howie, E.; Sy, S.; Ford, L.; Vicente, K.J. Human-computer interface design can reduce misperceptions of feedback. Syst. Dyn. Rev. 2000, 16, 151–171. [Google Scholar] [CrossRef]
  64. Machuca, J.A.D.; Carrillo, M.A.D. Transparent-box business simulators versus black-box business simulators: An initial empirical comparative study. In Proceedings of the 1996 International System Dynamics Conference, Cambridge, MA, USA, 21–25 July 1996. [Google Scholar]
  65. Ford, D.N.; McCormack, D.E.M. Effects of time scale focus on system understanding in decision support systems. Simul. Gaming 2000, 31, 309–330. [Google Scholar] [CrossRef] [Green Version]
  66. O’Neill, K. Real-time tragedies: A simulated ‘commons learning laboratory’. In Proceedings of the International System Dynamics Conference, Utrecht, The Netherlands, 14–17 July 1992. [Google Scholar]
  67. Langley, P.A. Building cognitive feedback into a microworld learning environment: Results from a pilot experiment. In Proceedings of the International System Dynamics Conference, Tokyo, Japan, 30 July–4 August 1995. [Google Scholar]
  68. Croson, R.; Donohue, K.; Katok, E.; Sterman, J. Order Stability in Supply Chains: Coordination Risk and the Role of Coordination Stock. Prod. Oper. Manag. 2014, 23, 176–196. [Google Scholar] [CrossRef]
  69. Sterman, J.D.; Dogan, G. “I’m not hoarding, I’m just stocking up before the hoarders get here.” Behavioral causes of phantom ordering in supply chains. J. Oper. Manag. 2015, 39–40, 6–22. [Google Scholar] [CrossRef]
  70. Young, S.H.; Yang, J.; Wang, S.F. Enhancing the learning effects of dynamic decision game on systems thinking. In Proceedings of the International System Dynamics Conference, Utrecht, The Netherland, 14–17 July 1992. [Google Scholar]
  71. Georgantzas, N.C. Cognitive biases, modeling and performance: An experimental analysis. In Proceedings of the International System Dynamics Conference, Chestnut Hill, MA, USA, 10–13 July 1990. [Google Scholar]
  72. Ricardson, G.P.; Rohrbaugh, J. Decision making in dynamic environments: Exploring judgments in a system dynamics model-based game. In Contemporary Issues in Decision Making; Borcherding, K., Larichev, O.I., Messick, D.M., Eds.; North-Holland: Amsterdam, The Netherlands, 1990; pp. 463–472. [Google Scholar]
  73. Andersen, D.F.; Maxwell, T.A.; Richardson, G.P.; Stewart, T.R. Mental models and dynamic decision making in a simulation of welfare reform. In Proceedings of the International Conference of the System Dynamics Society, Stirling, Scotland, 11–15 July 1994. [Google Scholar]
  74. Ritchie-Dunham, J.L. Informing mental models for strategic decision making with ERPS and the balanced scorecard: A simulation-based experiment. In Proceedings of the 19th International Conference of the System Dynamics Society, Atlanta, GA, USA, 23–27 July 2001. [Google Scholar]
  75. Yang, J. Facilitating learning through goal setting in a learning laboratory. In Proceedings of the International System Dynamics Conference, Cambridge, MA, USA, 21–25 July 1996. [Google Scholar]
  76. Donovan, S.; Güss, C.D.; Naslund, D. Improving dynamic decision making through training and self-reflection. Judgm. Decis. Mak. 2015, 10, 284–295. [Google Scholar]
  77. Güss, C.D.; Dörner, D. Cultural differences in dynamic decision-making strategies in a non-linear, time-delayed task. Cogn. Syst. Res. 2011, 12, 365–376. [Google Scholar] [CrossRef]
  78. Qudrat-Ullah, H. Empirical Evidence on Dynamic Decision-Making and ILEs. In Better Decision Making in Complex, Dynamic Tasks; Springer International Publishing: Cham, Switzerland, 2015; pp. 17–44. [Google Scholar]
  79. Perera, H.N.; Fahimnia, B.; Tokar, T. Inventory and ordering decisions: A systematic review on research driven through behavioral experiments. Int. J. Oper. Prod. Manag. 2020. [Google Scholar] [CrossRef]
  80. Größler, A. System Dynamics and Operations Management. In System Dynamics: Theory and Applications; Springer: New York, NY, USA, 2020; pp. 273–284. [Google Scholar]
  81. Qudrat-Ullah, H. Decision Making and Learning in Dynamic Tasks. In Improving Human Performance in Dynamic Tasks; Springer International Publishing: Cham, Switzerland, 2020; pp. 1–8. [Google Scholar]
  82. Yuen, K.F.; Wang, X.; Ma, F.; Li, K.X. The Psychological Causes of Panic Buying Following a Health Crisis. Int. J. Environ. Res. Public Health 2020, 17, 3513. [Google Scholar] [CrossRef]
  83. Shi, X.; Li, F.; Chumnumpan, P. The use of product scarcity in marketing. Eur. J. Mark. 2020, 54, 380–418. [Google Scholar] [CrossRef]
  84. Yang, J. Give me the leverage, I will be a good dynamic decision maker. In Proceedings of the 15th International System Dynamics Conference, Istanbul, Turkey, 19–22 August 1997. [Google Scholar]
  85. Purcell, Z.A.; Wastell, C.A.; Sweller, N. Domain-specific experience and dual-process thinking. Think. Reason. 2020, 1–29. [Google Scholar] [CrossRef]
  86. Kiekens, W. The Effect of Multi-attribute Utility Theory on Small to Medium-sized Enterprises’ Decision-makers’ Informative Level and Subsequent Judgment Decision-making Process in Comparative Country Risk Assessment. Bachelor’s Thesis, Rotterdam Business School, Rotterdam, The Netherlands, 2018. [Google Scholar]
  87. Scott-Trees, T.; Doyle, J.K.; Radzicki, M. Using cognitive styles typology to explain differences in dynamic decision making in computer simulation game environment. In Proceedings of the 1996 International System Dynamics Conference, Cambridge, MA, USA, 21–25 July 1996. [Google Scholar]
  88. Park, H.J.; Kim, J.; Yi, K.S.; Jun, K. Enhancing the performance in dynamic decision making: The adaptive model reconstruction using feedforward vs feedback decision strategy. In Proceedings of the International System Dynamics Conference, Cambridge, MA, USA, 21–25 July 1996. [Google Scholar]
  89. Chi, M.T.H. Two approaches to the study of experts’ characteristics. In The Cambridge Handbook of Expertise and Expert Performance; Ericsson, K.A., Charness, N., Feltovich, P.J., Hoffman, R.R., Eds.; Cambridge University Press: New York, NY, USA, 2006; pp. 21–30. [Google Scholar]
  90. Kirkwood, C.W. “The Beer Game”. In Business Process Analysis Workshops: System Dynamics Models; 1998; Chapter 4; Available online: http://www.public.asu.edu/~kirkwood/sysdyn/SDWork/work-4.pdf (accessed on 14 September 2016).
  91. Levinthal, D.; March, J.G. A model of adaptive organizational search. J. Econ. Behav. Organ. 1981, 2, 307–333. [Google Scholar] [CrossRef]
  92. Herriott, S.R.; Levinthal, D.; March, J.G. Learning from experience in organizations. Am. Econ. Rev. 1985, 75, 298–302. [Google Scholar]
  93. Worthy, D.A.; Gorlick, M.A.; Pacheco, J.L.; Schnyer, D.M.; Maddox, W.T. With Age Comes Wisdom: Decision Making in Younger and Older Adults. Psychol. Sci. 2011. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  94. Worthy, D.A.; Maddox, W.T. Age-based differences in strategy use in choice tasks. Front. Neurosci. 2012, 5. [Google Scholar] [CrossRef] [Green Version]
  95. Worthy, D.A.; Davis, T.; Gorlick, M.A.; Cooper, J.A.; Bakkour, A.; Mumford, J.A.; Maddox, W.T. Neural correlates of state-based decision-making in younger and older adults. NeuroImage 2016, 130, 13–23. [Google Scholar] [CrossRef] [Green Version]
  96. Eppinger, B.; Schuck, N.W.; Nystrom, L.E.; Cohen, J.D. Reduced striatal responses to reward prediction errors in older compared with younger adults. J. Neurosci. 2013, 33, 9905–9912. [Google Scholar] [CrossRef] [Green Version]
  97. Samanez-Larkin, G.R.; Worthy, D.A.; Mata, R.; McClure, S.M.; Knutson, B. Adult age differences in frontostriatal representation of prediction error but not reward outcome. Cogn. Affect. Behav. Neurosci. 2014, 14, 672–682. [Google Scholar] [CrossRef] [Green Version]
  98. Blanco, N.J.; Love, B.C.; Ramscar, M.; Otto, A.R.; Smayda, K.; Maddox, W.T. Exploratory decision-making as a function of lifelong experience, not cognitive decline. J. Exp. Psychol. Gen. 2016, 145, 284. [Google Scholar] [CrossRef]
  99. Worthy, D.A.; Cooper, J.A.; Byrne, K.A.; Gorlick, M.A.; Maddox, W.T. State-based versus reward-based motivation in younger and older adults. Cogn. Affect. Behav. Neurosci. 2014, 14, 1208–1220. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  100. Worthy, D.A.; Otto, A.R.; Maddox, W.T. Working-memory load and temporal myopia in dynamic decision making. J. Exp. Psychol. Learn. Mem. Cogn. 2012, 38, 1640–1658. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  101. Kurnianingsih, Y.A.; Sim, S.K.; Chee, M.W.; Mullette-Gillman, O. Aging and loss decision making: Increased risk aversion and decreased use of maximizing information, with correlated rationality and value maximization. Front. Hum. Neurosci. 2015, 9, 280. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  102. Nassar, M.R.; Bruckner, R.; Gold, J.I.; Li, S.C.; Heekeren, H.R.; Eppinger, B. Age differences in learning emerge from an insufficient representation of uncertainty in older adults. Nat. Commun. 2016, 7, 11609. [Google Scholar] [CrossRef] [PubMed]
  103. Eppinger, B.; Walter, M.; Heekeren, H.R.; Li, S.C. Of goals and habits: Age-related and individual differences in goal-directed decision-making. Front. Neurosci. 2013, 7, 253. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  104. Nicholson, C.F.; Fiddaman, T. Dairy policy and price volatility. In Proceedings of the 21st International Conference of the System Dynamics Society, New York, NY, USA, 20–24 July 2003. [Google Scholar]
  105. Pagel, E.; Nicholson, C.F.; Newton, P. Structural Changes in U.S. Dairy Farming and the Role of Government Policy. In Proceedings of the 20th Intentional Conference of the System Dynamics Society, Palermo, Italy, 28 July–1 August 2002. [Google Scholar]
  106. Leach, T. Dairy Report: More Milk, More Dumping. Dairy Herd Management, 17 May 2018. Available online: https://www.dairyherd.com/article/dairy-report-more-milk-more-dumping(accessed on 2 October 2020).
  107. Shanker, D.; Mulvany, L. Dairy Farmers Drowning in Surplus Milk. Dairy Business, 25 October 2018. Available online: dairybusiness.com/dairy-farmers-drowning-in-surplus-milk/(accessed on 2 October 2020).
  108. Rembert, E.; Durisin, M.; Chandler, A. Dairy Farmers Dumping Milk Worldwide are on Brink of Crisis. The Detroit News, 3 July 2020. Available online: detroitnews.com/story/business/2020/07/03/dairy-farmers-milk-worldwide-brink-crisis/5372654002/(accessed on 2 October 2020).
  109. Senge, P. The Fifth Discipline: The Art and Practice of Learning Organizations; Doubleday/Currency: New York, NY, USA, 1990; ISBN 0-385-26094-6. [Google Scholar]
  110. Sterman, J.D. Business Dynamics: Systems Thinking and Modeling for a Complex World; McGraw-Hill: New York City, NY, USA, 2000; ISBN 0-07-231135-5. [Google Scholar]
  111. Mitchell, D.P.; Norton, T.W.; Grenfell, R.I.; Woodgate, P.W. Barriers to implementation of natural resource management at the landholding level in the intensive agricultural zones of Southern Australia. Australas. J. Environ. Manag. 2007, 14, 103–110. [Google Scholar] [CrossRef]
  112. Lachapelle, P.R.; McCool, S.F.; Patterson, M.E. Barriers to Effective Natural Resource Planning in a “Messy” World. Soc. Nat. Resour. 2003, 16, 473–490. [Google Scholar] [CrossRef]
  113. Tàbara, J.D.; Pahl-Wostl, C.P. Sustainability learning in Natural Resource Use and Management. Ecol. Soc. 2007, 12, 3. [Google Scholar] [CrossRef]
  114. Forrester, J.W. 1965/1975. A new corporate design. Ind. Manag. Rev. 1965, 7, 5–17. [Google Scholar]
  115. Forrester, J.W. Collected Papers of Jay W. Forrester; Pegasus Communications: Waltham, MA, USA, 1975. [Google Scholar]
  116. Forrester, J.W. Learning through System Dynamics as Preparation for the 21st Century. Keynote Address for Systems Thinking and Dynamic Modeling Conference for K-12 Education; Concord Academy: Concord, MA, USA, 1994. [Google Scholar]
  117. Forrester, J.W. System dynamics as an organizing framework for pre-college education. Syst. Dyn. Rev. 2006, 9, 183–194. [Google Scholar] [CrossRef]
  118. Forrester, J.W. System Dynamics: The Classroom Experience Quotations from K-12 Teachers. Creative Learning Exchange. 2009. Available online: http://static.clexchange.org/ftp/documents/whyk12sd/Y_2009-02QuotationsFromTeachers.pdf (accessed on 2 October 2020).
  119. Forrester, J.W. Fireside Chat. In Proceedings of the 19th International Conference of the System Dynamics Society, Atlanta, GA, USA, 23–27 July 2001. [Google Scholar]
  120. Stuntz, L.N.; Lyneis, D.A.; Richardson, G.P. The Future of System Dynamics and Learner-Centered Learning in K-12 Education. In Proceedings of the International Conference of the System Dynamics Society, Palermo, Italy, 28 July–1 August 2002. [Google Scholar]
  121. Lyneis, D.; Stuntz, L.N. System Dynamics in K-12 Education: Lessons Learned. In Proceedings of the International Conference of the System Dynamics Society, Boston, MA, USA, 29 July–2 August 2007. [Google Scholar]
Figure 1. Conceptual model of the steps taken to complete this study.
Figure 1. Conceptual model of the steps taken to complete this study.
Systems 08 00037 g001
Figure 2. Image of the Beer Game database. Tabs of recorded data included participant orders (tab displayed) as well as inventory. Additional tabs include graphs of average, best, and worst performances, used as visual aids in the debriefing of participants at KRIRM and SDSUHC.
Figure 2. Image of the Beer Game database. Tabs of recorded data included participant orders (tab displayed) as well as inventory. Additional tabs include graphs of average, best, and worst performances, used as visual aids in the debriefing of participants at KRIRM and SDSUHC.
Systems 08 00037 g002
Figure 3. Beer Game model developed following [31,90]. The structure is identical except that the algorithm used to compute placed orders in the Kirkwood [90] model formulation was replaced with observed placed orders of participants (indicated by the red italicized import variables) using the record sheets inside the Beer Game database. Equations to replicate the model are provided in the references cited above as well as Appendix A.
Figure 3. Beer Game model developed following [31,90]. The structure is identical except that the algorithm used to compute placed orders in the Kirkwood [90] model formulation was replaced with observed placed orders of participants (indicated by the red italicized import variables) using the record sheets inside the Beer Game database. Equations to replicate the model are provided in the references cited above as well as Appendix A.
Systems 08 00037 g003
Figure 4. Comparative plots of observed and model expected team costs and ranks before discarding outlier teams from the dataset (a,b) and after (c,d). The black lines display x = y.
Figure 4. Comparative plots of observed and model expected team costs and ranks before discarding outlier teams from the dataset (a,b) and after (c,d). The black lines display x = y.
Systems 08 00037 g004
Figure 5. Distribution of team total costs from best (1) to worst (44) teams across the database relative to benchmark costs, with ranked teams 10, 20, 30, and 40 shown to illustrate the change in costs with progressively worse performances.
Figure 5. Distribution of team total costs from best (1) to worst (44) teams across the database relative to benchmark costs, with ranked teams 10, 20, 30, and 40 shown to illustrate the change in costs with progressively worse performances.
Systems 08 00037 g005
Figure 6. Illustration of effective inventories for the best worst teams (R-retailer; W-wholesaler; D-distributor; F-factory) in the database.
Figure 6. Illustration of effective inventories for the best worst teams (R-retailer; W-wholesaler; D-distributor; F-factory) in the database.
Systems 08 00037 g006
Figure 7. Comparison of mean effective inventories of King Ranch Institute for Ranch Management (KRIRM) and South Dakota State University Honors College (SDSUHC) participants. Upper and lower bounds represent 95% confidence intervals for each week.
Figure 7. Comparison of mean effective inventories of King Ranch Institute for Ranch Management (KRIRM) and South Dakota State University Honors College (SDSUHC) participants. Upper and lower bounds represent 95% confidence intervals for each week.
Systems 08 00037 g007
Figure 8. Comparison of mean effective inventories of King Ranch Institute for Ranch Management (KRIRM) and South Dakota State University Honors College (SDSUHC) participants. Upper and lower bounds represent 95% confidence intervals for each week.
Figure 8. Comparison of mean effective inventories of King Ranch Institute for Ranch Management (KRIRM) and South Dakota State University Honors College (SDSUHC) participants. Upper and lower bounds represent 95% confidence intervals for each week.
Systems 08 00037 g008
Figure 9. Corn market dynamics illustrating U.S. corn acreage and the mean area before and after 2007 (top left panel), the trend in crop productivity (top right panel), and the average annual corn prices (bottom panel) [all data provided by USDA NASS; quickstats.nass.udsa.gov].
Figure 9. Corn market dynamics illustrating U.S. corn acreage and the mean area before and after 2007 (top left panel), the trend in crop productivity (top right panel), and the average annual corn prices (bottom panel) [all data provided by USDA NASS; quickstats.nass.udsa.gov].
Systems 08 00037 g009
Figure 10. Dairy market dynamics illustrating U.S. dairy cow inventory (top left panel), the trend in dairy cow productivity (top right panel), and the average monthly farm milk price (bottom panel) [all data provided by USDA NASS; quickstats.nass.udsa.gov].
Figure 10. Dairy market dynamics illustrating U.S. dairy cow inventory (top left panel), the trend in dairy cow productivity (top right panel), and the average monthly farm milk price (bottom panel) [all data provided by USDA NASS; quickstats.nass.udsa.gov].
Systems 08 00037 g010
Table 1. Summary of results from research regarding individual’s abilities of task performance in dynamic decision-making environments.
Table 1. Summary of results from research regarding individual’s abilities of task performance in dynamic decision-making environments.
Dynamic Decision-Making CharacteristicKey Findings (with Sources)
Misperceptions of feedback
  • Individuals are generally unable to account for delays and feedback effects because of highly simplified mental models and poor ability to infer correct behavior of simple feedback systems [37].
  • Strategies were insufficiently adjusted to account for strength of feedback between information and material flows [38].
  • Misperceptions of feedback have been attributed to poor recognition of delays and preference to maintain static decision rules [32,40,59,60].
  • Misperceptions of feedback not only result in suboptimal performance but often lead to strategy and decision making that is in opposite direction of optimal changes in decision making [31,32,39,41].
Types, transparency, and scale of models; types of feedback
  • Transparency of model (including interface design) and prior knowledge of structural information can improve task performance in dynamic decision-making contexts [61,62,63].
  • Given an increasing time scale, participants are not likely to consistently or steadily improve performance [64].
  • Types of feedback (e.g., numerical, graphical, or both) are known to influence behavior [32,42,65,66,67].
Environmental and Contextual Characteristics
  • Coordination risk contributes to bullwhip effects since behaviors of other individuals are not known with certainty [68].
  • Stressors such as larger orders, backlogs, or late deliveries trigger hoarding and phantom ordering even though such behaviors are known to be irrational [69].
Table 2. Comparison of observed (recorded) and expected (modeled) team costs and ranks before and after discarding outlier teams from the analysis.
Table 2. Comparison of observed (recorded) and expected (modeled) team costs and ranks before and after discarding outlier teams from the analysis.
Full Dataset (n = 64)Adjusted Dataset (n = 44)
Costs
r20.90 0.97
slope coefficient p-value1.84 × 10−321.21 × 10−32
ave. error ($)414.3338.4
ave. error ($/week)11.89.7
ave. error (cases/week)15.812.9
root mean squared error149.297.0
Ranks
r20.89 0.98
slope coefficient p-value4.1 × 10−312.66 × 10−35
exact ranks (% of n)6 (9%)11 (25%)
within 1 rank (% of n)13 (20%)13 (30%)
within 2 ranks (% of n)8 (13%)10 (23%)
within 3 ranks (% of n)2 (3%)5 (11%)
error >3 ranks (% of n)35 (55%)5 (11%)
Table 3. Comparison of participant and benchmark costs identified by Sterman [31] as the upper bound estimate of optimal performance.
Table 3. Comparison of participant and benchmark costs identified by Sterman [31] as the upper bound estimate of optimal performance.
Team TotalRetailerWholesalerDistributorFactory
Benchmark comparison
mean (n = 44)$4802$412$1197$1564$1629
benchmark$204$46$50$54$54
ratio (mean/benchmark)23.59.023.929.030.2
t-statistic7.975.855.667.537.75
H0: mean cost = benchmarkp < 0.00+p < 0.00+p < 0.00+p < 0.00+p < 0.00+
Sterman [31] comparison
Mean (n = 11)$2028$383$635$630$380
ratio2.41.11.92.54.3
t-statistic4.810.462.774.656.15
H0: mean cost = Sterman [31]p < 0.00+p = 0.64p = 0.008p < 0.00+p < 0.00+
Quartiles
4th (top 25% of teams)$1410$174$273$421$542
3rd (50–75%)$2637$210$560$824$1043
2nd (25–50%)$5017$385$1143$1688$1801
1st (bottom 25% of teams)$10,143$879$2812$3323$3129
Table 4. Summary of experimental results from the database (n = 44). * Closed-loop gain is measured as change in output relative to that of the input.
Table 4. Summary of experimental results from the database (n = 44). * Closed-loop gain is measured as change in output relative to that of the input.
CustomerRetailerWholesalerDistributorFactory
Periodicity
Time to recover initial inventory-24272625
Date of minimum inventory-17212020
Date of maximum inventory-33353535
Amplification
Peak order rate (cases/week)816.923.839.059.5
Variance of order rate (cases/week)1.614.442.1138.1293.7
Peak inventory (cases)-69.8473.45152.80144.27
Minimum inventory (cases)-−4.02−56.05−64.27−56.00
Range (cases)-73.9129.5217.1200.3
Closed-loop gain * 3.234.958.7413.89
Phase lag
Date of peak order rate (week)518201819
Anticipated minimum inventory −1121
Table 5. Evaluation of results from the top 10 performing teams in the database used for comparison to teams reported in the literature. * Closed-loop gain is measured as change in output relative to that of the input, e.g., ∆Factory orders/∆Customer orders = (21.12 − 4)/(8 − 4) = 4.28.
Table 5. Evaluation of results from the top 10 performing teams in the database used for comparison to teams reported in the literature. * Closed-loop gain is measured as change in output relative to that of the input, e.g., ∆Factory orders/∆Customer orders = (21.12 − 4)/(8 − 4) = 4.28.
Team TotalRetailerWholesalerDistributorFactory
Top 10 performing teams (n = 10)$1278$172$228$335$543
Sterman [31] (n = 11)$2028$383$635$630$380
t-statistic−8.45−22.69−20.81−6.672.59
H0: mean cost = Sterman [31]p < 0.00+p < 0.00+p < 0.00+p < 0.00+p = 0.02
Periodicity
Time to recover initial inventory-22222222
Date of minimum inventory-12161614
Date of maximum inventory-24273027
Amplification
Peak order rate (cases/week)-10.612.115.921.1
Variance of order rate (cases/week)-2.84.713.327.9
Peak inventory (cases)-14.224.237.141.1
Minimum inventory (cases)-−2.2−3.4−9.7−12.7
Range (cases)-16.427.646.853.8
Closed-loop gain * 1.652.032.984.28
Phase lag
Date of peak order rate (week)-17131617
Anticipated minimum inventory −530−3
Table 6. Comparison of King Ranch Institute for Ranch Management (KRIRM) and South Dakota State University Honors College (SDSUHC) class performances.
Table 6. Comparison of King Ranch Institute for Ranch Management (KRIRM) and South Dakota State University Honors College (SDSUHC) class performances.
Team TotalRetailerWholesalerDistributorFactory
KRIRM (n = 38)$4988$400$1227$1644$1717
SDSUHC (n = 6)$3623$488$1008$1057$1070
t-statistic0.809−0.4750.3671.0051.095
H0: KRIRM = SDSUHC0.4230.6370.7150.3210.280
KRIRM
Inventory recovery time-24272625
Week of minimum inventory-17211920
Week of maximum inventory-35353535
Peak order rate (cases/week)-17.324.441.259.5
Mean order rate (cases/week)-9.511.916.921.2
Mean order rate post-inventory recovery-7.1 a7.2 b9.2 a13.8 b
Variance of order rate-13.842.2147.0304.3
Peak inventory (cases)-67.876.4163.4148.5
Minimum inventory (cases)-−4.4−54.7−64.8−54.6
Week of peak order rate-18211819
SDSUHC
Inventory recovery time-23272625
Week of minimum inventory-13202220
Week of maximum inventory-29353032
Peak order rate (cases/week)-19.325.244.360
Mean order rate (cases/week)-9.610.713.215.9
Mean order rate post-inventory recovery-6.0 a5.5 b5.0 a5.2 b
Variance of order rate-20.351.2123.6276.6
Peak inventory (cases)-95.354.598.3143.5
Minimum inventory (cases)-−2.7−65.5−69.5−64.7
Week of peak order rate-17201719
ap-value < 0.10, bp-value < 0.05.

Share and Cite

MDPI and ACS Style

Turner, B.L.; Goodman, M.; Machen, R.; Mathis, C.; Rhoades, R.; Dunn, B. Results of Beer Game Trials Played by Natural Resource Managers Versus Students: Does Age Influence Ordering Decisions? Systems 2020, 8, 37. https://0-doi-org.brum.beds.ac.uk/10.3390/systems8040037

AMA Style

Turner BL, Goodman M, Machen R, Mathis C, Rhoades R, Dunn B. Results of Beer Game Trials Played by Natural Resource Managers Versus Students: Does Age Influence Ordering Decisions? Systems. 2020; 8(4):37. https://0-doi-org.brum.beds.ac.uk/10.3390/systems8040037

Chicago/Turabian Style

Turner, Benjamin L., Michael Goodman, Rick Machen, Clay Mathis, Ryan Rhoades, and Barry Dunn. 2020. "Results of Beer Game Trials Played by Natural Resource Managers Versus Students: Does Age Influence Ordering Decisions?" Systems 8, no. 4: 37. https://0-doi-org.brum.beds.ac.uk/10.3390/systems8040037

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop