Next Article in Journal
Education for Sustainable Development: Emerging Themes from Adopters of a Declaration
Next Article in Special Issue
Learning to Navigate (in) the Anthropocene
Previous Article in Journal
Cooperation of Voltage Controlled Active Power Filter with Grid-Connected DGs in Microgrid
Previous Article in Special Issue
Do Pre-service Teachers Dance with Wolves? Subject-Specific Teacher Professional Development in A Recent Biodiversity Conservation Issue
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Changes in Pre-Service Teachers’ Values, Sense of Agency, Motivation and Consumption Practices: A Case Study of an Education for Sustainability Course

1
Mary Lou Fulton Teachers College, Arizona State University, Tempe, AZ 85306, USA
2
School for the Future of Innovation in Society, Arizona State University, PO Box 875603, Tempe, AZ 85287, USA
3
Biodesign Institute’s Sustainability Science Education Project, Arizona State University, Tempe, AZ 85287, USA
*
Author to whom correspondence should be addressed.
Sustainability 2019, 11(1), 155; https://0-doi-org.brum.beds.ac.uk/10.3390/su11010155
Submission received: 25 November 2018 / Revised: 25 December 2018 / Accepted: 26 December 2018 / Published: 29 December 2018
(This article belongs to the Collection Teacher Professional Development in ESD)

Abstract

:
Teacher education has a critical role to play as people around the world strive to reach the Sustainable Development goals. Education for sustainability (EfS) aims to motivate and prepare educators to create a more sustainable future through education. The purpose of this case study was to explore pre-service teachers’ changes in their values, sense of agency, consumption practices and motivation after participation in a required EfS course. Students were enrolled in a hybrid course that conveyed content through digital stories followed by reflections, in-class discussions and activities. Ninety-one undergraduate students completed pre- and post-course surveys. Students reported significant changes in their beliefs about the relevance of sustainability education, attitudes toward sustainable development, self-efficacy, locus of control and sustainable consumption practices. Qualitative analysis of an open-ended question served to triangulate quantitative findings. Results support the need and potential for EfS courses for educators, particularly in the United States where such courses are not typically required or even offered at most universities.

1. Introduction

Every person on the planet must work together to ensure a healthy and habitable biosphere for future generations—not just for humans, but for all species. Education provides a critical link between scientific knowledge and societal change [1,2,3,4,5]. Specifically, education for sustainability (EfS) aims to motivate and empower learners with the knowledge, skills and values needed to create a more sustainable future [3,5]. Education, and teacher education in particular, has a critical role to play in leading society toward a more sustainable way of life [2,3,6,7,8]. The Sustainable Development Goals outline a broad agenda for change, including specific goals for education [9]. Educators at all levels serve as the primary change agents in society, and they must be prepared and motivated to lead systemic shifts that ripple out into the world. This work begins by building an increased awareness and knowledge of how daily actions and purchasing decisions have broader systemic impacts. However, most educators are not prepared when it comes to sustainability, and typically are not required to take course in EfS as part of their teacher education program [10,11,12,13,14,15,16,17,18,19].

1.1. Education for Sustainability

The field of environmental education (EE) provided a rich foundation for EfS, emphasizing a deep understanding of and connection to the natural world, and bringing awareness to the impacts of humans on natural systems as well as concerns for environmental justice [20]. Prior work in the field of teacher education has focused on fostering environmental literacy, or the knowledge, skills, and dispositions that lead toward environmentally responsible behaviors [14,19,21,22]. EfS is an emerging field that focuses primarily on ensuring that future generations have the resources they need to thrive, with a strong emphasis on critical thinking [3,6,8,23].
To prepare teacher educators to meet the goals of EfS, institutions of higher education must include content specific to EfS and embrace pedagogies that help foster values and critical thinking skills needed to motivate action [17,24,25,26,27]. While EfS is becoming more prominent in teacher education programs around the world [5], it is relatively uncommon in programs across the United States [27]. Some programs may address education about sustainability, but it is important to note the distinction. Education about sustainability is focused on providing information about topics such as the causes and impacts of climate change, energy sources and their limitations, and biodiversity loss [28]. This knowledge is a first step in fostering sustainability since it allows people to make more informed decisions. Education for sustainability (EfS) builds on this knowledge and also promote values, thinking skills, and dispositions that may lead to engagement in building a more just and sustainable future [29,30,31]. An overarching goal of EfS is to develop a sustainability worldview, “a way of seeing and interacting with the world through the lens of sustainability” [31] (p. 58). According to Nolet, a sustainability worldview encompasses knowledge (an understanding of big ideas such as respect for limits and equity and justice), values (beliefs about what is important and meaningful), dispositions (tendency to act or think in certain ways), and agency (a feeling that one can enact positive change through choices and actions) [31]. The current study explores changes in values, sense of agency, consumption practices, and motivation among pre-service teachers over the course of a semester. Students were enrolled in a required EfS course where new ways of thinking are introduced, to foster critical thinking.

The Role of Motivation in EfS

A sustainability worldview is not a destination, but a journey. Affective domains of values, agency, and motivation are critical elements of education for sustainability to study and understand better, since knowledge alone does not often translate to actions [13,14,22,32]. Individuals have fundamental needs for sense of agency, feelings of competence or capability, and relatedness (connection to others). Learning environments can be designed that fulfill these needs [33]. When these needs are met, people are intrinsically motivated to learn and act [34,35]. Intrinsic motivation is “the inherent tendency to seek out novelty and challenges, to extend and exercise one’s capacities, to explore, and to learn” [35] (p. 70). Applied to the field of teacher education, cognitive evaluation theory suggests that teachers who feel competent and have a sense of agency will have authentic or intrinsic motivation and be more likely to persist and succeed in EfS in their classrooms [33]. Thus, fostering intrinsic motivation is critical. We extend prior work that highlights motivation and agency as essential goals of EfS by categorizing pre-service teachers’ reasons for motivation from their open-ended responses [22,32,36].

1.2. Impacts of EfS and EE in Teacher Education

Each course in EfS or EE is unique in its topics, pedagogical approaches, and outcomes. Teacher education researchers have often used surveys to assess student changes over the course of a semester or program [10,22,32]. For example, a group of researchers found that pre-service teachers in Australia who had participated in a 13-week EE unit reported feeling more confident about teaching EE, had more pedagogical content knowledge, and were motivated to teach about the environment [22]. Similarly, a group of early childhood and primary education majors improved in their EfS self-efficacy, interest, and familiarity with sustainability issues, and recognized EfS to be relevant to their teaching and learning after course participation [32]. However, attitudes toward sustainability issues did not change. Another study of pre-service teachers who took a required course in EfS found that students conceptualized sustainability with more complexity and gained skills in systems thinking over the course of the semester [37]. Thus, prior work has shown that knowledge, awareness, feelings of relevance about sustainability, and teaching efficacy are possible to change through course work. However, values toward sustainability or individual behaviors may be more difficult to change [22,32].

1.3. A Framework for Teachers

EfS courses should aim to help students develop sustainability competencies that empower and motivate them to be active and engaged citizens who examine society with a critical eye, evaluating existing institutions, policies and systems through a lens of sustainability [26,38]. The Sustainability Education Framework for Teachers provides a framework for analyzing and considering sustainability topics, problems, and solutions through a networked approach that includes four ways of thinking: futures, values, systems and strategic. The framework targets key EfS competencies and was designed to help educators and students understand the complexity of societal problems, engage with issues, and imagine and create solutions [23,39]. Each component of the framework is described below.
Futures thinking allows for anticipatory approaches and adaptively preparing for future changes, problems, and solutions [23,40]. Futures thinking integrates the ability to think imaginatively about the future and calls for review of how past decisions led to current conditions [41]. Values thinking focuses on recognizing the role of personal values in individual and collective decision making and encompasses concepts of justice, equity, social–ecological integrity, and ethics. Values thinking includes understanding how these concepts vary across and within cultures, and how integrating these concepts contributes to solving sustainability problems more holistically [42,43]. Systems thinking represents a configuration of parts connected and joined together by a web of relationships, flows, and networks, some of which might not be readily transparent or set on normalized time or space scales. Systems thinking is non-linear, and cause and effect networks might not be linked or connected with simple step-by-step chains [44]. Finally, strategic thinking considers possible solutions under a given set of assumptions, articulating potential alternative solutions, and challenging existing plans, assumptions, and alternatives. New solutions might be appropriate, and should be explored after reviewing the big picture (e.g., overall themes, trends, goals) in light of specific, local challenges [45]. These four ways of thinking can be used to address EfS in the classroom and can be implemented in conjunction with one another, sequenced, or used individually. Together, they comprise the Sustainability Education Framework for Teachers which guided course development.

1.4. Research Aim and Questions

A team of scientists and educators at Arizona State University (ASU) developed a required course for preservice educators with the goals and best practices for EfS in mind. In this case study, future teachers who were students in a semester-long, hybrid course were surveyed using a pre-post survey design. The following research questions guided our work:
(1)
To what extent did students’ values, motivation, sense of agency, and/or consumption practices shift over the course of the semester?
(2)
In what ways did the course motivate students to include EfS in their teaching?

2. Methods

This descriptive case study [46] reports results from a paired pre-post survey of pre-service teachers who were enrolled in a course called Sustainability Science for Teachers.
The survey was administered before the start of the semester in January and in May after course completion. Responses to an open-ended question were used to extend and explain results from quantitative analyses.

2.1. Participants

Participants were students who were pre-service elementary teachers in their third year of a four-year undergraduate degree program at ASU. They enrolled in one of four sections of the required fifteen-week course. Out of the 110 enrolled in the course, 105 consented to participate in the study. Results from 91 students (87%), who completed both surveys and consented to participate in our research study, are described below. Seventy-six students responded to the open-ended question. Students were 88% female, 12% male, 53% white, 30% Hispanic or Latino/a, 6% Asian, 3% Native American, 1% Native Hawaiian or Pacific Islander, 1% Arab, 3% mixed race, and 2% African American. Three instructors (two males and one female) taught the course, with the lead author teaching two of the four sections.

2.2. Context

The three credit, fifteen-week course, Sustainability Science for Teachers, was designed by an interdisciplinary team of scientists, educators, and design professionals. It was geared toward preparing elementary teachers to address sustainability endeavors with their future students leading to clear solutions linked to improving human health and well-being while reducing human exploitation of natural resources. The goal of the course was to develop sustainability literacy among preservice teachers by providing much needed EfS content knowledge as well as related pedagogical content knowledge [47]. The course used a flipped learning approach; the content was shared in the online portion through the use of digital stories that integrated vivid images and sounds along with rich narrative and compelling case studies [48]. Students watched videos related to the weekly topics, and then came to class for 75 min each week to debrief and learn pedagogical strategies to integrate the content into their future teaching. All instructors used the same the online content, were provided with weekly lesson plans, and met monthly to discuss pedagogical strategies.
The following topics were addressed during the semester: population, poverty, food, water, fossil fuels, new energy, ecosystem services, production, disposal, and governance. Through exploration of the topics, pre-service teachers learned about sustainability concepts, developed EfS competencies, and engaged with various pedagogical approaches to foster their ability to effectively teach EfS in the K-8 settings. Pre-service teachers’ roles both as citizens who can advocate for needed changes in society as well as their future role as sustainability educators were highlighted. Lessons varied each week, and included: a needs and wants cart sorting task; a philosophical chair debate on ecosystem services, a renew-a-bead activity where students graphed and reflected on diminishing nonrenewable resources over time; a systems thinking activity where students mapped out all of the inputs, outputs and components of the food system needed to produce a hotdog; an activity related to water systems where students could select different modes (visual, kinesthetic, music) to represent their knowledge from videos; an wind turbine engineering design challenge; small group discussion and lesson design activities. The final project for the course consisted of a student-designed digital artifact that represents a five-day unit on a sustainability topic of their choice. As previously described, four ways of thinking—strategic, futures, values and systems —provided an overarching framework for engaging with the course content. These critical thinking skills helped students think deeply about the content from a variety of perspectives, enabling them to imagine various scenarios for the future based on decisions made in the past and present, and to analyze systems to strategize the best levers of change in society.

2.3. Measures

A survey was developed to assess student changes in values, sense of agency, motivation and consumption practices. Our survey combined constructs from existing measures in EE or EfS in order to fully understand student changes (or absence of change) over the course of a semester. Values for sustainability was assessed using two subscales: perceived relevance of sustainability education (6 questions) and feelings of responsibility for sustainability development (7 questions). Sense of agency was assessed using three subscales: 5 questions about teaching efficacy, 5 questions about locus of control, and 1 question about changes needed in society. Sustainable consumption practices were assessed with 7 questions, and motivation was assessed with 2 questions—1 open-ended and one Likert-scale item. Each of the subscales and related items are described below.

2.3.1. Perceived Relevance of Sustainability Education

The survey included 6 items from a measure developed by Tomas, Girgenti, and Jackson [32]. Likert scale response options ranged from strongly disagree (1) to strongly agree (4). For example, students were asked to rate their level of agreement with statements such as: “It is important to include education for sustainability in my future classroom practice.”

2.3.2. Responsibility for Sustainable Development

An internationally validated scale from the Programme for International Student Assessment (PISA) science survey [49] was used to assess students’ responsibility for sustainable development. PISA survey items have been internationally validated and used in prior studies of preservice teachers. Students responded to seven questions on a 4-point Likert scale from strongly disagree to strongly agree. Items were focused on students’ beliefs about whether laws and policies should be changed to favor sustainable development. One statement asked students to evaluate the extent to which “Electricity should be produced from renewable sources as much as possible, even if this increases costs.”

2.3.3. Education for Sustainability Self-Efficacy

Five questions were selected from a measure developed by Tomas et al. [32] and were rated on a 4-point scale as follows: (1) I couldn’t do this, (2) I would struggle to do this on my own, (3) I could do this with a bit of effort, (4) I could do this easily. The questions assessed students’ perceptions of their ability to understand, teach about, and take action on environmental issues. For example, students were asked if they could identify the science that underlies a newspaper report on an environmental issue or bring about an improvement in the environment, if only in a small way, through their skills and knowledge.

2.3.4. Locus of Control

Students responded to five items from a study by Yavetz et al. [19] that assessed students’ internal locus of control, or a person’s belief in his/her ability to bring about environmental change through individual actions [50,51]. The responses used a 5-point Likert scale from strongly disagree to strongly agree. One item in this category was “I believe I can contribute to a more sustainable world through my personal behavior.” In addition, there were three reverse-coded items such as, “There’s no use trying to influence my family or friends on sustainability issues.”

2.3.5. Changes Needed in Society

Pre-service teachers responded to one question that was modified from a survey instrument created by Kennelly, Taylor, and Maxwell [22] concerning future vision for society. They were asked, “Which of the following do you believe will be necessary to achieve significant improvements in society? Please check all that you think are applicable.” Response options included government regulation, personal lifestyle changes, communities working together, and a radical restructuring of society.

2.3.6. Sustainable Consumption Practices

Seven questions were focused on individual consumption practices related to sustainability. These questions came from an unpublished sustainability literacy administered at ASU. Pre-service teachers noted the frequency with which they engaged in specific behaviors, such as recycling, choosing food based on its impacts, and conserving water, using a 4-point Likert scale (never, rarely, usually, always).

2.3.7. Motivation

We added 2 questions about motivation to the post survey after reflecting on the course and pre-survey as well as listening to students discuss their motivation throughout the semester. Questions that were asked included: “Rate your desire to include sustainability education in your future classroom” (very strong, strong, medium, weak, very weak) [22] and “In what way(s), if any, has learning about sustainability education altered your motivation to include it in your teaching?”

2.4. Data Analyses

Survey results were downloaded from the online survey tool into an SPSS file. Negatively worded items were reverse coded, and mean scores were computed for each scale. Cronbach’s alpha was computed to assess internal consistency for each scale. Internal consistency for all scales at both time points was adequate, ranging from 0.73 to 0.95. These results are reported below in Table 1. Dependent sample t tests were conducted to determine differences from pre- to post-survey for all Likert scale items. Cohen’s d was calculated for each difference, to better understand the magnitude of the effects [52]. For the results in Table 2, Cohen’s metrics for size of effects are reported to indicate whether an effect is small (0.2), medium (0.5) or large (0.8) [53].
Thematic analyses were used to characterize responses to the open-ended survey question. In the first phase of analyses, all authors read and open coded student responses, then met to discuss and define codes. Next, the first and second authors independently coded all data using these original codes, while also continuing to explore new themes that were identified inductively from the data [54]. Themes were condensed as needed. Next, a final codebook with definitions was created within Dedoose, an online data analysis software. and systematic deductive coding was then completed. Each individual student comment was treated as a unit of analysis, and several comments were coded with multiple themes. A final verification of the assigned codes occurred when the authors met to review each set of excerpts by code. Peer debriefing occurred throughout our analytic process as we reviewed data and summaries from Dedoose, discussed and refined code definitions, and selected the excerpts and categories that best characterized students’ responses. Responses are summarized below.

3. Results

Students made positive, significant gains in their sustainability-oriented values, agency, consumption practices, and motivation (Table 2). Responses from the open-ended question provided further insights about students’ values for sustainability and sustainability education, beliefs about the importance of sustainability and EfS, sense of responsibility, pedagogical skills, sense of agency, and increased knowledge and awareness. While the open-ended question asked students about whether learning about sustainability education motivated them to include EfS in their future teaching, their responses touched on a range of factors. For this reason, the open-ended question comments are reported by theme along with the quantitative survey results with which they most closely aligned. Table 3 displays summarizes qualitative comments by theme and frequency.

3.1. Values for Sustainability and Sustainability Education

Students indicated in a variety of ways that felt sustainability topics are important to learn and teach about. They made significant changes in the extent to which they perceived sustainability education as relevant in their classrooms, t(91) = 4.20. The changes showed a medium-sized effect, d = 0.55. Twenty participants talked about how important sustainability topics are in society and in their future classrooms. A typical response was: “The course has just showed me how important this topic is right now and how much we need it.” Ten stressed the importance of EfS for young children. For example, one student said, “I learned that sustainability is so important to teach to young students because they still have time to make changes and start early on it.” Three noted that EfS was surprisingly absent in their past schooling, for example: “I had never thought about integrating sustainability topics into my future classroom (just because nobody really does it and the idea has never been mentioned to me before) and now I cannot imagine my future classroom with the incorporation of sustainability topics.”

3.2. Sense of Responsibility

Students also made significant gains in their levels of responsibility for sustainable development t(90) = 2.77. Reported changes showed medium effects, Cohen’s d = 0.40. Responses to the open-ended question indicated that students were motivated to take action to help improve society and/or the planet. Four students indicated the importance of individual responsibility. For example, one student noted, “It has motivated me to become more involved as a citizen.” Nine students mentioned that they felt a desire to educate for sustainability. A typical response was, “Yes, I want my students to be educated citizens and do their part in saving the world.” Four students communicated a sense of urgency to educate for sustainability. One said, “It is extremely important to be teaching students these issues that are happening every day.”

3.3. Sense of Agency

Significant, positive changes in students EfS self-efficacy were reported, t(91) = 10.90. The magnitude of the change was large, Cohen’s d = 1.47. At the beginning of the semester, most students did not feel confident in their ability to do things such as interpret scientific information on a website about climate change or educate others about sustainability issues. By the end of the semester, students were much more confident in their abilities.

3.4. Pedagogical Skills

Twenty-one students described different ways that they felt prepared to teach about sustainability. A student noted, “It has increased my motivation because I have been given time to think about how I could incorporate this into my classroom and resources that I can use to help me more effectively teach this in the classroom.” Twelve mentioned integrating sustainability education across the curriculum. For example, “It made me realize that I can incorporate sustainability in any subject.”

3.5. Locus of Control

Students reported significantly higher locus of control for enacting change after course participation, t(91) = 4.02, Cohen’s d = 0.43. Students felt that their actions could make a difference more after taking the class than before the class.
Twelve students elaborated on their sense of agency that they can enact positive changes through choices and actions [31]. Two described how they could make a difference through individual actions. One student stated, “I’ve learned that my personal choices make a huge difference, whether through buying power or protesting.” Ten described ways they could make positive changes through their teaching. A student expressed a strong sense of agency about the impact she can have through education stating, “If we educate them on how much their habits can impact the environment from a young age, then the positive impact it could have is limitless.” Another student said, “A simple discussion can make all the difference.”
One of the strongest transformations described by a student captured several themes from our analyses, and showed a strong belief that she can enact change through her teaching:
It has completely changed my outlook on sustainability education. I can’t remember a single instance in elementary school where I was taught based on sustainability. I personally thought that it was a waste of time to even bring this sort of educational topic in the classroom but I have personal experience of how proper education in this subject matter can change outlooks and preconceived notions. I can influence the future generations to not only care for the planet better than their parents but also become problem solvers for the very scary issues facing the environment today.
Students were asked to indicate which actions they thought would be needed to achieve significant improvements in society. They were able to select one or more choices for actions (Table 4). Significant changes were not found in the overall number of responses that students selected, t(91) = 0. However, the number of students who chose each response slightly differed from pre to post survey. Over 90% of students selected personal lifestyle changes and communities working together in both the pre-and post-survey. Four more students selected personal lifestyle changes in the post-survey, while three fewer students selected communities working together in the post-survey. Seventy-seven percent of students selected government regulation in the pre- and post-survey (77%), and 40% selected a radical restructuring of society in the pre-survey, and one less in the post-survey (38%).

3.6. Sustainable Consumption Practices

Students reported a higher use of sustainable consumption practices at the end of the course than at the beginning t(87) = 3.96. d = 0.41 (Table 2). Changes in individual practices were analyzed to explore results further. Significant changes occurred in the following practices: choosing food conscientiously based on its impacts, t(85) = 4.31, conserving water, t(87) = 4.32, minimizing purchases that are not environmentally friendly, t(86) = 3.30 and conserving energy, t(86) = 2.29. Statistically significant changes were not reported for the use of alternative transportation, recycling, and minimization of waste generated. Recycling was the most prevalent practice reported at the beginning of the semester, suggesting less room for growth in this area.

3.7. Motivation

After the course, students were asked about their desire to include sustainability education in their future classrooms. Of the 90 students who responded, 51% of students (n = 46) reported a very strong desire, 32% reported a strong desire (n = 29), 15% reported a medium desire (n = 14), and 1% (n = 1) reported a weak desire. Most of the student responses to the open-ended question, summarized above and in Table 2, highlighted the variety of reasons why students were motivated to include sustainability education in their future classrooms. However, qualitative responses from two students describe a lack of motivation for sustainability education. One student said, “I know that one day this world is going to end because of what the bible says. If God created this world, then man shouldn’t be worried because God controls everything, we don’t.”

3.8. Knowledge and Awareness

Students’ levels of knowledge or awareness about sustainability concepts were not directly assessed in the survey. However, 12 students mentioned they were more knowledgeable and aware about sustainability concepts. Students made statements such as, “Learning about sustainability has given me more knowledge about the subject which has motivated me to want to include it in my teaching.” A different student said, “I feel like I know enough to teach sustainability in every subject taught in my class. Before this class, I was not comfortable because I did not have the knowledge, but now that I do I really want to teach my students what I now know.” Some students focused more on increased awareness than knowledge, for example: “I became more aware of other issues that were also important to teach to students.”

4. Discussion

David Orr [55] has emphasized that “the crisis we face is first and foremost one of mind, perception and values” (p. 27). In this statement, Orr reminds us that the changes needed in society today require more than just knowledge; they necessitate different ways of thinking and being in the world. Pre-service teachers reported significant changes in their perceptions, values and practices over the course of the semester, with effect sizes ranging from 0.40 to 1.47. This hybrid course, which includes weekly online digital stories with reflections, in-class discussions and activities, is a useful format for others to consider as they design EfS courses. This case study contributes to the body of work focused on preparation of teachers as sustainability change agents in society. In the section below, we highlight how these changes were similar to or different from related studies of EfS for pre-service teachers.

4.1. Value for Sustainability

Most participants in the current study felt that EfS was relevant and important for their students by the end of the semester. In another study, pre-service teachers in Australia typically started out high in the perceived relevance of EfS, and shifted in two of six areas [32]. However, students in that course did not change their attitudes towards sustainability issues. Effect sizes were not reported in that study for comparison. Goldman et al. found that Israeli students from teacher education colleges who majored in environmental fields reported positive changes in environmental knowledge and behaviors, but did not show positive changes in their values for nature or locus of control over the course of their undergraduate studies [56]. The authors suggested that students’ knowledge may be incomplete, and they may have lacked an understanding of “the importance of biodiversity and ecological integrity to human welfare” (p. 378). Together, prior studies suggest that attitudes about responsibility for sustainable development may require a depth of understanding that may be difficulty to attain without intensive and deliberate intervention. Future studies should report effect sizes for comparison between different contexts.

4.2. Agency

At the end of the course, students felt capable as sustainability educators, and empowered their teaching would make a difference in society. According to Didham and Ofei-Manu [57], “Teachers’ agency is a dynamic catalyst in effecting social change through schooling” (p. 100). Agency is a key driver of motivation [35] and a critical ingredient for educators in the current educational landscape [32]. The effect size for EfS efficacy was particularly large (d = 1.47). Students’ qualitative comments provided insight into their increased confidence. One student articulated that she was given time to think about how to incorporate sustainability into their teaching and resources for how to do this. Many mentioned that they felt capable of teaching sustainability across the curriculum. Throughout the semester, we included assignments that had students apply knowledge of course concepts to classroom settings. For example, they were asked to describe a short lesson that might incorporate a way of thinking or concept into a lesson in a specific grade level or subject. Students were also asked to create a digital artifact which included an instructional unit for a specific topic and grade level as a final assessment. These experiences to plan activities for themselves may have contributed to their increased sense of efficacy.

4.3. Changes Needed in Society

Unfortunately, students did not change considerably in the actions they felt were needed to make positive societal changes in the future. Ideally, students would recognize that ALL of the suggested changes are needed to sustain the planet and humanity in the future. It was surprising to see that three fewer students felt that communities working together was needed for societal change in the post-survey, when many had highlighted the important role of education in societal change through their open-ended responses. Students should recognize that beyond individual and community actions, large-scale systems and policies are a critical part of the solution to challenges such as climate change and deforestation. Future researchers who use the survey question designed by Kennelly et al. [22] should consider adding “restructuring the education system to emphasize education for sustainability” as another option for students to select, which would recognize the role of education in societal transformation. Reflecting on results from this survey motivated our instructor team to design a module on civic engagement, which emphasizes the importance of changes in government and communities as well as the restructuring of society to help address sustainability issues.

4.4. Individual Consumption Practices

Students reported significant positive changes in consciousness about food purchases and energy and water conservation. Consumption practices are often seen as a primary goal of environmental and sustainability education programs, yet are often difficult to change through intervention [10,21,30,58]. Providing students with tangible steps for enacting change, both individually and through education, may have empowered students. For example, after calculating their ecological footprint, students made lists of specific changes they could make in their lifestyles to reduce their carbon footprint. We recently extended this assignment to have them reflect on which changes were most feasible for them personally, and why.

4.5. Motivation

Students’ levels of motivation in the present study were similar to findings of Kennelly et al. [22] who reported that 87% of students had a very strong or strong desire to include sustainability in their classroom at the end of the course. Consistent with cognitive evaluation theory, motivation is an important driver that will help pre-service teachers persist in teaching and learning about EfS as they enter classrooms in the future [33]. As a result of the course, student motivation with respect to EfS was increased, and this finding highlights the importance of having a required course focused on sustainability for future teachers.
It is important to note, however, that two participants mentioned a lack of motivation, one of whom articulated religious reasons. For these students, the course was not transformative. Perhaps the ideas and beliefs discussed in class were too far from deeply held views. We continue to meet monthly as a team of instructors, to reflect on and improve the course, including ways to encourage diverse perspectives in class discussions. Prior work has shown that collaborative curriculum planning for EfS can help instructors reflect on and improve their teaching [59].

4.6. Limitations

As with any research, there are certain limitations to the current study. We did not have a control group, which would have strengthened the research design. In addition, any changes over the course of the semester may have inputs from a variety of individual and societal contexts, or other courses. However, most students mentioned this course specifically in their comments about motivation. Self-reported changes in sustainable consumption practices are subject to scrutiny and may not necessarily reflect long-term, meaningful changes. This presents an opportunity to follow up with course alumni to ask them about their current sustainability practices and what impact, if any, the course continues to have on their everyday lives. Because we recognize that people often over-report socially desirable behaviors in social science research [60], we made an effort to reduce the potential for desirability bias by using self-administered surveys instead of conducting interviews, and by assuring students of confidentiality in reporting the results.

4.7. Implications

ASU is a unique context to implement sustainability education into the curriculum for preservice teachers. Many university-wide initiatives are discussed in class and are considered through assignments so that students are aware that sustainability is part of a broader conversation at the university. This gives credibility to the ideas discussed in class and helps students see the relevance of these initiatives and actions.
How might a college course play a role in sparking the changes described here? Some of the course elements that may have prompted changes include digital stories, introducing ways of thinking that foster critical thinking, assignments that asked students to reflect on their sustainable consumption practices, and modeling of effective pedagogies. Carefully designed digital stories provided content knowledge and local and global case studies focused on sustainability challenges from economic, social, and environmental perspectives. Each week, students use different ways of thinking to think critically about the human-designed systems around them, with a sustainability lens. Finally, instructors modeled pedagogies such as philosophical chairs discussions, diagrammatic mapping of food systems and read-aloud texts that spark discussions about sustainability concepts. Seeing these pedagogies modeled may have contributed to students’ beliefs that they were capable of leading similar activities in the future.
Nolet [31] suggested that development of a sustainability worldview help us “begin to see that even small changes in our own thinking and behavior can matter profoundly for our own well-being as well as that of others now and in the future” (p. 63). The significant changes reported here suggest an emergent sustainability worldview. These shifts must come from within, as students assimilate new information, reflect on their own lives and priorities, and consider the role they want to play in society and as part of the natural environment. As a sustainability worldview emerges, along with a strong sense of agency and motivation, the hope is that pre-service teachers will persist and become successful change agents in society. A sustainability worldview embodies the four pillars of learning outlined in UNESCO’s Delors Report: learning to know (content knowledge and motivation to learn more), learning to do (to be effective educators), learning to be (sustainable consumption practices and a sense of responsibility), and learning to live together (an ongoing challenge in this complex world) [61].
As teacher educators, we are critically aware that teachers face many obstacles to EfS once they enter the profession, and will need supports from peers, principals and community members in their work ahead [62]. Further longitudinal work is needed to explore the extent to which the changes described here are short-term, or whether the values, sense of agency, motivation and behavioral shifts described here remain or diminish over time. Also, follow-up interviews and surveys could yield insight into individual differences in trajectories.
Many other researchers have called for innovative higher education curricula that foster interdisciplinary thinking, a sense of agency and a “planetary consciousness” [38] (p. 387), [25,31]. Such programs and pedagogies are desperately needed in order to optimize the role of education in meeting the sustainability challenges described by the Sustainable Development Goals [9]. The current study provides evidence that significant change related to EfS can occur through education, in courses with effective pedagogy and sufficient intensity or duration such as the semester-long course described. Teacher educators should continue to advocate for required courses in EfS. Nolet’s articulation of a sustainability worldview is visionary, albeit somewhat daunting in the present educational landscape. Educators must hold the tension of our current neoliberal education systems alongside our values for EfS and supporting educators’ sense of agency. Teacher educators can help learners develop a sustainability worldview through transformative pedagogy. And through education, these future teachers will play a critical role in transforming society.

Author Contributions

The first author led conceptualization, writing and analyses. The second and third authors assisted with analyses, writing and editing.

Funding

The APC was funded with contributions from the Biodesign Pathfinder Center and Mary Lou Fulton Teachers College at Arizona State University.

Acknowledgments

The authors and the research development team at the Biodesign Institute thank Arizona State University (ASU) for the generous Strategic Initiative Funds provided to form the Sustainability Science Education Project and to develop the Sustainability Science for Teachers course and other related projects at ASU. The project was developed under the leadership of Dr. Lee Hartwell. We thank the Pathfinder Center at the Biodesign Institute, the Mary Lou Fulton Teachers College, the School of Sustainability, and School for the Future of Innovation in Society located at ASU for advice and the many experienced students who helped with the project. We particularly thank all current and past Sustainability Science Education Project team members, colleagues, instructors, students, participants, and friends.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Steg, L.; Vlek, C. Encouraging pro-environmental behaviour: An integrative review and research agenda. J. Environ. Psychol. 2009, 29, 309–317. [Google Scholar] [CrossRef]
  2. UNESCO. Agenda 21. United Nations: Rio de Janeiro, Brazil, 1992. Available online: https://sustainabledevelopment.un.org/content/documents/Agenda21.pdf (accessed on 15 November 2018).
  3. UNESCO. United Nations Decade of Education for Sustainable Development 2005–2014, International Implementation Scheme, UNESCO, Paris. 2005. Available online: http://unesdoc.unesco.org/images/0014/001486/148654e.pdf (accessed on 15 November 2018).
  4. UNESCO. UNESCO Global Action Programme on Education for Sustainable Development, Goals and Objectives; UNESCO: Paris, France, 2014; Available online: http://en.unesco.org/gap/goals-and-objectives (accessed on 15 November 2018).
  5. UNESCO. Issues and Trends in Education for Sustainable Development; Leicht, A., Heiss, J., Byun, W.J., Eds.; UNESCO: Paris, France, 2018; Available online: unesdoc.unesco.org/images/0026/002614/261445e.pdf (accessed on 18 February 2018).
  6. World Commission on Environment and Development. Report of the World Commission on Environment and Development: Our Common Future; Oxford University Press: Oxford, UK, 1987. [Google Scholar]
  7. UNESCO. Bonn Declaration on World Heritage; UNESCO: Bonn, Germany, 2009; Available online: https://whc.unesco.org/document/147735 (accessed on 17 February 2018).
  8. UNESCO. Rethinking Education: Towards a Global Common Good? UNESCO: Paris, France, 2015; Available online: http://unesdoc.unesco.org/images/0023/002325/232555e.pdf (accessed on 18 February 2018).
  9. United Nations. Transforming Our World: The 2030 Agenda for Sustainable Development; United Nations, Department of Economic and Social Affairs: New York, NY, USA, 2015; Available online: http://www.un.org/ga/search/view_doc.asp?symbol=A/RES/70/1&Lang=E (accessed on 6 February 2017).
  10. Alvarez-García, O.; Sureda-Negre, J.; Comas-Forgas, R. Assessing environmental competencies of primary education pre-service teachers in Spain: A comparative study between two universities. Int. J. Sustain. High. Educ. 2018, 19, 15–31. [Google Scholar] [CrossRef]
  11. Andersson, K. Starting the pluralistic tradition of teaching? Effects of education for sustainable development (ESD) on pre-service teachers’ views on teaching about sustainable development. Environ. Educ. Res. 2017, 23, 436–449. [Google Scholar] [CrossRef]
  12. Ateskan, A.; Lane, J.F. Assessing teachers’ systems thinking skills during a professional development program in Turkey. J. Clean. Prod. 2018, 172, 4348–4356. [Google Scholar] [CrossRef]
  13. Effeney, G.; Davis, J. Education for sustainability: A case study of preservice primary teachers’ knowledge and efficacy. Aust. J. Teach. Educ. 2013, 38, 32–46. [Google Scholar] [CrossRef]
  14. Evans, N.; Stevenson, R.B.; Lasen, M.; Ferreira, J.; Davis, J. Approaches to embedding sustainability in teacher education: A synthesis of the literature. Teach. Teach. Educ. 2017, 63, 405–417. [Google Scholar] [CrossRef]
  15. Falkenberg, T.; Babiuk, G. The status of education for sustainability in initial teacher education programmes: A Canadian case study. Int. J. Sustain. High. Educ. 2014, 15, 418–430. [Google Scholar] [CrossRef]
  16. Kalsoom, Q.; Khanam, A.; Quraishi, U. Sustainability consciousness of pre-service teachers in Pakistan. Int. J. Sustain. High. Educ. 2017, 18, 1090–1107. [Google Scholar] [CrossRef]
  17. Qablan, A. Building capacities of educators and trainers. In Issues and Trends in Education for Sustainable Development; Leicht, A., Heiss, J., Byun, W.J., Eds.; UNESCO: Paris, France, 2018; pp. 111–132. [Google Scholar]
  18. Spiropoulou, D.; Antonakaki, T.; Kontaxaki, S.; Bouras, S. Primary teachers’ literacy and attitudes on education for sustainable development. J. Sci. Educ. Technol. 2007, 16, 443–450. [Google Scholar] [CrossRef]
  19. Yavetz, B.; Goldman, D.; Pe’er, S. Environmental literacy of pre-service teachers in Israel: A comparison between students at the onset and end of their studies. Environ. Educ. Res. 2009, 15, 393–415. [Google Scholar] [CrossRef]
  20. UNESCO-UNEP. Intergovernmental Conference on Environmental Education Final Report; UNESCO: Paris, France, 1978. [Google Scholar]
  21. Hollweg, K.S.; Taylor, J.R.; Bybee, R.W.; Marcinkowski, T.J.; McBeth, W.C.; Zoido, P. Developing a Framework for Assessing Environmental Literacy; North American Association for Environmental Education: Washington, DC, USA, 2011. [Google Scholar]
  22. Kennelly, J.; Taylor, N.; Maxwell, T.W. Addressing the challenge of preparing Australian pre-service primary teachers in environmental education: An evaluation of a dedicated unit. J. Educ. Sustain. Dev. 2008, 2, 141–156. [Google Scholar] [CrossRef]
  23. Wiek, A.; Withycombe, L.; Redman, C.L. Key competencies in sustainability: A reference framework for academic program development. Sustain. Sci. 2011, 6, 203–218. [Google Scholar] [CrossRef]
  24. Bertschy, F.; Künzli, C.; Lehmann, M. Teachers’ competencies for the implementation of educational offers in the field of education for sustainable development. Sustainability 2013, 5, 5067–5080. [Google Scholar] [CrossRef]
  25. Howlett, C.; Ferreira, J.A.; Blomfield, J. Teaching sustainable development in higher education: Building critical. reflective thinkers through an interdisciplinary approach. Int. J. Sustain. High. Educ. 2016, 17, 305–321. [Google Scholar] [CrossRef]
  26. Rieckman, R. Learning to transform the world: Key competencies in education for sustainable development. In Issues and Trends in Education for Sustainable Development; Leicht, A., Heiss, J., Byun, W.J., Eds.; UNESCO: Paris, France, 2018. [Google Scholar]
  27. McKeown, R.; USTESD Network. Reorienting Teacher Education to Address Sustainability: The U.S. Context; White Paper Series, No. 1. Indianapolis; United States Teacher Education for Sustainable Development Network: Indianapolis, IN, USA, 2013. [Google Scholar]
  28. Sterling, S. Higher education, sustainability, and the role of systematic learning. In Higher Education and the Challenge of Sustainability. Problems, Promise, and Practice; Corcoran, B.P., Wals, A.E.J., Eds.; Kluwer Academic Publishers: Dordrecht, The Netherlands, 2004; pp. 49–70. [Google Scholar]
  29. Buckler, C.; Creech, H. Shaping the Future We Want: UN Decade of Education for Sustainable Development (2005–2014): Final Report; UNESCO: Paris, France, 2014. [Google Scholar]
  30. Nolet, V. Preparing sustainability-literate teachers. Teach. Coll. Rec. 2009, 111, 409–442. [Google Scholar]
  31. Nolet, V. Educating for Sustainability: Principles and Practices for Teachers; Routledge: New York, NY, USA, 2016. [Google Scholar]
  32. Tomas, L.; Girgenti, S.; Jackson, C. Pre-service teachers’ attitudes toward education for sustainability and its relevance to their learning: Implications for pedagogical practice. Environ. Educ. Res. 2017, 23, 324–347. [Google Scholar] [CrossRef]
  33. Deci, E.L.; Vallerand, R.J.; Pelletier, L.G.; Ryan, R.M. Motivation and education: The self-determination perspective. Educ. Psychol. 1991, 26, 325–346. [Google Scholar] [CrossRef]
  34. Deci, E.; Ryan, R.M. Intrinsic Motivation and Self-Determination in Human Behavior; Plenum Press: New York, NY, USA, 1985. [Google Scholar]
  35. Ryan, R.M.; Deci, E.L. Intrinsic and extrinsic motivations: Classic definitions and new directions. Contemp. Educ. Psychol. 2000, 25, 54–67. [Google Scholar] [CrossRef]
  36. Mintz, K.; Tal, T. The place of content and pedagogy in shaping sustainability learning outcomes in higher education. Environ. Educ. Res. 2018, 24, 207–229. [Google Scholar] [CrossRef]
  37. Foley, R.; Archambault, L.; Hale, A.E.; Dong, H.K. Learning outcomes in sustainability education among future elementary school teachers. J. Educ. Sustain. Dev. 2017, 11, 1–18. [Google Scholar] [CrossRef]
  38. Wals, A.E.; Lenglet, F. Sustainability citizens: Collaborative and disruptive social learning. In Sustainability Citizenship in Cities: Theory and Practices; Horn, R., Fien, J., Beza, B., Nelson, A., Eds.; Routledge: New York, NY, USA, 2016; pp. 72–86. [Google Scholar]
  39. Warren, A.; Archambault, L.; Foley, R.W. Sustainability Education Framework for Teachers: Developing sustainability literacy through futures. values, systems, and strategic thinking. J. Sustain. Educ. 2014, 6, 23–28. [Google Scholar]
  40. Gibson, R.B. Sustainability assessment: Basic components of a practical approach. Impact Assess. Proj. Apprais. 2006, 24, 170–182. [Google Scholar] [CrossRef]
  41. Selin, C. Professional dreamers: The future in the past of scenario planning. In Scenarios for Success: Turning Insights in to Action; Shapre, B., Heiiden, K., Eds.; John Wiley & Sons: West Sussex, UK, 2008; pp. 27–52. [Google Scholar]
  42. Holifield, R.; Porter, M.; Walker, G. (Eds.) Spaces of Environmental Justice; Wiley-Blackwell: Chichester, UK; Malden, MA, USA, 2010. [Google Scholar]
  43. Kurtz, L. Encyclopedia of Violence, Peace, and Conflict; Academic Press: Cambridge, MA, USA, 2008; Volume 2. [Google Scholar]
  44. Meadows, D. Thinking in Systems: A Primer; Chelsea Green Publishing Company: White River Junction, VT, USA, 2008. [Google Scholar]
  45. Guston, D.H. Understanding ‘anticipatory governance’. Soc. Stud. Sci. 2014, 44, 218–242. [Google Scholar] [CrossRef] [PubMed]
  46. Yin, R.K. Case Study Research: Design and Methods; Sage: London, UK; Singapore, 2009. [Google Scholar]
  47. Shulman, L.S. Those who understand: Knowledge growth in teaching. Educ. Res. 1986, 15, 4–14. [Google Scholar] [CrossRef]
  48. Robin, B.R. Digital storytelling: A powerful technology tool for the 21st century classroom. Theory Pract. 2008, 47, 220–228. [Google Scholar] [CrossRef]
  49. Organization for Economic Cooperation and Development (OECD). Science Competencies for Tomorrow’s World. Volume 1: Analyses; OECD: Paris, France, 2007. [Google Scholar]
  50. Hungerford, H.R.; Volk, T. Changing learner behaviour through environmental education. J. Environ. Educ. 1990, 21, 8–21. [Google Scholar] [CrossRef]
  51. Peyton, R.B.; Miller, B.A. Developing an internal locus of control as a prerequisite to environmental action taking. In Current issues VI: The Yearbook of Environmental Education and Environmental Studies; Sacks, A.B., Ed.; Clearing-House for Science, Mathematics and Environmental Education: Columbus, OH, USA, 1980; pp. 173–192. [Google Scholar]
  52. Cohen, J. Statistical Power Analysis for the Behavioral Sciences, 2nd ed.; Lawrence Earlbaum Associates: Hillsdale, NJ, USA, 1988. [Google Scholar]
  53. Cohen, J. A power primer. Psychol. Bull. 1992, 112, 155–159. [Google Scholar] [CrossRef]
  54. Miles, M.B.; Huberman, A.M. Qualitative Data Analysis: An Expanded Sourcebook; Sage Publications: Beverly Hills, CA, USA, 1994. [Google Scholar]
  55. Orr, D.W. Earth in Mind: On Education, Environment, and the Human Prospect; Island Press: Washington, DC, USA, 2004. [Google Scholar]
  56. Goldman, D.; Yavetz, B.; Pe’er, S. Student teachers’ attainment of environmental literacy in relation to their disciplinary major during undergraduate studies. Int. J. Environ. Sci. Educ. 2014, 9, 369–383. [Google Scholar]
  57. Didham, R.J.; Ofei-Manu, P. Advancing policy to achieve quality education for sustainable development. In Issues and Trends in Education for Sustainable Development; Leicht, A., Heiss, J., Byun, W.J., Eds.; UNESCO: Paris, France, 2018; pp. 25–38. [Google Scholar]
  58. Schoolman, E.D.; Shriberg, M.; Schwimmer, S.; Tysman, M. Green cities and ivory towers: How do higher education sustainability initiatives shape millennials’ consumption practices? J. Environ. Stud. Sci. 2016, 6, 490–502. [Google Scholar] [CrossRef]
  59. Biasutti, M.; Makrakis, V.; Concina, E.; Frate, S. Educating academic staff to reorient curricula in ESD. Int. J. Sustain. High. Educ. 2018, 19, 179–196. [Google Scholar] [CrossRef]
  60. Krumpal, I. Determinants of social desirability bias in sensitive surveys: A literature review. Qual. Quant. 2013, 47, 2025–2047. [Google Scholar] [CrossRef]
  61. Delors, J.; International Commission on Education for the Twenty-first Century. Learning, the Treasure Within: Report to UNESCO of the International Commission on Education for the Twenty-first Century: Highlights; UNESCO: Paris, France, 1996. [Google Scholar]
  62. Merritt, E.G.; Archambault, L.; Hale, A.E. Sustainability education in elementary classrooms: Reported practices of alumni from a pre-service teacher course. Discourse Commun. Sustain. Educ. 2081, 9, 18–31. [Google Scholar] [CrossRef]
Table 1. Reliability of Survey Constructs.
Table 1. Reliability of Survey Constructs.
Constructnα at T1α at T2
Perceived Relevance of Sustainability Education910.810.87
Responsibility for Sustainable Development900.920.95
EFS Efficacy910.780.76
Locus of Control910.730.83
Sustainable Consumption Practices870.840.94
Table 2. Student Changes Before and After Sustainability Science Education Course.
Table 2. Student Changes Before and After Sustainability Science Education Course.
Pre-SurveyPost-SurveyRange
ConstructMSDMSD tCohen’s d
Perceived Relevance of Sustainability Education3.440.443.710.531–44.20 ***0.55
Responsibility for Sustainable Development3.210.663.470.651–42.77 **0.40
EfS Efficacy2.670.543.380.421–410.90 ***1.47
Locus of Control3.900.604.190.731–54.02 ***0.43
Sustainable Consumption Practices2.590.572.810.501–43.96 ***0.41
Note. * p < 0.05, ** p < 0.01, *** p < 0.001.
Table 3. Synthesis of Qualitative Findings by Theme for Open-Ended Question.
Table 3. Synthesis of Qualitative Findings by Theme for Open-Ended Question.
ConstructNumber of StudentsThemes
Knowledge or awareness12Knowledge or awareness about sustainability concepts/issues
Perceived relevance20Important to know or teach about
10Important for young children
3Missing from prior learning experiences
Sense of responsibility11Feel responsible in general
4Responsibility to educate for sustainability
10Individual sense of responsibility
6A sense of urgency to teach about sustainability
EfS self-efficacy21Ready/able to educate for sustainability
12Ready/able to teach sustainability concepts across the curriculum
Locus of control12Feeling that one can make positive changes through choices and actions
2Can make a difference through individual choices and actions
10Can make a difference through education
Table 4. Future Vision for Societal Change.
Table 4. Future Vision for Societal Change.
Type of Changes NeededPre-Survey n (%)Post-Survey n (%)
Government regulation70 (77%)70 (77%)
Personal lifestyle changes84 (92%)88 (97%)
Communities working together89 (98%)86 (95%)
A radical restructuring of society36 (40%)35 (39%)

Share and Cite

MDPI and ACS Style

Merritt, E.; Hale, A.; Archambault, L. Changes in Pre-Service Teachers’ Values, Sense of Agency, Motivation and Consumption Practices: A Case Study of an Education for Sustainability Course. Sustainability 2019, 11, 155. https://0-doi-org.brum.beds.ac.uk/10.3390/su11010155

AMA Style

Merritt E, Hale A, Archambault L. Changes in Pre-Service Teachers’ Values, Sense of Agency, Motivation and Consumption Practices: A Case Study of an Education for Sustainability Course. Sustainability. 2019; 11(1):155. https://0-doi-org.brum.beds.ac.uk/10.3390/su11010155

Chicago/Turabian Style

Merritt, Eileen, Annie Hale, and Leanna Archambault. 2019. "Changes in Pre-Service Teachers’ Values, Sense of Agency, Motivation and Consumption Practices: A Case Study of an Education for Sustainability Course" Sustainability 11, no. 1: 155. https://0-doi-org.brum.beds.ac.uk/10.3390/su11010155

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop