Integrated STEM for Teacher Professional Learning and Development: “I Need Time for Practice”

Abstract

This study compares three pre-collegiate teacher professional learning and development (PLD) integrated science, technology, engineering, and mathematics (STEM) experiences framed in astronomy. The study is set in the western United States (USA) and involves 60 pre-collegiate teachers (in the USA these are K-12 teachers) over the course of three years (June 2014–May 2017). During the PLDs, astronomy acted as a vehicle for pre-collegiate STEM teachers to increase their STEM content knowledge as well as create and implement integrated STEM classroom lessons. The authors collected quantitative and qualitative data to address five research questions and embraced social constructionism as the theoretical framework. Findings show that STEM pre-collegiate teachers are largely engaged with integrated STEM PLD content and embrace astronomy content and authentic science. Importantly, they need time to practice, interpret, translate, and use the integrated STEM content in classroom lessons. Recommendations for PLD STEM teacher support are provided. Implications of this study are vast, as gaps in authentic science, utilizing astronomy, PLD structure, and STEM integration are ripe for exploration.

1. Setting the Stage

A study in 2003 asked, ‘What about the teachers?’ and probed into studies on teacher understanding of astronomy [1]. At the time, there was a dearth of specific studies relating to pre-collegiate teachers (or K-12 teachers in the USA) and astronomy education, although some researchers had already begun investigating the need for pre-collegiate, astronomy education [2]. In 2008, another study highlighted the increase in research related to astronomy projects and future astronomy education research efforts, including investigating technological solutions, utilizing real scientific data, and exploring ongoing professional learning and development (PLD) [3]. Today, almost 20 years after questioning where teachers fit into astronomy education, researchers are exploring several teaching realms and extending studies to include expanded research questions and considerations [4,5,6,7]. This study adds to the existing literature and expands the literature by examining teacher professional learning and development with integrated STEM over multiple years. The purpose of the study was to focus on STEM integration as a central PLD tenet and demonstrating connections between STEM disciplines in solving real-world problems [8].

Pre-collegiate teachers instruct students from about age five through age 18, and the teachers explore how to engage these students during their teacher education program prior to certification and in PLD sessions after their certification. Most of the ‘before’ work occurs in an academic setting, while most of the ‘after’ work occurs in teacher PLD. In this article, ‘teacher(s)’ refers to pre-collegiate or K-12 teachers.

Astronomy PLDs, which lend themselves to physical manipulation and modeling, can utilize the USA’s Next Generation Science Standards (NGSS) [9]. NGSS include components of science and engineering practices (SEP) and crosscutting concepts (CCC) that enhance authentic science experiences to spark the imagination of teachers’ lessons [10,11]. Thus, there is a connection between astronomy, STEM disciplines, and the NGSS, even if a teacher is not teaching astronomy. The authors caution readers that astronomy and physics disciplines require content knowledge in all of the STEM disciplines (including computer science), and all too often authentic-science PLDs fail to embrace STEM in a cross-disciplinary manner. Instead, PLDs often present concepts in an isolated manner without showing the complexities that could create rich experiences for the teachers.

To add to this pre-collegiate STEM picture, current research encourages teachers and communities to incorporate extracurricular opportunities, classroom experiences, and hands-on projects to increase pre-collegiate students’ STEM interest [12,13]. Hence, authentic science, and what that term actually means, is currently in the spotlight [11]. In this article, ‘authentic science’ refers to a real-world problem viewed through the teachers’ questions, data collection, analysis, and community dissemination. Therefore, the problem facing researchers now is in the understanding of how to effectively use astronomy in authentic science settings in teacher PLDs to provide them with the support needed to translate the PLD content and pedagogy into useful STEM classroom experiences.

Because astronomy intrinsically encompasses cross-curricular concepts, such as light and energy, it is a natural approach to teach integrated STEM [14]. After an extensive literature review, researchers found STEM integration “emphasized the need for complex, authentic, or real-world problems [with] shared practices, skills, and concepts across disciplines as a motivation for integrating those disciplines [and] frequently encourage or even require student-centered teaching strategies and classroom structures, and collaboration and teamwork… [as] essential ingredients…” [14] (p. 11).

As such, the PLD teams selected astronomy as a vehicle to teach integrated STEM because students interested in STEM are often attracted to STEM through authentic science experiences [13]. Additionally, researchers have shown that astronomy is a vehicle to teach integrated science and math components [15]. As the authors structured the PLD they asked, ‘If astronomy is the “gateway science” to physics and coding, then wouldn’t the scaffolding, intersection, and support of the disciplines be vital for teachers to teach STEM disciplines well?’ Hence, the authors argue that by offering PLD focused on astronomy, teachers can strengthen their STEM content knowledge and their understanding of how the STEM disciplines integrate with one another.

Further, the authors argue carefully structuring astronomy-rich teacher PLD can offer the support needed for teachers to use astronomy as a focus for STEM lessons, including authentic science practices. Carefully structured PLDs would provide a translation of the pure STEM astronomy content into STEM discipline topics as presented in the NGSS [9]. For example, skills such as computer coding, telescope control, and database manipulation should be included in astronomy PLDs to position the PLD facilitators towards understanding what the teachers understand and what is actually working for them. One participant aptly stated, ‘I need time for practice,’ and this short sentence altered the PLD teams’ perceptions. Hence, what does the literature show researchers about PLD, authentic science, integrated STEM, and where to go next with teachers?

2. Literature Review

For the PLD leaders, including those in astronomy, researchers have identified general effective PLD practices—which should translate to the STEM classroom—including items such as core disciplinary content, materials, hands-on activities, real-world issues, reflection and collaboration [16,17]. Specifically, researchers include: (a) Improvement of content knowledge, pedagogy, and dispositions [18,19,20,21]; (b) Creation of quality instructional materials [19,22,23,24]; (c) Use of authentic science and inquiry practices [11,19,25,26,27]; (d) Consideration of socioscientific issues and links to other educational areas [19,27]; (e) Iterative cycles of use and reflection [19,20,28]; and (f) Partnership development [29]. Although not the focus of this article, the three PLDs described here fit these aforementioned criteria. These effective PLD practices are in addition to or expansion of the integrated STEM practices described in the previous section (including real-world problems, shared practices/skills/concepts, and student-centered teaching strategies).

In any PLDs conducted, along with the general attributes mentioned earlier, authentic science should be utilized if possible. As stated earlier, authentic science in this article is defined as a real-world problem viewed through the teachers’ focus on and questions about research, data collection, analysis, and community dissemination. Authentic science, according to Spuck [11] and other researchers, incorporates: (a) A real-world problem, (b) Exploring and summarizing information, (c) Using scientific instruments and technology, (d) Using mathematics in data analysis [30], (e) Analyzing evidence and using findings to form conclusions, (f) Developing and refining questions and presenting new questions as a result of the work, (g) Developing and refining procedures and methods, (h) Communicating the methods used and results of the work to others for critique, (i) Collaborating with others in meaningful ways throughout the process, and (j) Recording the results of participant work making it accessible to the broader scientific community [11,19]. Additionally, PLDs should last 50+ hours, which is the minimum suggested amount to enact teacher change [31].

Although the educational research community realizes that content knowledge alone does not directly equate with effective teaching [32], the community does know that teachers with less content knowledge are less confident and effective in some skills such as discussion and questioning [33]. Hwang, Hong, & Hao [34] found teachers value PLD when they acquire pedagogical content knowledge (PCK). They find PCK more valuable than content knowledge (CK) because of how closely related PCK is to the act of teaching. With the need for PLD to focus on PCK the researchers first asked themselves, what do teachers need to know about science content? “To teach students the knowledge and skills required for science proficiency, teachers need knowledge and skills that are congruent with them” [33]. Thus, teachers need content knowledge (CK) and the expertise to use them in engaging ways (PCK). With the ultimate goal being to increase teachers’ PCK the focus was on authentic science.

Subsequently, which STEM teachers engage students? It is likely teachers took science coursework implementing traditional teaching methods such as lecture and ‘cookbook-style’ laboratory experiences without authentic classroom connections [35,36]. Because teachers typically teach according to how they were originally taught, implementing authentic science inquiry practices could be difficult for a teacher who was taught using traditional methods such as lecture [19]. However, teachers can engage students in meaningful STEM experiences with learning from carefully structured PLDs [13]. To encourage active science teaching, some researchers have created authentic science, astronomy-based teacher PLDs to support STEM teaching. Additionally, teachers use the NGSS, or similar documents, to guide their STEM disciplines’ content curriculum. The evaluations from the three years of PLD experiences show that there is increased participant content knowledge, greater pre-collegiate student impact through teacher implementation, and an appreciation for authentic experiences for integrated STEM [25,35].

The gap in the literature that the authors’ address revolves around one central concept, making STEM integration a central PLD tenet and demonstrating connections between STEM disciplines in solving real-world problems [8]. Although researchers can identify characteristics that compose a quality STEM teacher, unknowns still include content knowledge gains on specific STEM items, perceptions of different PLD iterations, the comparisons of those iterations, and the use of authentic science experiences in STEM teacher planning and implementing which can only happen with an increase in teachers’ PCK. Although the PLDs in this study were structured around astronomy, there were daily attempts to expand on areas of STEM integration and show connections to other disciplines. As astronomy can be a vehicle to engage students, the authors wanted to know if teachers could transform PLD astronomy-based content, with explicit connections to STEM integration, to a variety of STEM classroom and activity uses.

3. Materials and Methods

This study utilized both quantitative and qualitative data collection over a three-year period in three separate PLDs interacting with 60 K-12 teachers in the western USA to address three questions. According to Decker and McGill [37] “reporting can be improved across all disciplines to improve the quantity of data needed to replicate studies and to provide complete data sets that provide for the comparison of collected data”. As such, the PLD authors of this work have included as much detail as possible in this section.

To fully investigate both the PLD structure and its use for integrated STEM, the authors of this study investigated several research questions. The study’s research questions include: (1) How effective were the PLDs at developing teacher knowledge of astronomy content? (2) How do K-12 teachers perceive astronomy PLDs? (3) How do K-12 teachers use astronomy in planning and implementing STEM classroom activities? (4) Does using an astronomy framed PLD promote using integrated STEM content in lesson planning? and (5) How do astronomy PLDs translate into teacher classroom activities in a variety of STEM disciplines to showcase integrated STEM?

3.1. Social Constructionist Theoretical Framework

With the need for an interactive nature in PLD settings [38], the authors embraced a social constructionism theoretical framework. As outlined by Koro-Ljungberg. Yendol-Hoppey, Smith, and Hayes [39], social constructionism is interpretivist in nature and describes the socialization, roles, dialogue, and transformation of the participants. Through this lens, data collection is through interactive means such as discourse analysis (not used in this study), group assignments (e.g., activities and project feedback), and archival materials (e.g., lesson plans). The validity of the material is through member checking and triangulation [40,41] and the participants negotiate to transform future practices. The social constructionist lens fits the three PLDs described in this research study as the authors encouraged collaboration and interaction, and they interpreted what the teachers experienced through quantitative and qualitative measures in social settings through several methods, which are described in the following section. The authors used mixed methods design and the data collected in one project (or phase) contributed to data collected in projects two and three (other phases) as an iterative process [42].

3.2. Participants

Participants included 60 STEM teachers in three PLDs conducted over three years. The first and second PLDs were named Launching Astronomy: Standards and STEM Integration (or LASSI), and LASSI was conducted for two years (2014 and 2015). The third PLD was named Robotics, Applied Mathematics, Physics, and Engineering Design (or RAMPED), and the first year of RAMPED was conducted in 2016. Teachers were invited to participate in LASSI and RAMPED by responding to an email invitation sent through the school districts’ listserv as well as direct email invitations from the PI, and they self-selected for the PLDs. A breakdown of demographics per experience is shown in

Table 1. Demographic breakdown: Teaching Level: (El)ementary, (M)iddle, (H)igh, (O)ther; Primary Subject Taught: (S)cience, (T)echnology, (E)ngineering, (M)ath, (O)ther/(A)ll; Gender: (F)emale and (M)ale.

Demographic	2014 LASSI (n = 8)	2015 LASSI (n = 22)	2016 RAMPED (n = 30)
Level (El, M, H, O)	2, 2, 2, 2	16, 3, 3, 0	6, 14, 9, 1
Primary Subject (S, T, E, M, O/A)	1, 0, 0, 3, 4	4, 0, 0, 1, 17	15, 1, 1, 3, 10
Gender (F, M)	4, 4	16, 6	19, 11
Level (El, M, H, O)	2, 2, 2, 2	16, 3, 3, 0	6, 14, 9, 1
Primary Subject (S, T, E, M, O/A)	1, 0, 0, 3, 4	4, 0, 0, 1, 17	15, 1, 1, 3, 10
Gender (F, M)	4, 4	16, 6	19, 11

.

3.3. Instruments

During the three separate PLDs there were seven types of data collected. Three of the seven types were quantitative instruments and four data points were qualitative components. The three quantitative components included: (a) pre/post-tests on astronomy content; (b) the Misconceptions-Oriented Standards-Based Assessment Resources for Teachers tests (MOSART: https://www.cfa.harvard.edu/smgphp/mosart/); and (c) the Science Teaching Efficacy Belief Instrument (STEBI: http://stelar.edc.org/sites/stelar.edc.org/files/Science-TE-2fbsc7e.pdf). The four qualitative components included: (d) open-ended questions; (e) anonymous notebook feedback; (f) archival lesson plans; and (g) classroom observations.

3.3.1. Mixed Method Instrument—(a) Pre/Post Astronomy Content Tests & (d) Open-Ended Questions

The pre/post-test on astronomy content tests were developed and examined by the LASSI and RAMPED project teams. Daily content pre/post questions, designed by session and team leaders, were administered at the start and end of the project. The pre/post-tests were created by the PLD teams. The PLD teams were comprised of the faculty leading the specific professional development session. The pre/post-tests were given via the computer by the faculty conducting the PLD (although the faculty could not see the answers or the raw data), and the external evaluator accessed and analyzed the data. The surveys and the open-ended questions were collected directly by the external evaluator. Participants used a code number (instead of a name) when submitting those scores and answers to the external evaluator. In this case, the faculty had no access to the raw data or when/how the participants answered the questions.

3.3.2. Quantitative Instrument—(b) MOSART Test

The MOSART test is an assessment developed by Harvard University, and the Astronomy MOSART was developed with support from the U.S. National Aeronautics and Space Administration (NASA) grant, and it is conceptual in nature. The MOSART project provides tests educators can use to assess their students’ understanding of science concepts. The tests can be used to assess one’s subject matter knowledge and what impact a PLD has had on enhance one’s subject matter knowledge. The MOSART test involved 20 items measuring concepts such as astronomy, chemistry, earth science, and physics. The psychometric properties are detailed on the MOSART website (https://www.cfa.harvard.edu/smgphp/mosart/testinventory_2.html).

3.3.3. Quantitative Instrument—(c) STEBI Survey

The STEBI is an assessment developed by Riggs and Enochs in 1989 [43]. The STEBI is used to assess one’s science teaching self-efficacy and outcome expectancy. It is comprised of 25 items measured on a 5-point Likert scale. There are two versions, one for in-service teachers and one for pre-service teachers. The version used in this study was geared toward in-service teachers. The LASSI and RAMPED PLD teams used the in-service STEBI teacher edition (http://stelar.edc.org/sites/stelar.edc.org/files/Science-TE-2fbsc7e.pdf).

3.3.4. Qualitative—(e) Notebook Feedback

Additional written data collected from participants included anonymous notebook feedback (in a hardcopy bound notebook). In the notebook, a free response space and a question were posed each day of the PLD so that the participants, at any time during the PLD, could write their thoughts about the day’s events (e.g., What do you want us to know about today?). Typically, the team read the notebooks at the end of each week, reflected, and made alterations to the following week’s schedule.

3.3.5. Qualitative—(f) Lesson Plans

Teachers were expected to create lesson plans based on the content they were exposed to in the PLD. This lesson plan was required to be submitted as part of their work in the PLD. The lesson plan was also intended to be implemented in the following academic year. The lesson plans were submitted by the teachers to an online site (http://uwyo.edu/wycs/lessons), where the lesson plans were evaluated. The lesson plans did differ from teacher to teacher as they were created to be implemented in the teacher’s classroom. However, each lesson plan contained the same basic components in various orders (i.e., pre-test, objectives, catch/hook, activity, review, post-test, standards connections).

3.3.6. Qualitative—(g) Classroom Observations

The classroom observations were conducted by teaching peers in the same PLD, and the written feedback, pictures, and notes from the observations were collected for analysis through Google forms. Each classroom observation was a minimum of 30 min long and the notes taken were guided by the following questions: What did you observe happening in the classroom?; What did you observe happening with the students?; How did the teacher conduct the activity/lesson?; Describe any lesson successes.; and Describe any lesson challenges.

4. Data Analysis

For research question number one (“What astronomy content do K-12 teachers learn?”) the team used the pre/post content questions as well as the MOSART survey data. For research question number two (“How do K-12 teachers perceive astronomy PLDs?”) the team used the STEBI survey, open-ended questions, and anonymous feedback notebook. For research question number three (“How do K-12 teachers use astronomy in planning and implementing STEM classroom activities?”) the team used the PLD lesson plans and classroom observations. For research questions four and five (“Does using an astronomy framed PLD promote using integrated STEM content in lesson planning?” and “How do astronomy PLDs translate into teacher classroom activities in a variety of STEM disciplines to showcase integrated STEM?”) the PLD team used the qualitative methods of open-ended questions, anonymous feedback notebook, lesson plans, and classroom observations.

The team performed a summary analysis of quantitative data collected from each of the surveys and pre/post-tests during all three PLDs. As a first pass, the authors summarized the prior analysis, which consisted of the use of a t-test to determine individual questions with significant changes between pre-to-post assessment [25,29]. Additionally, a matched paired analysis was performed to assess the effect size on the MOSART misconceptions test. Secondly, the authors used a t-test to determine if there were significant changes between groups of pre-to- post assessments during each PLD. Assessments were grouped by overarching subject matter, and p-values greater than 0.001 were deemed insignificant (and less than 0.001 were deemed highly significant). Additionally, the authors identified the percentage of technical and conceptual questions of each pre-to-post assessment subject matter group. Four team members coded the three years of qualitative data (e.g., lesson plans, open-ended questions, anonymous notebook feedback) creating major themes by frequency. The authors agreed on the trends that written feedback showcased. Three major themes, and their subthemes, emerged including: (A) STEM discipline content learning (S, T, E, M, and astronomy); (B) Perceptions—positive or negative—regarding the PLD; and (C) Astronomy use in STEM classrooms—in regard to planning, and implementing. The authors validated the data collected through member checking and triangulation of all the data sets. The PLD team talked with the teachers and reported external evaluator weekly summary results to them at the end of weeks one and two. The team asked for feedback on the legitimacy of the information that they were given. Lastly, the team asked if the themes and subthemes were correct. No participant identified any comments that he/she wrote, but a few comments were reclassified after discussion with the groups. For example, the comment that included, ‘I need time for practice’ was originally put into the STEM astronomy category, but after group discussion, it was moved to the ‘negative perceptions’ category. Hence, there is inter-rater reliability (>80%) as well as the validity of the data in regard to quantitative and qualitative measures as established through member checking and triangulation. The authors used the qualitative findings to understand teacher appreciation of the PLDs more effectively as well as their needs for creating and implementing astronomy, STEM, and integrated STEM lessons. Overall, the data sets were utilized to triangulate the findings.

5. The Professional Learning and Development Structure

Each of the three PLD sessions met during the summer over the course of three years (2014–2016). The sessions were held on a university campus and included teachers from around the region. Several of the teachers participated in more than one summer session. However, although a few teachers repeated from year to year, each set of teachers was unique from one year to the next. General format for the sessions included various topics covered over two-day periods. The two-day format would include content coverage, application, and lesson development time. Specifically, each session consisted of 1.5 days of engagement with the material and a domain faculty expert. The remaining half of a day was devoted to lesson planning time. The summer portion of the PLDs were two-week intensive experiences, but the PLDs persisted into the following academic years. After PLD participants (i.e., teachers in this study) completed the summer PLD sessions they were supported throughout the following academic year with lesson plan implementation. This support included close contact with the primary instructor during the PLD sessions and then virtual meeting sessions with the teachers as needed.

The first and second PLD experiences, both named LASSI (Launching Astronomy Standards and STEM Integration), immersed 30 teachers in 25 days of an astronomy research project over two summers (2014 and 2015). The reader can see some pictures and examples of LASSI activities on the following sites: http://uwyo.edu/wycs/lessons. The third PLD experience, named RAMPED (Robotics, Applied Mathematics, Physics, and Engineering Design), exposed another 30 teachers to 16 days of six sessions. Of the six RAMPED sessions, only one—named ‘space’—specifically addressed astronomy content. The reader can see some pictures and examples of RAMPED activities on the following site: http://uwyo.edu/wycs/lessons.

As an overview to the PLD designs, see

Table 2. Overarching PLD design for three years (2014–2016).

PLD Design: 8-Hour Days For 2 Weeks
2014 LASSI
Goal: Expose K12 teachers to astronomy lessons and show connection with all STEM content.
Emphasis: Astronomy exposure with STEM connections
Daily structure: Scheduled activities (~6 h/day) collaborative work time (~2 h/day).
Data collection & analysis: During on-campus activities with peers and faculty support
Data presentation: At the end of the two-week session to their peers, highlighting main “take-aways” and key content knowledge gains. No outsiders attended the presentations.
Examples: How can I use a solar system scaling activity to teach metrics, scale, and distance? How far away is Mars, and can we reach it in our lifetime? Would it be possible, with current technology, to colonize Mars?
Remaining participant need: Project work with space to fail
2015 LASSI
Goal: Encourage K12 teachers to create, explore, collect data, analyze data, & present their own astronomy research question or project; focus on data from an observatory or general topic. Use integrated STEM concepts.
Emphasis: Authentic science astronomy experience/project
Biggest change from 2014 LASSI: Focus shifted to independent authentic research projects
Daily structure: Collaborative work time with faculty support (~6 h/day) and one planned activity a day (~2 h/day).
Data collection & analysis: During peer collaborations on & off campus, some faculty support, minimal during activities.
Data presentation: At the end of the two-week session to peers and others, using electronic poster format, highlighting the main research question, data, analysis, and lessons learned. Several (~5) outsiders attended the presentations.
Examples: (Observatory) Analysis of spectra; (General) Does Venus have phases like the moon?; How can Jupiter’s moons be distinguished from each other?; Can the size of a celestial body be calculated using angular size and portions?
Remaining participant need: Project work with more structure and faculty supported space
2016 RAMPED
Goal: Expose K12 teachers to all aspects of STEM using different STEM disciplines (including astronomy), and assist them in creating classroom lessons showcasing their learning. Showcase integrated STEM activities.
Emphasis: Different and engaging STEM exposures
Biggest change from 2015 LASSI: Focus shifted to different content & technology exposure; less independent work time.
Daily structure: Planned activities (~6 h/day) and collaborative work (~2 h/day).
Data collection & analysis: During on-campus planned sessions with faculty support.
Data presentation: At the end of the two-weeks to their peers and others, using a printed poster, conference-like session, highlighting main the K12 classroom connections and lessons learned. Many (>15) outsiders attended the presentations.
Examples: Using SDSS, how can you create a list of potential changing-look quasars?; How can Raspberry Pis be used to teach K12 weather concepts?; How does NetLogo show and allow modeling manipulations of natural phenomenon?
Remaining participant need: Less areas of focus (max. 4)

. In the 2014 LASSI PLD, teachers explored different activities and authentic problem bases. In the 2015 LASSI PLD, teachers created a research question to pursue with faculty support [25]. Observing with the Wyoming Infrared Observatory (WIRO) telescope (http://physics.uwyo.edu/~WIRO/) and viewing at the STAR observatory (https://www.uwyo.edu/physics/observatories/) were participant experiences.

While in the 2016 RAMPED PLD, each teacher picked four of six sessions to attend during the 10 summer days (note that six additional PLD days were included in the academic year). Only of the sessions was strictly astronomy focused (‘space’). Importantly, for this article, the ‘space’ session is singled out of the six sessions for data analysis. For context, the other five RAMPED PLD sessions focused on computer science/modeling, virtual reality, Arduinos, Raspberry Pis, and robotics.

Although the PLD teams created these three PLDs at different times with different goals, all three—based on the literature—encompassed high-level content, engaging activities, and varying levels of authentic science. The same primary investigator (PI—first author) constructed all three PLDs in collaboration with higher education faculty and various school districts over the three years to use astronomy and STEM authentic science approaches for translation to STEM classroom use. Innovative technologies, such as web-based communities and Jupyter notebooks, which are web-based programming notebooks, were emphasized during the 2016 RAMPED PLD.

As the literature highlighted, all three PLDs notably and specifically addressed core disciplinary content (e.g., distances, energy, light, geology, stars, quasars, Sloan Digital Sky Survey—known as SDSS), creation of materials (lesson plans on scale), hands-on activities (e.g., chocolate chip pancakes & the universe; excel pattern recognition), real-world issues (e.g., how do we know it’s a quasar?), reflection (e.g., what isn’t working in the astronomy project, why, and what can I do about it?), and collaboration (e.g., peers, faculty on and off-campus, students, authors). The authentic science included in the PLDs mirrored the literature by using real-world problems (identified by both participants and faculty), while the K-12 teachers focused on research questions, data collection, analysis, and community dissemination.

6. Results

Results are shown here aggregated by research question and supporting data throughout all three years of the PLDs. In each of the following three sections, the first table highlights the various data sources on the left and contains three columns—one for each PLD (2014 LASSI, 2015 LASSI, and 2016 RAMPED). The individual cells contain either a (1) summary of that data along with any additional references to more detailed descriptions, or (2) note if that data was collected or omitted from this work.

6.1. Astronomy Content Knowledge Gains

The authors collected data on teacher content knowledge gains via pre-to-post tests and the MOSART survey (see Section 3.3.2) over the three PLD years (see

Table 3. Summary of quantitative data collected on teacher content knowledge gains, from pre-to-post and MOSART survey, with a reference to detailed pre-to-post data.

Instrument	Key Finding(s) of Teacher Participant Knowledge Gains across Projects
N Sample vs. Total	2014 LASSI	2015 LASSI	2016 RAMPED
	7 of 8	21 of 22	29 of 30
Pre-to-Post Question Content Gains (Section 3.3.1)	Only post data collected.	…experienced significant positive content knowledge gains (p < 0.001) in over 60% of content pre/post questions [22,26]. Potential relationship between significant change and technical vs conceptual assessment questions. (See Table 4)
MOSART Survey (Section 3.3.2) Result Post PLD	…had few misconceptions (most scored above 80%)	Not Collected	…had some misconceptions (most scored above 60%)

). Overall, all three PLDs showed gains in teachers’ content knowledge after the two-week PLD sessions. In 2014 LASSI PLD the MOSART test was used to assess misconceptions and astronomy knowledge of the STEM teachers. The teachers had few astronomy misconceptions after the summer PLD. The most prevalent teacher misconception was the primary source of energy for stars. The total possible score was 12, and one teacher received a three (low score) while six teachers received high scores (two with a 10, one with an 11, and three with a 12). Five additional questions from the MOSART were administered involving drawings, and the high score was five. Three teachers received a low score (one with a one, and two with a two), while five teachers received a high score (four with a four, and one with a five). Five of eight STEM teachers had a misconception that pertains to eccentricity of the ellipse representing the Earth’s path around the sun.

During the 2015 LASSI two-week PLD, the faculty presented a wide spectrum of astronomy and STEM topics, including: how to construct a telescope, identifying parts of the telescope, ways to use angular size and distance in STEM lessons, explaining Kepler’s Laws, the Doppler Effect and Newton’s laws, and how to use Arduinos. Other specific topics teachers explored included black holes, wormholes, circuits, Galilean moons, orbital resonance, applying spectral graphs, constellations, movement of light, wavelengths, relation to elements, information about astronomers (e.g., Tycho Brahe), electromagnetism, prisms, stars, galaxies, gravity, CCDs, wavelengths, Kirchoff’s circuit laws, and spectroscopy. The 2015 LASSI pre-to-post questions were skewed towards technical understanding. The test was neither differentiated for elementary and secondary teachers, nor did it take into account the different backgrounds of the teachers. The teachers’ content scores increased pre-to-post-tests on all but two topics (see

Table 4. Summary of significant (p < 0.001) teacher content gain in pre-to-post surveys and technical versus conceptual question ratio (grouped by content focus area).

Astronomy Content Gains (Pre/Post) (SeeSection 3.3.1)
2015 LASSI
Content Focus (3 to 5 questions each)	p-value	Technical vs. Conceptual Split
Foundations of Computing	<0.0001	60:40
Galileoscopes	Not Sig.	80:20
EM Spectrum, Gravity	<0.0001	80:20
Doppler Effect & Angular Size	Not Sig.	80:20
Stellar Classification & Gas Giant Planets	<0.0001	60:40
Galaxies, Quasars & Gas Giants Moon	<0.0001	50:50
Hubble’s Law, 3-Color Images, & Remote Observation	<0.0001	50:50
2016 RAMPED
Content Focus (3 questions)	p-value	Technical vs. Conceptual Split
Astronomy	Not Sig. (n = 2) <0.0001 (n = 1)	33:67

).

During 2016 RAMPED, five items, selected from each of four MOSART tests (Physics, Earth Science, Chemistry and Astronomy/Space), were pooled into a pre/post outcome assessment. Matched pair analysis indicated an increase in scores from pre-to-post that was statistically significant (t = 2.38, p = 0.0244) with a high correlation (r = 0.94) between the pre- to- post-test scores, therefore it was a very small effect size (0.15), and the difference is trivial. Additionally, half of the participants scored above 60% on the MOSART assessment.

During the 2016 RAMPED ‘space’ session, teachers were asked about repositories of astronomy data for future classroom use, and the pre-space session test showed only 15% could identify them, whereas the post-space session test showed that 90% could identify the correct databanks (p < 0.00001). The RAMPED session questions were skewed towards conceptual understanding.

6.2. Teacher Perceptions of Astronomy

The teams collected data on teacher astronomy perceptions from various sources over the three PLD years (see

Table 5. Overview of teacher astronomy perceptions findings based on various sources, with references to detailed data.

Data Source	Key Finding(s) of Teacher Participant Perceptions across Projects. Teachers
	2014 LASSI	2015 LASSI	2016 RAMPED
N Sample vs. Total	7 of 8	21 of 22	29 of 30
Pre-to-Post STEBI Survey Scores (Section 3.3.3)	Not Collected	…experienced a slight decrease in self-efficacy
Open-ended Questions (Section 3.3.1)	…enjoyed hands-on engagement but gained the most from success after challenging experiences. (See Table 6)
Notebook Feedback (Section 3.3.4)	Not Collected	…moved from excitement, to despair, to understanding. [22]
Aggregate of Content Gains Perceptions (Section 3.3.6)	Not Collected	…self-identified, with varying levels of detail—their own knowledge gains. (See Table 7)

,

Table 6. Excerpts of qualitative responses concerning teacher perceptions of astronomy PLDs, as reported in open-ended questions and anonymous notebook feedback.

Teacher Perceptions of Astronomy (Open Ended Questions + Notebook Excerpts)
2014 LASSI
“I enjoyed the hands-on part of the labs where we actually did some of the work we expect the students to do. I also liked when we were asked how we would adapt each part of the PD for use in our own class.” “Variety of instructors and ideas presented, some was challenging, some more basic—good mixture.” “This activity has students partake in research … similar to what astronomers do for research…”
2015 LASSI
“I have a pretty solid physics background (introductory only) and felt the concepts were interesting and helped elevate my knowledge.” “I appreciated having the opportunity to visit with experts. I learned a lot during my discussions.” “The topics were presented in a way that I could grasp the information even with my limited background experience in astronomy. I found every lesson very interesting and appreciated that I could relate most to my professional growth as a classroom teacher.” “The first week the material was really too complex for me, but during the 2nd week things came together and I felt really proud of the learning that had happened in the two-week PD.”
2016 RAMPED
“This was my first experience with the space concepts. I was overwhelmed; my table partner again, was invaluable to my comfort with the material. I need time for practice.” “The tutorials were good and helped to fill in some of the holes that were present for those that were not in the space session earlier. I do feel as though there was an expectation that we would quickly pick up Python and be able to extend on our own.” “I have never participated in a better, more relevant professional development.”

and

Table 7. Aggregation of qualitative teacher responses of astronomy content knowledge gains, as reported in open-ended questions and anonymous notebook feedback of qualitative responses concerning teacher perceptions of astronomy PLDs.

Teacher Self-Reported Astronomy Content Gains (Notebook & Open-Response Excerpts)
2014 LASSI
“I really learned a lot about astronomy today that I did not previously know.” “I found the morning sessions very informative and it helped me understand astronomy and waves.”
2015 LASSI
“Angular size was very helpful. I have a better understanding of how to measure things from a distance.” “I have a better understanding of gravity, how telescopes are constructed, and the kinds of Kepler’s Laws, and how to measure angular sizes and distance.” “I learned a lot about spectra! I feel confident that I can tell the difference between a star, galaxy and quasar. I can confidently tell the temperature and how something is redshifting.” “I have learned about quasars, the many different moons that orbit the planets in our solar system...their characteristics and interesting facts, how the temperature has affected the formation of the planets in our Solar System, remote observing and 3-color images.”
2016 RAMPED
“I have had the opportunity to learn about astronomy data repositories that even the public has access to! How neat!” “I learned about which attributes can be measured from point sources in images, and how they can be used to differentiate quasars from stars.” “I know that the Sloan Digital Sky Survey (SDSS) is an incredible resource for myself and my students.” “This week I finally realized how computer science fits into STEM, computer science was foundational to the experiments we were doing with the Sloan data.”

). Overall, all three PLDs had high teacher perception value, and a ratio of three positive comments for every negative comment. In the 2014 LASSI PLD, each day the STEM teachers rated the usefulness of the day’s events that were listed on the daily agenda. Overall, LASSI met the PLD needs and interests of participating teachers at a high level based on the mean quality rating of each session at 4.0 or higher on a scale from 1 to 5 (5 high). Ninety percent of the teachers reported being highly engaged in the sessions at least 75% of the time.

The most highly rated events were those that involved teachers in hands-on activities. These activities included: (A) The sidewalk solar system, (B) Solar system size and scale, (C) The distance lab, (D) The crawl of the crab, (E) Math connections, (F) Writing lesson plans and (G) Lesson planning and collaboration. Collaboration is an important aspect of the social constructionism framework that was employed in these PLDs, and the teachers appreciated interacting and learning with their colleagues. All of the teachers were interested in leaving the PLD with lessons they could use in their classrooms. Time spent collaborating on lesson planning was always rated ‘very useful.’ The STEM teachers wanted to develop lessons in astronomy that they could take back to their classrooms and immediately use. Session activities that included food were popular because teachers reported that, in general, their students like these kinds of activities. One teacher wrote about a mathematics session activity, “I can see some possibilities for integrating it…preferably with food (fractions). What fraction is a Cheez-It of a graham cracker?” Science and mathematics topics were the focus of the summer (without a specific computer science presence).

Additionally, nearly two-thirds (64%) of the teachers reported that they anticipated that what they learned in the PLD sessions would impact their teaching in their own classrooms in 2014-15. After each PLD session, participating teachers responded to the question, ‘How much do you anticipate what you learned during today’s PLD session will impact your teaching in your own classroom in 2014-15?’ All the responses were aggregated by theme. Based on this pooled data 64% of the teachers anticipated that their own classrooms would be impacted ‘quite a bit’ (45%) and ‘very much’ (19%) by what they learned during the PLD session. Although 2% said that they weren’t sure or ‘not at all,’ the other 33% said their classrooms would be ‘somewhat’ (9%) and ‘moderately’ (24%) impacted.

In the 2015 LASSI PLD, the overall quality of the second week of the PLD was rated higher than the first PLD week. On a scale of 0 to 4 (0—poor, 1—fair, 2—good, 3—very good and 4—excellent), the mean overall quality ratings were 2.32 for the first week and 3.67 for the second week. Based on the data, the lower mean rating of the first week’s session was likely due, at least in part, to teachers’ opinions of the level of complexity of the topics included each week: the topics were rated as ‘just right’ by 41% of the teachers in week 1 and 76% of the teachers in week 2.

Premium experiences were applied. The teachers praised the interactions with the astronomers at the WIRO (http://physics.uwyo.edu/~WIRO/), and the hands-on expert explanations. Also, on the top of the list, the teachers valued the opportunities to make things, use new resources (e.g., Makey Makey; https://makeymakey.com/), and develop lesson plans for STEM classes from the content learned during the PLD. Remote viewing, engaging presentations, stellar lectures, expert guest speakers, energetic astronomers, the planetarium, the Jelm Mountain Observatory (http://physics.uwyo.edu/~WIRO/), break-out sessions, hands-on activities, developing lesson plans, and the 3-color images lesson were all highlights of the second week. Computer science was specifically a focus of the PLD, as teachers needed to write code in order to obtain data on stars and quasars. The STEM teachers used code to transform the raw data retrieved from the telescope.

Thirteen teachers (59%) were enthusiastic about the lessons and topics they were taking back to their own classrooms, including: making pin-hole telescopes to view the total eclipse, using the Bayer filter in photography, using remote telescopes, using spectrums to reinforce the importance of graphing, sharing information about women in astronomy, general information about the solar system, introducing astronomy vocabulary, using linear regression to find Hubble’s constant, using sinusoidal functions in trigonometry when studying wavelength, frequency, and energy, and how to determine size and mass of a planet. One teacher was thinking about starting an after-school astronomy club. Eight teachers (36%) were either not sure how they would use what they learned during the PLD or could only think of a few ways that they could use the information in their own classrooms.

Eleven 2014 LASSI teachers (50%) planned to use some combination of the following from the PLD: remote telescope operations such as Sloan Digital Sky Survey (SDSS), Stellarium, and Micro-Observatory Robotic Telescopes, plus online databases. A few teachers intended to have students build and/or use Galilean telescopes, and one emphatically stated that phases of the moon could be observed without a telescope. One did not know what he/she would have students use in lieu of the WIRO, STAR, and rooftop telescopes, while four replied that they could not use their projects with the grade levels that they teach.

One teacher pulled out the popular culture aspect of one presentation and activity and explained, “I liked the discussion on UFOs. It’s helpful to discuss the difference between science and pseudoscience with [students] and what better time to discuss this difference then when asked about UFOs. I will use [the expert’s] statement [with my classes]. It is impossible to disprove that something does not exist, and therefore, the question of whether something does not exist is pseudoscience not science.”

In 2015 LASSI, the STEM teachers were asked, ‘In what ways are you a more knowledgeable educator as a result of this summer’s focus on guided inquiry and authentic research experiences?’ Although a couple of the teachers commented on their better content knowledge in astronomy, and one noted a better understanding of the importance of independent research and its processes, far more teachers’ remarks pointed to other areas including (1) better understanding of and appreciation for the integration of mathematics and sciences areas and the integration of STEM into their classrooms, (2) a better grasp of what constitutes successful guided inquiry, and (3) how to improve and use classroom discussions more effectively. The teachers noted that effective STEM integration and appropriate guided inquiry could impact student learning and career decisions and encourage students’ ownership of their learning. One respondent said that a better understanding of the inquiry-based approach derived from having become ‘a responsible participant in their own learning’ during the project experiences.

In 2016 RAMPED, the percentages of teachers who were either ‘somewhat’ or ‘very satisfied’ with the PLD at the end of week one and end of week two were 93% and 97%, respectively. The likelihood that teachers would recommend more PLD like RAMPED increased from a mean of 8.86 (end of week one) to 9.07 (end of week two) on a scale of 0 to 10 (0 not at all likely and 10 extremely likely). The STEM teachers made connections with other teachers who they did not know earlier, enthusiasm for working collaboratively with one another increased, and by the end of the second PLD week the teachers were excited about being able to continue to network with collaborators they met during the PLD in order to share ideas and learn from one another on how to implement project-based activities so that students could be engaged and learning high-level science. A teacher expressed an example of the support teachers can offer each other and the increased comfort with one another that they had at the end of the PLD’s second week, “These connections allow me expanded collaboration. I am able to bounce ideas off of teachers not just housed in my district but others as well. They give me the insight I haven’t before seen.”

Some teachers struggled with the amount of and complexity of content, including the computer science components. Teachers rated the extent (0 to 3, not at all = 0 and large extent = 3) to which the space session was useful to them. The mean rating for space was 1.95. Within the space session, the STEM teachers utilized computer science by obtaining and manipulating SDSS data through development of Python code to automate the process and visualize the results.

Specifically, regarding the ‘space’ session, 20 teachers chose to attend the summer session, and of those teachers, 14 felt that the space session was useful to a ‘moderate’ (45%) or ‘large’ (25%) extent. Five of the teachers felt the ‘space’ PLD was ‘not at all useful’ (5%) or to a ‘small extent useful’ (20%). One teacher stated it would have been helpful to have the first session be about programming and then one or two sessions about how to use that programming with the technology from the sessions. The teacher stated, “Programming was very overwhelming and got in the way of learning the technology.” However, another teacher commented that “the Space session showed [me] how coding relates to [astronomy].”

Based on data from the teachers, the strengths of the project were the depth of content, exposure to cutting edge technology that can be used in the classroom, involvement in programming and coding, knowledgeable presenters and team, impressive organization, applicability of the content to the classroom, collaborations, enhanced problem-solving skills, acquisition of technology skills, availability of (patient) individual help, the fun of living in a dorm, resources available for loan to the classroom, intelligent discussions, and exposure to many STEM areas. One teacher summed up the experience and stated, “I have never participated in a better, more relevant PLD. The applications for my classroom will really support me in helping my students to understand and apply 21st-century skills.” After the 16 days of the 2016 RAMPED PLD, 97% of the teachers reported that they were ‘moderately’ or ‘extremely’ satisfied with the experience. One teacher noted, “Great insight into where we need to support our students and prepare them for these amazing experiences. The instructors were beyond amazing and supportive, the content was interesting, and the use of hands-on materials and the problems [presented] were very beneficial….” The external evaluator exclaimed, ‘I have never seen a project with such high participant satisfaction. And I’ve evaluated hundreds of projects.” The authors include these evaluation scores and quotes to situate the conclusions.

The STEBI was used as a second teacher perception assessment. Mean scores decreased from pre-to-post on the STEBI for personal science teaching efficacy belief. None of the pre/post changes on the STEBI were statistically significant.

6.3. Evidence of Astronomy Planning and Implementation

The authors collected data on teacher planning and implementation via participant PLD developed lesson plans and classroom observations over the three PLD years (see

Table 8. Summary of collected data for the planning and use of astronomy in the classroom, with references to detailed data.

Instrument	Key Finding(s) of Teacher Participant Classroom Use across Projects. Teachers
	2014 LASSI	2015 LASSI	2016 RAMPED
N Sample vs. Total	7 of 8	21 of 22	29 of 30
Lesson Plan	… planned use of (1) Foundational math, (2) Astronomy concepts, and (3) Associated technology. (See Table 9)
Lesson Plan Topics (See Table 10)	Use of Science and Math dominated. Astronomy in 5 of 6 science lessons.	Used a balanced approach, emphasis on Science. Astronomy in 13 of 16 science lessons.	Use of Technology dominated. Astronomy in 8 of 15 science lessons.
Classroom Observations	Not Collected	See Table 10

). Overall, there was a shift in PLD lesson planning from science and mathematics, to science and astronomy, and finally technology and astronomy. Moreover, teachers vested in astronomy (such as the 2014 LASSI participants) were more likely to create astronomy laden lesson plans, while those only interested in astronomy were less likely to include specific astronomy topics in their STEM lesson plans (such as the 2016 RAMPED participants). One may expect all teachers to create lessons focused on astronomy since that was the focus of the first two PLDs. However, this was only found in lesson plans for astronomy teachers. Other STEM teachers were able to create lesson plans not directly related to astronomy based on the PLD content. This illustrates the value in offering PLD in a field not directly related to one’s content area and the value of STEM integration.

During 2014 LASSI, the teachers focused on utilizing science and mathematics in their lessons (see Figure 1 and

Table 9. Aggregation of qualitative teacher responses on planned STEM lessons with bolded integrated STEM aspects, as reported in lesson planning documents.

Astronomy Planning and Use (Lesson Plan)
2014 LASSI
“[I will connect] knowledge about astronomy… to [my] Common Core State Standards-based math classes.” “There are many astronomy connections that I can use in my classroom and it was very beneficial to have time to work on how to incorporate the astronomy ideas and activities from LASSI for my students.” “One of the things that I found to be useful in today’s sessions was [how to] use technology within the classroom. This is an area that I believe to be weak in my school.”
2015 LASSI
“Understanding the material of astronomy on a deeper level will help me teach my second-grade students at their level and allow me to answer some of their advanced questions. Creating a lesson plan has been a great take-away that I will be able to implement into my classroom.” “I can use the information in black holes and wormholes in my classroom to fit with the space curriculum as well as in my astronomy club.” “In 5th grade part of the NGSS is the stars and planets so I will use what I [learned] … in my classroom.” “The concepts of wavelength and frequency as it pertains to light would be useful in a Trigonometry class.” “I could use all of the topics in different ways. Hubble’s Law and constant fit well in a linear functions unit. Wavelength, frequency, and energy fit well in a Trigonometry class after sinusoidal functions.”
2016 RAMPED
“Great application problem with eclipse [in 2017].” “I see a benefit in physics & astronomy [classrooms].” “I like how the idea of algorithms was brought down to an elementary school level [in the] discussion of sequencing.” “I came away with an incredible amount of curriculum, ideas and connections that I can use in my classroom!” “The applications for my classroom will really support me in helping my students to understand and apply 21st-century skills.”

). One teacher focused on student understanding of scale by ‘walking the solar system.’ Another teacher emphasized light, specifically having students investigate visible, infrared, and ultraviolet light. All teachers used standards, objectives, pre/post essential questions, a catch, activity, review, and assessments in their lesson planning.

The 2015 LASSI teachers used science and astronomy on equal footing (see Table 9 and

Table 10. Breakdown of seven observed lessons with astronomy connections, where the frequency counts are shown in parentheses.

Astronomy Use (Classroom Observations of Lessons)
2016 RAMPED
Level	Astronomy Content	Primary Subject
Elementary (3)	Orders of Magnitude (2) Space Book (1)	Math (2) Reading (1)
Middle School (1)	Diffusion of Gasses, spatial patterns, analyzing spectra (1)	Chemistry (1)
High School (3)	Space, Gasses and Coding (1) Plasma (1) Elliptical Orbits/Ellipse (1)	Physics (2) Math (1)

). This group of teachers created lesson plans where students could use constellation-viewers, engage in spectroscopy labs, and explore the inverse square law with lasers. Again, all teachers used standards, objectives, pre/post essential questions, a catch, activity, review, and assessments in their lesson planning.

During 2016 RAMPED there was another shift in lesson plans, but this time the teachers focused heavily on technology (see Figure 1 and Table 10). One would expect to see a focus on technology integration with that being the focus of the PLD. However, some PLD never make an impact on the actual classroom [37]. This PLD did show potential for an impact on the teacher’s classroom by evidence of the lesson plans. Again, all teachers used standards, objectives, pre/post essential questions, a catch, activity, review, and assessments in their lesson planning. A teacher stated, “I’ve learned so much, have used the instruction this year and have plans to use it next year. Next year two [sic] other teachers will also be using the instruction.”

The teachers recorded ways they planned to use what they learned in the sessions in their own classrooms. Four teachers (13%) were not sure about how they would use what they learned in their own classrooms and eight teachers (27%) made general statements of classroom inclusion. For example, one teacher described, “I plan to use SDSS as an introduction hook and as enrichment.”

Classroom visits and lesson plans showed the team that teachers can and do use astronomy to teach STEM lessons (see Figure 1). Classroom observers, peer leaders, were invited to 21 of the 30 STEM teachers’ classrooms where the STEM teacher picked the lesson and content to showcase. There was no requirement to show classroom astronomy use, but instead the teachers were asked to show the incorporation of something new that they learned from the PLD. Of the 21 classrooms visited, seven of the 2016 RAMPED teachers (33%), used astronomy in the lesson presented. The observers noted six specific instances where the STEM teachers applied astronomy as a vehicle for STEM.

Teachers that used astronomy in classes (see Table 10) included: (a) A high school chemistry teacher that used SDSS, and the Jupyter Notebook technology, to teach chemical gas laws; (b) Two elementary teachers that used mathematics and the order of magnitude, introduced in the space session, to teach about the numbers of red blood cells and other parts of the human body; (c) A high school physics teacher that used space materials as a catch for plasma content and other physics topics; (d) A high school calculus teacher that used space orbits to teach about the ellipse; (e) A middle school teacher that used material distribution in space to teach about diffusion and movement of gas particles, observing spatial patterns, and analyzing data and graphs; and (f) An elementary teacher that read from a children’s space book to introduce a writing lesson’s components. Overall, observational findings indicate that astronomy, which is outside of teachers’ primary area of expertise, is utilized about a third of the time after exposure to astronomy PLD experiences.

7. Discussion and Conclusions

Overall, the PLD teachers learned new STEM content, perceived that the astronomy PLDs were beneficial, and used astronomy in classroom lessons approximately a third of the time, although not in the integrated STEM fashion that the authors would have liked to observe. As found in previous research, the STEM teachers in these three PLDs were caught in an ebb and flow of engagement [37] and seemed to cling to some pieces of the PLD and disregard others. The PLD team (the faculty and team members who designed the PLD) allowed the teachers to grapple with concepts, exactly as their students would in the classroom, by using patience, positive reinforcement, and challenging questions instead of answers, as authentic science experiences. Similar to the Socratic method, this is the same approach that STEM teachers are encouraged to use in their own classrooms. This approach showed mixed results, as some of the teachers initially disengaged with the process because of the unease they felt with creating research questions, collecting data, analyzing the data, and disseminating to the community. Especially early in the PLD, some of the teachers wanted short, clearly directed activities without having to create questions or consider alternatives. The PLD team faced these challenges and made adjustments for each summer, achieving various levels of success.

The PLD teams have started to understand the PLD dynamics and how astronomy fits into integrated STEM education. The teams saw that content learning was directly tied to the activities and experiences created for and/or experienced by the teachers. This corroborates the literature on learning [37]. The PLD value teachers perceive might best be shown in the application to their classroom. Teachers find PLDs useful when the content is relevant for their classroom context [44,45,46]. However, when asked what they learned about astronomy, most of the participants’ statements were general in nature, and they cited general background astronomy knowledge, optics, angular size and distance, telescope construction and use, object identification (e.g., star, quasar, galaxy), waves, gravity, Kepler’s Laws, SDSS data repository, and spectra. The general nature of the statements speaks to the value of the PLD for STEM teachers, but this potential was left unexplored in many instances. Teachers were able to find value from the PLD, no matter the specific courses they taught, but not through specific content or to the extent that the PLD team wanted to observe. True authentic science experiences were included and contributed to knowledge gains, but the authors question the longevity and depth of the astronomy or integrated STEM learning due to the common content utilized. These areas could be ripe for future research directions.

Teacher perceptions of the PLDs were consistently high. The teachers noted that the PLD ideas and concepts presented were useful and relevant, the topics were complex, the variety of facilitators and experts was helpful, the challenge of learning difficult material was stimulating, and the hands-on experiences that mimicked student work was insightful. The teachers agreed that disseminating the results allowed them to synthesize the PLD concepts. Thus, the STEM teachers wanted and valued the astronomy PLDs, although not one of the teachers specifically taught astronomy classes (which made this field rich in integrated STEM possibilities). Again, this finding speaks to the value of PLD geared towards content and activities that can be integrated into teachers’ classrooms. STEM teachers wanted involvement with astronomy activities and projects that provided information that could be used in the classroom. The authors value the high ratings of the PLDs, and anticipate more changes in the PLDs, but also know that there is room for improvement. For example, after this study, the most beneficial STEM PLD organization seems to include: (1) use of (astronomy) authentic science projects, (2) structured support for content and use of technology, (3) providing extension activities utilizing content and technology for practice, and (4) providing space for participant practice, collaboration, and sharing ideas for skillset development. This structure is conceptual and not linear and allows the PLD providers a frame to build activities, practice time, and other elements onto the complete PLD that participants experience. Thinking of integrated STEM frameworks [14,47] when creating the PLD can also provide a guide for aspects to consider and emphasize. Planning with these concepts in mind could alleviate some challenges. For example, although the use of science projects and activities speaks to an increase in teachers’ pedagogical content knowledge (PCK), which was the ultimate goal of the PLD, more subject integration was expected. Acquiring a deeper level of PCK translates to teachers having more ideas to represent complex ideas [45], but more work on how this happens is required.

In general, STEM teachers’ use of astronomy in the classroom varied from ‘infrequently’ to ‘often’, considering their feedback and observations in the classroom. The authors anticipated that the majority of lesson plans would show new STEM integration, specifically with an astronomy focus. However, the team noticed that the teachers’ lesson plans focused on connecting astronomy only to the content that they had previously taught, using astronomy in advanced questioning or enhanced activities/projects, connecting astronomy content to their standards (e.g., NGSS), using the new technologies presented, and/or engaging students in new skills correlating with their existing content. With a third of the teachers showcasing an astronomy connected lesson during observation, the authors are both encouraged about the astronomy topic use and concerned that more teachers did not identify a connection to showcase the astronomy content or use it to teach their STEM subject. Unfortunately, not one teacher implemented an observed lesson with an authentic, real-world science focus. The authors are left wondering if the PLDs emphasized authentic science enough (as defined earlier in this work), or if perhaps the task of using authentic science in pre-collegiate classrooms seemed overwhelming to place into a classroom context? With Kanli [46] stating that the “importance given to astronomy teaching in science and physics education has been gradually increasing,” STEM teachers should engage students inside and outside of the classroom in STEM opportunities that include astronomy, but the best path to this outcome is still under review.

What are the lessons learned from the comparisons in these three PLDs? With all of the data presented, the authors argue that careful structuring of PLD leads to stronger content knowledge and use of content (astronomy in this instance) to teach various disciplines (STEM disciplines in this instance). How can PLD facilitators check on the authentic science aspect of an astronomy PLD? One way might be to pay special attention to the dissemination, or presentations, given by the teachers. When the PLD teams listened closely for STEM teacher content gains, perceptions, and future planning in their presentations, opportunities to improve the PLDs emerged. One such instance was when the PLD team used only oral presentations in the 2014 LASSI PLD, then oral, electronic, and group posters in the 2015 LASSI PLD, and finally oral and printed individual posters in the 2016 RAMPED PLD. In the final iteration, the STEM teachers stated that they participated in a meaningful, classroom changing PLD, and the authors believe that this was due in part to the changing nature of the dissemination skillset where the teachers had to ‘make sense’ of what they learned in order to present to others. Future PLD iterations will include teacher products showcasing all STEM subjects.

Another important discussion waits at the intersection of using an astronomy PLD to assist teachers with integrated STEM classroom applications. Did the astronomy structured PLD work to deliver integrated STEM content? The answer is partly (see Table 10), because the teachers did use all of the STEM disciplines to create their lessons, but they were not utilized in a consistent or evenly distributed manner. As one content area was incorporated more, another content area was not used as much. In all three years, the pattern of using some disciplines was accompanied by using others less. It is possible that the teachers focused on what was emphasized with the PLD group on a particular day or over a particular week, but refined research questions and data gathering methods would be required to address this speculation. After the teachers planned a lesson, were they able to translate the astronomy content into observable integrated STEM discipline lessons? Here the outlook was less optimistic as only 7/21 (33%) of teachers implemented an observable integrated STEM lesson, and only 6/21 (29%) used astronomy as the vehicle to showcase the STEM discipline. The STEM discipline was also secondary to the astronomy content addressed, which was not the intended delivery mechanism for the teachers. The number of observable integrated STEM lessons using astronomy could have been impacted by school curricula demands, pressure to address standards without a real-world application, or a multitude of other options, but as a group, the majority of teachers could not find a means of using astronomy in authentic, integrated ways to make STEM more engaging in their classrooms. This reality is disappointing to the authors, but it encouraged the PLD teams to realize more explicit connections to STEM disciplines (and showcasing integrated STEM) were needed to enable teachers to translate the STEM spaces into activities for their classrooms. PLD barriers to teaching lessons that lead to improved student success encompass teacher lack of knowledge base, formative feedback, and district factors, and although these challenges were addressed in the PLDs there is still room to understand why content and skillset transfer is slow (such as improvement science education—or ISE—which focuses strongly on context through six principles) [48]. Other studies show that teacher awareness of discipline connections and finding integration valuable [49] as well as bridging a discipline-based to problem-based approach and teachers’ professional mindset [50] are also factors that should be considered. The authors of this article are experimenting with using explicit non-examples and examples of both integrated STEM lesson plans and teaching recordings to showcase successful STEM teacher authentic, integrated STEM lessons through purposeful iterations. These example lessons show real-world applications, integrated skillsets, and apply student-centered pedagogies as the STEM disciplines are teased apart for critique. Educational researchers should consider creating studies with a focus on the transfer of integrated STEM to teacher lesson plans, implementation, and student learning.

After reviewing the results of this study, the PLD team offers that researchers must focus on assisting teachers in translating the concepts of astronomy (or any discipline) seamlessly into the content and skills that their students should understand, and PLD leaders must give teachers time to practice, fail, and repeat. The practice should occur outside the classroom and then inside the classroom. Showing the opportunities and providing extended time to try them are basic ideas that could be overlooked in PLDs. Experiences could include skillsets such as coding, collecting telescope data, analyzing data sets, and the like. The content STEM teachers learn, the astronomy opportunities they experience, their perceptions while learning, and then how PLD facilitators support those STEM teachers through ‘content to classroom’ translations, is an important aspect to consider when creating an astronomy PLD for integrated STEM use. The transfer of content knowledge from facilitators to teachers is vitally important if STEM teachers are to gain the knowledge needed to use astronomy to inspire their students with authentic projects and experiences in STEM subjects. This transfer occurs with hands-on experiences and time to fail and try again. At the end of the three summers, there were clearly teacher content gains in knowledge, positive astronomy PLD perceptions, and integrated STEM classroom uses, but the authors believe that more could be achieved especially using PLDs to kickstart integrated STEM spaces in classroom activities. While the outlook for using astronomy themed PLDs has promise, there are limitations that must be considered when recreating or utilizing this work.

8. Implications, Limitations, and Future Directions

Teachers, PLD facilitators, and higher education faculty and their partners need to examine astronomy for classroom STEM enhancements, as it can provide a means for teachers to spark student interest and engage in hands-on projects and experiences. Implications of this research include gaps in authentic science, online learning platforms/environments, astronomy PLD structure, and STEM integration. Future researchers should consider focusing on the exploration and solutions in light of NGSS and teacher interest and involvement in astronomy PLDs.

Limitations of this work fall into five main categories. First, participants come mostly from one state and they self-selected to learn about astronomy as a means to teach STEM. Thus, this might be a participant pool that is not representative of the general STEM population. Second, satisfying the content needs of teachers over a wide range of student ages (5 to 18 years) was a challenge in all three years. Therefore, explicitly anticipating the different content and technology needs of elementary, middle, and high school teachers would benefit a PLD team. Third, teacher observations were conducted by different peer project leaders, and no official protocol was used to evaluate the observed sessions, although an outline of what to observe was included in the online form. Thus, comparing and contrasting the observed lessons was difficult and does not allow the team to analyze the lessons as completely as desired. Fourth, a limitation in interpreting the content gains is the short time between the administration of the pre-test and post-test. Lastly, the authors worked with the participants as education and science faculty, and they could have swayed the results due to interactions, lack of interactions, and/or misconceptions and miscommunication.

Future teacher astronomy PLD questions include: (1) Why do teachers utilize lower-level astronomy content in lessons? Is it an issue in understanding the astronomy content?; (2) Do the STEM teachers require more exposure to authentic science astronomy PLDs in order to create more robust STEM lessons that use authentic science?; (3) Why are a majority of STEM teachers seemingly reluctant to engage in astronomy laced authentic science projects by creating their own research questions, investigating and analyzing the data, and disseminating their findings?; (4) Since they are the experts of their content and classrooms, why do STEM teachers desire others (e.g., PLD leaders) to map out the astronomy connections with STEM content and standards that are taught in their classrooms?; (5) What are the impacts of using PLDs focused on science versus mathematics with STEM teachers?; (6) What type of authentic astronomy PLD experiences do teachers connect to the classroom most readily?; (7) How do integrated STEM experiences impact teachers’ planning of classroom integrated STEM lessons?; and (8) What impacts teachers’ use of classroom integrated STEM lessons?

In the end, the authors reflect on one quotation from a participant; “I need time for practice”. It is important for both the authors and all educators to remember that ‘I need time for practice’ is a small statement with a big message. For PLD teams, they must form purposeful, structured time for teacher experiences and feedback while focusing on both explicit STEM content and integration, but also showcasing and supporting how the PLD translates into STEM classroom contexts.