Active Learning in STEM Education with Regard to the Development of Inquiry Skills

Abstract

Active learning, represented by inquiry-based science education (IBSE) strategies, is considered essential for students to develop skills and knowledge to prepare for the challenges of the 21st century world. The success of IBSE, and the resulting development of inquiry skills in particular, can be enhanced by various factors. This study is focused on the synergetic effect of the implementation of IBSE through well-designed inquiry activities across STEM-related disciplines, enhanced by digital technologies and formative assessment tools, delivered by teachers educated in this field. The corresponding research based on a quasi-experimental design evaluated the effect on the development of inquiry skills that were identified before and after a period of consistent implementation of IBSE, using a written test of inquiry skills as the main research instrument. The research findings on the sample of 2307 upper secondary school students confirmed a low initial level of inquiry skills, however a statistically significant improvement in students’ inquiry skills with medium size effect was identified. The detailed analysis shows the largest impact in the skill of determination of accuracy and statistically significant differences between genders without practical importance, however no difference was identified with regard to the number of inquiry activities undertaken.

1. Introduction

This study was initiated as part of the long-time intensive effort aimed at the implementation of innovations in STEM education at the national level combined with active learning strategies. Our partnership in large European projects, such as ESTABLISH [1] and SAILS [2] served as a professional starting point for intensive activities related to active learning represented mainly by inquiry-based science education (IBSE). A lot of attention was also paid to the assessment strategies, development of the respective instructional materials enhanced by digital technologies, and teacher professional development in this field. In addition, the IT Academy large-scale national project [3] provided a platform for a wide and consistent implementation of the IBSE methodologies designed. Even though there are many studies on the role of specific factors influencing the success of IBSE, in this case, the authors investigated the effect of IBSE—implemented across several STEM-related disciplines in a targeted and consistent manner, enhanced by digital technologies and formative assessment tools, and delivered by educated teachers—on the inquiry skill development. In the following sections, the theoretical background, goals, methods, results, and discussion pertaining to this research are presented.

1.1. Active Learning and IBSE

Active learning strategies and their implementation into teaching practice are still in focus by science educators and science education researchers. One of the key reasons for this relentless interest are the radical technological changes in the 21st century and implemented the growing set of shared problems and challenges that humans are facing [4,5]. Addressing these challenges requires a set of skills and knowledge to properly prepare learners for active engagement in the 21st century world [6]. The labour market needs people equipped with analytical thinking, digital skills, and communication skills [7].

Active learning engages students in the process of learning through activities and/or discussion in class, as opposed to passive listening to the teacher. It emphasizes higher-order thinking and often involves group work [8]. In active learning, students are required “to select, organize, and integrate information, either independently or in groups” [9]. It involves students in doing things and thinking about what they are doing [10]. McConnell and others [11] combine these aspects to formulate the following components of active learning: “(1) students participate in activities (either by doing or observing them) in addition to (or instead of) listening to direct instruction; (2) activities provide opportunities for student reflection on their learning or facilitate student–instructor interaction and assessment of learning; and (3) peer-to-peer interaction occurs as students complete the activity”.

There is a wide range of active learning strategies. Michael and Modell as cited in [12] as well as Hood Cataneo [13] list some of the strategies for science education, e.g., problem-based learning, cooperative/collaborative learning/group work, think-pair-share or peer instruction, and inquiry-based science education strategies. In an IBSE teaching and learning environment, learners are placed into situations in which they are invited to solve problems, formulate and answer questions independently, i.e., they follow the same procedures as scientists. Like scientists, students also investigate the world around them through active experimentation or by modelling the behaviour of an object or a system. This approach to learning requires a variety of skills often referred to as the science process or inquiry skills. Kuhltau [14] stresses that inquiry approach is the most efficient way to learn in the 21st century since students engaged in inquiry develop a variety of different competencies, knowledge, and skills. IBSE skills can be defined using different frameworks.

Table 1. Framework of inquiry skills [19].

Stage	Inquiry Skills
Conception, planning and design	1.1 Formulate the question, define the problem. 1.2 Formulate the hypothesis or expectation to be tested. 1.3 Design experiment (which variables, which relationship). 1.4 Design observation and/or measurement procedures (incl. lab-apparatus selection; experiment set-up) for each variable. 1.5 Predict results of the experiment.
2. Implementation	2.1 Manipulate apparatus. 2.2 Observe/measure. 2.3 Record results. 2.4 Calculate during the execution. 2.5 Explains or make decisions on the experimental techniques.
3. Analysis and interpretation	3.1 Transform results into standard form (i.e., tables, graphs). 3.2 Determine the relationships between variables based on e.g., graphs, tables, text and formulas. 3.3 Determine the accuracy of experimental data (identify possible sources of errors). 3.4 Compare experimental data to the hypothesis/expectation. 3.5 Discuss limitations/assumptions of the experiment. 3.6 Propose generalizations of experiment results. 3.7 Formulate new questions/problems. 3.8 Draw conclusion.
4. Communication	4.1 Share and present results in front of the class. 4.2 Discuss/defend results/form arguments. 4.3 Create a formal report about the results gained.
5. Application and follow-up	5.1 Predict based on the results obtained. 5.2 Formulate hypothesis for a follow-up. 5.3 Apply the experimental technique to a new problem.

shows a framework of inquiry skills for experimental activities elaborated based on inquiry skills taxonomies [15,16,17,18]. This framework arranges skills into five stages of the inquiry process included the respective skills.

There have been a large number of studies conducted in the field of IBSE investigating its effect on students’ achievement in various fields. The results show its positive effect on conceptual understanding and inquiry skills development [19,20,21,22,23,24], as well as on other aspects of learning, e.g., curiosity [25] and attitudes and motivation towards science [26].

1.2. STEM Education and IBSE

Although inquiry-based learning originated in science education [27], where students are involved in authentic scientific practices (e.g., planning and designing experiments and collecting data), it also occurs in mathematical or technological contexts [28,29,30]. This approach to learning mirrors the procedure and thinking that scientists, engineers, and innovators use in the real world. As a result, inquiry-based learning naturally suits related disciplines such as science, technology, engineering and mathematics, or STEM disciplines, as defined by the National Science Foundation [31]. Even though there had been initiatives aimed at improvement much earlier, the call for the increased attention towards STEM education began in the early 2000s as a result of the need for today’s students to be prepared for a technology-based future and a changing workforce [32]. It is also viewed as a response to low levels of students’ interest and engagements in STEM-related disciplines, leading to declining enrolments and concerns about shortages in people taking up STEM-focused careers [33]. This is in contrast with the fact that “employment of STEM skilled labour in the European Union is increasing in spite of the economic crisis and demand is expected to grow. Around 7 million job openings are forecast until 2025. Demand for STEM skills concerns both upper-secondary and university graduates” [34].

The simple definition of STEM education is based on the idea of educating students in four specific areas (science, technology, engineering, mathematics). More detailed definitions are ambiguous, i.e., there is no single, universally accepted definition of the STEM education model. It can be implemented according to the number of disciplines involved, ranging from STEM 4.0 (integration of all four disciplines in problem-solving or project development) [35,36,37] to STEM 1.0 (disciplines are taught separately in each subject and independently).

The shape of this integration may vary. As suggested by Attard and others [38], the foundations of a STEM approach are formed within the disciplines of mathematics and science that are critical if students are to successfully apply the knowledge and skills from the individual disciplines to STEM-related learning. Shaughnessy [39] highlights the mathematics and science in his definition of STEM education: “STEM education refers to solving problems that draw on concepts and procedures from mathematics and science while incorporating the team work and design methodology of engineering and using appropriate technology”. Kelley and Knowles [40] define integrated STEM education as “the approach to teaching the STEM content of two or more STEM domains, bound by STEM practices within an authentic context for the purpose of connecting these subjects to enhance student learning”.

Nevertheless, what is considered important is that either separated or integrated approach should be based on applying constructivist strategies to learning, such as problem-based, project-based or inquiry-based approaches to learning. In this way students explore and come to their own understanding by answering questions and solving problems [38] and drawing evidence-based conclusions while collaborating with others [41]. Inquiry originated in the science disciplines and has long been considered as an effective approach to teaching and learning, particularly within STEM areas. A number of studies point to the benefits of an inquiry-based approach [37], e.g., increased student knowledge and skills in STEM subjects [42] or selected inquiry skills development [19], increased positive attitudes about STEM and STEM careers [43,44], increased understanding of how STEM activities apply to day-to-day life [45] as well as increased problem-solving ability [46].

Some studies point to the gender differences in relation to attitudes towards science. Girls are less interested in science education than boys [28,47,48,49] which usually results in fewer women choosing to study math, science and technology [50]. While the differences between boys and girls remain the same when it comes to field-of-study preference, gender differences in student’ achievements have decreased over the past few decades [51,52] as can be seen in the regular PISA assessment [53,54,55,56,57]. In PISA 2018, boys scored a few points higher than girls in mathematics. On the other hand, girls scored slightly better in science. In PISA 2018 reports [57,58] as well as in the meta-analysis by Reilly et al. [59], the authors suggest that gender differences in mathematics and science are generally small and boys and girls now show similar levels of performance.

1.3. Digital Technologies and IBSE

Several researchers claim that meaningful use of digital technologies can contribute to an increase in students’ motivation for active learning [60,61] and to enhanced subject attainment and attitude to learning [62]. The increased motivation can be linked to enhanced beliefs in their abilities to undertake and engage in learning activities [63]. Moreover, some specific digital tools have strong potential to enhance inquiry. In physics, biology, chemistry, or mathematics they can serve as powerful tools to support experimentation as well as modelling, which are two interconnected methods used by scientists to explore the world around us [64]. Digital technologies such as datalogging, video measurement and simulations enable students to collect, organize, and analyze data from real, virtual or even remote-controlled experiments [65]. With the assistance of computer simulations, students can analyze complex or even microscopic phenomena [66]. Such visualization tools implemented in inquiry activities improve students’ representational competence and conceptual understanding of chemical systems [67]. Computer simulations can also assist students in the development of specific inquiry skills, e.g., identifying and establishing relationships between variables [68,69]. In addition, students can construct computer models independently and compare their outcomes with the results of real experiments [70] to test their models. Students manipulating representations in computer simulations may understand science concepts more deeply because they can observe direct consequences of the changes they make [71]. In mathematics, dynamic tools such as GeoGebra enable students to learn abstract concepts in an interactive and explorative way [72] and support students in making connections between representations [73]. They help students to explore and conjecture mathematical relationships, generate appropriate examples and counterexamples from which students can investigate patterns and develop arguments [74]. Using these interactive dynamic tools provides advanced techniques to visualize and investigate mathematical structured relationships in both mathematics and science teaching [75] and contributes to developing learners’ visual and spatial intuition [76]. It helps to enhance students’ investigative, analytical and interpretative skills [77].

Widely available mobile technologies have been already successfully used in inquiry-based science learning [78]. The smartphone can be a powerful device to collect data, especially with the various sensors that the modern smartphone carries [79,80]. Moreover, students can now go beyond just using mobile technology and can also be active developers of tools for exploring the world around them, thanks to their knowledge of informatics and coding, in particular [81,82].

There are many more digital technologies that can enhance learning. Nattland and Kerres [83] divide them into four categories: drill and practice programs, tutoring systems, intelligent tutoring systems, simulations and hypermedia systems. Their main benefit is connected with their common feature of interactivity based on getting regular feedback, individual learning pace and interaction with the learning environment by allowing manipulation of the presented information [72,84]. Hillmayr and others [72] collected a large number of studies published since the year 2000 investigating how the use of technology can enhance learning in secondary school mathematics and science. Based on the comprehensive meta-analysis they conclude that digital tool use had a positive effect on student learning outcomes. The results of the meta-analysis study show the largest effect sizes from the use of simulations. Karich and others [85], who also conducted research into the effect of simulations, stress that the main reason for this result might be the greater extent of learner control and learning through discovery and exploration. This result also provides evidence for the importance of this specific digital tool for the success of IBSE.

The effective use of digital tools can be influenced by various factors. Results of some studies show more positive attitudes towards digital technologies in boys than in girls [86,87]. The authors of several meta-analysis and comparative studies [88,89,90] suggest that male students hold more positive attitudes towards technology, use technology more actively, and have higher technological self-efficacy.

1.4. Formative Assessment and IBSE

A very important aspect of successful learning is giving regular and constructive feedback to the learner. The regular and interactive evaluation of students’ work with regard to the learning goals and indicating next steps in teaching is known as formative assessment [91]. There are a large number of studies aimed at perspectives on formative assessment in learning [92]. It is agreed that formative assessment is essential to the implementation of IBSE [93]. It should be embedded at every stage of inquiry-based learning [94]. Some formative assessment strategies are designed to directly develop inquiry skills e.g., making predictions, making inferences, drawing conclusions, or question generating [95].

Several research studies suggest that formative assessment supports student development of inquiry skills [96,97]. Other studies point to an important aspect in developing inquiry skills connected to the type of feedback in formative assessments. According to one intervention study [98], students who received information on the learning goal, learning level, and opportunities for improvement achieved a statistically significant increase of their inquiry skills in comparison to students who received only some or none of the mentioned information. As noted in the previous section, digital technologies can help in providing regular feedback on the basis of interaction with the learning environment while respecting the individual’s learning pace [72].

1.5. Teaching and Learning Materials and Teachers’ Professional Development in IBSE

The success of IBSE can be influenced by various factors discussed in the previous sections. Nevertheless, one of the most important elements of the successful implementation of IBSE is the teacher and their pedagogical content knowledge (PCK) in this field [99]. Implementation of IBSE needs motivated and educated teachers. Teachers should be convinced about its positive effect on students’ achievements. Some studies point to several barriers identified in IBSE, such as lack of teacher training, lack of time, lack of materials, lack of support, overemphasis on assessing content learning rather than process learning, inquiry approach is too difficult and far more time consuming [100,101]. As a result, it is essential to provide teachers with high-quality teaching and learning materials as well as to provide ongoing support through in-service and pre-service professional development in IBSE [102]. The results of a research project aimed at the effect of implementation of guided inquiry showed that for all three MAP tests [103] aimed at Scientific Practices, Science Concepts, Science Composite, the students of teachers participating in professional development had significantly higher than expected improvement relative to the comparison group containing students of non-participating teachers. Marshall and others [104,105] also stress that teachers need to devote significant time, at least initially, to lesson planning and must reflect often and deeply on their practice, which involves review and evaluation of the degree of success of their instruction.

Teachers also need support in implementing innovative approaches to teaching with the use of digital technologies. Hähkiöniemi [73] claims that for the use of technology-enriched inquiry-based mathematics teaching it is necessary to prepare pre-planned teaching units. They should involve not only instructional materials for students but also detailed description and comments for teachers on methods used in guiding students through the inquiry lesson.

Results of research conducted by Lee and others [106] show that well-designed inquiry science units can improve student understanding of complex topics across science courses and teaching contexts. It also shows that there are many ways for professional development to improve inquiry outcomes through inquiry workshops, curriculum development experience, mentoring, and peer collaboration.

1.6. Research Problem and Questions

As discussed in the previous sections, inquiry skills development in students is considered an important means for preparing learners for active engagement in the 21st century world. This is the main reason why, among many other active learning strategies, inquiry-based learning is in the main focus of this research. There have been a large number of studies conducted in the field of IBSE investigating its effect on students’ achievement and different elements influencing the success of IBSE (teacher, teaching and learning materials, digital technologies, formative assessment, implementation in STEM education). The existing studies generally investigate the individual effect of these elements and are aimed mainly at reviewing conceptual understanding or attitudes and motivation towards science; however, few studies focus on inquiry skills in particular or investigate the development of inquiry skills.

In the light of the existing research, the main research problem of this study concerns the efficacy of IBSE with regard to inquiry skills development. Unlike the studies investigating the role of specific elements on the success of IBSE, this research the focuses on the synergetic effect of implementation of well-designed inquiry activities across several STEM-related disciplines enhanced by digital technologies and formative assessment tools delivered by teachers educated in this field (the concept hereinafter referred to as “consistent implementation of IBSE across several STEM-related disciplines”). Attention is also paid to specific factors that can influence the success of IBSE.

The research addressed several questions:

• What is the effect of IBSE, if implemented consistently across several STEM-related disciplines, on the level of selected inquiry skill group development?
• What is the effect of IBSE, if implemented consistently across several STEM-related disciplines, on the development of selected inquiry skills?
• How do different factors (gender and number of implemented inquiry activities) influence the level of inquiry skills development?

2. Materials and Methods

2.1. Research Design

This research was conducted in the framework of the large national project IT Academy—Education for the 21st century that has been running in Slovakia from 2016–2022 [3]. The main project goals emerged from the imbalance between the current goals of the biology, physics, chemistry, geography, mathematics and informatics curricula emphasizing IBSE and active learning strategies and the lack of instructional materials and educated teachers in this field. The main project goals were:

• Development of inquiry activities for teaching biology, physics, chemistry, geography, mathematics and informatics in the form of complex teaching and learning materials based on the active learning and inquiry-based learning approaches enhanced by digital technologies and formative assessment tools. The inquiry activities were designed in accordance with the upper secondary curriculum goals. For successful implementation, an instructional model was needed. With regard to other studies [107,108], the 5E model was proposed as an appropriate model to design a lesson based on the IBSE approach (Figure 1). Each lesson plan was developed based on a pre-agreed structure common for all subjects. It started with an overview of the learning objectives with explicitly specified inquiry skills developed in the lesson using the framework in Table 1, teaching aids and materials needed for the lesson. Subsequently, the expected misconceptions and designed level of inquiry were addressed. This overview was followed by a detailed description of the teaching and learning scenario. The concept described served as the common framework for all subjects. Respecting the results of studies regarding the limitations of minimal guidance instruction [109], the lesson plans were designed at a lower level of student independence with the support of learning materials usually in the form of student worksheet and teacher guidance. The activities (mostly at the guided inquiry level including the lesson plans and complementary materials) were designed based on the agreed criteria by experts in education from universities as well as experienced secondary school teachers motivated to implement IBSE. These teachers cooperate with the universities as mentors in pedagogical practicum of student-teachers at secondary schools or participate at various in-service teacher trainings. Altogether, approximately 470 lesson plans were designed.
• Design and implementation of in-service teachers’ educational programmes aimed at IBSE enhanced by elements presented in Figure 1. In this way a large number of teachers were educated in the field of IBSE. Specific 50 h educational programmes designed for biology, physics, chemistry, geography, mathematics and informatics teachers were completed by almost 400 upper secondary school teachers.
• Implementation of IBSE teaching and learning materials in the classroom by teachers educated in the field of IBSE in two subsequent cycles. Even though the materials were designed by experts, their implementation in practice provided meaningful feedback from teachers. Their comments based on direct experience and analysis of students’ outcomes helped authors to make minor edits and improvements in the teaching and learning materials.

The IT Academy national project and implementation of its goals provided an opportunity for conducting this research between March 2018–December 2021. In order to answer the research questions, the following research design was implemented, presented in the structure in Figure 2. Firstly, more than 300 schools across different regions of Slovakia were contacted. They participated in the project voluntarily at different levels of involvement, e.g., they had access to the teaching and learning materials to use in the classroom or their teachers could join teacher educational programmes. Finally, 57 upper secondary schools (in many European countries known as grammar schools) participated in this research. Selected science, mathematics and informatics teachers from the participating schools completed teacher educational programmes and served as tutors for other colleagues from the same school. These teachers implemented the designed teaching and learning materials into practice. The experimental classes were selected based on the curriculum content and availability of teaching and learning materials. Since the schools as well as the experimental classes were not selected truly randomly, the research followed the quasi-experimental design.

Students from the experimental classes were exposed to coherent, intentional inquiry-based teaching and learning across STEM-related disciplines, i.e., biology, chemistry, physics, geography, mathematics and informatics. Even though geography is considered a bridge between natural and social science, it was included in the STEM group because a large part of its curriculum covers physical geography (branch of natural science). The IBSE approach has a large potential in this field as well [110].

During a certain time period, teachers were implementing the designed high-quality teaching and learning materials enriched by digital technologies and formative assessment tools. Each experimental lesson was registered through an online project portal. For each completed lesson, teachers answered an online questionnaire in order to provide feedback. They reflected on how they managed to reach the intended goals, implemented formative assessment tools and digital technologies, and how much assistance students needed to effectively engage in inquiry activities. They also reported any problems they or their students faced during the lesson. In addition, teachers submitted filled-in worksheets or other products collected from five students. It allowed for a better control of the whole process and information about the ongoing experimental instruction was made visible to the research team.

Before and after experimental instruction the level of inquiry skills development was tested using the same research instrument described in the following section. The pre-test was made available between March 2018–March 2020 when schools gradually joined the research. In order to minimize the practice effect of the pre-test [111], a four-month time interval was set as the minimum time to elapse between the pre-test and post-test administration.

2.2. Research Instrument

A test of inquiry skills was the main research instrument used in this study in order to monitor and assess the level of inquiry skills development. The test was developed earlier on the basis of existing testing instruments [112,113,114] and inquiry skills taxonomies [16,17,18]. It consists of 14 test items that are distributed over different inquiry skills and subject matter (

Table 2. Distribution of test items across different inquiry skills and subject matter.

Inquiry Skill	Number of Test Items	Subject
Formulate hypothesis to be tested	2	physics, geography chemistry
Design experiment (identify independent and dependent variables and their relationship)	4	physics, informatics, physics, chemistry
Transform data to standard forms (i.e., tables or graphs)	2	physics, mathematics, informatics, mathematics
Determine relationship between variables (based on tables)	2	physics, mathematics, informatics, mathematics
Determine relationship between variables (based on graphs)	2	physics, mathematics, mathematics
Determine accuracy (identify possible sources of errors)	2	physics, biology

). Most test items are multiple-choice items with one or two correct answers. In two questions students select from a set of variables to identify appropriate variables and their relationship. The students’ answers were evaluated on the basis of agreed criteria. Each answer was assigned a value of 0–1, point which means that students could achieve a test score in the range of 0–14 points. For multiple choice items with one correct answer students got 1 point for the correct choice, but in all other cases (even when the correct answer was combined with another option) they got 0 points. Multiple-choice items with two correct answers were scored with 1 point for choosing just two correct answers and 0.5 point for choosing just one correct answer. In all other cases a score of 0 points was assigned. A detailed description of the test design can be found in [115,116]. The online version was prepared and administered through the LMS Moodle system.

2.3. Research Sample

The sample involved in the research included students aged 15–18 from the first, second or third grades at various upper secondary schools (including grammar schools). All the STEM-related subjects are compulsory for students in the first three years of study while the fourth grade curriculum differs based on students’ individual interest in specific subjects. Altogether, 109 classes from 57 upper secondary schools with more than 8700 students were involved, i.e., they answered the pre-test or post-test, and they were exposed to experimental teaching. Firstly, students who completed the test in a period less than 15 min and more than 90 min were excluded. These time limits were based on the estimated time needed for test completion that was set at 30–40 min assuming that students read and answer the test items with comprehension. Since there are always students who can complete the test much faster, the lower time limit was set at half of the lower estimate, i.e., 15 min. The upper limit was set at 90 min with regard to the fact that an upper secondary school lesson lasts 45 or even 90 min. Students who did not correctly answer a single test item or students with some missing answers were also excluded from the sample. Next, only those students who completed both pre-test and post-test were finally involved in the research that resulted in the final number of 2307 students. To study the effect of inquiry activities number (IAN) between pre- and post-test, the sample was divided into two group: up to 5 and more than 5 activities.

Table 3. Sample sizes of the research sample reflecting gender and IAN.

	Gender		IAN
	Boys	Girls	≤5	>5
Sample size	956	1351	1384	923
%	41.4	58.6	60	40

represents sizes of the studied sample according to gender and IAN.

2.4. Ethical Consideration

The schools that participated in this research were involved in the IT Academy national project. At the beginning of the project, a large number of schools across Slovakia were approached. The schools that were interested in cooperation signed a contract under which they agreed with the implementation of the developed teaching and learning materials and the evaluation of their effectiveness in the classroom. These schools were awarded the “IT Academy partner” status. The students were informed and agreed that the test results concerning their inquiry skills would be statistically processed for research purposes. In terms of result evaluation, each student participant was assigned a code to maintain confidentiality of their personal information.

2.5. Data Analysis

The performance of students in the test was expressed as a percentage. The overall test performance as well as performance in six inquiry skills were analyzed. First, the main features of sample distribution of overall performance in pre- and post-test were analyzed using the basic descriptive statistics. Next, the paired t-test and two-way analysis of variance (ANOVA) were used to address the questions of overall improvement. Finally, for testing the improvement in six inquiry skills the Hotelling T2 test was used. All statistical analyses were performed in R [117,118].

3. Results

The overall score of pre- and post-test basic descriptive statistics are shown in

Table 4. Descriptive statistics of pre- and post-tests scores.

		Pre-Test (%)				Post-Test (%)
		Mean	SE	Std. Dev.	Median	Mean	SE	Std. Dev.	Median	Mean Gain
Overall		27.12	0.32	15.25	25.00	35.11	0.36	17.30	32.14	7.99
Gender	Boys	30.18	0.54	16.63	27.68	39.69	0.59	18.21	37.50	9.51
	Girls	24.96	0.38	13.80	23.21	31.86	0.43	15.85	28.57	6.90
IAN	≤5	27.10	0.41	15.21	25.00	34.99	0.47	17.44	32.14	7.89
	>5	27.16	0.50	15.33	25.00	35.27	0.56	17.10	32.14	8.11

Std. Dev.—standard deviation, SE—standard error of the mean.

and its distribution is represented in Figure 3 and Figure 4.

To study the relative improvement, the difference between post-test and pre-test scores was calculated and analyzed. The distribution of overall improvement is shown in

The average improvement between pre-test and post-test is 7.99%. For the whole sample the mean score has shifted from 27.12% to 35.11% (median shifts from 25.00% to 32.14%). A paired t-test with the alternative that the performance is greater for post-test showed this shift to be statistically significant (t = 22.89, df = 2306, p < 0.001) with a 95% confidence interval (CI) (7.41, 100). The Cohen’s d for this improvement is 0.48, which means moderate effect [119].

Considering the question of the effect of gender and IAN on the student’s gain a two-way ANOVA was conducted. The normality of the residuals was rejected; however, ANOVA is known to be robust to this assumption as long as homogeneity is not rejected. Nevertheless, using Levene’s test the homogeneity was also rejected (test statistic = 3.19, p = 0.02); to deal with heteroskedasticity we used White-adjusted heteroscedasticity corrected standard errors. Moreover, we checked these ANOVA results with a permuted Wald-type statistic, which allows non-normal error terms as well as heteroscedastic variances [117]. Gender is a statistically significant factor affecting the total gain (test statistic = 11.52, p = 0.0009), while IAN is not (test statistic = 0.058, p = 0.81). Gender and IAN do not interact significantly (test statistic = 0.92; p = 0.33), which means that the IAN increases the gain for boys in the same way as for girls.

Boys have higher mean gain comparing to girls, however, the coefficient η² being equal to 0.004 indicates that it is only statistically significant without practical importance.

Finally, we considered improvement in six inquiry skills given in Table 2. The results are summarized in

Table 5. Results of difference between pre- and post-test in six inquiry skills.

		Mean (SE)	Std. Dev.	95% CI
				Lower	Upper
Overall		7.99 (0.35)	16.75	7.30	8.67
Inquiry skill	Formulate hypothesis	0.27 (0.64)	30.64	−1.41	1.96
	Design experiment	6.34 (0.52)	24.96	4.97	7.71
	Transform data	7.56 (0.81)	38.93	5.42	9.70
	Determine relationship (based on tables)	12.05 (0.91)	43.66	9.65	14.45
	Determine relationship (based on graphs)	8.73 (0.86)	41.24	6.47	11.00
	Determine accuracy	14.59 (0.88)	42.38	12.26	16.92

and in Figure 5 the mean gain is shown. The Hotelling T² test confirms a statistically significant improvement in all inquiry skills together (T² = 605.24, df₁ = 6, df₂ = 2301, p < 0.001). In addition, a paired t-test with Bonferroni correction also revealed an improvement in every considered inquiry skill except the skill Formulate hypothesis (Figure 5).

Here we present and analyze the results of one test item aimed at the skill Formulate hypothesis (

Table 6. Example of test item focusing on formulating hypothesis with its evaluation.

Marie wondered if the earth and oceans are heated equally by sunlight. She decided to conduct an investigation. She filled a bucket with 1 kg of soil and another bucket with 1 kg of water. She placed them so each bucket received the same amount of sunlight. Which of the following options represent possible hypotheses that Marie could test to get the answer to her question? 1
	pre-test	post-test
(a) The longer the soil and water are in the sun, the warmer they become.	4.6%	4%
(b) Water and earth are warmed differently by the sun.	16.8%	14.1%
(c) Different amounts of sunlight are received by water and soil at different times of the day.	2.9%	2.6%
(d) How are water and soil heated in sunlight?	7.3%	9.1%
(e) Water and earth are warmed equally by the sun.	8.7%	9.1%
(a) and (b)	18.1%	13.2%
(b) and (e)	10.4%	12.8%
(b) and (c)	6.5%	4.2%
(b) and (d)	6.0%	8.6%

¹ Question is modified from the original TIPSII test item [113].

). Its main goal is to select all possible hypotheses that enable finding the answer to the presented research question. This test item has two correct answers ((b) and (e)). The average pre-test score reached the value of 23.12% while in the post-test students reached 24.47%. This result shows a low initial level of students’ achievements and almost no practical improvement. This fact can also be seen in the frequencies of selected answers that did not change significantly in post-test compared to pre-test.

All the offered answers involve key words such as water, earth, sun and also temperature and time, which can be found in the answers, even though they are not mentioned explicitly. Students were expected to decide what relationships between which variables they should observe in order to answer the research question.

The answers express certain experience from everyday life. Nevertheless, some students were not able to choose the answers that represent the hypotheses clearly linked to the research question. This is the case of the (a) answer (it is a hypothesis but not corresponding with the research question) that was selected by a fairly large number of students in combination with the correct (b) answer in both pre- and post-test. We assume that the frequency of the (a) answer choice was influenced also by the fact that this option was clearly and simply formulated and understandable for many students and moreover, it was connected with the (b) answer. Some students did not distinguish between a hypothesis and a research question. This fact can be seen in selecting answer (d) separately or in combination with the correct option (b). The selection of a single correct answer ((b) or (e)) instead of both two correct answers ((b) and (e)) that were mutually exclusive indicates that students may have believed that it is sufficient to test just one of these hypotheses.

4. Discussion and Conclusions

The aim of this study was to assess the effect of IBSE on the level of inquiry skills development. The results show that the implementation of IBSE has an impact on the development of inquiry skills. Implementing IBSE through well-designed inquiry activities across several STEM-related disciplines enhanced by digital technologies and formative assessment tools delivered by educated teachers in this field proved to be effective with regard to inquiry skills development. Having a closer look at the individual research questions we can draw the following conclusions that are discussed below.

4.1. Effect of IBSE on Inquiry Skill Group Development

As the detailed analysis of test results shows, there is a statistically significant improvement in students’ inquiry skills of almost 8% (Table 4). This improvement corresponds to an effect size (Cohen’s d) of 0.48 that means that the experimental intervention has a medium effect on inquiry skills development [119], i.e., the improvement has medium practical significance. These benchmarks given by Cohen in 1969 are used widely across social sciences. Recently, mainly for educational research, Kraft [120] revised these benchmarks and proposed 0.2 as the lower bound for a large effect size. However, according to the research design and several limitations of our study, the interpretation of the given effect size according to Cohen was opted for.

This practically significant improvement is in line with various studies on the effect of IBSE on inquiry skills, however, they mainly focus on the implementation in a single STEM-related subject (physics, chemistry, biology, ecology, mathematics) or integrated science [121,122,123,124,125,126,127,128,129,130]. Inquiry skills improvement was identified when implementing a virtual physics lab [128] or simulation-based inquiry instruction in chemistry [129] that also confirms the role of digital technologies in IBSE. Unlike the aforementioned studies, this research investigates implementation of IBSE across several STEM-related subjects. The question is whether the subject difficulty might affect the gained results. It was assumed that the inquiry approach to teaching and learning was crucial, and the subject content did not play a determining role in this case. The inquiry activities in all subjects were designed based on the agreed framework to target the development of inquiry skills in all subjects concerned. Results of our previous study aimed at consistent implementation of IBSE across three STEM-related subjects (mathematics, physics and informatics) with a much smaller research sample of 300 students show a statistically significant improvement of 7% [19]. This is in line with other similar studies aimed at STEM education. Consistent STEM approach to education provides opportunities to solve problems that require the use and therefore also the subsequent development of inquiry skills. As suggested by Pahrudin [131], STEM integrated with answering questions and solving problems can improve scientific literacy, creativity and learning outcomes. All these studies including this research show that inquiry skills development needs systematic and consistent implementation of IBSE, and that without targeted inquiry teaching students’ inquiry skills stay underdeveloped.

Another question that may emerge in discussion is the achieved value of inquiry skills improvement. Apart from the above mentioned facts the research design also took into account that teachers and their pedagogical content knowledge are crucial for the success of IBSE [99,104,132]. Respecting this important fact, teachers completed educational programmes aimed at IBSE and the principles of instructional materials design. On the other hand, even though the online feedback was collected after each experimental lesson, we had no direct influence on whether teachers actually implemented the inquiry activities applying IBSE strategies in a consistent way as designed in the instructional materials. We assume that there is still some open space for teachers to enhance their motivation towards IBSE and their skills to implement IBSE successfully, and also with regard to digital technologies as well as formative assessment tools. As emerged from the discussions with teachers who participated in the research, teachers expressed their need for ongoing support to improve inquiry outcomes. This is in line with the outcomes of other similar studies [106] which suggested that teachers’ professional development can be conducted in many ways, such as inquiry workshops, curriculum development experience, mentoring or peer collaboration. Moreover, Marshall and others [104] even stress that teachers need to devote time to lesson planning and regular reflection on their own practice. Reflecting the results of this and other research, we followed up with regular teacher training webinars as well as meetings dealing with IBSE implementation.

Implementation of IBSE is a complex and long-term process that requires systematic and ongoing support of teachers in stimulating and developing their pedagogical and professional development. Moreover, it needs changes in educational policy also including the curriculum design that should reflect IBSE approaches connected with the content, learning goals, teaching and learning strategies as well as the assessment. For science education, in particular, effective classroom organization with a smaller number of students and science laboratories with appropriate equipment are also important factors for success.

4.2. Effect of IBSE on Selected Inquiry Skills Development

In addition to the analysis of overall test results, the study considered improvement in six inquiry skills (Table 2). The results show significant improvement in every considered inquiry skill except the skill Formulate hypothesis (Table 5, Figure 5). Larger improvement was identified in the skills Determine accuracy, Determine relationships based on tables and graphs compared to the skills Design experiment and Transform data to standard forms (i.e., tables or graphs). Three skills representing the highest, lowest and medium improvement are discussed below.

The skill of Determining accuracy is one of the important inquiry skills when conducting experiments and collecting experimental data. Development of this specific skill does not need to be the center of attention of each of the STEM-related subjects. On the other hand, experimental inquiry activities in physics, chemistry or biology offer many opportunities in this field. Nevertheless, as also presented in the research of Čipková and Karolčík [133], students of biology had difficulties in identifying the accuracy of experimental data and possible sources of error that distort the results of an experiment. For the development of this skill students need to be directly involved in manipulating the measuring device or equipment and in data collection. Balanay and Roa [134] in their study report that a student-centered approach and particularly inquiry-based learning in conducting science experiments helps to develop students’ scientific skills involved in recognizing possible errors. The results of these studies correspond with this research showing a low initial level of development of 24.47%, however, this research study identified the highest improvement here of almost 15%. This result confirms the fact that targeted IBSE teaching and learning help to develop inquiry skills. Students in this research were likely exposed to activities involving identifying possible sources of errors across several STEM-related subjects, which can be considered as the main reason for the students’ large progress in this skill.

The inquiry skill Formulate hypothesis can be classified as a higher-order inquiry skill. The low initial level of its development of 21.29% as well as no practical improvement (only 0.27%) after the IBSE implementation suggests that the development of this skill may not be sufficiently stimulated at school. Inquiry activity usually starts with the research question to which the hypotheses should be formulated. However, students at this point may still have limited knowledge about the investigated phenomenon and when formulating a hypothesis, they often apply only their intuition. Another reason may be found in the fact that the implemented inquiry activities are predominantly designed at the confirmation or guided inquiry level where the teacher helps students a great deal and far fewer activities are based on a more open inquiry level with higher level of students’ independence. As a result, students are not only unable to formulate hypotheses independently, but they cannot even select the correct answer from the offered options. Some students also mix the research question with the hypothesis. The achieved results are in line with other similar studies. Kadir, Lucyana and Satriawati [135] report that even after two cycles of teaching and learning based on an open inquiry approach students still have difficulty in developing hypotheses and drawing conclusions. A similar outcome is presented by Arantika, Saputro and Mulyani [126] who found that even though learning is done by the inquiry model through activities on formulating hypotheses, students still have difficulty in this skill. As also argued by Zeidan and Jayosi [136], formulating hypotheses belongs to integrated process skills that are more difficult to improve compared to other skills.

The skill of designing experiments involves identifying variables (independent, dependent and control variable) connected with the research question or testing the corresponding hypothesis. When designing experiments students need to be clear which variables to measure or observe, what variable to manipulate and what variable(s) to control [137]. Nevertheless, many studies show that students have difficulties in designing experiments. If the research question or hypothesis does not include variables named explicitly, students usually fail to identify concrete variables that they could measure or observe [138]. Sometimes they vary too many variables not realizing that only one variable can be changed in order to see its effect [139]. As suggested by Hmelo and others [140], students need help to overcome the mentioned difficulties. They need to be guided in the processes of inquiry learning, i.e., when designing experiments students are provided with scaffolds in the form of guiding questions and hints that lead them through gradual experimental design as well as appropriate variables selection [141]. Research by Ganajová and others [97] points to the importance of formative assessment integrated in IBSE for designing experiments’ skill development. The research involving experimental and control group monitoring six inquiry skills resulted in an interesting outcome where the experimental group achieved a statistically significant difference just in the designing experiments’ skill. The initial level of development of designing experiment skill in this paper’s research was 32.35% with a significant improvement of 6.34%. This result suggests that the implementation of IBSE helped students to improve this skill. As presented earlier, the implemented inquiry activities were designed mostly at the confirmation or guided inquiry level. It means that students were provided with detailed guidance in the form of hints and questions that students answered before moving to another step, and similarly in the experimental design stage of the experimental activity. Moreover, the activities were enhanced by formative assessment tools.

The results from this research complement and strengthen the results of the above-mentioned studies. It strongly suggests that IBSE at the confirmation or guided inquiry level can help not only in the Designing experiments skill but is effective in all other monitored inquiry skills with a demonstrated practical improvement.

4.3. Different Factors Influencing the Effect Size

In PISA testing within the period of 2012–2018 [55,56,57] a statistically significant difference between the learning outcomes of boys and girls in Slovakia was identified in favor of boys, however, the average gender gap for OECD countries has decreased over this period. Looking at the OECD countries, in 2018 boys achieved higher results in mathematics than girls. On the other hand, girls outperformed boys in science. These gender gaps identified in PISA 2018 were very small and girls and boys showed similar levels of performance [57].

Considering inquiry skills development, some studies also indicate differences between boys and girls. As reported by Wang, Guo and Jou [128] differences were observed in the development of scientific inquiry skills during the experimental course between students of different genders. Boys performed better in process skills and comprehensive skills, while girls performed better in learning attitude and communications.

In this research we identified a statistically significant difference in the improvement of inquiry skills between girls and boys in favor of boys, however this difference proved to be at the borderline of practical significance. This result is in line with the trends of narrowing the gender gap identified in PISA testing [57] and other studies [59] over the past decades.

When considering the effect of completed inquiry activities, the results show that it practically did not affect the gains. As discussed previously, we see still significant potential for improvement in teachers’ professional competencies and their pedagogical mastery [132].

Beside this, number of inquiry activities increases the gain for boys in the same way as for girls. We can conclude that IBSE based on an inductive and constructivist approach to learning can be considered as an appropriate model of education in STEM-related subjects, as also stressed by Harlen [93] who claims that learning through scientific inquiry should be part of the experience of all students without any preferences.