An Experimental Study on the Effectiveness of Students’ Learning in Scientific Courses through Constructive Alignment—A Case Study from an MIS Course

Abstract

This study aimed to explore students’ effectiveness in scientific courses that have adopted the framework of constructive alignment. The researchers conducted an experimental study in the education sector to compare two different teaching models—traditional and following constructive alignment—and used statistical tools to analyse differences in students’ learning effectiveness. The course “Management Information System” (MIS) was specifically chosen to investigate how constructive alignment initiatives used in the course influenced students’ learning effectiveness. Two groups of students were selected as the control group and the experimental group, respectively. In the experimental group, the intended learning outcomes, teaching and learning activities, and assessment tasks were always aligned by the instructor. The learning effectiveness of the two groups was evaluated by the Course Experience Questionnaire Survey and academic grades. After this study, it was found that significant improvement—in terms of students’ learning experience scores and academic grades—was seen in the experimental group compared with the control group. This study has further verified that implementing a constructive alignment template can significantly improve students’ learning effectiveness in scientific courses, hence providing theoretical and practical references for teaching and learning in scientific courses.

1. Introduction

With more countries and regions entering the era of massification of higher education, more attention is being paid to the teaching quality of institutions. Good teaching is believed to help cultivate students’ high-level cognitive skills that elite students can spontaneously use [1]. However, students attending universities today are no longer only the academic elite; they share classrooms and study spaces with students who are of different statuses (p. 192, [2]). Therefore, higher education institutions demand paradigmatic and epistemic change brought about by transformative learning [3,4]. Previous studies have also argued that in the process of teaching and learning, learning outcomes should be the product of students’ activities and experiential processes rather than teachers’ instruction alone; students should construct knowledge and meaning initiatively [5,6]. Although the university teaching model is gradually shifting focus from instructors’ teaching to students’ learning, there are still many obstacles. Much empirical research remains at the institutional level and lacks specific analysis of the characteristics of the curriculum [7,8].

In response to this shift, John Biggs proposed constructive alignment principles for higher education in 1996 [9]. Constructive alignment aims to provide the majority of students with a highly challenging and supportive academic environment [2]. The framework involves two aspects of theoretical connotations. First, from the instructors’ perspective, alignment is manifested as an instructional design with intended learning outcomes, teaching and learning processes, and assessment tasks. The teaching and learning activities, designed by instructors to support and guide students with the intended learning outcomes, can be proven to be efficient through students’ participation in later assessment tasks [10]. Second, from the students’ perspective, learners are expected to follow the constructive approach to learning; they are to actively construct knowledge and meaning by changing their cognitive thinking modes to assimilate new knowledge, information, and methods [11]. Therefore, constructive alignment principles focus on the following questions in the instructional design of the curriculum: what are the specific learning outcomes that students need to achieve in their learning process? To achieve these intended outcomes, what learning activities and learning experiences should students have? How do teachers determine if their students have learned effectively?

Biggs focused on constructing a student-centred learning environment using the framework of constructive alignment principles—including presage, process, and product—without deliberately discriminating between humanities and social sciences in terms of curriculum nature and models of teaching and learning. It is generally believed that in humanities and social sciences courses, an autonomous learning atmosphere for students is more likely to form with the involvement of learning outcomes. Additionally, the teaching and learning methods can be more diversified [8,12]. Conversely, due to the nature of their curriculum, scientific courses tend to place more emphasis on deterministic knowledge results. Billett [13] points out that teaching and learning are commonly regraded as synonyms; the absence of teaching would impact the quality of the learning experience. Based on the information above, this study intends to explore the effectiveness of students’ learning in scientific courses using constructive alignment principles. This study conducted an experimental study in the education sector to compare two different teaching models—traditional and following constructive alignment—and used statistical tools to analyse differences in students’ learning effectiveness. The results provide theoretical and practical references for teaching and learning in scientific courses.

2. The Framework of Constructive Alignment Principles

2.1. Intended Learning Outcomes

The first step of constructive alignment is to devise intended learning outcomes [14]. University-level dialogues have increasingly focused on students’ learning outcomes, involving students’ skills, competencies, continuous improvement, retention, etc., [15]. Central to the idea of intended learning outcomes is the belief that education should be planned around the competence that students intend to develop rather than the content that teachers intend to teach (p. 2295, [16]). Biggs and Tang proposed the Structure of Observed Learning Outcomes Taxonomy (SOLO Taxonomy, 1982) to promote students’ learning from surface to deeper levels and clarify learning outcomes for students’ understanding levels. This taxonomy was based on Bloom’s taxonomy of educational objectives. It divided students’ understanding levels into five categories from low to high: prestructural, unistructural, multistructural, relational, and extended abstract. Accordingly, it is important to wisely select suitable verbs to describe students’ future competencies in a subject related to developing the intended learning outcomes [17]. Corresponding verbs reflect the gradation of setting learning outcomes from lower-order to higher-order thinking, demonstrating the increasingly challenging process of thinking.

With this system, previously randomly used verbs with relatively vague meanings—such as comprehend, appreciate, know, understand, recognise, and others—are replaced by verbs with more explicit meanings, such as identify, analyse, apply, interpret, compare, critique, or evaluate. Intended learning outcomes composed using these verbs can better reflect the specificity of intended learning activities and the richness of knowledge content. Furthermore, these learning outcomes could also help students to answer the question of what they should be able to demonstrate at the end of learning experiences. These student responses, in turn, provide enlightenment for educators, enabling them to conduct more student-centred teaching practices as encouraged by constructive alignment (p. 125, [18]).

2.2. Teaching and Learning Activities

When intended learning outcomes are established, teaching activities need to be consistent with the intended learning outcomes to ensure that they are directly related to learning. Constructivism implies students’ ability to actively construct knowledge and meaning, form opinions and connections, and develop skills and abilities during learning. Non-elite students in college classrooms show relatively passive coping mechanisms and exhaustive manners of learning when compared to their elite counterparts’ learning initiatives. Therefore, teaching and learning activities in college should provide richer learning environments and more organised activities to enable students to stimulate their active learning and constructive consciousness. According to Hailikari [11], formative teaching assessments that require students’ active participation clearly encourage students to adopt more sophisticated learning methods; meanwhile, the opposite is true for more traditional courses (p. 1).

Thus, students’ positive behaviours when making learning decisions are more important than teachers’ behaviours [19]. The term “teaching and learning activities” is adopted by constructive alignment because it can better demonstrate the relationship between students and teachers in the classroom [1]. With constructive alignment as the basis, teaching and learning activities have promoted the construction of teacher–student learning communities. In this learning environment, during the teacher–student interaction, teachers give students timely feedback according to the different dimensions and standards of observation. They adjust students’ proficient learning content and intended learning outcomes to provide students with enlightening and challenging learning tasks, enabling them to conduct their learning innovatively. Simultaneously, some view these learning interactions as formative assessment processes that help students improve their own self-regulated learning through assessment [20].

2.3. Assessment Tasks

At the final stage, assessment tasks of a constructive alignment model must be consistent with the teaching activities and intended learning outcomes. Sally Brown [21] has pointed out that assessment tasks must be authentic to meet the corresponding intended learning outcomes and should relate to one or several of the outcomes directly [22]. Additionally, the assessment tasks during the learning process should be more engaging than the post-instructional assessment. From the perspective of students, the assessment should define the actual content of the curriculum and what teachers intend to teach (p.187, [23]). This view corresponds with Elton’s belief that the assessment, rather than curriculum objectives or curriculum content, determines students’ learning content and method [24]. From the teachers’ perspective, the intended learning outcomes are seen as the central pillars of the aligned teaching system, and the summative assessment is the final stage of teaching–learning events. This viewpoint differs from the perspective of students, who hold that the assessment tasks begin with the teaching activities [14].

Many college instructors are still accustomed to norm-referenced assessments when choosing or designing a final assessment, signifying that education is selective. Students’ results are always compared to those of their peers and the normally distributed curve of students’ grades. Alternatively, within the concept of constructive alignment, a criterion-referenced assessment aligned with the intended learning outcomes is identified as the most suitable assessment method. This style of assessment can also help students make positive, meaningful choices that help them enhance their learning [25].

3. Literature Review

3.1. Constructive Alignment

The literature related to constructive alignment involves a range of professional disciplines [1,3,4,6,8,26,27], because the purpose of constructive alignment is to promote the improvement in students’ learning effectiveness across different courses. Although the importance of constructive alignment has been widely recognised, there are few studies on the effectiveness of its implementation. Many of these studies are purely theoretical, focusing on topics such as assessing learning outcomes in interactive learning spaces [28], exploring the relationship between constructive alignment and different criteria levels [29], discussing the international vision of intended learning outcomes in programmes [30], or examining the different theoretic models of constructive alignment in different contexts [1,31]. However, some of the other research studies directly explore the application of constructive alignment in scientific courses [32,33,34].

3.2. Empirical Research on Constructive Matching

Many countries, such as Malaysia, have already begun implementing empirical research on constructive alignment. However, most of the implementation in these countries remains at the institutional level, for example, studying the matching extent of the curriculum systems and graduate capabilities of institutions [22,35]. From the micro-level, there are other various types of research. The constructive alignment model was used in Harvey and Kamvounias’s study [36] to investigate how the content of subject outlines could be consistent with graduate attributes. Teachers’ personal experiences were examined to determine the achievement degree of teaching assessments and intended learning outcomes. However, before implementing this operation, subject coordinators were asked to “tick” some suitable learning outcomes from a list related to the graduate attributes. This led to the similarity of many choices about subject learning outcomes and a lack of analysis of subjects’ characteristics. This task also failed to distinguish humanities and social sciences subjects from scientific courses. In Sumison and Goodfellow’s study [37], a detailed constructive alignment template was adopted to map the relationship between curriculum and generic skills required by students in the Bachelor of Education programme. This study was in depth and meticulous, encouraging academic staff to reflect on their teaching practice. However, the researchers did not consider students’ learning outcomes. Additionally, only education courses were studied, while scientific courses were ignored in the scope of research. Furthermore, Simper [29] used thematic analysis to discover some troublesome aspects from the perspective of academic staff, including instructor expectations, lack of consistency, differentiation of performance, and more. Contributing to this research, the present study focused on assessment thresholds of constructive alignment and differentiation of standards. However, it still lacked a holistic exploration and the perspective of students.

3.3. Research on Constructive Alignment from the Perspective of Students

Treleaven and Voola [38] studied the effectiveness of the implementation of constructive alignment from the students’ perspective. The researchers integrated graduate attributes with a post-graduate course in marketing, using the matrix map to record the matching degree of graduate attributes with intended learning outcomes, learning activities, and assessment tasks. Students’ grades and feedback data (both quantitative and qualitative) were used to evaluate the effectiveness of the study. However, this study still had limitations, such as only having an experimental group without a control group, which resulted in the unclear causation of “changes of students’ performance and satisfaction” with constructive alignment implementation [38]. In another study, Higgins et al. [39] used the online questionnaire to investigate the experience of level-5 students in Research-informed Teaching (RiT) activities. Based on the analysis of the questionnaire, it supported the effectiveness of this RiT activity within the curriculum in terms of student learning and research skill development. However, this study did not consider the adoption of standard experimental methods.

3.4. The Application of Constructive Alignment in Science Courses

In contrast, there is a lack of empirical research on the impact of constructive alignment on students’ learning effectiveness in scientific education. Brabrand and Dahl [40] used the method of comparison to investigate the proportion of learning outcomes with different understanding levels in SOLO taxonomy when setting intended learning outcomes for courses such as computer science, mathematics, and other natural sciences. However, this study did not analyse the achievement effect of different learning outcomes. Later, Maxworth [41] used an extended constructive alignment model in an engineering electromagnetics course to establish three levels of learning outcomes and evaluation methods. He elaborated on the implementation paths and advantages of this study simultaneously in detail. However, this model lacked rigorous data verification. In a study by Gynnild et al. [42], researchers matched the project exercises with the tasks of the final examination in science and engineering courses, focusing on the specific problems of concepts and algorithms. Results show that students’ ability to cope with conceptual and calculative questions improved significantly in this study. Although this study gathered observations of specific empirical data, it lacked the discussion of the overall model of constructive alignment. A field study in the second year of a self-retained undergraduate course in the Geology (BSc) programme at Royal Holloway University has proven constructive alignment’s desirability and typical achievement [43]. Obada et al. [34] continuously investigated how science and engineering courses could be taught more effectively in their universities. After the initial deep dive into a few courses taught by the science and engineering faculties of selected Nigerian universities, the researchers adopted a reverse design approach in five case studies. However, two of these did not use an experimental method.

This study, with the reference of the studies above and the acquisition of data and research perspectives, tried to explore the effectiveness of the constructive alignment principle through teaching experiments. The research hypothesis was as follows: compared to the traditional model, implementing constructive alignment strategy in scientific courses can significantly improve students’ learning effectiveness.

4. Methodology

This study adopted an experimental methodology. The course “Management Information System” (MIS) at a tertiary institution in a southern city of mainland China was selected as the implementation case based on the research objectives. The MIS course at this university is a science-oriented, interdisciplinary course. It is offered in the third semester of the undergraduate study, which involves different disciplines, such as computer science management, database technology, operations research, communication, and network technology. This study used the experimental group to explore the impact of implementing a constructive alignment theoretical framework on students’ learning effectiveness in the MIS course.

4.1. Experimental Procedure

Step 1: Select the experimental and control groups. The MIS course is a large course consisting of several small classes from the same grade. This study chose two successive MIS courses as the experimental groups. A total of 108 students (34 males and 74 females) from the class of 2019 served as the control group (3 small classes were combined), and 62 students (18 males and 44 females) from the class of 2020 served as the experimental group (2 small classes were combined).

Step 2: Control the irrelevant variables. To achieve better experimental results, this study attempted to control all variables other than the independent variable (implementation of constructive alignment principle) and dependent variable (learning effectiveness) as much as possible. First, researchers aimed to accomplish this by ensuring the age and proportion of male and female students in the two courses were roughly the same. Secondly, these students primarily originated from the same province in southern China, and their college entrance examination scores fell mainly between the score requirements of attending first-tier and second-tier universities. Thirdly, the courses were taught by the same instructor—who used the same teaching topics, materials, and reference data—and the total amount of instructional and assessment time was 18 weeks. Finally, the assessment criteria of the course were reviewed and verified by the programme coordinator and other academic staff, which helped counter the inconsistency of grading standards during the performance evaluation process.

Step 3: Implement the experiment. In the control group, the teacher adopted the traditional teaching style based on the courseware and supplemented by classroom Q & A, computer practice, and other supplementary instructional methods. There were only two assessments throughout the semester: a mid-term computer experiment and the final test. Students’ final grades were given as a combination of their scores on these two assessments and their usual classroom performance.

Alternatively, the teacher in the experimental group adopted constructive alignment teaching templates. The instructor started the course by introducing and explaining the abstract of curriculum contents, the concrete intended learning outcomes, teaching and learning activities, and all formative and summative assessment tasks; simultaneously, they answered students’ questions.

Step 4: Set up the measurement methods. The objective of this study was to investigate the difference in learning effectiveness between the control group and the experimental group. The primary measuring methods were considered from the students’ perspective with two variables: students’ course learning experience scores and their academic performances. Using both students’ course learning experience scores and their academic performances as learning outcomes variables, the validity of the constructive alignment can be understood not only from students’ subjective experiences but also from the quantified scores. Therefore, to improve the credibility of this research, the qualitative and quantitative analyses are combined.

Step 5: Conduct statistical analysis. The experimental results were analysed and processed using IBM SPSS Statistics 26 so final discussions and the formation of conclusions could be carried out.

4.2. Description of the Experimental Group Variable Manipulation

The research team communicated with the instructor before the experiment. Researchers informed the teacher about the principle and template of the constructive alignment but let the instructor set intended learning outcomes, organise instruction, and arrange assessments by himself.

The core feature of the constructive alignment template is to set and implement the intended learning outcomes. According to Biggs’s SOLO taxonomy, the instructor used “concrete” verbs to describe intended learning outcomes in the experimental group to express cognitive categories and levels that students needed to achieve, such as identify, describe, compare, interpret, generalise, classify, reflect, apply, evaluate, design, create, and more. Here, learning outcomes were divided into three categories: technological, informational, and managemental, which reflect the increasing degree of challenge faced by students in terms of cognitive levels. For example, when the teaching objective in the control group was to “understand the theoretical basis of MIS and to appreciate the process and methods of system analysis and design”, the intended learning outcomes were changed to “elaborate the theoretical basis of MIS and apply the system analysis methods and design into actual situations”. In addition, the instructor also adopted several levels of intended learning outcomes for different stages of instruction. It is argued that, as teaching projects advance from the primary to the advanced levels, the breadth of knowledge used in the implementation of instructional plans would be adjusted accordingly, and the integration of knowledge would also become more purposeful [44].

Compared to the control group, the teaching and learning activities of the experimental group focused more on a student-centred approach, emphasising the formation of a teacher–student learning community and insisting that activities align with intended learning outcomes. In the learning process, the experimental group adopted a case-driven learning method with the help of a flipped classroom. Students were asked to read case materials before classes and formed cooperative learning groups during class time. With the framework of “research skill development, RSD”, teachers guided students in developing research skills during the learning process of each case study [45]. The experimental group also developed simulated projects to align learning with intended learning outcomes. For example, for the intended outcome of analysing the system business process, students were first asked to describe their phase of information processing. Then, the teacher would generalise their language. With the help of the teacher’s adjustment and supplemental elements, students formed the prototype of the business process and completed the flowchart. Because students’ enthusiasm was fully mobilised in this process, they wanted to participate positively. The project achieved the intended learning outcomes of interpreting, analysing, and demonstrating the business needs of enterprises with the help of the integrated use of case analysis, problem guidance, situation design, group project cooperation, and other comprehensive methods under the collaborative community. Earlier studies also investigated the role of cooperative learning environments in improving students’ higher-order skills [46] and enhancing critical thinking and reasoning abilities [47]. Case-based learning with cooperation has also long been proven to facilitate student discussion and improve their advanced cognitive reasoning skills [48]. Teaching and learning activities in the experimental group were committed to providing a learning atmosphere aligned with their intended learning outcomes.

Formative independent and cooperative learning activities were provided as assessment tasks for the experimental group. While the formative assessment focused on teacher feedback on different perspectives and students’ improvement, students’ learning process was also viewed as belonging to the teachers’ assessment process. Additionally, since students would achieve one or several learning outcomes during the learning process, students’ continuous assessment grade was formed by two parts of usual performance (attendance, motivation, etc.) and independent learning and cooperative practice (cases, projects, integrated application, etc.). Teachers focused on students’ achievement of academic tasks rather than just simply grading [49]. The comprehensive case analysis was conducted as the summative assessment. A criterion-referenced assessment was adopted, setting five levels (excellent, good, medium, passed, and failed) according to different evaluative dimensions. The corresponding levels students could attain were determined by their performance scores and were not established in advance based on the normal distribution curve [50].

4.3. Measurement and Statistical Analysis

The MIS Course Experience Questions (online), with reference to the more mature student rating scale, presented questions divided into two parts: basic information and scale testing (5-point Likert scale, see item in

Table 1. Basic information of participating students in the experimental and control groups.

	Gender		Age	Course of Study	Scores of College Entrance Examination		Year of Study	Place of Residence
	Male	Female	(19–20 Years Old)	MIS	Above the Threshold Score of First-Tier Universities	Between the Threshold Score of First-Tier Universities and Second-Tier Universities	The Second Year	Guang Dong	Outside Guang Dong
Control group size	31	73	104	104	15	89	104	104	0
Experimental group size	14	41	55	55	6	49	55	52	3

).

In the basic information section, the questions gathered information on students’ grades, gender, places they took the college entrance examination, and their scores. The scale testing part of the questionnaire contained 16 questions about intended learning outcomes, teaching and learning activities, and learning assessments for students to rate based on the principle of constructive alignment. Each question item was treated independently in the statistical analyses since each had addressed specific elements of the three components of constructive alignment. Therefore, the results of the statistical analysis helped us determine whether the implementation of constructive alignment was effective or not in each specific section.

After obtaining consent from the instructor and class counsellors, the researcher asked students from the control and experimental groups to complete these questionnaires in Wechat after finishing the course. The study ran from September 2020 to January 2022, lasting more than one year for two classes of students. In January of 2022, after the experimental group completed the course, all the collected data were summarised and statistically analysed to determine the difference between students’ experiences in the control and experimental groups.

Additional evidence of the effectiveness of constructive alignment was found in the final assessment score of the two groups of students. Since the overall score distribution of the experimental group was unknown or skewed after implementing the constructive alignment framework, it was an appropriate choice to convert the students’ specific scores into ordinal variables with good robustness. Therefore, the academic performance was assigned as follows: 59 or below = 1, failed; 60–69 = 2, passed; 70–79 = 3, medium; 80–89 = 4, good; 90 or above = 5, excellent. This study used the nonparametric statistical approach of ordinal variables to investigate the differences between the experimental and control groups and the changing trend of the proportion of students at all levels.

5. Results

After excluding five invalid questionnaires, the effective response rates of the two groups that completed the students’ Course Experience Questions were 96.3% and 88.7%. Irrelevant variables, such as age, gender percentage, place of residence, and college entrance examination scores, were controlled for with the participants’ characteristics in Table 1. This study adopted exploratory descriptive statistics based on different groups and tested for normality using the Shapiro–Wilk test (

Table 2. Exploratory descriptive statistics for different groups.

Question Items	Group	N	M	SD	Mdn	Skewness	Kurtosis	S.W. (p)
1. The teacher proposes teaching objectives or learning outcomes at the beginning of the course.	1	104	3.770	0.895	4	−0.234	−0.347	0.287
	2	55	4.040	0.693	4	−0.214	−0.483	0.053
2. The learning outcomes of each class are specific and clear.	1	104	3.680	1.017	4	−0.222	−0.389	0.156
	2	55	4.020	0.680	4	−0.357	−0.412	0.112
3. Students can understand the teacher’s teaching intentions in the learning process.	1	104	3.630	1.025	4	0.389	0.370	0.172
	2	55	4.020	0.782	4	−0.059	−0.759	0.032
4. The teacher encourages class discussion.	1	104	3.580	0.972	3	0.028	0.107	0.258
	2	55	3.960	0.719	4	−0.055	−0.734	0.043
5. The teacher invites students to share their knowledge and experiences.	1	104	3.400	1.010	3	0.389	0.370	0.172
	2	55	3.980	0.828	4	−0.059	−0.559	0.075
6. The teacher returns assignments promptly.	1	104	3.860	0.989	4	0.136	0.198	0.356
	2	55	4.020	0.652	4	−0.089	−0.625	0.033
7. The teacher invites comments on his/her own ideas.	1	104	3.560	1.003	3	−0.334	−0.265	0.254
	2	55	4.040	0.719	4	−0.251	−0.312	0.124
8. The teacher has a genuine interest in students.	1	104	3.760	0.930	4	0.278	0.262	0.122
	2	55	4.110	0.762	4	−0.074	−0.654	0.042
9. The teacher relates to students as individuals.	1	104	3.520	0.924	3	−0.133	−0.343	0.223
	2	55	3.890	0.832	4	−0.254	−0.385	0.087
10. The teacher is enthusiastic about his/her subject.	1	104	3.910	0.936	4	0.112	0.232	0.168
	2	55	4.160	0.739	4	−0.037	−0.725	0.023
11. The learning modules are well organized.	1	104	3.550	0.994	3	0.156	0.226	0.321
	2	55	4.050	0.650	4	−0.178	−0.325	0.134
12. The learning modules are useful in enhancing my understanding of the subject/analytic ability/practical skills.	1	104	3.560	1.032	3	0.088	0.207	0.245
	2	55	4.040	0.693	4	−0.066	−0.456	0.088
13. The various learning modules’ components (e.g., lectures, tutorials, seminars, etc.) are well integrated.	1	104	3.600	0.950	3.5	0.333	0.245	0.243
	2	55	4.050	0.756	4	−0.025	−0.418	0.078
14. The relative weightings of learning modules assessment (e.g., assignments, tests, exams, etc.) are appropriate.	1	104	3.530	1.070	4	−0.185	−0.456	0.093
	2	55	4.110	0.685	4	−0.062	−0.215	0.154
15. The course has adopted an assessment method that combines quantitative scoring and qualitative feedback.	1	104	3.630	0.996	4	0.142	0.332	0.068
	2	55	4.040	0.693	4	−0.055	−0.764	0.036
16. The course assessment can reflect students’ learning process.	1	104	3.540	1.023	3	0.288	0.270	0.164
	2	55	4.110	0.658	4	−0.049	−0.325	0.066

Note: M = mean; SD = standard deviation; Mdn = median; S. W. = Shapiro–Wilk statistic.

). The statistical results show that the kurtosis and skewness of the scores of each question item in different groups followed the normal trend; the p-value of the S-W statistics was mostly greater than 0.05, indicating that the first and second sample groups met the assumption of normality, respectively.

The results in

Table 3. Comparison of students’ learning experience in MIS.

Question Items	Control Group M(SD)	Experimental Group M(SD)	t (* p < 0.05)	df	Cohen’s d
1. The teacher proposes teaching objectives or learning outcomes at the beginning of the course.	3.770 (0.895)	4.040 (0.693)	2.084 *	136	0.34
2. The learning outcomes of each class are specific and clear.	3.680 (1.017)	4.020 (0.680)	2.476 *	148	0.39
3. Students can understand the teacher’s teaching intentions in the learning process.	3.630 (1.025)	4.020 (0.782)	2.634 *	137	0.43
4. The teacher encourages class discussion.	3.580 (0.972)	3.960 (0.719)	2.844 *	140	0.44
5. The teacher invites students to share their knowledge and experiences.	3.400 (1.010)	3.980 (0.828)	3.874 *	130	0.63
6. The teacher returns assignments promptly.	3.860 (0.989)	4.020 (0.652)	1.240	149	0.20
7. The teacher invites comments on his/her own ideas.	3.560 (1.003)	4.040 (0.719)	3.465 *	143	0.55
8. The teacher has a genuine interest in students.	3.760 (0.930)	4.110 (0.762)	2.545 *	130	0.41
9. The teacher relates to students as individuals.	3.520 (0.924)	3.890 (0.629)	2.495 *	157	0.47
10. The teacher is enthusiastic about his/her subject.	3.910 (0.936)	4.160 (0.528)	1.718	157	0.33
11. The learning module is well organized.	3.550 (0.994)	4.050 (0.650)	3.863 *	150	0.60
12. The learning module is useful in enhancing my understanding of the subject/analytic ability/practical skills.	3.560 (1.032)	4.040 (0.693)	3.476 *	148	0.55
13. The various learning module components (e.g., lectures, tutorials, seminars, etc.) are well integrated.	3.600 (0.950)	4.050 (0.565)	3.320 *	133	0.58
14. The relative weightings of learning modules’ assessment (e.g., assignments, tests, exams, etc.) are appropriate.	3.530 (1.070)	4.110 (0.592)	4.151 *	151	0.67
15. The course has adopted an assessment method that combines quantitative scoring and qualitative feedback.	3.630 (0.996)	4.040 (0.513)	2.973 *	145	0.52
16. The course assessment can reflect students’ learning process.	3.540 (1.023)	4.110 (0.658)	4.262 *	151	0.66

Note: M = mean; SD = standard deviation.

show that the average scores of the 16 items improved. After an independent sample t-test, when p < 0.05 except for “the teacher returns assignments promptly” and “the teacher is enthusiastic about his/her subject”, the scores of other items had a statistically significant improvement. The statistically significant Cohen’s d measure ranged from 0.34 to 0.67, with a relatively high effect, indicating a larger difference between the experimental and control groups. The average scores of two items demonstrating no statistically significant improvement also showed a positive direction trend. In addition, the standard deviation of each item score of the experimental group was smaller compared to the control group. This finding indicates that the statistical dispersion of students’ scoring in the experimental group was smaller, and the consistency of course experience was higher.

When deciding how to measure the second learning effectiveness variable, researchers considered that the ordinal variable represented the student’s performance. Therefore, the study adopted the Nonparametric Sum of Ranks Tests (Mann–Whitney Test) to study the changes in the performances of the experimental and control groups.

The statistical results in

Table 4. NPar Sum of Ranks Test Statistics.

	Mann–Whitney U	Wilcoxon W	Z	Asymp. Sig. (2-Tailed)
Value	2577.500	8463.500	−2.586	0.010

show the value of ASM. Sig. (two-tailed) was 0.010, less than 0.05, proving that the academic performance of the experimental group had a statistically significant improvement compared to that of the control group.

To further verify the specific differences between various groups of students in different grades, the study examined the differences between the experimental and control groups in each grade; this was determined using Pearson chi-square statistics after weighting the number of percentages for each grade (

Table 5. Chi-square test for each grade between the two groups.

Comparison of Each Grade between the Two Groups	TOTAL N	Chi-Square Test Statistic	df	Asymp. Sig. (2-Tailed)
Excellent	10	3.600	1	0.058
Good	55	4.091	1	0.043
Medium	63	1.921	1	0.166
Passed	48	0.083	1	0.773
Failed	24	2.667	1	0.102

). The statistical results show that the significant differences between the groups were mainly in the good grade (p = 0.043 < 0.05), while the differences in the other grades were not that significant.

As shown by the sample data in Figure 1, compared to the control group, the experimental group showed a decrease in the proportion of students in the low-scoring band (failing, passing, and moderate) and an increase in the proportion of students in the high-scoring band (good and excellent). Together with the statistically significant increase in good grades, these findings ensure a significant increase in the experimental group compared to the control group.

6. Discussion

The experiment was designed to observe the impact of the constructive alignment initiative on student learning effectiveness. The connotation of constructive alignment is composed of three intentionally aligned elements: intended learning outcomes, constructive nature of teaching and learning processes, and learning assessments. For the specific results of the experiment, this study was interpreted according to these three components. The results show that after the implementation of constructive alignment, the students in the experimental group showed significant improvement in their academic performance and learning experience scores compared with the control group. After controlling for other variables’ influence as much as possible, this result better reflected the effectiveness of implementing constructive alignment and verified the earlier hypothesis proposed by this study.

After examining the results of students’ intended learning outcomes (from item 1 to item 3), the experimental group gave a clear, satisfied response towards learning outcomes and the teaching intentions of teachers. The central element of constructive alignment is setting intended learning outcomes and using Bloom’s and SOLO taxonomies as frames of reference [51]. Previous research has confirmed that using specific verbs for intended learning outcomes can be closely related to learning activities and assessments [15,16,18]. The statistically significant improvement in the scores of corresponding items also showed that students could lead their own learning using specific verbs in their studies, “relating” and “extending” their knowledge and abilities to a higher level.

The curriculum design in the teaching and learning processes included forms of learning, teaching methods, teacher–student relationships, course learning, structures, and other aspects. The primary goal of constructive alignment is to keep teaching and learning activities aligned with intended learning outcomes and form a learning community. This study adopted case and project teaching methods used in the teaching and learning process, hoping to stimulate students to take the initiative in knowledge construction. Simultaneously, formative assessments involving feedback and improvement were added to classroom teaching, strengthening the viewpoint that “assessment activities are the primary learning activities in higher education” [52] and emphasising the importance of integrating formative assessment with teaching and learning activities. Some studies have provided empirical examples of how formative assessment may influence an individual student’s use of self-regulated learning skills or criteria [20,53]. These findings coincide with the empirical results of this study. During the process of cooperating, questioning, commenting, and thinking in the case and project analyses, students expanded their thinking, developed a great interest in learning, showed their unique ability, and were often able to “have an epiphany”. Recent studies also confirmed that perceiving teaching–learning activities as being aligned with intended learning outcomes was associated with enjoyment of the course and usefulness ratings [54]. The significant improvement in the scores of students in the experimental group from items 4 to 14 could further indicate that the teaching and learning process in the experimental group was more constructive than the traditional teaching model of the control group.

Finally, the assessment model of the experimental group tends to be a criterion-referenced assessment, which could better reflect the “qualitative” characteristic of the standard. For example, in the comprehensive case analysis of the final assessment, students can obtain specific grades when they meet the corresponding standards of each assessing dimension [14,25]. The students in the experimental group not only had a better learning experience but also had a statistically significant improvement in their final scores compared to the control group. In addition, the experimental group combined the formative assessment in daily learning with the final summative assessment. Students in this group showed significant improvement in their experience scores on the assessment. Furthermore, because the assessment was aligned with the intended learning outcomes and teaching and learning activities, it also reflected the students’ learning content, which was supported by the significant improvement in students’ scores on the last question item in the experimental group.

7. Conclusions

This study adopted an experimental research methodology to investigate students’ perspectives on their learning effectiveness after the implementation of constructive alignment in an MIS course. Researchers obtained results with statistical significance. According to Bloxham and Boyd [52], the structures and characteristics of courses are the key to supporting and reinforcing students’ ability to take an initiative and sophisticated approach to their learning, and form clear structures as well as the knowledge bases to promote students’ integration of content and expansion of new topics that relate to previous knowledge. The experiment in this study illustrated the effectiveness of implementing constructive alignment by verifying the changes in student scores on the course experiences and final assessment. Through independent sample t-tests, students’ scores on the course experiences in the experimental group showed statistically significant improvements. A nonparametric Mann–Whitney test for ordinal variables and a chi-square test on each grade also confirmed that students from the experimental group improved significantly in their performances. The evidence from both tests verified our research hypothesis, explaining that the implementation of constructive alignment could be productive for students’ learning in scientific courses.

In today’s era of massification of higher education, many students from tertiary education find themselves unsuited to the university environment, lacking confidence in their learning ability and feeling doubtful about whether they will be able to complete their studies [55]. The constructive alignment theory can help improve students’ learning effectiveness by providing an operational framework for science education through the close interrelation of intended learning outcomes, teaching and learning activities, and learning assessment. By verifying the effectiveness of the experimental group after implementing the theoretical framework of constructive alignment, this study provided empirical evidence and insight into the teaching and learning processes of scientific education.

Nevertheless, this study also has limitations. On the one hand, the present study did not consider uncontrollable factors, such as the need to change specific teaching content due to the changing teaching methods. On the other hand, the study only compared the experimental and control groups; it did not consider whether the improvement in students’ academic performance and experience scores was affected by other factors (such as environmental change or professional interest). This limitation also shows that the constructive alignment principle tends to emphasise the establishment of theoretical models, but the factors related to the external environment and emotional perspective have not been considered. In future in-depth research, we can further study the impact of constructive alignment on students’ academic performance in different disciplines to explore whether the results of this study can be extended to other situations. In addition, to make the research more scientific, researchers should control for the interference factors that may be involved in the experiment. In short, there is still a broader and more in-depth research space available for studying the impact of constructive alignment on students’ learning results. Additionally, we should pay more attention to the scientific design of the experimental process and other influencing factors within the experimental group to improve the theory and practice of the teaching process in scientific education.