Adaptation of Kirkpatrick’s Four-Level Model of Training Criteria to Evaluate Training Programmes for Head Teachers

Abstract

Training programmes are evaluated to verify their effectiveness, assess their ability to achieve their goals and identify the areas that require improvement. Therefore, the target of evaluators is to develop an appropriate framework for evaluating training programmes. This study adapted Kirkpatrick’s four-level model of training criteria published in 1959 to evaluate training programmes for head teachers according to their own perceptions and those of their supervisors. The adapted model may help evaluators to conceptualise the assessment of learning outcomes of training programmes with metrics and instruments. The model also helps to determine the strengths and weaknesses of the training process. The adaptation includes concrete metrics and instruments for each of the four levels in the model: reaction criteria, learning criteria, behaviour criteria and results criteria. The adapted model was applied to evaluate 12 training programmes for female head teachers in Saudi Arabia. The study sample comprised 250 trainee head teachers and 12 supervisors. The results indicated that the adapted Kirkpatrick evaluation model was very effective in evaluating educational training for head teachers.

1. Introduction

Much of the research on the effectiveness and development of schools has high-lighted the vital role that head teachers play in making schools more effective and education more successful by carrying out their duties and responsibilities [1,2,3,4,5,6]. The primary role of head teachers at the school level is to promote and support teaching and learning; however, a head teacher must set a multitude of goals to achieve such a broad, all-encompassing aim [7]. Today’s head teachers are tasked with ever-increasing responsibilities and multifaceted roles, which include communication with students, staff and parents, community management and communication, administrative and financial duties, crisis and conflict management, curriculum monitoring and professional development opportunities and evaluations for teachers [7,8].

According to Bush and Glover [9] and Pheko [10], recent studies have revealed that the effectiveness of schools is linked to the leadership training of secondary school head teachers. Consequently, Morrison [11] emphasises the significance of head teachers’ training, highlighting that all candidates for head teacher posts are teachers and, as a result, few will be qualified or experienced in management. Therefore, Owings and Kaplan [12], and Hutton [13], confirm that it is essential for head teachers to receive training and professional guidance to develop their leadership skills and function effectively. Consequently, educational institutions organise training programmes that help head teachers to perform their tasks and responsibilities effectively. The final stage of training is evaluation of the training, which plays a critical role in measuring training outcomes; however, this aspect is commonly either marginalised or ignored [14].

Although evaluation of training plays a vital role in measuring training outcomes, it is a source of frustration for institutions, who may struggle to make sense of the approaches, various requirements and necessary evidence to conduct it [15,16,17]. The inadequate and poorly executed appraisal system may be a barrier to training effectiveness [18]. The lack of awareness of or access to methods and tools for the evaluation process is another possible reason for inadequate evaluations [19]; however, the process of assessment does not need to be overly complicated [17].

This article proposes a systematic and practical approach to evaluating training in the domain of education, specifically, training programmes for head teachers, by adapting Kirkpatrick’s four-level model of training criteria, which is a popular framework used for evaluation of training in business organisations, published in 1959. This article provides a theoretical basis for the proposed adaptation of the model for the evaluation of training for head teachers. In addition, it presents a case study of successful use of this model to evaluate training programmes for head teachers in Saudi Arabia.

2. Background

2.1. Definition and Purposes of Training Evaluation

Many definitions of training evaluation have been used in the literature to describe multiple aspects of evaluation and what it measures [20]. In terms of the effectiveness of a training programme, Brown ([21], p. 820) defines the evaluation of training as “a process that may be used to determine the effectiveness and/or efficiency of instructional programmes”. In terms of its ability to provide feedback, evaluation is defined as “any attempt to obtain information (feedback) on the effects of a training programme and to assess the value of the training in light of that information” [22], p. 16). Moreover, some authors refer to the approach of conducting an evaluation and therefore define it as a systematic procedure for collecting and analysing information from the training process to determine the efficiency and effectiveness of training interventions [23,24,25]. However, the evaluation is used in different ways, with various implications; it occurs at multiple levels, for example, in classrooms, programmes, courses, general education and in institutions [26].

The literature highlights several benefits of training evaluation, one of which is that evaluation is a form of quality control for training [27]. Therefore, evaluation is perceived as an integral part of a continuous cycle of quality assurance that includes the philosophy of the programme [28]. Evaluation is employed to verify the effectiveness of training and, as such, is considered to be the most appropriate method to achieve this, as asserted by Bramley, Cheng and Ho, Tennant et al., Khandker et al. and Farjad [29,30,31,32,33]. Effectiveness is defined as the attainment of a desired target [34,35].

A second benefit of evaluation is in determining the achievement of organisational objectives [21]. Khandker et al. [32] state that the evaluation of training programmes provides a picture of the ability of programmes to achieve their goals, while Gertler et al. [36] note that the results of an evaluation help “to determine whether the training and development programme justifies the cost” ([21], p. 18), which is always a consideration for companies and institutions. Since training for employees places a financial and administrative burden on institutions, they need to know the results of the training to ensure that the financial outlay will be reflected in enhanced performance in the workplace. This helps institutions to prevent wastage and estimate the return on the monetary investment in training of their staff [37]. According to Kirkpatrick and Kirkpatrick [38], training programmes should be evaluated to determine whether or not they should be continued.

In addition, Gertler et al. [36] state that training evaluation is necessary for policymakers to be able to make informed decisions, such as to discontinue inefficient programmes and expand efficient ones. Moreover, evaluation identifies the areas in training that need further improvement and may provide an insight into methods for improving training programmes [18,39]. Therefore, in this regard, Topno [22] and Rampun et al. [39] recommend that training processes must be evaluated to determine the effectiveness of different components of the training and development programme.

2.2. Kirkpatrick’s Model

Kirkpatrick proposed four levels for evaluating training in his model: reaction criteria, learning criteria, behaviour criteria and results criteria. Kirkpatrick’s four-level model has contributed significantly to the enhancement of training assessment thinking and practice. It has shifted the focus of training assessment practice on to the results [40,41] and inspired the development of several other assessment models [40,42,43]. Therefore, most of the evaluation models found in the literature are based on the Kirkpatrick model [43,44,45]. It is the best-known and most widely used framework for classifying evaluation [18,40,46]. Furthermore, it is simple, practical and easy to understand [40], and provides a structure for evaluation. It is a recognised approach and does not require an inordinate amount of time to administer [47]. The Kirkpatrick model has made considerable contributions to the theory of evaluation and its practice [48], as well as being highly cited in academic research [37].

Although Holton [43], and Alliger and Janak [49], have criticised the hierarchical nature of the model, studies and empirical results do not provide sufficient evidence to support this assumption of its hierarchical nature [40]. In addition, this model has been criticised due to the assumption of causality between its levels and its importance in terms of the increasing levels of learning outcome. However, Kirkpatrick and Kirkpatrick [50] argue that levels can be measured in any order without assumption of causality. Despite the criticism of the Kirkpatrick model, it is still widely used and many researchers and practitioners have found it very useful [37,51]. However, several models to evaluate training were proposed, such as Warr et al. [52], the systems approach by Bushnell [53], Kaufman and Keller [42], Holton [43] and Phillips [51]. According to Tamkin et al. [46], some of these can be considered direct descendants of Kirkpatrick’s model, as they adopt much from the original model and extend it either at the front, by including training design or needs analysis, at the back, by adding evaluation of societal outcomes, or, sometimes, at both ends. Some models extended it with the addition of an evaluation of the return on investment, but this can be included within the Results level (Level 4). Therefore, most of the evaluation models found in the literature are based on the Kirkpatrick model [37,44,45]. However, most existing models of training evaluation are very complicated, costly to implement and time-consuming [37].

The four levels of Kirkpatrick’s model are outlined briefly in the next sections.

2.2.1. Level 1: Reaction

This level describes the trainees’ feelings and impressions regarding the training programme [54] and offers information about whether or not trainees found the training programme valuable [55]. The most common way to evaluate trainees’ reactions is the end-of-course evaluation questionnaire [38,55,56].

2.2.2. Level 2: Learning

This level assesses “the extent to which participants change attitudes, improve knowledge and/or increase skill as a result of attending the program” ([50], p. 22). Learning measures are quantifiable indicators for the learning that has taken place after the training programme [55,57] and they are typically assessed by the self-evaluation of participants regarding their learning [55,58,59].

2.2.3. Level 3: Behaviour

This level evaluates how participants transfer the knowledge and skills to the workplace. Positive final results cannot be expected unless a positive behavioural change has occurred. Therefore, it is important to know whether the knowledge, skills and/or attitudes learned in the programme transfer to the job [50]. According to Kirkpatrick and Kirkpatrick [38], at Level 3, the evaluator has to make the decision whether to use interviews or questionnaires or both; however, it is not necessary for evaluation at this level to be elaborate or scientific.

2.2.4. Level 4: Results

This level measures the effect of training on the organisation in terms of reduced cost, improved quality and improved quantity.

3. Adaptation of Kirkpatrick’s Model to the Educational Setting

According to Huber [60], the growing number of professional development programmes delivered in the education sector increases the need for evaluation strategies that are able to measure the effectiveness of training accurately from both objective and subjective perspectives. While schools and school districts often share similar professional development programmes, there is no unified approach to measuring the effectiveness of such programmes. However, one of the models most frequently used to evaluate educational programmes is Kirkpatrick’s model [17,61,62,63]. Although Kirkpatrick’s model was originally developed to evaluate the training process in organisations, it can also be used in the academic context [50].

Many authors have adapted Kirkpatrick’s model for use in academic contexts by determining metrics and assessments geared to specific learning environments, which shows the potential of its use in this area [17,48]. Ruiz and Snoeck [55] explain that Kirkpatrick’s model is applicable to various types of educational programmes and to various national and multinational contexts. As Bewley and O’Neil [62] assert, the Kirkpatrick model has been successfully used for evaluation in many different training and educational settings. Similarly, Heydari et al. [63] confirm that although all models have some deficiencies, the Kirkpatrick model is suitable and has an acceptable performance record for assessing educational programmes.

Of the four levels in Kirkpatrick’s model, the reaction and learning criteria focus on what occurs within the training programme; therefore, they are considered internal criteria. Behavioural and results criteria are seen as external criteria as they focus on changes that occur outside (and typically after) the programme [17]. External criteria are likely to be influenced by factors other than learning, for example economic or organisational contexts [64,65,66].

3.1. Reaction Level

The reaction level can measure one dimension, such as satisfaction, or multiple dimensions, for example training materials, content, delivery methods, trainer, timing, instructional activities and improvement [67]. However, there appears to be some consensus regarding the multidimensionality of reactions [68], and most research related to the evaluation of trainees’ reaction levels has assessed multidimensional constructs [69]. Whereas Kirkpatrick’s original work was unclear about the types of question that trainees should be asked in relation to their reactions and how the reaction concept should be used, Brown, Morgan and Casper, and Tan et al. [67,70,71] support the notion that the nature of reaction is multidimensional, as it measures trainees’ satisfaction levels with all aspects of the training event [56]

Therefore, researchers have proposed broader categorisations for reactions. For example, Lee and Pershing [72] proposed up to 11 dimensions of reaction, which included elements related to the training environment and methods used for the delivery of training. In addition, Phillips [51] identified up to 15 types of reaction dimensions (e.g., reactions to instructional materials, trainees’ motivation to learn, relevance of training and training facilities).

Moreover, the reaction elements reported in the literature include trainee characteristics, training design and work environment [73], training content, methods and trainer skills [74] and the trainer, food, facilities and training material [56]. These categories might include, for example, elements of practicality and comfort, such as whether the training venue can be easily reached, whether the training venue has suitable areas equipped for coffee breaks, whether the classroom is adequately lit and whether it is of a suitable size [14], as well as seating arrangements and physical distractions [75]. Nikandrou et al. [76] include the training objectives, environment, methods, trainer and training content, whereas Ruiz and Snoeck [55] suggest the training topic, the quality of the materials used and the quality of the trainer, and Sitzmann et al. and Saks and Burke [25,77] refer to trainer performance, training environment and training components, such as the content, goals, process, material and design and delivery of training.

3.2. Learning Level

There are three aspects of learning outcomes: skill-based, cognitive and attitudinal [78]. Cognitive learning outcomes are associated with acquisition of knowledge [79], skill-based learning outcomes are concerned with the acquisition of technical or motor skills and attitudinal learning consists of aspects such as goals, motivation and attitude that are connected with the training programme’s objectives [78]. At the end of the training programme, if the participants’ knowledge has increased or they show a different attitude and improved skills, it is considered that learning has taken place and the trainees have gained something from the training [80,81].

3.3. Behavioural Level

This level evaluates the effects of training on the work performance of the participants in their work environment. Methods for evaluating this level include supervisor-based or objective indicators of performance [17,55,66], observation, self-assessment, performance record review and peer review [18,52]. Since this level evaluates trainees’ behaviour, a quantitative study may not be sufficiently flexible for gaining an understanding of complex human behaviour [82]. Kirkpatrick suggested the combined use of quantitative and qualitative methods of investigation for this level when evaluating training in the education sector, examples of which are found in Reference [83]. Regarding the best time to measure behavioural change in trainees, Kirkpatrick [64] recommended that post-training evaluation should be carried out at least three months after the training, although he acknowledges that some participants may not change their behaviour for six months or may change for a while before reverting to previous behaviours. Similarly, May and Kahnweiler [84] suggested that trainees need sufficient time to put their new skills into practice after training. However, Axtell et al. [85] reported that the amount of learning transferred after one month is a strong predictor of the amount transferred after a year.

3.4. Results Level

At Level 4, collecting, organising and analysing data can be difficult and more costly and time-consuming than at the other three levels; however, the outcomes are often very worthwhile for their value to the organisation [17,22,86]. Consequently, results from this level are the most sought after by stakeholders and the most difficult to provide [81].

Since this study is in the field of education, the evaluation at the results level differs from that of the business training sector, since the academic environment differs in certain aspects, such as stakeholders and results; therefore, it is necessary to adapt the process of evaluation to the academic field [87]. According to the Kirkpatrick model, the results level focuses on the measurement of financial benefits; however, influence in education cannot be measured directly in terms of money [88]. Therefore, the results criteria in education might include a wide range of outcomes, such as alumni employment and succession in the workplace, admission to tertiary education, servicing of underprivileged groups, working to promote peace and justice, responsible citizenship, literary or artistic accomplishments and personal and family stability—most of these results benefit both the individual and society [17]. To determine the criteria for evaluating results of training in this study, it is necessary to consider the desired outcomes from the perspectives of multiple stakeholders; therefore, we need to understand who is to benefit from training [44,89]. In this study’s assessment of the head teachers’ training process, it seems that two parties benefit from the training: the head teacher who needs to enhance their leadership skills, and the community surrounding the head teacher, which includes students, teachers and others. In this study, the effect of training programmes on head teachers was evaluated through supervisors’ perceptions of the impact of training on three parties: (1) head teachers, in terms of their personal development, the development of their leadership skills and their enhanced ability to communicate with teachers and the community, (2) teachers who work in trainee head teachers’ schools, since head teachers have an influence on the performance of teachers and play a role in supporting their teachers’ professionalism [90,91,92] by establishing positive conditions that are conducive to teachers collaborating and working to become better teachers [92] supporting their teachers’ professional development, instructional practices [91] and creativity and providing encouragement for teachers [92] and (3) students, since head teachers have an influence on the performance and results of students, as the existing literature on school quality designates the head teacher as the entity responsible and accountable for ensuring the continued academic progress of students [90]. Schleicher [5] states that although a number of studies have suggested that development programmes for head teachers influence student achievement only indirectly, they show that head teachers who participate in these programmes change practices within the school that lead to better learning and teaching and, thus, enhanced outcomes. Therefore, the effect of training programmes for head teachers on the students was evaluated.

Table 1. Adaptation of Kirkpatrick’s model of evaluation of training programmes to head teachers.

Level	Learning in Training Programmes for Trainee Head Teachers	Measurement Instruments
Reaction	Trainees’ reaction to trainers, training delivery and training environment	Questionnaire immediately after completion of training
Learning	Direct measures of learning outcomes achieved by trainees (knowledge, skills and attitudes)	Questionnaire immediately after completion of training
Behaviour	Measures of change in performance of trainees after completing training	Questionnaire that includes both open-ended and closed-ended questions, as well as interviews or observations to collect data from trainees and/or their supervisors three months after the completion of the training programme
Results	Personal development, development of leadership skills, effect on teacher development and effect on student achievement	Interviews with supervisors, conducted three months after the completion of the training programme

summarises the four-level model adapted to training programmes in the educational setting and provides some examples of linking specific instruments and indicators to corresponding criteria.

4. Research Methods

The current study used a convergent parallel mixed-methods design to evaluate training programmes, as mixed methods are an accepted standard approach in human resource development research [93] and Mizikaci [94] recommends that researchers should use statistical analysis alongside qualitative research methods to provide deeper analysis and information when evaluating training programmes. The adapted model consists of four levels, with two of them (reactions and learning) evaluated by quantitative data (a survey) with trainee head teachers. The behaviour level was evaluated by qualitative and quantitative data (open- and closed-ended questions) with trainee head teachers because a quantitative study may not be sufficiently flexible to facilitate an understanding of complex human behaviour or to address sensitive matters [82]. The results level was evaluated by qualitative, semi-structured interviews with supervisors. However, the effectiveness of training programmes was evaluated through trainees’ and supervisors’ perceptions about these levels, thus, there were no fixed measurable approaches to evaluate behavioural change or outcomes in this study.

One of the considerations in the mixed-methods approach is priority. According to Bryman [95] one of the key decision criteria is around which method, qualitative or quantitative, is the principal data gathering tool or, alternatively, should they have equal weight? The balance of methods for this study must therefore be driven by its research question and the philosophical assumptions that underpin it. Hence, the measurement of effectiveness is related to increasing positive behaviour among the trainees of the trainees, which is a positive result of training, as explained in the literature. This was explored through qualitative methods, with the trainees reporting their opinions and perceptions and the supervisors confirming the positive results of the training in interviews. Therefore, this research gave priority to qualitative methods to support this exploration. Johnson and Onwuegbuzie [82] note that for a mixed-method design, the findings must be integrated at some stage. So, the qualitative might inform the quantitative phase or, if the quantitative and qualitative phases are carried out simultaneously, the findings must be integrated at some point. This study started by collecting quantitative data, then, after three months, qualitative data was collected. The two datasets were analysed separately and then the information was integrated for the interpretation of the overall results to provide a comprehensive analysis of the effectiveness of training programmes for female head teachers in Saudi Arabia. This paper is influenced by Makrakis and Kostoulas-Makrakis’ work [96] for the analysis and the bridging of quantitative and qualitative research methods in order to achieve complementarity of the quantitative and qualitative data.

The instruments of the adapted model were designed in two stages before applying them to the study, as follows.

4.1. First Phase

In the first phase, based on the literature and adapted model, a survey was developed to gather information about evaluation training programmes. The survey was designed based on the levels adapted from the Kirkpatrick model (reaction and learning) through close-ended questions, while the behaviour level includes close-ended questions and open-ended questions. The fourth level (results) was performed from the semi-structured interviews. The trainees’ reaction was designed as a multidimensional construct that included 21 elements classified under three dimensions: trainer, training delivery and training environment. The second level comprised seven items asking the trainees about their knowledge, skills and attitudes after they attended the training programmes. The changes at the behavioural level were evaluated three months after the completion of the training programmes, through mixed methods: eight items and four open questions. The items were rated on a Likert scale ranging from ‘1′ to ‘5′ (where 1 = strongly disagree, 2 = disagree, 3 = neutral, 4 = agree, 5 = agree strongly).

4.2. Second Phase

This phase focused on testing the prototype before application. After the model was adapted, its validity was tested through three actions. The first involved determining the constructs to be measured, based on the literature. Second, face validity was assessed by three specialists in training and training evaluation at the Ministry of Education (MOE) in Saudi Arabia. These experts were asked to give their opinions on the questionnaire items and interview questions, especially the items in each concept, and on the importance of each question for measuring its representativeness and suitability in relation to the concept to be measured. After assessing the face validity of the interview questions, the specialists suggested some improvements. These questions were reworded to improve clarity or comprehensiveness, for example the question about the impact of training programmes on the performance of head teachers: in this case the existing question about ‘what?’ and ‘how?’ was supplemented with a question on ‘why?’.

The third action was conducting a pilot study with 20 head teachers undergoing training, who were asked to respond to and evaluate all the items. Based on the results of the pilot study, additions were made to clarify meaning for two questions and one item in the learning level was deleted. The final revised model consisted of 39 items.

This study used Cronbach’s alpha to establish the reliability of measurements involving multiple items. The Cronbach’s alpha for the questionnaire data showed high reliability for all the constructs and also for the internal consistency of the dimensions, confirming that the internal reliability of the survey was very high (

Table 2. Reliability analysis using Cronbach’s alpha.

Level		Number of Items	Cronbach’s Alpha	Cronbach’s Alpha of Questionnaire Data
Reaction	Trainer	7	0.891	0.954
	Training delivery	10	0.893
	Training environment	4	0.813
Learning		8	0.909
Behaviour		7	0.983

). In addition, the correlations across the parts were reliable. Cronbach’s alpha result shows that the adapted model was a good model with high reliability of its internal consistency.

4.3. Methodology and Context of Case Study to Apply Adaptation of Kirkpatrick’s Four-Level Model

4.3.1. Context

The usefulness of the adapted model for evaluating training programmes for head teachers can be illustrated by the example of such an evaluation for female head teachers in Saudi Arabia. Considerable efforts have been made by the Ministry of Education in Saudi Arabia to support training and develop its programmes (which aim to develop the capabilities of head teachers and improve their skills, knowledge and abilities in all fields): it has provided various training programmes for head teachers every year to train them in technical, supervisory and organisational areas [97], and established a number of training centres in every region for its employees, including head teachers, teachers and administrators. The centres’ employees arrange the training process and manage their centre, and each training centre is provided with equipment, tools and training resources [98].

Training programmes are held according to the training needs of head teachers. The Educational Training Department of the MOE has established a mechanism that identifies the training needs of head teachers in two ways. The first is through supervisors’ observation of the training requirements of head teachers and their needs or through an evaluation performance process of head teachers. Supervisors are those who work in a higher position than the head teachers and who supervise and evaluate their work. Second, training needs are identified through head teachers’ requests for training programmes to develop specific skills [97]. After a training need is identified by a training centre, a training plan is prepared, and a suitable trainer is chosen. The training plan varies from one academic year to another depending on different training needs. In 2017/2018, the training plan included programmes on the following skills: research skills, teaching skills, management skills, technical skills and interpersonal skills. The data for this study was collected in the second semester in 2017/2018, during which there were 12 training programmes for head teachers. Therefore, this study evaluated 12 training programmes in total.

4.3.2. Participants

Each training programme was evaluated through the perceptions of the trainee head teachers and their supervisors using the adapted model. The sample comprised 250 trainee head teachers and 12 educational supervisors working as trainers and supervisors for head teachers. All participants were assigned pseudonyms to ensure anonymity. It is important to emphasise that the education system in Saudi Arabia is segregated according to gender in all schools and universities. As this is accepted practice for a conservative Islamic society, article 155 of the Policy of Education in the Kingdom of Saudi Arabia (1969) prevents the mixing of males and females throughout the different stages of their education, with the exception of nursery school. Therefore, this study was limited to female head teachers. Since the model included both qualitative and quantitative data, SPSS software was used to analyse the quantitative data and NVivo to analyse the qualitative data. Thematic Analysis (TA) was the method chosen for analysing the transcripts. TA’s particular popularity originates in its ease of use and accessibility as a research tool [99] that is both theoretically and empirically flexible. The TA process employed for the analysis involved six stages, as described by Braun and Clarke [99]. The first step focuses on familiarity with the entirety of the data; therefore, the recorded interviews were transcribed using Microsoft Word. Step two consists of coding general structural aspects, such as numbering questions and answers, and generating the initial codes. Descriptive coding was used to summarise issues in a word or short phrase, using NVivo software. In step three, the answers to open-ended questions and interviews were coded, with each one analysed separately. Meanwhile, connections between the codes were searched for and, as a result, some codes were renamed, and similar codes merged. In step four, the themes were organised in the order of the search structure. The fifth phase of TA consisted of defining the themes and describing the scope and content of each theme and sub-theme. The final phase of TA involved producing a report to facilitate a discussion of the findings.

Table 3. Overview of demographics of study participants: 250 female head teachers in Saudi Arabia, 2017/2018.

Variable	Classification	N
Age	25–29 years	6
	30–34 years	33
	35–39 years	95
	40+ years	116
Experience	less than 5 years	80
	5–9 years	44
	10–14 years	35
	15+ years	91
Highest qualification	Diploma	50
	Bachelor’s degree	192
	Master’s degree	8
	PhD	0
School sector	Early childhood school	23
	Primary school	108
	Lower secondary school	63
	Upper secondary school	56

provides an overview of the demographics of the participants, while the results are presented in the next subsection.

5. Outcomes from Applying the Adapted Kirkpatrick Model in a Case Study

5.1. Reaction Level

We used questionnaires to measure the reaction level immediately after completion of the training.

Table 4. Descriptive statistics for participants’ reactions to the trainer (250 female head teachers in Saudi Arabia, 2017/2018). SD = standard deviation.

Statement	Mean	SD
The trainer was an effective communicator with trainees.	4.15	0.663
The trainer prepared the scientific material in an appropriate manner and in accordance with the objectives of the training programme.	4.07	0.696
The trainer prepared training activities appropriately and in accordance with the objectives of the training programme.	4.00	0.799
The trainer used appropriate training methods that were compatible with the course objectives.	3.98	0.784
The trainer achieved the goals of the programme.	3.97	0.763
The trainer gave trainees an opportunity to discuss and ask questions.	4.22	0.758
The trainer was able to use available training equipment.	4.14	0.730
Weighted Mean and SD	0.5774	4.076

,

Table 5. Descriptive statistics for participants’ reactions to training delivery (250 female head teachers in Saudi Arabia, 2017/2018).

Statement	Mean	SD
The training took place at a suitable time for me.	3.75	0.915
The subject content in the programme was relevant to my job.	4.24	0.717
The training programme combined theory and practice.	3.83	0.829
The content of the training programme included up-to-date theory and practical information.	4.03	0.765
The material was presented in a manner appropriate to the target group’s training needs.	3.94	0.789
The audio-visual aids were effective.	3.95	0.958
The length of the training programme was suitable and adequate.	4.01	0.894
The handouts provided will help me to meet all my training needs.	3.38	1.177
The training programme was linked to my training needs and my current job tasks.	3.99	0.857
I feel that the programme will help me do my job better in the future.	4.06	0.815
Weighted Mean and SD	0.627	3.918

and

Table 6. Descriptive statistics for participants’ reactions to the training environment (250 female head teachers in Saudi Arabia, 2017/2018).

Statement	Mean	SD
The organisation of the training room was appropriate for the nature of the training, as the distribution of tables was appropriate.	3.70	1.080
The training techniques were appropriate for the training situation.	3.92	0.943
The facilities (such as toilets) were suitable.	3.72	1.173
The services provided (meals, drinks) were suitable.	3.45	1.286
Weighted Mean and SD	0.902	3.696

provide descriptive statistics for the trainees’ reaction level, which included three dimensions: trainer, training delivery and training environment.

The literature confirms the importance of satisfaction with the trainer for achieving effective training [77]. Findings from this study show that among the three dimensions, namely the trainer, training delivery and training environment, the trainees had the highest level of satisfaction with the trainer, with a weighted average score of 4.076. More than 70 percent of participants reported that they strongly agreed or agreed with all the statements related to aspects of the trainer that were evaluated. A number of studies have highlighted the importance of the quality and efficiency of the trainer and their style if the success of the training programme is to be assured. An effective trainer can be very influential and make a difference in achieving training success [100,101] since the trainers play a role in trainees’ learning transfer. Marsh and Overall [102] found that if a trainee liked their instructor, they were more likely to be satisfied and motivated to do better in the course. Therefore, it is clear that satisfaction with the trainer plays a role in the trainees’ transfer of the skills and knowledge delivered through the training programme [103]. The trainer, instructor style and human interaction have the strongest effect on trainees’ reactions [77]. Morgan and Casper [70] also asserted that the trainer is of high importance in trainees’ overall perceptions of the training. This is consistent with the results of this study, which show that the trainees’ satisfaction with the trainer predicts good reactions of the trainees to training. However, three of the participants believed that the changes after training programmes were not positive: for example, #246 reported that “trainers lack efficiency in skills of presenting the training content and in discussion and dialogue with trainees”, #205 believed that “the training programmes lack sufficient preparation by the trainer” and #174 stated that “trainers in training centres do not have training skills”.

The above results show that, in the current study, the reactions of trainees to the trainer were satisfactory. This shows that the trainers were of a good standard and were chosen well by the training centres. This finding is consistent with the findings of Hassan et al. [104] and Yusoff et al. [105] in studies that also indicated that trainees were satisfied with the trainers as they assessed their trainers positively.

Training delivery refers to the training scheduling, the programme length, the training content, the methods of providing training, training equipment and technology resources [25,38,49,51,55,106]. In this study, training delivery ranked second in predicting the satisfaction of the trainees, which indicates its importance and priority in the opinion of the participants. The findings show that the average score for the responses of participants was 3.918, which suggests that the trainees’ opinion of the training delivery was satisfactory.

However, four participants believed that the changes after training programmes were not positive, since only the lecture method was used in the training programmes and they lacked practical application and practice. For instance, #8 believed that “the majority of the training programmes are theoretical and devoid of practice; therefore, they are extremely detached from the practical field and vision of the Ministry of Education”, and #184 reported that “most of the training programmes cannot be implemented practically in the field because they are theoretical”. This corresponds to the finding of Albahussain’s [107] study, that the most popular training methods used by Arab organisations are seminars, conferences and lectures, and that of Albabtain’s [108] study, that the methods used in the training of educational leaders in the Ministry of Education in Saudi Arabia still depend on the lecture. Studies in the Saudi context have indicated that the reasons for the prevalence of the lecture approach in training programme delivery in Saudi Arabia are the prevailing culture that imposes the method of indoctrination, the conviction of senior management regarding the success of traditional methods of training, such as the lecture approach, and the trainers’ lack of ability to use other approaches [108,109].

Although the lecture style is effective in training for many types of tasks and skills [66], it can be an unengaging and ineffective training delivery method [110]. Pashiardis [111] suggested that the most effective training is that which combines different methods, since using multiple methods keeps learners interested, arouses curiosity and leads to enhanced understanding and retention, as individuals learn in different ways. Therefore, using a variety of training methods will increase the likelihood that learners will have been touched by at least one method [56]. According to Gauld and Miller [112] and Browne-Ferrigno and Muth [113], training content should combine theoretical and practical aspects, as well as the transfer of new knowledge and skills, since trainees measure the usefulness of training based on its balance of theoretical and practical content. Similarly, when trainees perceive an imbalance between theoretical and practical training, their satisfaction will generally be low [14]. Therefore, training centres and trainers must consider the diversity of methods in the implementation of training programmes to achieve greater training effectiveness for trainees. All the elements above suggest that correcting the misconceptions of senior management about the effectiveness of the lecture method and recruiting qualified trainers may address this problem. Trainees’ satisfaction with the training programmes offered to them has also been reported by other studies on training and its evaluation. For example, Chang and Chen [114] and Yusoff et al. [105] found that most respondents were satisfied with training delivery and that training programmes were effective.

The training environment is “all about the condition or surrounding of the medium the training programme takes place in” ([115], p. 34). The training environment is key factors responsible for the successful implementation of a training programme is the training environment [115]. It includes the suitability of the physical facilities, equipment, classrooms and accommodation [106,116]. If the training environment is unsatisfactorily prepared, it will impact on the intake of participants or distract them [117], which might influence next year’s intake if potential trainees hear bad reports.

In this study, the results indicated that the participants were satisfied with the training environment, as the average response for training environment was 0.902. However, among the elements assessed at the reaction level, the training environment had the lowest level of satisfaction.

The participants were satisfied with all the statements in this section, with the highest satisfaction relating to the statement that the techniques and tools in the training environment were good and appropriate. This may be due to the fact that the training centres do not suffer from a lack of financial resources to equip them, since they are affiliated with the MOE and are especially for training MOE employees, including teachers and head teachers. Therefore, MOE manages and finances them and provides them with full support and the necessary technical resources and tools [118].

However, one participant (#174) believed that the changes after training were not positive, since the training environment was not clean and was uncomfortable, which negatively affected her attitudes towards the training. The satisfaction of the trainees with the training environment predicts a good reaction to the training, as indicated by the results of this study and other studies. Therefore, if a suitable and comfortable training environment is lacking, this may negatively affect trainees’ attitudes towards the training [50].

Therefore, it is evident that the training environment positively influences the learning outcomes [56,68,106]. Consequently, the training environment’s role is critical in terms of the emphasis and usefulness of the knowledge gained and the training programme’s success [119,120]. This is why the trainees and supervisors highlighted that an inappropriate training environment was one of the barriers to the effectiveness of the training.

The findings on this theme are consistent with those of previous studies [121] in that the trainees believed that the training environment was satisfactory and that the training environment was important in ensuring participants’ comfort during training.

In summary, trainee female head teachers generally reported satisfaction with the training programmes they received through the training centres in Saudi Arabia. Trainees’ satisfaction with the training included satisfaction with the trainers, the training delivery and the training environment, which reflect the efforts made by the training centres in Saudi Arabia to implement successful training programmes.

However, since the training process is continued every academic year and the programmes vary from one academic year to the next, the training content, method of implementation and trainers differ from one programme to another. This means it is crucial to continue the evaluation process for training programmes to maintain the level of trainees’ satisfaction with training.

Table 4, Table 5 and Table 6 show to what extent the trainee head teachers had a positive reaction in this first level of Kirkpatrick’s model. The study results showed that trainee female head teachers had positive reactions to and were satisfied with all three dimensions of the training programmes, since each dimension had a mean score greater than 3.60 out of 5.

5.2. Learning Level

We used questionnaires to measure the learning level immediately after completion of the training.

Table 7. Descriptive statistics for participants’ perceptions of the impact on their learning and knowledge (250 female head teachers in Saudi Arabia, 2017/2018).

Statement	Mean	SD
My knowledge and information developed as a result of the training.	4.17	0.627
Through the training programmes, I learned about some laws, theory and practices and learned information I did not know before.	4.08	0.767
Training programmes provided me with practical skills in my field that I did not have before.	3.98	0.836
Training programmes provided an opportunity for the exchange of new information, knowledge and experiences among participants.	4.24	0.743
Training programmes helped me to succeed in my work in a way that I would not have been able to before.	3.94	0.887
The training programmes motivated me and made me interested in learning more.	4.05	0.800
The training programme has helped to change my attitude towards the topic and training area.	3.98	0.786
Weighted Mean and SD	0.653	4.061

presents the trainee female head teachers’ perceptions of the impact of the training programmes on their knowledge and learning.

The results of the study show that participants believed that their knowledge, information and practical skills had improved as a result of undertaking training programmes as they learnt about some laws, theory and practices, or discovered information related to their job that they did not know before. Moreover, 21 participants and all the supervisors who were interviewed confirmed this result as a positive change after training. One participant (#70) asserted that “the training programmes give us accurate information and a clear explanation of the information and knowledge help us in our work”. A supervisor (#10) highlighted that “the training programmes affect work in many aspects, as they inform the head teachers about work updates”.

The aims of the training are to raise the levels of awareness and understanding of the trainees through the development of skills, knowledge and trends that are required by the institution [122], and to provide trainee head teachers with information and skills commensurate with the nature of their work. Therefore, the belief of trainees and the confirmation of supervisors that training programmes have a positive impact on trainees’ knowledge and learning indicate that the implementation of training programmes for female head teachers has achieved the result at the learning level, which aims to continue their education by providing them with information, management skills and technical and educational theories that better enable them to do their work [123].

5.3. Behavioural Level

The behaviour of the trainees was evaluated three months after completion of the training. At this level, the opinions of respondents were investigated using both qualitative and quantitative methods, via questionnaires with open-ended and closed-ended questions.

Table 8. Descriptive statistics for participants’ opinions on behaviour (250 female head teachers in Saudi Arabia, 2017/2018).

Statement	Mean	SD
The training programmes helped me to organise my role as head teacher more effectively.	4.10	0.769
The training programmes inspired me to improve my achievement.	4.07	0.840
The training programmes increased my ability to perform well in my job role.	4.07	0.831
The training programmes helped me to develop leadership behaviour.	4.16	0.768
The training programmes developed some aspects of my behaviour.	4.08	0.767
The training programmes helped me to prove myself in my work as head teacher.	4.07	0.813
My job behaviour changed after completing the programme.	4.00	0.848
Weighted Mean and SD	0.702	4.077

presents the results of the closed-ended questions.

The introductory part of the qualitative investigation included a closed-ended question that examined whether participants felt that their behaviour change was positive after completing the training course or not. Subsequently, a further in-depth investigation was conducted to determine their reasons for choosing a positive or negative response, and the impact of the changes on them and their jobs.

As shown in Figure 1, 238 participants (95.2 percent) believed that the changes after training were positive, and among the participants who believed that the changes after training were positive, 75 gave reasons for their choice. All the answers were analysed using NVivo Pro 11 software, since there was a large variety of answers. The symbol (N) in front of each value in Figure 2 indicates the sample size of participants who mentioned that.

We divided these changes into two themes: (1) positive impact of the training on the personal development of head teachers and (2) positive impact on their work. The first theme included items such as increased self-confidence, with trainee #42 saying that she felt “self-confident after training” and #49 stating that her “self-confidence increased after training”. Moreover, communication skills with others, such as teachers, students and their parents, were enhanced. For example, one trainee (#75) believed that “training developed our communication skills with staff members and students in school”. Moreover, eight supervisors referred to the development of trainee head teachers’ skills in communicating with others as a result of training. Under the second theme, positive impact on work, the major positive change that the participants noticed in their behaviour after the training and reported in this study was the motivation to transfer learning to their work, with 31 trainees stating that they intended to use and apply the training content in their job. For example, one trainee (#13) stated “I felt motivation for work after the training programme”, and another (#25) asserted that “the training gave me strong motivation to work according to education systems”. They may now be confident in using the skills and aware of where demonstration of the new skills in work situations is appropriate [124], or they believe that the new training knowledge and skills are helpful in solving work-related problems and meeting frequent job demands [124].

Increased motivation to work is one of the most significant results indicating the success of a training programme [125], and it is the main goal for designers of training [126].

In addition, the trainee head teachers believed that the training programmes helped them to perform their duties and responsibilities as head teachers in several ways. First, they explained that training helped them to organise their work, which is necessary since they have many overlapping jobs and responsibilities, such as organising meetings and agendas, discussing protocols of the educational process, following up in meetings and implementing measures [127]. Another aspect that trainees reported was that the training helped them to improve their creativity at work. Creativity is generally defined as the production of new, useful ideas or solutions to problems [128]. Therefore, the trainees believed that the training programmes and what they included, and the exchange of experiences with peers, inspired them to be creative at work and find solutions to the problems they face.

Moreover, ten trainee head teachers believed that training helped them to perform their tasks in the correct manner, which enhanced the quality of their work, and four trainees believed that training facilitated their performance of tasks. The trainee head teachers might be new head teachers in the position and feel that leadership tasks are onerous and difficult; therefore, the training programmes help them by teaching them how to fulfil their role. Finally, two trainees believed that the training helped them to deal with new technology in education, which two supervisors cited as a positive impact of training. The ability to use new technology, including devices, software and electronic technologies, in their work is a skill head teachers should have [129] to facilitate their job performance.

In summary, the responses suggest that most of the respondents believed that training programmes developed their behaviour by improving their skills and enhancing the character traits they need as head teachers.

Since training has a positive influence on the performance of trainees, training-related changes should result in improved job performance of trainees and other positive changes, such as acquisition of new skills, attitudes and motivation [130]. The Ministry of Education’s efforts to train, support and induct head teachers provide a strong foundation to enable them to acquire the required leadership skills and to help them change their behaviour positively.

Conversely, 12 participants (4.8%) did not believe that the changes after training were positive. They attributed this to four barriers that affected positive behavioural changes after training: the limited professional skills of the trainer, the type of training delivery, content of training that includes repetition and lack of content diversity of training and the lack of preparedness of the training environment.

5.4. The Effect of Trainees’ Characteristics on Learning and Behavioural Change

This subsection will explore whether there is a relationship between independent variables (the age, qualifications and experience of participants) and dependent variables, namely change in the learning and behaviour of training participants, by using a stepwise linear regression test. The results on the relationship between the age and change in the learning and behaviour of participants are displayed in

Table 9. Descriptive statistics for the effect of trainees’ age on self-reported learning and behavioural change (250 female head teachers in Saudi Arabia, 2017/2018).

Age	R	R2	F	Sig.
Learning change	0.068	0.005	1.139	0.287
Behavioural change	0.062	0.004	0.954	0.330

.

Table 9 illustrates that there is no correlation between age and learning and behaviour change, since R² has very small values. In addition, the regression analysis is not statistically significant: for learning, F = 1.139 and Sig. (significance level) = 0.287 (>0.05), and for behaviour, F = 0.954 and Sig. = 0.330 (>0.05). This result indicates that the effectiveness of training programmes and self-reported change in the learning and behaviour levels of the trainees neither depends on nor is influenced by the age of the trainees.

Table 10. Descriptive statistics for the effect of trainees’ experience on self-reported learning and behavioural change (250 female head teachers in Saudi Arabia, 2017/2018).

Experience	R	R2	F	Sig.
Learning change	0.044	0.002	0.481	0.489
Behavioural change	0.121	0.015	3.675	0.056

explores the influence of the head teachers’ experience on learning and behaviour after training programmes. Table 10 illustrates that there is no correlation between experience and responses to learning and behaviour, since the R² has very small values.

In addition, regression is not statistically significant, with F = 0.481 and Sig. = 0.489 (>0.05) for learning, and F = 3.675 and Sig. = 0.056 (>0.05) for behaviour. These results show that the number of years of experience of head teachers does not influence the degree of self-reported change affected by training programmes through learning and behaviour.

Table 11. Descriptive statistics for the effect of trainees’ qualifications on self-reported learning and behavioural change (250 female head teachers in Saudi Arabia, 2017/2018).

Qualifications	R	R2	F	Sig.
Learning change	0.114	0.013	3.281	0.071
Behavioural change	0.170	0.029	7.379	0.007 **

** Correlation is significant at the 0.01 level (2-tailed).

explores the correlation between head teachers’ qualifications and responses to learning and behaviour. It confirms that there is a weak positive correlation between trainees’ qualifications and self-reported change in their behaviour after training, since R = 0.170 and R2 = 0.029, which indicates that the level of qualifications explains 3 percent of the variance in behaviour.

Furthermore, the regression is statistically significant, with F = 7.379 and Sig. = 0.007 (<0.05). Therefore, the result shows that the change in the behaviour of participants after training is affected by their level of qualifications while the result for learning is not significant.

Table 12. Regression coefficients (250 female head teachers in Saudi Arabia, 2017/2018).

Model	Unstandardised Coefficients		Standardised Coefficients	t	Sig.
	B	Standard Error	Beta
(Constant)	4.561	0.183		24.874	0.000
Qualification	0.264	0.097	0.170	2.716	0.007

shows the extent to which qualifications affect behaviour.

Table 12 displays the coefficients of regression models, which indicate that the behaviour of participants was affected by their qualifications, since an increase of one unit in the level of qualifications led to an increase in participants’ behaviour scores of 0.264.

Therefore, a positive relationship was found between participants’ behavioural change after training and their qualification level. The more qualified participants were, the more they were able to modify their behaviour after training as compared with those who were less qualified; that is, the trainees’ qualification level appears to affect their impact in terms of the training outcomes. Trainees with a high level of education tend to be more motivated learners and accomplish more [131]; therefore, they are more ready to accept new studies and modern theories and transfer them to the workplace.

The literature indicates that the trainees’ personal characteristics exert an influence on or play a critical role in the level of variance in training outcomes [132]. These characteristics include demographic variables such as age, degree held and experience [133]. Such a difference was not found in this study in relation to age and experience; however, the results of this study agree with the literature indicating that trainees’ qualifications affect training outcomes.

This finding is significant, as it demonstrates the need for training centres and training organisers to intensify their training efforts for diploma holders (the least qualified head teachers) and to encourage these trainees to raise their level of education. Moreover, the finding highlights the need for special, targeted training courses for this group.

5.5. Results Level

The final level was explored through interviews with the 12 supervisors. Figure 3 shows the impact of training programmes on head teachers, their work, their teachers and their students.

Supervisors’ perceptions were based on their observations and supervision of the head teachers, job performance and students’ results and achievements. In interviews, the supervisors discussed in depth the impact of the training on head teachers as the primary beneficiaries of the training process. The first aspect, their personal development in building their leadership skills, promotes building of self-confidence and enhancement of social communication skills. This confirms the beliefs of head teachers, as they reported these positive changes in their responses, as discussed in the previous section. The second aspect raised was the development of leadership by providing head teachers with information about regulations and educational laws, developing their administrative skills and enhancing their professional competence and communication skills. These supervisors believed that the training programmes had satisfactory outcomes at the level of head teachers, as supervisor #4 asserted that “training has a positive effect on the improvement and development of the administrative work of the head teachers, as there is data and information at the end of each academic year to measure the results of training on the work of head teachers, and our data indicates the qualitative and quantitative improvement in the administrative behaviour of the head teachers in the school and in the work as a whole”.

Regarding the impact of head teachers’ training on teachers, only four supervisors reported on how the training of head teachers impacted teachers who work in their schools. For example, one supervisor (#2) reported that “if the head teacher has training, this is reflected in her behaviour with her teachers. She supports them to develop their skills and removes barriers to their success in teaching”.

The impact of head teachers’ training on their teachers was noted through their enhanced support of the performance of teachers by inspiring them, which removes barriers to their success, improves their creativity and encourages them to achieve success in their job. This is consistent with the literature, which emphasises the importance and influence of a school principal on her teachers and their professional performance [90]. Similarly, it aligns with Karaköse’s [134] assertion that the behaviour and attitudes of leadership influence the actions, attitudes and perspectives of staff.

More attention was given to the ways that training of head teachers were reflected in the achievements of their students. Ten supervisors reported that the training reflected positively on the level of students’ results. The supervisors indicated this outcome through evidence drawn from annual evaluations of head teachers and students.

Other supervisors explained how the training programmes had improved students’ achievement; for example, supervisor #4 reported that “the results of the training and the development of the head teachers are reflected in the teachers who push students to attain high and satisfactory achievements”. In addition, supervisor #8 asserted that “for head teachers who undertake training, that is reflected in their school students. We find that their achievements and results are high because after training, the head teachers create a school climate for students to succeed and encourage students to achieve and they have more contact with parents to support students”.

In general, the analysis showed that programmes provided by the Ministry of Education to train head teachers in Saudi Arabia and prepare them for their roles provided a strong foundation, enabling them to acquire the required leadership skills. However, there were some barriers to the effectiveness of training programmes. Therefore, there is a need to identify the measures that can help to overcome these barriers and to design effective training that helps with the transfer of knowledge to the work environment.

6. Limitations

This study encountered several limitations at all levels of the research that must be taken into consideration. Only female participants were asked to respond to the questionnaire and participate in interviews in the study. The reason for the exclusion of male participants is the strict separation of males and females in education environments in Kingdom of Saudi Arabia KSA for religious and cultural reasons. This means that a female researcher could not conduct a study with male participants or carry out interviews with them. Therefore, by focusing on female head teachers, this study represented one view, which is that of female head teachers, whereas male head teachers may have different views of the training process and highlight different issues.

Another limitation is that this study only assessed the impact of individual characteristics of trainee head teachers (age, qualifications and experiences) on training outcomes and did not consider the effects of other factors, such as environmental factors, on training outcomes. Therefore, further research could examine the effects of other factors.

Finally, this study evaluated four levels in the Kirkpatrick model through trainees’ and supervisors’ perceptions about these levels, thus, there were no fixed measurable approaches to evaluate behavioural change in this study.

The above limitations do not affect the results of the study, however, and are highlighted to guide future researchers to achieve improvement in this field.

7. Conclusions

This study adapted the Kirkpatrick evaluation model (published in 1959), which includes four levels of training outcomes, to evaluate training programmes for head teachers.

The case study illustrated that the adapted Kirkpatrick model was able to clarify the trainee head teachers’ and supervisors’ perceptions of the training process outcomes. Moreover, Kirkpatrick’s evaluation model was able to determine the strengths and weaknesses of the training process of head teachers. Therefore, the use of Kirkpatrick’s four-level model to evaluate training programmes for female head teachers in Saudi Arabia was effective, and the results of this study demonstrate how the model can be used in this context to evaluate perceptions of training.

Based on findings of this empirical study using the adapted Kirkpatrick model, it can be argued that this framework provides a more holistic understanding of the key constructs that ensure the effectiveness of training for female head teachers and is a concrete assessment model that can provide insights into the effectiveness of the training programmes for head teachers provided by the Ministry of Education in Saudi Arabia.

The results of this study indicated that the adapted Kirkpatrick model was very effective for evaluating the educational training of head teachers. The adapted model may help evaluators to conceptualise the assessment of learning outcomes for training programmes, using metrics and instruments. The model helps to determine the strengths and weaknesses of the training process. It gives educators a concrete assessment model for gaining insights into the effectiveness of training programmes.

The programmes of the MOE in Saudi Arabia to train head teachers and prepare them for their roles as head teachers are effective and do provide a strong foundation to enable them to acquire the required leadership knowledge and skills to fulfil their responsibilities.

In future research, it would be beneficial when conducting a similar study to obtain ideas on what training topics would be needed for head teachers.

In future work, we plan to automate this adapted model by setting up a computerised system to evaluate the training process utilising approaches such as fuzzy logic or other artificial intelligence algorithms (decision tree, neural network or others) or combinations between them. It enables and facilitates the evaluation process for stakeholders (including trainees, supervisors, trainers and training providers), in order that the training programmes can be dynamically and continuously improved.