Validation of a Football Competence Observation System (FOCOS), Linked to Procedural Tactical Knowledge

Abstract

The use of instruments for the evaluation of a player’s procedural tactical knowledge (PTK) in sociomotor sports, such as football, is a line of research of growing interest since it allows a pertinent description of the player’s football competence. The aim of this study is to configure and validate an ad-hoc observational tool that allows evaluating the player’s PTK, understood as football competence, from the observation, coding and recording of the roles, the actions of the acquired subroles and the operational and specific principles of football in the attack and defense phases. Based on the Delphi method, a field format coding instrument was designed and validated where each criterion is a system of categories, exhaustive and mutually exclusive. The results showed excellent content validity (9.02 out of 10), and high values of intra-observer stability (k = 0.747) and inter-observer agreement (k = 0.665). Generalizability analysis showed an excellent reliability (G = 0.99). Additionally, the construct validity of the tool was calculated through a small-sided game Gk + 4v4 + Gk, using two independent samples: semi-professional and amateur players. The results reflected significant differences (α < 0.05) between both samples in the variables total score, offensive score and defensive score. Therefore, this study provides a valid and reliable instrument that allows data collection in a rigorous and pertinent way, as well as their analysis and evaluation in attack and defense according to the roles of the players and based on the motor behaviors that they perform using the subroles that they acquired, associated with the technical dimension, along with the principles that they develop in parallel, in support of the tactical dimension.

1. Introduction

The construction of instruments for the evaluation of the tactical knowledge of the players in sociomotor sports [1], as is the case of football, is a line of research of increasing interest due to the importance that tactic dimension assumes in training and performance [2]. In this sense, the instruments proposed for the tactical evaluation of the player have been developed and classified into two perspectives according to the type of tactical knowledge that has been evaluated. One perspective refers to declarative tactical knowledge (DTK), that is, “knowing what to do”, through knowledge of the rules, positions, functions, offensive and defensive strategies, and understanding of the technical-tactical logic of the game [3]. The perspective of the procedural tactical knowledge (PTK) is intimately linked to the particular motor action [4,5,6], that is, “to know how to do”. The latter, the tactical dimension of behavior, is decisive in a sport like football, with a very complex logic due to its high unpredictability and randomness of events [7], and refers to the player’s performance in the context of the game [8] or to football competence [9].

To analyze and assess the behaviors of the players, several methods have been used, as can be observed in systematic reviews on match analysis carried out in soccer [10] and other team sports [11,12,13,14]. From the observational methodology [15], there is a wide variety of instruments to assess PTK in football, such as “Performance Assessment in Team Sports” (TSAP) [16], “Game Performance Assessment Instrument” (GPAI) [17], “Procedural Tactical Knowledge Test” (KORA) [18], validated by Memmert [19], “System of Tactical Assessment in Soccer” (FUT-SAT) [20], “Game Performance Evaluation Tool” (GPET) [21] and “Instrument for the Measurement of Learning and Performance in Football” (IMLPFoot) [22]. These tools are articulated around the tactical variables or game principles [23] that each author takes into consideration, resulting in very different configurations, and reflecting the difficulty that exists when evaluating the motor behaviors that are developed by the players in the collective games [24].

The most important limitations of these tools are as follows: focusing only on the attack phase (TSAP, KORA, GPET), or solely on the evaluation of the player with the ball (TSAP); not covering all the possibilities that the player has to respond in every situation (TSAP, KORA, GPAI, GPET, IMLPFoot); not using game principles to classify the behaviors carried out by the player (TSAP, GPAI, KORA, IMLPFoot); resorting to game principles, without analyzing the tactical behaviors that the player displays (FUT-SAT). It can be said that no tool offers a complete coding system around the roles and subroles of the football player, which allows the deployment and analysis of motor behavior based on these, leading to a relevant evaluation of the player’s football competence.

Taking into account the limitations presented by the tools shown, it is interesting to note that the dual structure of sociomotor games leads to understand that the sociomotor roles that the participant can assume can be different when attacking and defending [25]. Taking as a reference the concept of “game center” [26], different roles can be identified depending on the relationship of closeness between ball and player [27]. However, this level of specification does not seem sufficient to allow a rigorous and detailed analysis of the game action, for which it is necessary to go to the subroles associated with each of the player’s roles. Some studies [28,29] have delved into the study of motor behaviors in football through the use of sociomotor subroles, allowing to appreciate, in the case of each player, the particular orientation they make of their role [30] and the consequent possibilities of action. The possibilities of action of the players can be framed within the game principles. The current literature includes various types of game principles: operational principles [31], fundamental principles [32,33] and specific principles [5,32,33,34,35,36,37], in addition to the principles associated with the game model to be transmitted, linked to a certain way of playing. They can all can facilitate the framing of tactical behaviors shown by teams and players, contributing to the design of instruments that reflect with greater specificity and superior relevance the events that happen in a match or training [38,39].

Given all of the above, the objective of this study is to configure and validate an observation tool designed ad-hoc to evaluate a player’s PTK, understood as football competence, based on the observation, coding and recording of the roles, the own actions of acquired subroles and the principles adopted by the players in the game. To achieve this objective, guaranteeing the validity and reliability of the data collected is important so that the performance analysis can effectively fulfill its intentions and purposes [40]. For this, it seems necessary to determine the degree of validity and reliability of ad-hoc tools from various dimensions. This is, on the one hand, the content, construct and criterion validity [41], and on the other hand, the stability and agreement of the instrument, based on its intra-observer and inter-observer reliability. The results of the present study allow the applicability of the instrument not only in the scientific field, but also in the field of evaluation of pedagogy and sports training.

2. Materials and Methods

2.1. Design

The study responds to an observational design that is punctual, idiographic and multidimensional [42]. It is idiographic because it is focused on a study unit, the player in particular; it is punctual because the data collection is carried out in a single session, and not throughout a season; and it is multidimensional, because the effectiveness index of the tactical behaviors displayed by the player is studied based on various criteria. Therefore, for data collection, it is necessary to configure an ad-hoc instrument, which, conditioned by the structure of the observational design, will be a field format coding instrument where each criterion is a system of categories, exhaustive and mutually exclusive [43]. For all this, the data type is sequential and event-based, since the observer collects the order of events, not their duration, and only one behavior can take place at a time [44].

2.2. Participants

During the design and reconfiguration stages of the instrument that gave way to its validation, a total of 31 experts contributed their conclusions via “Google Forms” in three different phases (n = 6, n = 8, n = 17). Experts had to meet at least two of the following three requirements: (1) have more than 10 years of experience training, (2) be graduates in physical activity and sports sciences with a specialty in football, (3) and be active coaches with a minimum qualification of professional level.

To establish the construct validity of FOCOS, two similar small-sided games (SSG) Gk + 4v4 + Gk were recorded and analyzed using two independent samples: eight semi-professional players (21.68 ± 1.38 years old), who were active in Spanish Second Division B playing in the reserve team of a “La Liga” club, and eight amateur players from a club of the last category of federated football in Madrid (25.30 ± 2.15 years old). Goalkeepers were not considered in any of the samples. In addition, for the reliability process, two observers were trained in the use of the tool, joining the experimenter, who acted as the third observer.

2.3. Coding Instrument

The Football Competence Observation System, FOCOS, was developed taking as reference various studies around the classifications of operational [31] and specific principles [5,32,33,34,35,36,37] as well as the roles [27] and subroles [28,29]. The new tool is formed by the combination of a field format and exhaustive and mutually exclusive category systems, based on six criteria (phase, role, own action of the subrole, operational principle, specific principle and result of the action) that appear in

Table 1. Criteria and category systems of the Football Competence Observation System (FOCOS).

Phase	Role	Own Action of the Sub-Role	Operational Principle	Specific Principle	Result of the Action
Attack	Attacker with the ball Attacker without the ball in the game center Attacker without the ball out of the game center	Ball control Driving Dribble Passing Shooting Move off-the-ball Positioning	Maintain ball possession Progress towards rival area Achieving the goal	Penetration Offensive coverage Depth mobility Width and length Offensive unity	Successful Improvable Wrong
Defense	Defender in the intervention space Defender in game center Defender out of game center	Tackling Interception Dissuading Relocating	Regain Possession Prevent opponent’s progre ssion Protect the goal	Delay Defensive coverage Balance Concentration Defensive unity	Successful Improvable Wrong

.

Taking the observation system proposed, the observer analyzes the volume and effectiveness of the behaviors that the player is displaying based on the criteria described. For this, the observer must know in detail the definitions of the categories (see

Table 2. Codes and definitions of observation categories.

Criterion	Category	Code	Definition for Observation
Phase	Attack	AT	The team of the player to be analyzed has possession of the ball
	Defense	DF	The team of the player to be analyzed does not have possession of the ball
Role	Attacker with the ball	Acb	The player to be analyzed has possession of the ball
	Attacker without the ball in the game center	Asb:c	The player to be analyzed is the 1st or 2nd partner closest to the player with possession (including goalkeeper)
	Attacker without the ball out of the game center	Asb:a	The player to be analyzed is not the 1st or 2nd partner closest to the player with possession (including goalkeeper)
	Defender in the intervention space	Dacb	The player to be analyzed is the closest to the attacker with the ball
	Defender in game center	D:c	The player to be analyzed is the 2nd or 3rd opponent closest to the attacker with the ball (including goalkeeper)
	Defender out of game center	D:a	The player to be analyzed is not the 1st, 2nd or 3rd opponent closest to the attacker with the ball (including goalkeeper)
Actions of the acquired subroles	Ball control	RECEP	✓ Successful tactical behavior: Profiling oneself with body orientation to receive, maintaining the ball after the first contact
			❖ Wrong tactical behavior: Failing to profile prior to receive the ball or not holding the ball after the first contact
	Driving	COND	✓ Successful tactical behavior: Moving with the ball by making at least 3 contacts after the previous control and maintaining possession
			❖ Wrong tactical behavior: Moving with the ball by making at least 3 contacts after the previous control and losing possession
	Dribble	DRIB	✓ Successful tactical behavior: Surpass the adversary, avoid the opponent’s tackle while maintaining possession of the ball, or receive foul having the ball
			❖ Wrong tactical behavior: Losing possession of the ball when trying to surpass the opponent
	Passing	PASE	✓ Successful tactical behavior: Maintaining possession of the ball or creating a dangerous situation by sending the ball to a teammate
			❖ Wrong tactical behavior: Losing possession of the ball or not creating a dangerous situation by trying to send the ball to a teammate
	Shooting	TIRO	✓ Successful tactical behavior: Kicking the ball towards the goal with the intention of scoring or receiving a foul while attempting to score
			❖ Wrong tactical behavior: Failure to kick the ball on goal or lose possession of the ball at the time of preparation to kick on goal
	Move off-the-ball	DESM	✓ Successful tactical behavior: Move to a favorable area to score a goal or to a free space in front of the ball giving a passing line to the teammate
			❖ Wrong tactical behavior: Moving in front of the ball to an occupied space and/or not giving a passing line, except for areas conducive to scoring a goal.
	Positioning	POSI	✓ Successful tactical behavior: Support your partner with the ball, either on his back, left or right; or give width-length to the team without intention to receive immediately.
			❖ Wrong tactical behavior: Being too far from the teammate with the ball, without giving width or length to the team, not being able to offer pass options or offensive coverage in case of loss.
	Tackling	ENTR	✓ Successful tactical behavior: Contact the ball or the opponent, avoiding being overtaken by dribbling or making him miss
			❖ Wrong tactical behavior: Being overtaken by dribbling or making a foul
	Interception	INTER	✓ Successful tactical behavior: Changing the direction of the ball by cutting, clearing, or deflecting a pass or shot
			❖ Wrong tactical behavior: Trying to cut, clear or deflect a pass or shot without contacting the ball or preventing it from reaching its destination (opponent or goal)
	Dissuading	DISU	✓ Successful tactical behavior: Condition the opponent’s attack without contacting the ball or the opponent, preventing progression or finishing on goal
			❖ Wrong tactical behavior: Trying to condition the opponent’s attack without contacting the ball or the opponent, not preventing progression or finishing on goal
	Relocating	RECOL	✓ Successful tactical conduct: Protect the goal by marking or watching for potential receivers, or being on the last defensive line
			❖ Wrong tactical behavior: Neglecting the protection of the goal by not marking or watching for potential receivers, or not being in the last defensive line
Result of the action	Successful	YES	The tactical behavior performed by the player according to his subrole is successful
	Improvable	DUDA	The tactical behavior performed by the player according to his subrole cannot be identified as clearly successful or wrong.
	Wrong	NO	The tactical behavior performed by the player according to his subrole is wrong
Operational principle	Maintain ball possession	AT1	For the player with the ball: Control from behind or at the same height as the previous action in the own field; pass, drive or dribble without advancing towards the opponent’s goal.
			For the player without the ball: Move or position behind the ball or at the same height in the own field, or in the rival field without the immediate objective of receiving to score.
	Progress towards rival area	AT2	For the player with the ball: Control ahead of the previous action, or driving, passing, dribbling moving towards the opponent’s goal (except for all behaviors in rival field or against the last defender or exceeded this one)
			For the player without the ball: Moving in front of the ball in the own field or positioning in the rival field without the immediate objective of receiving to score a goal.
	Achieving the goal	AT3	For the player with the ball: Control, drive, pass, or dribble in the rival field or against the last defender (or past this one) and shoot
			For the player without the ball: Move in the rival field with the immediate objective of receiving the ball.
	Regain Possession	DF1	Tackling the attacker with the ball or intercepting a pass
	Prevent opponent’s progression	DF2	Relocating in the last defensive line, or dissuading in the opposite field by not allowing the advance of the ball, giving coverage to the teammate, balancing the center of play or reducing the effective playing space
	Protect the goal	DF3	Relocating increasing the protection of the goal, dissuading in the own field when the attacker with the ball is going to shoot or intercept a shot
Specific principle	Penetration	AT:P1	The player to be analyzed displays a behavior with the ball with the intention of progressing towards the opposite goal or finishing directly (shooting) or indirectly (assisting a teammate in the end zone)
	Offensive coverage	AT:P2	The player to be analyzed does not have possession of the ball, and is in the center of the game behind the ball or at the same height
	Depth mobility	AT:P3	The player to be analyzed does not have possession of the ball and is in front of it with the intention of receiving
	Width and length	AT:P4	The player to be analyzed displays a behavior with the ball without the intention of progressing towards the opposite goal or finishing directly (shooting) or indirectly (assisting a teammate in the end zone)
			The player to be analyzed does not have possession of the ball, is not in the center of the game, or in the last defensive line, and does not intend to receive immediately
	Offensive unity	AT:P5	The player to be analyzed does not have possession of the ball and is in the last line of the team without including the goalkeeper
	Delay	DF:P1	The player to be analyzed is the closest to the attacker with the ball
	Defensive coverage	DF:P2	The player to be analyzed is in the game center behind the teammate who performs the delay
	Balance	DF:P3	The player to be analyzed goes to the game center with the intention of creating superiority or is in the game center ahead of the ball
	Concentration	DF:P4	The player to be analyzed is not in the game center and is in the last line of the team without including the goalkeeper
	Defensive unity	DF:P5	The player to be analyzed is not in the game center and is not in the last line of the team without including the goalkeeper

). Volume and effectiveness are two performance indicators that have also been used in TSAP [16].

2.4. Procedure

The instrument design, the validity and reliability processes were carried out in six stages (see Figure 1) following the procedures used in other recent observational tools [22,45,46]: (a) bibliographic review and provisional design of the tool observation system, (b) consultation with experts, reconfiguration of the tool observation system and choice of reference formats for the game protocol, (c) content validation of the coding instrument, (d) simplification of the coding instrument and validation of this process, (e) development of intra-observer and inter-observer reliability processes in addition to generalizability analysis and (f) calculation of construct validity. Finally, the quality of this process was assessed using the methodological quality checklist for studies based on observational methodology (MQCOM) [47].

In the first stage, the provisional selection of the criteria and observation categories that make up the tool was carried out through a bibliographic review of the main evaluation tools of the PTK [16,17,18,20,21], as well as studies and observation tools designed from the football player’s subroles [28,29].

In the second stage, the observation system was gradually modified after consultation with experts. Using a Likert scale of 1–10, they were asked about: (a) degree of agreement, regarding clarity of language in the definition of the criteria and categories of the tool; (b) degree of importance and adequacy, based on practical and theoretical relevance, when the criterion or category to evaluate was part of the tool; (c) considerations, comments and observations about each criterion and categories of the tool. In this way, the criteria and categories were reconfigured, shaping the observation system of the tool and subjecting it to a new expert judgment, until passing the validation process in the third phase.

Parallel to this process, and taking the observation system proposed, an ad-hoc observational tool was designed for the coding and data collection process using the “LINCE software” [48]. Subsequently, templates were designed using Microsoft Excel 365 (Microsoft Corporation, Washington, USA) for the analysis and treatment of the data obtained, which would also be adapted during the process until obtaining the final version. From the observation tool, several simulations were performed and codified using different protocols and players to identify possible aspects to improve, which could be added to the experts’ judgment.

The choice of the reference formats that would serve as a protocol for the analysis of the PTK of the players tried to respond both to 7-football (for players U12), and to 11-football (from U13). For this, the player’s theoretical individual space of interaction was considered; that is, 300 m² for 11-football and 200 m² for 7-football [49]. These values served as a reference for the construction of the protocols considering the age of the players to be analyzed. Two protocols, based on SSG Gk + 4v4 + Gk, were established according to the football modality. As a result of this, easily identifiable spaces were established within the playing field, as well as playing times, in order to minimize the effect of fatigue during the protocol, establishing the following game formats that would serve as a reference for its realization (see

Table 3. Reference formats for carrying out the protocol.

Category	Players	Goals Sizes	Space: Length × Width	Individual Space of Interaction	Duration: Series × Minutes/Break between Series
Up to U12	GK + 4v4 + Gk	6 × 2 m	Half field (7-football) 25 × 30 m (minimum)	94 m2	2 × 4′/1′
From U13	Gk + 4v4 + Gk	7.32 × 2.44 m	Double area (11-football) 33 × 40 m	165 m2	2 × 4′/1’

).

In the third stage, the content validity of the instrument was established from the last group of experts (n = 17) through the Content Validity Coefficient (CVC) [50]. Once the opinion of this last group of experts was obtained, the categories of the observational system with average values < 0.70, in terms of degree of agreement or degree of acceptance, were eliminated (n = 0); the categories with values between 0.70 and 0.80 were reformulated following the proposals of the experts (n = 1) and the categories with average values greater than 0.8 were accepted (n = 36) [51]. In this sense, practically all the categories had average values above 0.80 since the tool had undergone a rigorous configuration process before reaching this point. However, based on the considerations provided by the experts, a new category was included within the criterion result of the action (category = improvable). Football is a sport of maximum uncertainty, where unrepeatable behaviors occur. This new category seems important, when the observer cannot identify with certainty whether the behavior performed by the player is successful or not.

In the fourth stage, to simplify the instrument and increase its agility, the number of criteria in the analysis tool was reduced to two, unifying the phase, role, own action of the subrole, operational principle and specific principle in a single criterion called “general tactical behavior”, and maintaining the criterion “result of the action”. To carry out this process, the networks of mutually compatible categories were validated, discarding those combinations that were impossible in the game (examples: an attacker without the ball could never make a pass, or a defender could never perform the specific principle of penetration). Once this was complete, the 315 combinations of categories of the criteria in attack and the 180 combinations in defense were presented to a last group of experts (n = 13). The experts had to show their degree of agreement and acceptance through a Likert 1–10 scale with those combinations proposed as compatible by the experimenter, propose new compatible combinations if any, and accept or reformulate the general tactical behavior name proposed for each one. From this process, combinations with values below 8 out of 10 should be discarded or reformulated following the contributions and comments of the experts [50]. In the case of the tool, a combination that did not reach the predetermined values was discarded, a new one was approved and 11 general tactical behaviors’ names were reformulated after consultation with experts, even though all of them had exceeded the predetermined values. After this process, 21 attack and 10 defense combinations were proposed as compatible, providing an identifying name for each in the form of general tactical behavior.

Table 4. General tactical behaviors in the network of compatible category combinations in attack and defense.

	Phase	Role	Own Action of the Sub-Role	Operational Principle	Specific Principle	General Tactical Behavior	Combination Code
1	Attack	Attacker with the ball	Ball control	Progress towards rival area	Penetration	Control the ball ahead of previous action (*)	P Recep Pen
2	Attack	Attacker with the ball	Ball control	Maintain ball possession	Width and length	Control the ball at the same height or behind the previous action (*)	M Recep Esp
3	Attack	Attacker with the ball	Ball control	Achieving the goal	Penetration	Control the ball in the rival area or in front of the last defender (or surpassed this one)	F Recep Pen
4	Attack	Attacker with the ball	Driving	Progress towards rival area	Penetration	Driving the ball forward (*)	P Cond Pen
5	Attack	Attacker with the ball	Driving	Maintain ball possession	Width and length	Driving the ball backwards, right, or left (*)	M Cond Esp
6	Attack	Attacker with the ball	Driving	Achieving the goal	Penetration	Driving the ball in the rival area or in front of the last defender (or surpassed this one)	F Cond Pen
7	Attack	Attacker with the ball	Dribble	Progress towards rival area	Penetration	Dribble to beat the rival (*)	P Drib Pen
8	Attack	Attacker with the ball	Dribble	Maintain ball possession	Width and length	Dribble without progress avoiding rival tackle (*)	M Drib Esp
9	Attack	Attacker with the ball	Dribble	Achieving the goal	Penetration	Dribble in the rival area or in front of the last defender (or surpassed this one)	F Drib Pen
10	Attack	Attacker with the ball	Passing	Progress towards rival area	Penetration	Pass the ball forward (except to assist)	P Pas Pen
11	Attack	Attacker with the ball	Passing	Maintain ball possession	Width and length	Pass the ball backward, right, or left (except to assist)	M Pas Esp
12	Attack	Attacker with the ball	Passing	Achieving the goal	Penetration	Assist teammate to score goal	Asistir
13	Attack	Attacker with the ball	Shooting	Achieving the goal	Penetration	Shoot at goal	Tirar
14	Attack	Attacker without the ball in the game center	Move off-the-ball	Progress towards rival area	Depth mobility	Move giving close option ahead of the ball	P Desm Cerca
15	Attack	Attacker without the ball in the game center	Move off-the-ball	Achieving the goal	Depth mobility	Appear in a space suitable to scoring a goal (near the teammate with the ball)	F Desm Cerca
16	Attack	Attacker without the ball in the game center	Positioning	Maintain ball possession	Offensive coverage	Take care of the back of the partner with the ball or give option close to the right/left	M Posi Esp
17	Attack	Attacker without the ball out of the game center	Move off-the-ball	Progress towards rival area	Depth mobility	Move away from the ball appearing between rival lines or behind the defense	P Desm Lejos
18	Attack	Attacker without the ball out of the game center	Move off-the-ball	Achieving the goal	Depth mobility	Appear in a space suitable to scoring a goal (away from the teammate with the ball)	F Desm Lejos
19	Attack	Attacker without the ball out of the game center	Positioning	Progress towards rival area	Width and length	Give depth to the attack by positioning in length	P Posi Prof
20	Attack	Attacker without the ball out of the game center	Positioning	Maintain ball possession	Width and length	Give amplitude to the attack by positioning in width	M Posi Ampl
21	Attack	Attacker without the ball out of the game center	Positioning	Maintain ball possession	Offensive unity	Relocate in coordination with the teammates on the last line	M Posi UO
22	Defense	Defender in the intervention space	Tackling	Regain Possession	Delay	Make a tackle to the rival	Entrar
23	Defense	Defender in the intervention space	Interception	Regain Possession	Delay	Intercept, clear or divert a pass	Interc
24	Defense	Defender in the intervention space	Interception	Protect the goal	Delay	Block a shot	Bloq
25	Defense	Defender in the intervention space	Dissuading	Prevent opponent’s progression	Delay	Redirect the opponent’s attack	Redir
26	Defense	Defender in the intervention space	Dissuading	Protect the goal	Delay	Do not give the opponent a shot option without entering him (avoid possible shot)	Disu Tiro
27	Defense	Defender in game center	Dissuading	Prevent opponent’s progression	Defensive coverage	Take care of the partner’s back in the intervention space in a staggered manner	Cober
28	Defense	Defender in game center	Dissuading	Prevent opponent’s progression	Balance	Move to create superiority in the game center or mark/watch opponents	Equi
29	Defense	Defender out of game center	Dissuading	Prevent opponent’s progression	Defensive unity	Create uncertainty in the last opponent line or reduce the effective playing space	Reple
30	Defense	Defender out of game center	Relocating	Protect the goal	Defensive unity	Relocation in the last defensive line reducing the effective playing space	Recol
31	Defense	Defender out of game center	Relocating	Protect the goal	Concentration	Increase the protection of the goal, marking or watching opponents	Concen

(*) Except in the rival field.

shows the network of combinations described.

After the observational system was validated, the observation tool was codified again, this time using the new “LINCE PLUS software” [52].

In the fifth stage, the inter-and intra-observer reliability process were performed. For this, the procedures developed in other works were followed [53,54,55,56]. First, the conceptual and registration protocol for motor behaviors was developed. Secondly, two observers were trained according to said protocol, and carried out the analysis of a determined player independently, who was previously analyzed by the experimenter. Third, inter-observer reliability was calculated, and the behaviors analyzed as different between observers were discussed and re-analyzed. Five weeks later, through the test-retest reliability method, an observer repeated the analysis process and the results obtained were compared with their previous analysis to calculate intra-observation reliability. Given the nature of the data analyzed and to control their quality, the TG (Generalizability Theory) [57] was applied from the modeling of the different sources of variability or facets (e.g., observers and categories of the taxonomic system), designing two possible models: Categories:Observers [C:O] and Observers:Categories [O:C].

Finally, in the sixth stage, once a high content validity for the instrument was obtained and the reliability processes were overcome, the construct validity of the instrument was calculated, in its perspective of discriminant validity, to measure the degree of the instrument to distinguish between groups of players that are expected to be different [58].

2.5. Application

After using FOCOS to carry out the PTK analysis of the players taking part in the selected protocol, the data obtained from each player were transferred to Excel templates designed ad-hoc to obtain the resulting scores and to perform the consequent evaluation. In these templates, data processing is performed to obtain the volume and the effectiveness index of each variable within the criteria studied. Volume is understood as the number of times the player develops tactical behaviors in which each category is involved, while the effectiveness index is represented by the volume of successful tactical behaviors divided by the number of tactical behaviors deployed by the player in the category of analysis studied.

Once the effectiveness indices have been obtained for each category, the offensive and defensive effectiveness indices are calculated, as well as a global effectiveness index. This global effectiveness index represents the player’s PTK level. In short, general scores are obtained for these last three mentioned variables, together with the specific scores of the variables that represent the categories of the role criteria, own action of the subrole and operational and specific principle of the FOCOS. All these specific scores are also compared with the average scores of all the analyzed players, allowing the determination of the player’s PTK level in each variable with respect to the teammates in their group. In addition, the scores of the variables are shown in the form of general tactical behaviors in game-play situations in which the player has developed them.

2.6. Data Analysis

The coding instrument has been evaluated in relation to the quality of the data required of any observational research that purports to be scientific [59]. To do this, the content validity of the instrument has been approached qualitatively, through consensual agreement [60] of a group of experts, through the Delphi method and using the content validity coefficient [50]. It has also been analyzed quantitatively, by calculating intra-observer reliability, using Cohen’s kappa; and inter-observer reliability, using the fleiss kappa index. Furthermore, the construct validity has been calculated using Student’s t-test for independent samples.

3. Results

The verification of the quality of the observational data allows for subsequent objective studies, and in this way, the adoption of original strategies for their application in training [59]. The results are described in the following sections.

3.1. Content Validity of the FOCOS

To calculate the Content Validity Coefficient [50], the averages of the two factors used with the expert groups were calculated, following the Delphi methodology: the degree of agreement (8.74 out of 10) which reflects the clarity of the language (to what extent do you consider the definition to be well developed and exclusive with respect to the other categories of the criterion?), and the degree of adequacy (9.3 out of 10) which represents practical and theoretical relevance (to what extent do you consider that the category should be part of the criterion?). From these two factors, the total content validity of the tool was obtained (9.02 out of 10), concluding that it is a very high validity. In the same way, the criterion “General tactical behavior” was also validated. In this process, the global content validity was also very high (9.4 out of 10).

3.2. Construct Validity of the FOCOS

The construct validity of the instrument was calculated, in its perspective of discriminant validity, to measure the degree of the instrument to distinguish between groups of players that are expected to be different [58]. Using the reference formats, the protocol was carried out with two independent samples. Although all variables were analyzed, the overall total score, the total offensive score and the total defensive score obtained by semi-professional players were compared with the scores obtained by amateur players, since they reflect a more global vision of the players’ football competence. The data were analyzed using the Student’s t-test for independent samples and the results showed significant differences (α < 0.05) between both groups in these three variables (see

Table 5. Differences between semi-professional football players and amateur football players.

Criteria	Variable	Semi-Professional	Amateur	(sig.) T	Cohen’s d
Role	Attacker with the ball (efficiency)	0.88 ± 0.06	0.75 ± 0.12	0.011 *	1.37
	Attacker without the ball in game center (efficiency)	0.87 ± 0.11	0.81 ± 0.08	0.242
	Attacker without the ball out of the game center (efficiency)	0.96 ± 0.06	0.77 ± 0.15	0.006 *	1.66
	Defender in the intervention space (efficiency)	0.63 ± 0.10	0.51 ± 0.14	0.078
	Defender in game center (efficiency)	0.89 ± 0.09	0.71 ± 0.18	0.019 *	1.26
	Defender out of game center (efficiency)	0.92 ± 0.06	0.54 ± 0.30	0.004 *	1.76
Own action of the subrole	Ball control (efficiency)	0.96 ± 0.04	0.85 ± 0.19	0.161
	Driving (efficiency)	0.88 ± 0.35	0.48 ± 0.47	0.077
	Dribble (efficiency)	0.88 ± 0.35	0.58 ± 0.42	0.147
	Passing (efficiency)	0.80 ± 0.11	0.70 ± 0.12	0.107
	Shooting (efficiency)	0.81 ± 0.22	0.64 ± 0.31	0.21
	Move off-the-ball (efficiency)	0.88 ± 0.12	0.78 ± 0.07	0.097
	Positioning (efficiency)	0.94 ± 0.05	0.79 ± 0.09	0.002 *	2.06
	Tackling (efficiency)	0.31 ± 0.32	0.65 ± 0.35	0.068
	Interception (efficiency)	0.59 ± 0.21	0.46 ± 0.43	0.43
	Dissuading (efficiency)	0.78 ± 0.12	0.58 ± 0.13	0.006 *	1.59
	Relocating (efficiency)	0.92 ± 0.07	0.61 ± 0.29	0.012 *	1.47
Operational principle	Progress towards rival area (efficiency)	0.88 ± 0.11	0.78 ± 0.07	0.061
	Maintain ball possession (efficiency)	0.93 ± 0.05	0.77 ± 0.11	0.003 *	1.87
	Achieving the goal (efficiency)	0.87 ± 0.07	0.72 ± 0.09	0.001 *	1.86
	Prevent opponent’s progression (efficiency)	0.82 ± 0.08	0.62 ± 0.15	0.004 *	1.66
	Regain Possession (efficiency)	0.71 ± 0.24	0.70 ± 0.19	0.917
	Protect the goal (efficiency)	0.74 ± 0.08	0.48 ± 0.19	0.004 *	1.78
Specific principle	Penetration (efficiency)	0.86 ± 0.07	0.72 ± 0.11	0.006 *	1.52
	Offensive coverage (efficiency)	0.69 ± 0.43	0.89 ± 0.11	0.224
	Depth mobility (efficiency)	0.88 ± 0.12	0.78 ± 0.07	0.097
	Width and length (efficiency)	0.98 ± 0.04	0.85 ± 0.13	0.017 *	1.35
	Offensive unity (efficiency)	0.64 ± 0.44	0.53 ± 0.39	0.606
	Delay (efficiency)	0.63 ± 0.10	0.51 ± 0.14	0.078
	Defensive coverage (efficiency)	0.87 ± 0.12	0.66 ± 0.31	0.089
	Balance (efficiency)	0.96 ± 0.06	0.63 ± 0.34	0.019 *	1.35
	Concentration (efficiency)	0.77 ± 0.34	0.61 ± 0.33	0.349
	Defensive unity (efficiency)	0.96 ± 0.06	0.48 ± 0.32	0.001 *	2.08
General tactical behavior	Control the ball ahead of previous action (*) (efficiency)	0.95 ± 0.06	0.97 ± 0.07	0.59
	Control the ball at the same height or behind the previous action (*) (efficiency)	0.75 ± 0.46	0.75 ± 0.46	1
	Control the ball in the rival area or in front of the last defender (or surpassed this one) (efficiency)	0.85 ± 0.35	0.55 ± 0.39	0.126
	Driving the ball forward (*) (efficiency)	0.88 ± 0.35	0.38 ± 0.52	0.041 *	1.13
	Driving the ball backwards, right, or left (*) (efficiency)	0.88 ± 0.35	0.38 ± 0.52	0.041
	Driving the ball in the rival area or in front of the last defender (or surpassed this one) (efficiency)	0.00 ± 0.00	0.31 ± 0.46	0.074
	Dribble to beat the rival (*) (efficiency)	0.38 ± 0.52	0.38 ± 0.52	1
	Dribble without progress avoiding rival tackle (*) (efficiency)	0.50 ± 0.53	0.50 ± 0.53	1
	Dribble in the rival area or in front of the last defender (or surpassed this one) (efficiency)	0.38 ± 0.52	0.25 ± 0.40	0.598
	Pass the ball forward (except to assist) (efficiency)	0.71 ± 0.35	0.52 ± 0.38	0.291
	Pass the ball backward, right, or left (except to assist) (efficiency)	0.92 ± 0.13	0.66 ± 0.44	0.141
	Assist teammate to score goal (efficiency)	0.74 ± 0.35	0.63 ± 0.32	0.515
	Shoot at goal (efficiency)	0.81 ± 0.22	0.64 ± 0.31	0.21
	Move giving close option ahead of the ball (efficiency)	0.85 ± 0.19	0.67 ± 0.32	0.192
	Appear in a space suitable to scoring a goal (near the teammate with the ball) (efficiency)	0.95 ± 0.09	0.84 ± 0.18	0.164
	Take care of the back of the partner with the ball or give option close to the right / left (efficiency)	0.69 ± 0.43	0.89 ± 0.11	0.224
	Move away from the ball appearing between rival lines or behind the defense (efficiency)	0.84 ± 0.35	0.19 ± 0.37	0.003 *	1.8
	Appear in a space suitable to scoring a goal (away from the teammate with the ball) (efficiency)	0.50 ± 0.53	0.57 ± 0.40	0.762
	Give depth to the attack by positioning in length (efficiency)	0.75 ± 0.46	0.62 ± 0.51	0.591
	Give amplitude to the attack by positioning in width (efficiency)	0.86 ± 0.35	0.39 ± 0.42	0.029 *	1.22
	Relocate in coordination with the teammates on the last line (efficiency)	0.64 ± 0.44	0.53 ± 0.39	0.606
	Make a tackle to the rival (efficiency)	0.31 ± 0.32	0.65 ± 0.35	0.068
	Intercept, clear or divert a pass (efficiency)	0.81 ± 0.37	0.58 ± 0.50	0.314
	Block a shot (efficiency)	0.16 ± 0.23	0.00 ± 0.00	0.074
	Redirect the opponent’s attack (efficiency)	0.75 ± 0.13	0.43 ± 0.23	0.004 *	1.71
	Do not give the opponent a shot option without entering him (avoid possible shot) (efficiency)	0.10 ± 0.20	0.37 ± 0.26	0.041 *	−1.16
	Take care of the partner’s back in the intervention space in a staggered manner (efficiency)	0.87 ± 0.12	0.66 ± 0.31	0.089
	Move to create superiority in the game center or mark/watch opponents (efficiency)	0.96 ± 0.06	0.63 ± 0.34	0.019 *	1.241
	Create uncertainty in the last opponent line or reduce the effective playing space (efficiency)	0.83 ± 0.36	0.40 ± 0.43	0.043 *	1.08
	Relocation in the last defensive line reducing the effective playing space (efficiency)	0.97 ± 0.05	0.34 ± 0.38	0 *	2.32
	Increase the protection of the goal, marking or watching opponents (efficiency)	0.77 ± 0.34	0.61 ± 0.33	0.349
Overall scores **	Total Average	8.11 ± 0.67	6.43 ± 0.86	0.001 *	2.18
	Offensive Average	9.01 ± 0.68	7.38 ± 1.08	0.003 *	1.81
	Defensive Average	7.20 ± 0.86	5.48 ± 1.58	0.017 *	1.35

Semi-professional football players (n = 8); amateur football players (n = 8); (*) except in the rival field; (**) average of general tactical behaviors × 10.

). Cohen’s d-effect size [61] was also calculated to assess the magnitude of the difference between both groups. Differences based on effect size are referred to descriptively as very large (d ≥ 2), large (2.0 > d ≥ 1.2), moderate (1.2 > d ≥ 0.6), small (0.6 > d ≥ 0.2) and trivial (0.2 > d ≥ 0). [62] The results showed values between 1.08 and 2.32, except for one variable that showed significant differences in favor of the amateur group.

3.3. Intra-Observer Reliability

To calculate the intra-observer stability index, test-retest reliability was used by applying Cohen’s kappa to the data extracted from the observation of a player with a difference of five weeks between both records. In relation to the records made, it should be clarified that some error of omission in the record of any category may cause a mismatch between records, causing a possible underestimation of the concordance coefficient [63]. To avoid this, and before proceeding to calculate the Cohen’s kappa index, a filter was developed manually, matching those identifiable behaviors through their temporal registration. Once this process had been carried out, the results showed an agreement index of 0.747, which could be valued as good [64] regarding an observational tool with these characteristics.

3.4. Inter-Observer Reliability

The inter-observer reliability of FOCOS was calculated following the same manual filtering process that was used in the intra-observer reliability calculation. To calculate the inter-observer concordance coefficient for more than two observers (n = 3), Fleiss kappa was applied. The values obtained (k = 0.766) showed a good agreement.

3.5. Generalizability Analysis

The generalizability analysis was carried out in the SAGT v1.0 build 218.0.1 software program [65], using two possible models: Categories/Observers and Observers/Categories (see

Table 6. Estimated values of the relative coefficients (ξρ²_(δ)) and absolute (ξρ²_(Δ)) of generalizability for the designs Categories:Observers [C:O] and Observers:Categories [O:C].

	Sum Squares Type III	Degrees of Freedom	Mean Squares	Standard Error	%
Observers [O]	0.0	2	0.000	0.010	0
Categories [C]	11,389.1	34	334.972	26.318	100
CO	144.7	68	2.127	0.360	0
Designs	[C:O]	ξρ2(δ) = 0.994 y ξρ2(Δ) = 0.994
	[O:C]	ξρ2(δ) = 0.001 y ξρ2(Δ) = 0.001

).

The [C:O] design was used to calculate the inter-observer reliability. The relative generalizability coefficient is associated with high reliability in the generalization precision of the results (close to 1). To assess construct validity, the [O:C] design was used. The generalizability coefficients were found to be close to 0 (for both coefficients, relative and absolute). The possible sources of variance showed that most of the variability (100%) was associated with the categories facet, being null in the rest of the facets: Observers (0%), and Observers:Categories (0%). This reveals that the established categories are heterogeneous and, therefore, exclusive within the configured taxonomic system.

4. Discussion

The aim of the present study is to describe the steps carried out to design and assess the validity and reliability of a new proposal of an ad-hoc observational tool. The developed instrument allows us to analyze and evaluate the player’s PTK, both in attack and defense, unlike other tools such as GPET [21] and KORA [18] which focus only on the offensive phase, or TSAP [16] which exclusively analyzes the player when he has the ball.

Based on the record of the motor behaviors developed by the player, FOCOS allows evaluating their performance based on several criteria: the roles, the own actions of the acquired subroles, the operational principles and the specific principles. In this sense, the complete analysis of the behaviors that the player can develop during his performance is another advantage of FOCOS compared to other tools. FUT-SAT [20] does not evaluate the behaviors displayed by the player, and although TSAP [16], GPAI [17], KORA [18], GPET [21] and IMLPFoot [22] evaluate certain behaviors, they do not cover all the possibilities that the player has to respond to any game-play situation. Furthermore, the use of sociomotor roles and subroles to classify tactical behaviors is another contribution of the tool, allowing a more rigorous analysis. In addition, the evaluation of both operating principles and specific principles represents a great advantage over other tools such as FUT-SAT [20] which is focused only on specific principles, GPET [21] which analyzes operational principles, or TSAP [16], GPAI [17], KORA [18] and IMLPFoot [22] which are not articulated around game principles.

Another advantage of the tool is its protocol because it has a sustainable and easily applicable game format. FOCOS uses a SSG Gk + 4v4 + GK in football double area (for U13 players or older) or half football field-7 (for U12 players). Regarding this fact, different game formats are used in other tools: 3v3 without goals in KORA [18], Gk + 3v3 + Gk in FUT-SAT [20] and IMLPFoot [22]; Gk + 5v5 + Gk in GPAI [17], and from 2v2 to 7v7, in GPET [21] according to the age of the players. The use of SSGs that guarantee the representativeness [66] of the football game seems to be something on which most of the authors agree. In this study, Gk + 4v4 + Gk has been used because it is a game format that facilitates the occupation of the entire space in depth and width. Spaces of greater width than length have been used, since the interaction contexts [63] generated by the teams during a match usually have this characteristic, and the SSGs have the particularity of facilitating that all the players participate actively due to their proximity to the game center.

For everything mentioned, it is understood that the knowledge of the strengths and weaknesses of each player by the coach or coaching staff can be used to optimize the teaching-training processes from the subroles (divergent learning) or principles (convergent learning). The subroles represent, in one way or another, the most applicable version of the player’s technique in the tactical context that is presented, and they are related to exploratory capacity, while the principles are closely linked to learning a set of action rules common to any game model within the tactical framework that presents football as a sociomotor sport.

Respecting the applicability of the instrument, several possibilities can be found: (1) within a team, the player’s football competence could be periodically analyzed, allowing to evaluate his evolution compared to himself and his teammates; (2) also, the level of football competence of new players who train with a team on a trial basis could be assessed; (3) in recruitment days, those players who show an adequate level of football competence in the eyes of coaches and scouts could be evaluated in detail, in order to identify possible sports talents; (4) could also be used to complement the analysis performed using positional data tools. Considering this fact, the positional data focuses on the team, analyzing variables such as team length, team width and team surface area [67], while FOCOS is focused on the player, analyzing aspects already mentioned that are conceptually closer to those managed by coaches.

Regarding the limitations of this study, it can be noted that it was decided not to calculate the criterion validity of the tool, understood as concurrent or concomitant validity; that is, the degree of correlation between two measures of the same concept, at the same time and in the same subjects [68]. For this, FOCOS would have to be compared with an external criterion that intended to measure the same, but there are no tools in the scientific literature with the level of depth that FOCOS presents. This level of complexity implies several limitations: the deep knowledge of the tool and the game to be able to use it, the large volume of information that is handled, the temporary and human resources for its use on a large scale, as well the impossibility of applying it in real time.

5. Conclusions

As conclusions of the study, it should be mentioned that the coding instrument presented shows optimal validity and reliability values. It is the first instrument collected in the scientific literature, which is structured interactively based on the roles, the actions of the subroles, the operational principles and the specific principles of the game of football. It can fully analyze, both in attack and defense phases, the player’s procedural tactical knowledge, understood as football competence. It is able to analyze and evaluate the player in detail from a technical-tactical point of view, based on the motor behaviors that he performs using the subroles that he acquired, associated with the technical, and the principles that he develops in parallel, in support of the tactical dimension. This aspect represents something pioneering within the range of observational instruments directed towards the analysis of the player’s PTK. Based on these conclusions, the instrument could be used for scientific purposes to carry out possible research projects or specific studies, as well as by clubs, performance analysis departments and coaches to analyze and evaluate their players in detail, and thus improve their teaching and training processes.