Detection of Abnormal Patterns in Children’s Handwriting by Using an Artificial-Intelligence-Based Method

Abstract

Using camera-based algorithms to detect abnormal patterns in children’s handwriting has become a promising tool in education and occupational therapy. This study analyzes the performance of a camera- and tablet-based handwriting verification algorithm to detect abnormal patterns in handwriting samples processed from 71 students of different grades. The study results revealed that the algorithm saw abnormal patterns in 20% of the handwriting samples processed, which included practices such as delayed typing speed, excessive pen pressure, irregular slant, and lack of word spacing. In addition, it was observed that the detection accuracy of the algorithm was 95% when comparing the camera data with the abnormal patterns detected, which indicates a high reliability in the results obtained. The highlight of the study was the feedback provided to children and teachers on the camera data and any abnormal patterns detected. This can significantly impact students’ awareness and improvement of writing skills by providing real-time feedback on their writing and allowing them to adjust to correct detected abnormal patterns.

1. Introduction

The actors involved in teaching–learning analyze students’ needs during their school life. Among the most critical needs is the ability to write; since the mid-twentieth century, a series of action guidelines and public policies focused on reading and writing have been built that create spaces for educational practices. However, learning to read and write has elements that must be identified as they are typical of the age of the people who start learning [1]. Children usually begin writing between the ages of 3 and 4. At this point, they can scribble and form bare traces with pencils or crayons. Their writing skills improve as they age, and they begin to write more complex letters, words, and sentences. However, it is essential to remember that each child develops at their own pace, and some may start writing before or after the age of 3 or 4. In addition, some children may face writing challenges due to coordination problems, excessive muscle tension, or other difficulties and may require additional support to develop their writing skills [2].

Writing ability refers to the ability to generate language through various means, such as typing or texting. On the other hand, handwriting refers specifically to the ability to create italics or print hyphens by using a handwriting tool. That is, while writing can include various forms of written communication, handwriting is a specific skill that focuses on handwriting and its quality. Therefore, it is essential to keep this difference in mind when discussing detecting abnormal patterns in handwriting, as this skill is crucial in developing cognitive and communication skills in children.

Excessive muscle tension or lack of coordination also affects a child’s writing. For example, suppose a child overly tenses their hand or arm muscles while writing. In that case, they may experience muscle fatigue and hand pain, affecting their ability to write effectively and comfortably. Additionally, a lack of coordination can render it difficult for a child to execute precise movements to write clearly and legibly. Suppose a child is experiencing problems with muscle tension or incoordination while writing. In that case, working with an occupational therapist who can help them improve their fine motor control and posture while writing may be helpful. It may also be beneficial to offer the child writing tools such as colored pencils or special grips to help reduce muscle tension and improve writing accuracy.

Excessive muscle tension or lack of coordination when writing is due to various reasons, including lack of practice; if a child does not have enough experience or training in writing, they may have difficulty coordinating the movements necessary to write fluently and precisely [3]. In some cases, incoordination and muscle tightness may result from developmental problems, such as autism spectrum disorders, neuromotor development disorders, or Tourette syndrome. Poor writing posture can also lead to excessive muscle tension in the hand, wrist, arm, or shoulder, rendering writing difficult. In addition, there are problems related to uncorrected vision problems, which hinder the child’s ability to see what they are writing [4]. As well as emotional factors, in some cases, this may result from the child’s stress, anxiety, or frustration when facing the task of writing.

To identify if a child has problems with excessive muscle tension or lack of coordination in writing, some signs and symptoms that can be observed, such as incorrect pencil grip, if the child holds the pencil too tightly or too loosely, or if they have difficulty changing their grip, may be a sign of excessive muscle tension. If the child’s handwriting is difficult to read, inconsistent in size and shape, or if letters and words are skipped, it may be a sign of a lack of coordination. If the child complains of fatigue or pain in the hand, wrist, or arm after writing for a short period, it may indicate excessive muscle tension. Another symptom is if the child is slow to write or takes longer than they should to complete writing tasks; it may be a sign of a lack of coordination.

Some software programs are specifically designed to assist in evaluating handwriting and identifying problems with excessive muscle tension or lack of coordination. These programs use motion-tracking technology to analyze a child’s handwriting, pencil grip, writing speed, pressure on the page, posture, and other factors. Examples of writing assessment software include WriteWell, a writing assessment software that uses motion-tracking technology to analyze the quality of writing and provide detailed feedback on speed, readability, and other aspects of writing [5]. SENSE-Writing uses a digital pen and tablet to generate a child’s report and provide feedback on pen grip, speed, pressure on the page, and other aspects of writing. KiddyWriting uses a digital pen and tablet to generate a child’s report and provide feedback on pen grip, speed, pressure on the page, and other aspects of writing.

This work proposes using artificial intelligence (AI) algorithms to identify and analyze student muscle tension patterns. These algorithms can analyze children’s movement patterns and muscle tension and detect areas of weakness or excessive muscle tension. For example, motion sensors or tablets can measure the hand while the child writes. The collected data can be analyzed by using AI algorithms to identify the child’s movement patterns and muscle tension. AI algorithms identify patterns that indicate a lack of coordination or excessive muscle tension and can provide recommendations to improve a child’s writing. By designing an AI algorithm, it is possible to evaluate children’s handwriting and provide feedback on the writing’s legibility, speed, and accuracy. By analyzing children’s writing patterns, the software can detect problems such as excessive muscle tension or lack of coordination and provide feedback to improve writing.

2. Materials and Methods

The development of the method uses several concepts that form the basis for designing an algorithm that can identify the software that can detect problems such as excessive muscle tension or lack of coordination for writing in children. The description of materials begins with a general review of the related works that were considered vital in developing this proposal.

2.1. Review of Related Works

Writing is a complex skill involving a series of cognitive and motor processes that render it possible to generate language by producing graphemes or written symbols. Writing development begins early, learning the fine motor skills needed to hold a pencil and render basic strokes. As children get older and learn to write letters and words, writing becomes an increasingly sophisticated skill that requires good eye–motor coordination, short-term memory, attention, and language skills. Dysgraphia, on the other hand, is a specific learning disorder characterized by persistent difficulties in acquiring and automating writing skills, affecting the legibility, fluency, and coherence of writing. Children with dysgraphia may have difficulties writing letters and numbers, problems with space and size, and slow and disorganized writing. Often these problems negatively affect the quality of their academic work and their self-esteem.

Research in the area of dysgraphia has led to the identification of different subtypes of dysgraphia, such as motor dysgraphia, which refers to problems in planning and executing hand and arm movements necessary to write; spatial dysgraphia, which refers to difficulties in the spatial organization and orientation of written symbols on a page; and phonological dysgraphia, which refers to problems in the identification and production of speech sounds that are related to written characters. To address the challenges of dysgraphia, different intervention approaches have been developed, including occupational therapy, sensory integration therapy, writing therapy, and spelling therapy. These interventions can help children develop fine motor skills, improve visual–motor coordination, and develop strategies to improve organization and planning in writing. In addition, technological tools such as cameras to detect abnormal patterns in handwriting have been developed, which may provide valuable information for the diagnosis of and early intervention in dysgraphia.

Numerous studies have focused on writing development and dysgraphia in children. For example, [6] established that handwriting involves a complex process that requires the coordination of fine motor skills, the planning and organization of ideas, and the transcription of spoken language into written form. In addition, some studies suggest that dysgraphia may result from low muscle tone or poor eye–motor coordination [7]. Other studies point to central nervous system dysfunction as a contributing factor [8].

Regarding dysgraphia screening tools, a recent study by [9] evaluated using an artificial-intelligence-based tool to screen for dysgraphia in school-age children. The results showed high accuracy in dysgraphia detection and suggested that artificial-intelligence-based tools may be helpful for early assessment and intervention in dysgraphia.

In works such as [10], the relationship between fine motor skills and handwriting was examined in children with and without developmental coordination disorder (DCD). The results indicated that children with DCD had more excellent hand and arm muscle tension during the writing and had significantly lower writing quality than children without DCD. In addition, fine motor skills were cleverly related to writing grades in both groups of children. In [11], handwriting problems in elementary school children in Turkey were examined. Children with handwriting problems were found to have inadequate writing posture, weak pencil grip, a tendency to press too hard on the page, and slower writing speed. Furthermore, it was observed that children with handwriting problems also had difficulties with fine motor skills.

The work [12] identified the relationship between handwriting and cognitive development in preschool and school-age children. The results showed that handwriting quality was significantly related to children’s working memory, attention, and visual–spatial processing. Children with handwriting problems were also found to have limited working memory and attention spans compared to children without handwriting problems. In [13], a systematic review of the existing literature on handwriting difficulties in children with developmental disorders was conducted on coordination (DCC). Children with DCC were found to have significantly lower handwriting quality than children without DCC, and handwriting problems in DCC children were related to poor hand–eye coordination.

Papers that use artificial intelligence techniques to identify writing problems in children, such as [14], point out that they use an artificial neural network to develop a classification model for detecting developmental dysgraphia in children. The results showed that the model was approximately 93% accurate in detecting dysgraphia. Similarly, in [15], machine learning techniques were used to develop a system for detecting handwriting difficulties in children. The system analyzed movement data from the child’s hand while writing and used machine learning algorithms to identify patterns that indicate writing problems. In [16], motion sensors were used to monitor the kinematics of writing movement in children with developmental coordination disorders. They used a machine-learning model to identify writing problems in real-time. The system provided real-time visual feedback to help children improve their writing.

In [17], an intelligent tutoring system was developed that uses machine learning techniques to analyze children’s writing and provide real-time feedback to improve writing. The results showed that the system significantly improved the children’s writing. Finally, in [18], machine learning techniques were used to analyze children’s writing and detect dysgraphia. The results showed that the model was 93% accurate in detecting dysgraphia. These works demonstrate that artificial intelligence techniques can detect children’s writing problems and provide personalized interventions.

2.2. Writing Problems in Children

Handwriting problems in children can affect the quality and efficiency of handwriting. Excessive muscle tension and lack of coordination are two common problems. Excessive muscle tension refers to excessive tension or stiffness in the muscles that are used for writing. This can render typing uncomfortable and painful, leading to slow and unclear typing. In addition, children who have excessive muscle tension may have difficulty controlling pencil pressure and may over-squeeze the pencil when writing.

The idea of capturing the biomechanical complexity of handwriting through the term “(in)coordination” is somewhat oversimplified. Children can have difficulty writing due to a lack of fine motor coordination and a variety of factors, including problems with planning, attentional control, working memory, and cognitive processing. Therefore, it is essential to recognize that handwriting is a complex neurocognitive skill that involves multiple interconnected cognitive and biomechanical processes. Therefore, any evaluation of handwriting must consider this complexity and use appropriate measurement tools and techniques to reliably capture the various aspects of the writing process.

2.3. AI Tools and Techniques to Detect Typing Problems

AI is being increasingly used in education and rehabilitation to help improve children’s ability to write. While there is no specific AI algorithm to identify writing problems specifically, there are AI technologies that can help assess children’s writing and provide real-time feedback to improve their ability. By analyzing children’s writing patterns, the software can detect problems such as excessive muscle tension or lack of coordination and provide feedback to improve writing. In addition, there are also AI tools that can give personalized writing therapy for children. These tools use machine learning algorithms to adapt to each child’s needs and abilities and provide individualized writing therapy that improves writing accuracy, speed, and legibility.

In this work, Python is used as the programming language, considering that this is a tool with greater penetration in using AI libraries to solve any user need. For example, various Python packages can help detect typing problems, such as excessive muscle tension or lack of coordination [19]. One of the most widely used packages for handwriting analysis is “PyDTW”, which calculates the distance between two handwriting sequences. The DTW (Dynamic Time Warping) algorithm used by PyDTW can detect abnormal typing patterns that may indicate problems such as excessive muscle tension or lack of coordination. In addition, PyDTW can compare a child’s handwriting with a reference handwriting model and detect significant discrepancies. Another valuable package for handwriting analysis is “OpenCV”, which is used for image processing. OpenCV can analyze a child’s handwriting for coordination problems, such as crooked lines or mis-shapen letters. This proposal uses several libraries to detect children’s writing problems and provide real-time feedback to improve their ability. Finally, specific recommendations are established for tutors to help identify writing issues in students and take corrective measures in real-time; these libraries are detailed in Appendix A.

2.4. Method

Figure 1 represents the flowchart, which covers the stages considered as part of detecting writing problems in children by using AI techniques. First, data acquisition includes regular and abnormal strokes representing abnormal writing patterns in children. In the next phase, the write speed for each writing stroke is calculated from the data. Writing speed can be calculated by dividing the stroke length (in mm) by the stroke duration (in seconds). Based on descriptive statistics of the data, write speed thresholds are established, which are considered normal and abnormal [20]. The average and standard deviation of write speeds can be used to set points, considering that strokes of which the rates are above or below specific multiples of the standard deviation are considered abnormal.

In the next phase, the write speed thresholds are applied to the data, and the write traces that exceed the established abnormal thresholds are identified as bizarre. To evaluate the algorithm’s accuracy in detecting abnormal patterns, the calculation of sensitivity, specificity, positive predictive value, negative predictive value, and other performance evaluation metrics of the algorithm are included [21]. Based on the results obtained in the validation, adjustments and optimizations are performed on the algorithm to improve its accuracy and performance in detecting abnormal patterns in children’s writing. Finally, the results obtained are presented, and feedback is provided to the tutors to carry out corrective actions for the students who require it.

2.4.1. Data Collection

For data collection, it is necessary to clearly define the variables to be measured, the indicators, and the population or sample that participates in the study. In addition, it is essential to have an adequate methodology for data collection, which can include various techniques such as surveys, interviews, observation, and document analysis, among others. Other essential aspects are ethical aspects, such as the informed consent of the participants, the confidentiality of the information, and the protection of the data collected. In this work, informed consent was requested from the parents of the participants. In addition, the work includes only those images that do not contain data that identify the study participants. Therefore, the photos or tables that are used do not have data that reveal the personal information of the participants.

Therefore, detailed information about how children write, including their posture, movement, and writing rhythm, was collected in this phase [22]. For this, cameras or motion sensors can be used. In this work, cameras and tablets were used. Therefore, the strategic location of the devices has been considered to capture children’s writing from various angles. Furthermore, these cameras are set to record at a suitable speed to obtain a clear and detailed image of the child’s handwriting.

In classrooms, high-speed, high-resolution cameras can be placed at different angles to capture the movement of children as they write [23]. These cameras can be placed in a fixed location or can be portable to track the child’s activity. In addition, it is essential to consider adequate lighting within the classroom environment to render the captured image clear and sharp. In addition, the data generated on the tablets through writing helps data acquisition.

The

Table 1. Characteristics collected by the tablets and the camera related to muscle tension and lack of coordination.

Characteristic	Related with	Collection Method
Write speed	Muscle tension, lack of coordination	Tablets
Writing tilt	Muscle tension, lack of coordination	Tablets, camera
Irregular spacing between words	Lack of coordination	Tablets, camera
Pen pressure	Muscle tension	Tablets
Write duration	Muscle tension	Tablets, camera
The direction of the strokes	Lack of coordination	Tablets, camera
Other relevant parameters	Muscle tension, lack of coordination	Tablets, camera

presents the characteristics collected by tablets and cameras about muscle tension and lack of coordination in writing. These features include typing speed, slant, irregular word spacing, typing duration, stroke direction, and other relevant parameters. These data allow us to identify potential problems related to muscle tension and lack of coordination in students at the time of writing. This information is helpful for early detection, addressing potential writing difficulties, and providing appropriate intervention to improve writing skills in affected students.

2.4.2. Data Analysis with AI

Once the handwriting data have been collected, they must be processed to detect abnormal patterns. For this, various signal processing and computer vision techniques were used. These data identify abnormal patterns that could indicate excessive muscle tension or lack of coordination. On the other hand, computer vision was used to analyze the images captured by cameras during the writing process. Through image processing techniques, movement and posture patterns that could indicate handwriting problems can be detected [24]. Machine learning is used to train models that can identify abnormal patterns in children’s writing. Through classification and regression techniques, specific patterns can be detected that indicate problems with muscle tension or lack of coordination. Once the models are trained, they can see these patterns in real time and provide immediate feedback to children to improve their writing.

Figure 2 shows the flowchart considered for the design of the data analysis model, in which an image classification and processing algorithm are used. Data collection: The process begins with data collection as children write, including information about their posture, movement, rhythm, etc. Data that are collected through cameras undergo a series of pre-processing techniques to improve data quality [25]. For example, filters were applied to remove noise from the signal, normalize the data to be on a standard scale, etc. In the next phase, the relevant characteristics for the analysis are extracted. These characteristics are writing frequency, writing pressure, and writing speed. Once the relevant factors have been removed, these are analyzed to detect abnormal patterns in the children’s writing. Machine learning techniques, such as classification or regression algorithms, are used to identify abnormal patterns.

Write frequency refers to the number of times an act of writing is performed in each period. For example, if a student performs ten writing acts in one minute, the writing frequency would be ten writings per minute. As for the duration of the breaks, it refers to the time the student takes between each act of writing. For example, if a student performs an act of writing, takes a 5 s pause, and then performs another act of writing, the break would be 5 s.

These parameters are relevant to assess the rhythm and fluency of writing. For example, the frequency of writing can indicate how quickly the student can perform acts of writing, while the length of the breaks can reveal the pause or interruption in the writing flow. Both tablets and cameras can collect these data and are part of the comprehensive assessment of students’ writing skills.

The test data set must be large and diverse enough to represent a variety of children’s writing patterns and conditions. Finally, the algorithm is run on the test data set, and its ability to detect abnormal patterns in children’s handwriting is evaluated.

2.4.3. Set Speed Thresholds

To establish the speed thresholds, it is necessary to calculate the writing speed from the data considering the duration and length of children’s writing strokes [26]. The data of 10 writing strokes of children between 7 and 11 were considered for the calculation. Data include stroke duration in seconds and stroke length in mm.

Table 2. Table with calculation of writing speed.

Streak	Duration (s)	Length (mm)	Speed (mm/s)
1	0.5	5.6	11.2
2	0.8	7.9	9.875
3	1.2	10.2	8.5
4	0.6	6.8	11.333
5	1.0	8.5	8.5
6	0.7	7.1	10.143
7	0.9	9.3	10.333
8	0.4	4.7	11.75
9	1.1	11.5	10.455
10	0.8	7.8	9.75

shows the time and distance of writing strokes. In this, the writing strokes vary between 0.4 s and 1.2 s. The sizes of the writing strokes vary between 4.7 mm and 11.5 mm. The shortest writing strokes have a duration of 0.4 s and a length of 4.7 mm, while the most extended strokes have a time of 1.2 s and 11.5 mm.

The writing speed is calculated by dividing the length of the stroke by the duration of the stroke, obtaining values in mm/s. The writing speed varies between 8.5 mm/s and 11.75 mm/s. The slower writing strokes have a writing speed of 8.5 mm/s, while the faster writing strokes have a rate of 11.75 mm/s. This write speed calculation is essential for establishing write speed thresholds based on descriptive statistics. Writing speed is critical in assessing children’s writing and can help identify abnormal patterns or problems in developing writing skills.

The write speed threshold was set based on several factors. First, it was based on the existing literature that has reported on the writing speed of children of different ages and levels of handwriting ability. In addition, the ease of execution of the handwriting movements for the children was considered, and the threshold was set at a level considered achievable for most children of the age and ability included in the study. The children’s typing speed also evaluated the algorithm’s success rate, and the threshold was adjusted accordingly to maximize the algorithm’s accuracy.

Based on the data in the write speed calculation table, write speed thresholds can be set to identify abnormal traces. These thresholds can be determined by using descriptive statistics, such as the mean (average) and standard deviation of the data. For example, the low write speed threshold is set to mean write speed minus one standard deviation. This represents a significantly lower-than-average write speed, which may indicate slow or poor writing. For instance, if the mean write speed is 10 mm/s and the standard deviation is 0.5 mm/s, the low write speed threshold could be set to 9.5 mm/s.

To set the high write speed threshold is set as the mean of the write speed plus one standard deviation. This represents a significantly higher-than-average write speed, which may indicate fast or atypical writing. If the mean write speed is 10 mm/s and the standard deviation is 0.5 mm/s, the high write speed threshold could be set to 10.5 mm/s. These write speed thresholds are then used to identify abnormal strokes in the children’s handwriting or accurate data. For example, if a child’s handwriting stroke’s writing speed is below the low or above the high, it could indicate an abnormal handwriting stroke that requires further evaluation or follow-up. It is essential to adjust and optimize these thresholds based on the specific data and the context of the study or assessment of writing in children.

2.4.4. Apply Thresholds and Detect Abnormal Traces

To apply the write speed thresholds that were previously established in a data set to detect abnormal traces, the data in

Table 3. Writing speed data table in mm/s for a group of children.

Child	Stroke	Duration (s)	Length (mm)	Writing Speed (mm/s)
1	A	0.8	10	12.5
2	B	1.2	12	10
3	C	0.9	9	10
4	D	1.5	15	10
5	E	0.7	8	11.4
6	F	0.5	5	10
7	G	1	8	8
8	H	1.3	18	13.8
9	I	1.1	11	10
10	J	0.9	7	7.8

are shown with parameters such as write speed in mm/s.

With previously established write speed, thresholds are applied to the data to identify abnormal traces, as presented in

Table 4. Application of thresholds to identify abnormal traces.

Child	Stroke	Duration (s)	Length (mm)	Writing Speed (mm/s)	Abnormal
1	A	0.8	10	12.5	No
2	B	1.2	12	10	No
3	C	0.9	9	10	No
4	D	1.5	15	10	No
5	E	0.7	8	11.4	No
6	F	0.5	5	10	No
7	G	1	8	8	Yes
8	H	1.3	18	13.8	Yes
9	I	1.1	11	10	No
10	J	0.9	7	7.8	Yes

.

• Low write speed threshold: 9.5 mm/s
• High write speed threshold: 10.5 mm/s

In the table, traces G, H, and J are identified as abnormal, as their write speed is below the low write speed threshold of 9.5 mm/s or above the high write speed threshold of 10.5 mm/s. The other strokes are not abnormal since their writing speed is within the established ranges. However, it is essential to consider that determining deviant strokes or applying writing speed thresholds may vary depending on the context and the population under study. Once abnormal tracings are identified, further analysis can be performed to investigate the possible causes of these abnormalities, such as motor problems, neuromuscular deficits, or cognitive dysfunction.

2.4.5. Statistical Analysis

To evaluate the algorithm for detecting abnormal patterns in writing in children, several parameters are involved depending on the specific nature of the algorithm and the data used. The parameter that is considered to measure the efficiency of this proposal is precision; this is an indicator of the proportion of correct predictions compared to the total number of predictions made [27]. This is calculated as the quotient between the number of accurate predictions and the total number of predictions. Sensitivity is the algorithm’s ability to detect abnormal patterns, while specificity refers to the algorithm’s ability to identify typical patterns correctly. These parameters are essential in assessing whether the proper balance between sensitivity and specificity is desirable. The positive predictive value is the proportion of true positives compared to the total optimistic predictions. In contrast, the negative predictive value is the proportion of true negatives compared to the full pessimistic predictions. These parameters are essential to understand the ability of the algorithm to form accurate predictions in the positive and negative classes.

Another important parameter for evaluation is the receiver operating characteristic (ROC) curve, which is a graphical representation of the sensitivity versus the specificity of the algorithm at various classification thresholds. It allows for evaluation of the algorithm’s performance over a range of points and selection of the optimal topic that balances sensitivity and specificity [28]. Cross-validation is a technique that allows evaluation of the algorithm’s performance in various data sets, which helps to estimate the generalization capacity of the algorithm. It can be used to split the data into training and test sets or perform k-fold cross-validation to better assess the algorithm’s performance. Depending on the specific nature of the problem of detecting abnormal patterns in children’s writing, there may be additional relevant metrics. For example, if it is a binary classification problem, precision, F1-score, accuracy, or area under the ROC curve, among others, can be used. The equations that are used for these parameters are:

$$\mathit{\text{Accuracy}}=\frac{(\mathrm{T}\mathrm{P}+\mathrm{T}\mathrm{N})}{(\mathrm{T}\mathrm{P}+\mathrm{T}\mathrm{N}+\mathrm{F}\mathrm{P}+\mathrm{F}\mathrm{N})}$$

(1)

$$\mathit{\text{Recall}}=\frac{\mathrm{T}\mathrm{P}}{(\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{N})}$$

(2)

$$\text{Specificity}=\frac{\mathrm{T}\mathrm{N}}{(\mathrm{T}\mathrm{N}+\mathrm{F}\mathrm{P})}$$

(3)

$$\mathit{\text{Positive Predictive Value}}=\frac{\mathrm{T}\mathrm{P}}{(\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{P})}$$

(4)

$$\mathit{\text{Negative Predictive Value}}=\frac{TN}{\left(T\mathrm{N}+\mathrm{F}\mathrm{N}\right)}$$

(5)

In the evaluation, it is essential to select the appropriate metrics based on the specific problem and the available data and consider the results together to obtain a complete review of the performance of the algorithm; in Figure 3, the flow chart used for the process is presented for an algorithm evaluation [29]. In addition, it is essential to consider that the review of algorithms for detecting abnormal patterns in children must be carried out in collaboration with professionals who are specialized in writing and child development.

The evaluation process begins with dividing the data set into training and test sets: one to train the algorithm and the other to evaluate its performance (test assigned). This division allows evaluation of how the algorithm generalizes to previously unseen data. In the next phase, the algorithm is trained by using the training set, which contains examples of writing in children with normal and abnormal patterns. During training, the algorithm learns to detect un-natural patterns in children’s handwriting [30]. Once trained, the algorithm is used to generate predictions on the test set, which contains examples of writing in children that were not used during training. Finally, the algorithm predicts whether each documented instance in the test set has abnormal patterns.

Next, several evaluation metrics are calculated to measure the algorithm’s performance in detecting abnormal patterns in children’s writing. These metrics include precision, sensitivity, specificity, positive predictive value, negative predictive value, and ROC curve [31]. These metrics provide a quantitative measure of the algorithm’s performance in detecting abnormal patterns in children’s handwriting. In addition, cross-validation can be performed, which involves dividing the data set into k subsets (k-fold cross-validation), training and evaluating the algorithm on each subset, and averaging the evaluation results. This helps obtain a more robust and reliable evaluation of the algorithm’s performance. Finally, the results are presented, and the algorithm is adjusted if necessary.

3. Results

The system was applied in a primary school that was interested in identifying possible writing problems in its students. According to the information acquired from the tutors, they determined that some children had difficulties in writing, such as irregular strokes, excessive pencil pressure, or inappropriate speed. With the system, it is hoped to detect possible abnormal patterns in children’s writing early to provide them with adequate intervention and support. For this, reporting data from 71 primary school children were collected. Of the total population, 46 were men, which corresponds to 65%, while 25 girls made up 35% of the total population.

Table 5. Descriptive data of the population considered in the study.

Age of the Participants	Number of Participants	Percentage
7–9 years	15	21%
9–10 years	31	44%
10–11 years	25	35%
Total	71	100%

presents the general data of the people participating in this study.

In the application, the writing data of the 71 children were collected by using digital tablets that recorded the children’s writing with a digital pen and a camera to identify the parameters included in the report. The data provide writing speed, pen pressure, writing tilt, and other relevant parameters. These data are preprocessed and segmented for analysis.

Table 6. Results obtained from the evaluation of parameters in the recognition of problems.

Result	Amount
Writing samples processed	210
Writing samples with abnormal patterns detected	42
Percentage of writing pieces with abnormal patterns detected	8.40%
Types of abnormal patterns detected	Delayed writing speed, Excessive pen pressure, Irregular slant, Lack of word spacing
Distribution by school grade of the samples with detected abnormal patterns
1st grade	8
2nd grade	12
Grade 3	10
4th grade	7
5th grade	5
Percentage of agreement between abnormal patterns detected by the algorithm and teacher observations	85%
Number of children referred to exceptional education specialists or occupational therapists for intervention	36
Follow-up results	30% improvement in writing skills in children who received the intervention
Additional data obtained from the camera
Average write time	1.5 s per word
Average pen pressure	45 g
Average pencil tilt	12 degrees
The average spacing between words	1.2 cm
Comparison of camera data with abnormal patterns detected by the algorithm to confirm detection accuracy	95%
Feedback was provided to children and teachers on camera data and abnormal patterns detected	Yes

shows the results obtained from applying the writing anomaly verification algorithm.

The table provides the results from when 210 writing samples were processed. The algorithm detected abnormal patterns in 42 writing samples, representing 8.4% of the samples processed. Abnormal patterns included delayed writing speed, excessive pen pressure, irregular slant, and lack of word spacing. In addition, abnormal patterns were identified in writing samples from students from various grade levels, with 8 examples in 1st grade, 12 in 2nd grade, 10 in 3rd grade, 7 in 4th grade, and 5 in 5th grade. The percentage of coincidence between abnormal patterns detected by the algorithm and teacher observations coincided by 85% with teachers’ words in detecting abnormal patterns in the children’s writing. According to the results, 36 children were referred to exceptional education specialists or occupational therapists for intervention based on the abnormal patterns detected by the algorithm.

For the children referred to specialists, a 30% improvement in writing skills was observed. In addition, additional data from the monitoring camera were obtained in the evaluations, including average writing time (1.5 s per word), average pen pressure (45 g), average pen tilt (12 degrees), pen spacing, and intermediate between words (1.2 cm). From this, the algorithm’s detection accuracy was 95% when the camera data were compared with the detected abnormal patterns. With these results, feedback was provided to the children and teachers on the data from the camera and the abnormal patterns detected by the algorithm. These results indicate that the camera-based handwriting verification algorithm that was used in the study showed effective detection of anomalous patterns in children’s handwriting, allowing early identification of handwriting problems and appropriate intervention to improve writing skills and writing in affected children.

The evaluation of the system’s effectiveness is presented in

Table 7. Results obtained from the evaluation of the abnormal patterns in children’s handwriting identification algorithm.

Metrics	Value
Sample used	210 samples
Accuracy	92%
Sensitivity	85%
Specificity	96%
Positive predictive value	89%
Negative predictive value	95%
F1-Score	0.87
Area under the ROC curve	0.92
False positive rate	4%
False Negative Rate	15%

, for which the 210 writing samples of the children were used. The results indicate that the algorithm has a high accuracy of 92%, a sensitivity of 85%, a specificity of 96%, a positive predictive value of 89%, and a negative predictive value of 95%. The F1-Score is 0.87, and the area under the ROC curve is 0.92, indicating good model performance in binary classification. The false positive rate is 4%, and the false negative rate is 15%.

The 15% false negative rate indicates a significant percentage of abnormal traces that the algorithm does not detect. One reason that has been identified is that established writing speed thresholds are unsuitable for all children, especially those with atypical writing skills. To reduce the false negative rate, some measures are considered, such as:

• Adjust typing speed thresholds for each child based on their writing skills. This could be achieved by using an adaptive approach based on individual child performance.
  • Add more features and measurements to assess writing quality, such as pen pressure, rate of pressure change, and stroke direction.
  • Improve the accuracy of the sensor that is used to capture write data. This could be achieved by using more advanced and precise sensors or by conducting more rigorous tests to assess the quality of the data captured.
• In general, it is essential to remember that no algorithm is perfect, and there will always be limitations and areas for improvement. The detection of abnormal strokes in handwriting is a constantly evolving and improving area of research.

The ROC curve evaluates the ability of the model to distinguish between positive and negative classes. The curve is plotted by using the sensitivity and specificity values obtained by varying the decision threshold of the model. The area under the curve (AUC) is used to measure the model’s ability to discriminate between classes. It is interpreted as the probability that the model correctly classifies a random sample from the positive class and a random sample from the negative type. The ROC curve in Figure 4 shows that the model performs well in classifying the positive and negative classes since the AUC is 0.97, which indicates a high probability that the model correctly classifies a sample randomly. Furthermore, the curve shows a reasonable balance between sensitivity and specificity, which suggests that the model can correctly type both classes without significant bias. The choice of the appropriate decision threshold depends on the application’s specific requirements and the relative cost of misclassification of each class.

4. Discussion

The results obtained from the handwriting verification algorithm show high effectiveness in accuracy, sensitivity, specificity, positive predictive value, and negative predictive value. The algorithm’s accuracy is 92%, which indicates that the algorithm can correctly classify 92% of the writing samples of the children in the model that was used [32]. Furthermore, the sensitivity of 85% suggests that the algorithm can accurately detect 85% of the cases with abnormal writing patterns, which indicates that the algorithm performs well in detecting abnormal writing.

The specificity of 96% is high, indicating that the algorithm can correctly identify 96% of standard write cases, which reduces the false positive rate and improves the algorithm’s ability to avoid incorrectly classifying the script of normal children’s writing as abnormal. The positive predictive value of 89% indicates that the algorithm has a high probability of giving a correct prediction when it identifies a write as abnormal. It is essential to minimize false positives and ensure that anomalous write detections are accurate [33]. The 95% negative predictive value indicates that the algorithm has a high probability of giving a correct prediction when it identifies a write as usual. Minimizing false negatives and ensuring that regular write detections are accurate are essential.

The F1-Score of 0.87 is a measure that combines the accuracy and sensitivity of the algorithm and shows a good balance between the algorithm’s ability to classify both abnormal and typical cases of writing correctly. The area under the ROC curve of 0.92 indicates that the algorithm performs well in discriminating between abnormal and normal writing cases [34,35]. However, a false negative rate of 15% is observed, which means that there is a 15% rate of bizarre writing instances that the algorithm does not detect. This could indicate that the algorithm still has room for improvement in sensitivity, which could lead to insufficient early detection of writing problems in some children.

Some limitations of the write verification algorithm need to be considered. First, the accuracy and effectiveness of the algorithm largely depend on the quality and quantity of sample data used in training. The algorithm’s performance may negatively affect if the sample data are limited or skewed. In addition, the algorithm is based on signal processing and computer vision techniques, which means that it is subject to possible errors in the processing or interpretation of the writing by the system [36]. Additionally, external factors such as image quality, lighting, and camera position can affect handwriting verification through a camera, which could influence the algorithm’s accuracy.

The handwriting verification algorithm developed in this study can be a complementary tool for evaluating students’ writing progress. It could also be used in clinical settings to detect potential neuromotor or developmental disorders that may affect writing in children [37,38]. However, it is essential to note that the handwriting verification algorithm developed in this study is complementary and should not replace assessment and diagnosis by health or education professionals. Writing is a complex process that can be influenced by various factors, including cognitive and emotional factors.

Teacher assessments are an essential source of feedback in the AI use process. Teachers must be trained in using AI and understand how it works, what kind of data they can use, and how to integrate it into their teaching activities. In addition, teachers must understand the learning objectives and the skills that students are expected to acquire with AI. Teacher feedback can also help identify areas in which the AI teaching process needs improvement or adaptation to meet specific student or curriculum needs. Teachers can provide valuable information on the effectiveness of the tools and techniques used in the teaching process, which can help identify best practices and areas for improvement. Combining student data and teacher feedback can provide a more complete and accurate view of the success of the AI use process.

The use of technology such as cameras and tablets to detect abnormal patterns in children’s handwriting has proven to be a promising tool in education and occupational therapy. However, it is essential to note that there are limitations to the ability of these technologies to capture specific data, such as muscle tension during typing. Although muscle tension is a critical factor in handwriting production, it cannot be directly recorded through the technology used in this study. Instead, muscle tension is inferred from the collected data streams, which can be problematic due to one-too-many assignments and the multitude of degrees of freedom at each level of control of the neurocognitive processes involved in muscle tension handwriting production. Furthermore, the inference of muscle dynamics from digitized kinematics also presents additional challenges. Digitized kinematics refers to the analysis of the movements of objects and the body by recording and measuring positions and velocities over time. However, digitized kinematics cannot capture the complexity of muscle dynamics, such as electromyographic activity and the distribution of muscle forces in the production of handwriting. Therefore, caution is needed when interpreting the data collected through these technologies and considering their limitations in capturing specific information.

Although the use of technologies such as cameras and tablets helps detect abnormal patterns in children’s handwriting, it is essential to be aware of the limitations of these technologies in capturing specific information, such as muscle tension. In addition, it is necessary to consider the full neurocognitive context in which handwriting occurs and how the technologies used in this study may have difficulty capturing the complexity of these processes. More research is needed to address these limitations and to develop new technologies that can capture more precise information about the neurocognitive processes involved in the production of handwriting.

5. Conclusions

The results show that the algorithm used to detect abnormal patterns in children’s writing was adequate, with a detection accuracy of 95% when comparing the camera data with the detected abnormal patterns. This indicates that the algorithm is a reliable and accurate tool for identifying abnormal patterns in children’s writing, which is essential for early and appropriate intervention. However, it has been determined that the results obtained are limited since the numbers of children in the different age groups are low to draw a general conclusion about the method’s performance in each age group. In addition, the false cases may be concentrated mainly in one group, which could affect the overall results. Therefore, more data and analysis are needed to draw more precise conclusions about the method’s performance in different age groups.

The feedback provided to the children and teachers on the data from the camera and the abnormal patterns that were detected turned out to be an effective strategy for improving the student’s writing skills. The possibility of having real-time information about their writing and adjusting correctly meant that abnormal patterns can significantly impact the awareness and improvement of children’s writing skills, highlighting the importance of feedback in the process of intervention. Follow-up results from the study showed a 30% improvement in writing skills in children who received intervention based on the abnormal patterns detected by the algorithm. This indicates that early detection and appropriate intervention can positively impact affected children’s writing skills, reinforcing the algorithm’s usefulness and effectiveness in the intervention process.

The camera-based handwriting verification algorithm’s application for detecting abnormal patterns in children’s handwriting has important implications for special education and occupational therapy. Early detection of abnormal patterns in writing may allow education and occupational therapy professionals to intervene early and tailor to the needs of each student, which may improve the effectiveness of the intervention and facilitate progress in writing skills. However, it is essential to note that the study has some limitations. For example, the sample of students tested was limited, and the results may not be generalizable to other populations. In addition, the study relied on a unique camera-based handwriting verification algorithm, so it is essential to consider various approaches and technologies for abnormal handwriting pattern detection in future research.

The application of a camera-based handwriting verification algorithm raises ethical and privacy concerns. It is essential to protect student data, ensuring it is obtained and used ethically and legally. Therefore, obtaining informed consent from students and parents or guardians is necessary before using camera-based handwriting verification technology and ensuring compliance with applicable privacy policies and regulations. It is important to note that the success rate of our dysgraphia detection method has not been validated by using a standardized dysgraphia assessment. Therefore, future studies are needed to validate and improve our process by using standardized rating scales to compare results and establish a more accurate success rate.