Measuring the Functional Abilities of Children Aged 3-6 Years Old with Observational Methods and Computer Tools

Summary

We present a protocol to use a computational tool to record and analyze the functional abilities of children aged 3-6 years old. The protocol facilitates the comparison of these abilities throughout their development and can be used to assess developmental difficulties.

Abstract

The analysis of functional abilities and their development in early childhood (0-6 years old) are fundamental aspects among young children with certain types of developmental difficulties that can facilitate prevention, through programmed interventions adapted to the needs of each user (student or patient). There are, however, few investigations to date, that have analyzed the use of automated tools for recording and interpreting the results of the initial assessment. Here, a protocol is presented to examine the functional abilities in early childhood in young children, aged between 3-6 years old, with intellectual disabilities, but the protocol can also be used for ages 0 to 6 years. The protocol makes use of a computer application, eEarlyCare, that facilitates the interpretation of the results of systematic observations, which are recorded in natural environments by professionals trained in early intervention. The software can be used to analyze 11 functional areas (Food Autonomy, Personal Care and Hygiene, Dressing and Undressing Independently, Sphincter Control, Functional Mobility, Communication and Language, Daily Life Routines, Adaptive Behavior and Attention) and a total of 114 different behaviors. Its use facilitates the analysis of the observed abilities and greatly assists early intervention. Compared to other observational methods, it allows a more efficient use of personal and material resources. The use of the computer application facilitates the recording of the observation results, which helps with organization and reflection on the observations. The software displays the observation results on-screen compared to normal developmental parameters. This information can be referred to for decision-making about the most suitable intervention program for each user (student or patient). Likewise, clustering techniques are applied to analyze the relation between the type of intellectual disabilities and functional development identified with the software, a relation that is intended to serve as a guide for early-care professional intervention.

Introduction

Observation at early ages: what and how to observe

Early childhood assessment in ordinary family contexts and at school is performed using the observational method. Hence, the evaluator must adhere to a precise observational process, the key to an accurate diagnosis and, therefore, to a successful training¹. There are many development inventories that provide guidelines for assessment: The Portage Guide², the Brunet Lézine Scale³, and the Battelle Developmental Inventory⁴, among others. These tools are based on internationally agreed standards set by the scientific community in the field of human evolutionary development. Although these tools analyze developmental areas (Psychomotor, Cognitive, Communication and Language, and Autonomy and Socialization), recent studies⁵ have proposed new tools that can also analyze these areas. These studies point out that the observational method from birth provides pointers of immense utility to early intervention and for the early detection of pathologies. However, the observational processes at these ages are complex, as they depend on behavioral observations recorded in natural contexts, which are not always easy to carry out.

Within this framework, the assessment of the acquisition of functional abilities at early ages is of great interest to parents, educators, and therapists alike. Any such assessment is of relevance for children who have been diagnosed or who are at risk of developing some disability. Early detection of developmental disorders is essential for the early diagnosis and intervention. Observational study from birth will provide indicators of this early detection and intervention⁵. Currently there are various tools (development inventories, scales, tests, etc.) for measuring development at those ages. The instruments that can currently be applied are developmental inventories, some of which are standardized. However, some of those instruments may require knowledge of psychometric techniques and the results are not automatically displayed on screen. For this reason, it is important to develop other tools that are easier to use and interpret.

   

Preparation of software for recording and interpreting data from the processes of contextual observation processes at early ages

The software development was, therefore, considered of relevance, which would assist the observers (therapist, educator, etc.) to record and to interpret the results of their observations. This protocol and software, eEarlyCare, can be used both in educational centers that work with children with disabilities and in therapeutic intervention centers aimed at this group. This is why from now on the term "user "will be used, which includes both students and patients, depending on the place where the intervention is carried out. In particular, a software that could facilitate the recording and the interpretation of data gathered in natural contexts from the observation of functional abilities among children from 0 to 6 years old. This software, eEarlyCare, is based on the functional abilities scale⁶ [Scale for the measurement of Functional Abilities in children between 0-6 years old] (SFA); this scale includes the measurement of 11 areas of development (Food Autonomy, Personal Care and Hygiene, Independently and dress and undresses, Sphincter control, Functional mobility, Communication and Language, Interactive symbolic play, Daily life routines, Adaptive behavior). Also, it was in turn, inspired by the Portage Guide², the Pediatric Evaluation of Disability Inventory (PEDI)⁷, and the works of Bronson⁸, as well as Whitebread and Basilio⁴ on social skills at ages 0-6, the Brunet-Lézine Scale³, developmental inventories for children aged 0-6⁷, and the Assessment Scale of the precursors⁹ to social skills. This tool is a computer application that is used for registering the results of each user assessment in longitudinal follow-ups (quarterly, monthly, annual, etc.). It is a referential aspect for the therapist with regard to the intervention, and for other professionals who work with children in early childhood with suspected dysfunctionalities. In addition, the software¹⁰ can automatically produce comparisons between the development of the functional abilities of different users, regardless of whether they are at the same intervention center, thereby facilitating the definition of common aspects for collaborative work.

In particular, this software is based on mainstream technologies (e.g., Windows Presentation Foundation Development -WPF-¹¹), a technological innovation that integrates advanced graphics to produce accurate graphic results¹² and a positive computer user-experience. The quality of the graphs improves the visualizations and the interactivity available with other tools such as spreadsheets. The application can store the data locally on relational databases and upload the information onto the cloud to be shared. In addition, the classic client-server architecture is also supported. These features make it easy to record the data that are collected from the observations and to process the results for visualization. Also, once registered, it is very easy to export the data. This allows the data to be used in powerful statistical packages to apply data mining techniques, such as supervised (classification and/or regression) and unsupervised (grouping) machine learning.

In this study, the classification techniques are of specific utility for personalized learning¹³. The architecture can be seen in Figure 1 and Figure 2. In Figure 1, the functionality of cloud storage is used as a safeguard, in case of data security problems and potential loss and corruption of data when exchanged between applications. Additionally, the software can also operate in a classic network-based client-server architecture with a database (Figure 2) with all data exchanges taking place between clients (these are concepts that are used in the field of computer science). These platforms provide authentication mechanisms and restricted access, which ensure privacy and data protection, while facilitating interaction with fully developed applications. The final result is an application interface that is designed¹⁴ for early-care professionals, so that they can use Learning Analytics techniques in a simple way and refer to them, in order to follow up the degree of each student’s development in each evaluation area of the scale¹⁵.

The application also offers a general profile of each user in each of the functional areas and subareas (Table 1). It also produces a comparison between all users at any one center. In brief, it produces a personalized analysis of the intervention needs of the different users. In addition, it helps early-care professionals with their intervention programs, as it can flag up areas in which users might or might not show similar developmental patterns. All these results guide the type of intervention programs that can be used together, rather than those that have to be designed individually. The data that are linked to this interface are the SFA scores of the users that are measured on a Likert scale from 1 to 5. Those scores can be compared with the maximum developmental age scores linked to each SFA dimension. The software can also link the chronological age of each user to the developmental age of each SFA dimension; a relevant aspect for the detection of intervention areas from among which to prioritize the areas for treatment.

Functional area	Functional sub-area
1. Food Autonomy	1. Food Texture
	2. Use of utensils
2. Personal Care and Hygiene	3. Dental Hygiene
	4. Hairstyle
	5. Nasal Care
	6. Handwashing
	7. Face and body wash
3. Independently dresses and undresses	8. Dressing and undressing (waist upwards)
	Dressing and undressing (waist downwards)
4. Sphincter control	9. Sphincter control
5. Functional mobility	10. Upper extremity
	11. Transfers in WC
	12. Transfers on a chair
	13. Mobility and bed transfers
	14. Mobility in the bathtub
	15. Mobility indoors
	16. Transport of objects
	17. Outdoor Mobility
6. Communication and Language	18. Word comprehension
	19. Understanding phrases
	20. Functional use of communication
7. Resolution of tasks in social contexts	21. Solving Problems
	22. Information about yourself
8. Interactive and symbolic play	23. Interactive Play
	24. Interaction with the peer group
	25. Play with objects
9. Daily life routines	26. Guidance in time
	27. Homework
10. Adaptive behavior	28. Self-harm behavior
	29. heteroagressiveness (harming others)
	30. Destruction of objects
	31. Disruptive behaviour (cry, shout, laugh without reason)
	32. Stereotypes
11. Attention	33. Attention

Table 1: List of functional areas and subareas.

Figure 1: Architecture of the proposal of automation of the correction of the Scale functionality of cloud storage. Please click here to view a larger version of this figure.

Figure 2: Architecture of the proposal of automation of the correction of the Scale functionality classic network. Please click here to view a larger version of this figure.

Therapists and intervention professionals can use this assessment protocol and the software implementation for the assessment of functional abilities and their development in early childhood between 0 and 6 years of development. The software can be used with children within that age range, although it is especially useful for children with suspected impaired development of functional abilities. It is likewise especially useful at Special Education Centers. The Research question is whether, having observed the functional abilities of children, the use of a computer tool will facilitate the recording and the interpretation of the results for the therapist.

Protocol

This protocol was performed in compliance with the procedural regulations of the Bioethical Committee of the University of Burgos (Spain). Prior to their participation, both the students and their parents and in some cases legal tutors had all provided their informed consent and had been fully made aware of the objectives of study. No financial compensation was offered for their participation.

1. Participant recruitment

1. Recruit children between 0-6 years of age with a history of special educational needs related to alteration in the areas of motor development, cognitive, personal autonomy, and socialization and with moderate to severe intellectual disabilities. A total of 11 participants (7 boys and 4 girls) were recruited for the present study.
2. Include children who have been diagnosed with an intellectual disability (moderate or severe) in accordance with the Diagnostic and Statistical Manual of Mental Disorders (DSM-5)¹⁶ and attend a Special Education Center for their schooling (Table 2).
   NOTE: In each case (depending on the detection of the problem and the place where the intervention is performed), the diagnosis should have been confirmed by a pediatric neurologist at a hospital or a psychologist within a multidisciplinary team, in accordance with DSM-5. Ideally, there should be no fewer than 15 participants, since the insertion of the observation results is laborious.

Sex	n	Mage	SDage	Rank age (months)	Disability•		Schooling
					degree
					a	b	c	d
Boys	7	95.2	11.77	86-114	4	3	2	5
Girls	4	83.5	23.56	45-112	1	3	2	2
NOTE:* Degree of disability: a = moderate intellectual disability; b = severe intellectual disability; Type of Schooling: c = Combined Schooling; d = Schooling in a Specific Special Education Center; M = Mean age in months; SD = Standard Deviation. * The children were diagnosed using the criteria of DSM516

Table 2: Characteristics of the sample.

3. Exclude children with normal development in functional areas (motor, cognitive, personal autonomy and socialization development) and children with an Intellectual functioning limitation.

2. Data collection

1. Collect data on the development of the children in different functional areas (Food Autonomy, Personal Care and Hygiene, Dressing and Undressing Independently, Sphincter Control, Functional Mobility, Communication and Language, Daily Life Routines, Adaptive Behavior and Attention).
2. Perform the observation using the Scale for the measurement of functional abilities (SFA)⁵ in children between 0-6 years old (SFA) (Table of Materials) and conduct the observation in a natural context (e.g., school settings).
3. Record observations throughout the week in different natural environments that reflect the daily life of the students at the center (Table 1) (e.g., while playing, etc.).
   1. Let the teacher or the therapist directly attending the children at the educational institutions record the observations. Gather the observations for each functional area.
      NOTE: Prior to data observation, three training sessions should be conducted for teachers or therapists. These sessions must be conducted by a psychologist or a professional with experience in early-childhood assessment.
4. Input the results into the software for the assessment of functional abilities in early childhood between 0-6 years old. The details on how to use the software is provided in step 3.

3. Experimental procedure

1. Assigning a student to a teacher in the software
   NOTE: This step has to be carried out by the director or coordinator of the center where the intervention takes place.
   1. Log in to the software package with a username (professional) and password and select the language: English or Spanish.
   2. Enter the student data information from the records held by the director of the center.
   3. Fill in the following fields for each student: Name, Surname, Code, Sex, Date of Birth, Developmental Age, Primary Diagnosis, Secondary Diagnosis, where appropriate, and observations that are relevant to medication data, allergies, and other information of interest to the management of the center.
   4. Enter the teacher or therapist information from the records held at the center.
   5. Assign the students to a group with a teacher (or therapist) by clicking on Classroom. Go to the column Students, choose the student to be assigned to the classroom, and click on Teachers and select one.
   6. Allocate each group of students and their teacher (or therapist) to a classroom by clicking on Teachers, input dates, name, surname, identification code, mail, passwords and observations. Click Accept.
      NOTE: The director or coordinator of the center assigns a role for each teacher or therapist to carry out the evaluations of each student or patient (as explained above, it depends on the type of center). The director or coordinator of the center can also consult the evaluations that the teacher or therapist has made.
2. Use of the software by the teacher
   NOTE: The teacher or therapist performing the assessment can then select an academic year and/or a term. The scale offers the possibility of selecting different functional areas for each term (Table of Materials).
   1. Log in to the software with the username (professional) and the password previously assigned by the director of the center.
   2. Enter the results of the assessments completed in natural environments for each participant assigned to the classroom.
   3. Choose a student in the software by clicking on his/her name, and start the assessment of the different functional areas (detailed in step 2.3.1).
      NOTE: The teachers or therapists can stop the assessment of each student at any time and continue at another time, having saved the data that they have registered.
   4. Perform the comparative analysis between the development of each user and the expected development at that chronological age.
      1. Once the data is registered, select the column Evaluation by right clicking. Then select the year and the trimester.
      2. Select the column Students and select the students of a class of whom to make the evaluation. Select the column Areas clicking on the area or subarea that needs to be evaluated.
      3. Click on the Maximum tab. It gets the information about the students and the comparison with the development expected for their age.
         NOTE: This offers a user (student or patient) or development profile for a given functional area with a comparative link to the highest expected scores for the chronological age of the user (student or patient).
   5. Compare the different functional areas of each user from the classroom. An analysis of the functional areas of each user can be performed with the software. Following step: once the data is registered, select the column Evaluation by right clicking. Then select the year and the trimester. Select the column Students and select the students of all classes. Select the column Areas and click on the area or subarea to evaluate.
      1. Click on the Maximum tab. It gets the development of the students and the comparison with the development expected for their age.
3. Exporting data from the software
   1. Select the user data and functional areas, and export the database. Select the column Excel to get the database. Export the database in the statistics program or library of choice.
   2. Import the data into statistical packages and libraries such as SPSS, Weka, Python's scikit-learn, etc. and perform clustering analysis. Here, analysis with SPSS is detailed as below.
      NOTE: Clustering or cluster analysis is an ‘unsupervised’ machine-learning technique, and, within k-means, it is a grouping method, which aims to partition a set of n observations into k groups, in which each observation belongs to the group with the closest mean value. In this experiment, k-means clustering was used to check the clusters of children and their functional development measured with SFA.
      1. Select the option Analyze and Classify followed by the option k-means cluster in the statistical package.
      2. Select crosstabs under Descriptive Statistics, and the following two variables: the cluster variable of ‘belonging to a group’ and the degree variable (‘moderate’ or ‘severe’).
         NOTE: This correspondence is important, because it offers the teacher or therapist information on the homogeneous functional development of users that goes beyond the diagnosis, providing information to propose similar intervention programs in some areas of functional development. This option is expected to facilitate full use of the educational or therapeutic service and its personal and material resources.
      3. Select the Descriptive Statistics option and select crosstabs and the Cohen Kappa Coefficient option.
         NOTE: The Cohen Kappa coefficient is a statistic which measures inter-rater agreement for qualitative (categorical) items. It is generally thought to be a more robust measure than simple percent agreement calculations, as k accounts for the possibility of the agreement occurring by chance. This coefficient provides an indicator of the reliability of the relationship found between the diagnostic classification in moderate-to-severe intellectual disabilities and the clustering of the developmental results obtained with the software.
   3. Use the spreadsheet to generate the spider chart and specific bar graphs for the groups of children with moderate and severe intellectual disabilities.

Representative Results

The present study recruited 11 participants with confirmed diagnoses [both moderate (development quotient = DQ 40-65) and severe (DQ 39-60) intellectual disabilities]. The protocol was tested in a pilot project over 20 months at a Special Education Center. All the diagnoses for this study had been confirmed by a multidisciplinary team in accordance with DSM-5, using the Brunet Lézine Scale, as the degree of affectation was beyond the scope of other scales such as the Wechsler Preschool and Primary Scale of Intelligence (WPPSI). However, the ages of the participants slightly exceeded the chronological age of 6 (Table 2). The actions carried out and the application times in this study can be found in Table 3.

Functional area	Time	Location	Responsible*
Food Autonomy	1 week	During meals in natural contexts.	Teacher or Therapist
Personal Care and Hygiene	1 week	In care and hygiene activities in natural contexts.	Teacher or Therapist
		(center or home)
Dressing and undressing independently	1 week	During the task of dressing and undressing in natural contexts.	Teacher or Therapist
		(center or home)
Control of sphincters	1 week	During the daytime in the natural context.	Teacher or Therapist
		(center or home)
Functional mobility	1 week	During the activities of daily life in natural contexts.	Teacher or Therapist
		(center or home)
Communication and language	1 week	During the activities of daily life in natural contexts.	Teacher or Therapist
		(center or home)
Resolution of tasks in social contexts	1 week	During activities of daily living in natural contexts	Teacher or Therapist
		(center or home)
Interactive and symbolic play	1 week	Situations of symbolic play in natural contexts.,	Teacher or Therapist
		(center or home)
Daily life routines	1 week	During the activities of daily life in natural contexts.	Teacher or Therapist
		(center or home)
Adaptive behavior	1 week	During the activities of daily life in natural contexts.	Teacher or Therapist
		(center or home)
Attention	1 week	During the activities of daily life in natural contexts.	Teacher or Therapist
		(center or home)
NOTE: *Either the teacher or the therapist will contrast the results of the observations gathered at the center with the information gathered from the family, to see whether there are any discrepancies. Prior agreement and consent were always sought from the family, before observation at home in natural contexts that were recorded in video format for subsequent analysis. The number of observations was established according to the type of functional ability, establishing the criterion of different observations during one week in a natural context (center or home).

Table 3: Process to observation in different functional areas.

A comparative analysis can be performed once the professional (teacher or therapist) enters the data in the software. The development of users (students or patients) from the same classroom on the different dimensions of the scale is shown.

In this study, we present some examples of data analysis with the observational protocol coupled with the use of the software. We first performed a cluster analysis, in order to check whether the diagnoses of both medium and severe intellectual disabilities of the children with special educational needs corresponded to their actual development. Using the k-means method, we found 2 clusters belonging to either group. Cluster 1 and Cluster 2 integrated 55% and 45% of the sample, respectively. A cross-table was then constructed to study the relationship between membership of a group cluster and the variable ‘degree of disability’. It was found that all the subjects grouped in Cluster 1 belonged to group a) (moderate intellectual disabilities) and Cluster 2 included all the students belonging to group b) (severe intellectual disabilities) and there was only one with moderate intellectual disabilities (Table 4). A Cohen Kappa coefficient = .820, p = .006 was obtained.

		Cluster case number		Total
		1	2
Disabilities	a	5	0	5
	b	1	5	6
Total		6	5	11

Table 4: Cross Table Disability Cluster Case Number.

The software can, therefore, record the development of the functional abilities of the children under treatment at the same (education or therapy) center and can determine similar and different levels of development. This aspect is very important, as it will facilitate the application of similar intervention programs among children with similar needs, which will in turn facilitate personalized intervention and efficient use of resources.

Progressing with this analysis, a study can be performed of the abilities of the participants in each functional group (disability moderate vs. disability severe). Both Spider and specific bar graphs were used for this purpose see Figure 3 and Figure 4.

Figure 3: Comparative analysis between development of each user (student or patient) and the expected development in children with several intellectual disabilities.
(A) Food Autonomy, (B) Personal Care and Hygiene, (C) Independently dress and undresses, (D) Sphincter control, (E) Functional mobility, (F) Communication and Language, (G) Interactive and symbolic play, (H) Daily life routines, (I) Adaptive behavior. Blue line indicates skill acquisition score. Garnet line indicates maximum score. Error bars included in A, B, C, D, E, F, G, H and I. The error bars are graphical representations of data variability, an error bar indicates the uncertainty of a value. The x-axis represents the participants (with several intellectual disabilities) and the y-axis represents the scores they have obtained in each functional area. The spider graph shows the comparative development of the participants in each functional area. Please click here to view a larger version of this figure.

Figure 4: Comparative analysis between development of each user (student or patient) and the expected development in children with moderate disabilities.
(A) Food Autonomy, (B) Personal Care and Hygiene, (C) Independently dress and undresses, (D) Sphincter control, (E) Functional mobility, (F) Communication and Language, (G) Interactive and symbolic play, (H) Daily life routines, (I) Adaptive behavior. Blue line indicates skill acquisition score. Garnet line indicates maximum score. Error bars included in A, B, C, D, E, F, G, H and I. The error bars are graphical representations of data variability, an error bar indicates the uncertainty of a value. The x-axis represents the participants (with moderate intellectual disabilities) and the y-axis represents the scores they have obtained in each functional area. The spider graph shows the comparative development of the participants in each functional area. Please click here to view a larger version of this figure.

Discussion

When using the software, it is recommended that the teacher or therapist record the development of each child's functional abilities over at least for two sessions. The scale contains 114 items, 11 functional areas, which are divided into 33 functional subareas and the assessment should be the result of observation in natural contexts. In addition, the software allows early-care professionals to record evaluation data with ease and to analyze the data displayed in individual and group graphs, highlighting the development of the users, students or patients, assigned to the classroom in all functional areas and subareas.

The software also facilitates data exportation from a center and classroom to text files (e.g., .csv, .json,) and other spreadsheet formats (e.g., .xlsx), facilitating their transfer to statistical packages, and specific Machine Learning libraries, where it is possible to perform other more complex analyses, such as cluster analysis, mean differences with parametric and/or non-parametric statistics, and reliability analyses of the software for that sample, among many others. In this case, we applied non-parametric statistics with statistical software packages.

The use of the software was only tested at one center, so the work is underway to extend its application to other service centers for users with disabilities. Similarly, and from a perspective of continuous improvement, work is also underway to develop a second phase of the software, in which the possibility will be offered of implementing individualized early stimulation programs, based on the detection of the most affected functional area or areas of development that will be automatically generated. These programs will include the behaviors to be implemented in hierarchical order of acquisition need, as well as specific guidelines for intervention, the necessary materials, time and space for implementation and generalization activities, and a monitoring and evaluation template similar to the one presented for the assessment phase.

It is worth noting that the use of a protocol for the observation of functional abilities that offered no automated processing of the information to be interpreted by the professional, could neither obtain the results for their interpretation in real time, nor conduct the statistical analyses that would guide the subsequent practice of therapeutic intervention.

As a general conclusion, it may be noted that the use of the software has facilitated both the recording and the interpretation of the results by the therapists. Those functional aspects were highly valued in a very positive light, both by the director of the center and the therapists participating in this study.

This method has very few limitations on its implementation, in terms of both material resources (computer characteristics, application requirements) and personal resources (degree of prior knowledge for use of the software application and for the interpretation of the results by the professional recording the observations). The use of the application is not complicated. The most important difficulty can be found in the processing of the database with statistical packages, at which point more sophisticated data calculation and computation abilities are required.

Similarly, and from the perspective of continuous improvement, work is underway to develop a second phase of the software, in which the possibility will be offered of implementing individualized early stimulation programs based on the detection of the most affected functional area or areas of development that will be automatically generated. These programs will include the behaviors to be implemented in hierarchical order of acquisition need, as well as specific guidelines for intervention, necessary materials, time and space for the implementation, and the generalization of activities, and a monitoring and evaluation template similar to the one presented in the assessment phase.

The use of the tool presented here and the proposed improvements in addition to the field work in institutions that work with children with some development affectation will serve the work of researchers in this field and also for programmed use from government institutions, since it facilitates both the registration and the interpretation of the results.

Materials

Name	Company	Catalog Number	Comments
eEarlyCare software	Authors and University of Burgos. Register number 00/2019/3855		Computer application to implement SFA
Scale for the measurement of functional abilities in 0-6 years old (SFA)	Authors and University of Burgos. Register number 00/2019/4253		Scale for the measurement of functional abilities in 0-6 year olds