Behavioral Engagement Assessment in University Classrooms via Deep Learning-based Video Object Detection

Summary

This study assesses students’ behavioral engagement by detecting their behaviors in university classroom videos using a deep learning algorithm. Seven behaviors that are positively correlated with learning engagement are identified, and a scoring model is used to assess the overall engagement of individual students and the class.

Abstract

This study aims to assess students’ learning engagement in university classrooms using deep learning-based video object detection. To do so, via correlation analysis, this research first identified seven classroom behaviors presenting highly positive correlation with learning engagement as indicators to measure students’ learning engagement; then it collected 30 synchronized videos of real classroom teaching from 6 classes from Shandong University of Science and Technology (SDUST) and divided them into a training set and a test set. After the seven behaviors were manually annotated in the training data, a machine learning algorithm was then trained in a supervised manner on this set. Once trained, the model generated initial annotations for the remaining unlabeled data. To achieve more accurate and efficient classroom behavior recognition, this study selected two representative algorithms, namely, Faster R-CNN and YOLOv5s, for behavior detection experiments. Based on a comparison of their detection performance in terms of accuracy and time cost, YOLOv5s was selected for classroom behavior detection in this study. Finally, this study used the focus group method to assign scores to each behavior and develop a three-level learning engagement scoring model. Based on automatically measured behavioral data, the model enables real-time, automatic assessment of learning engagement at both the individual and class levels.

Introduction

Learning engagement, referring to students’ participation, concentration, and efforts in learning, consists of behavioral engagement, emotional engagement, and cognitive engagement¹. Students’ learning engagement reveals their learning situation and helps teachers adjust teaching strategies². However, classroom learning engagement is mainly assessed by students’ self-reports and teachers’ observations³. The high teacher-student ratio in China’s universities means it takes teachers a great deal of time and effort to use traditional manual methods for learning engagemen....

Protocol

All videos were obtained and used in compliance with current ethical practices of human research as prescribed by Shandong University of Science and Technology. The approval has been received for the usage of the data.

First, correlation analysis was conducted to identify students' behaviors that could serve as indicators of behavioral engagement. After the dataset was established by frame extraction and divided into a training set and a test set, two representative algorithms for object detection were run in the training set and compared in terms of accuracy and efficiency. Finally, the algorithm with better performance (YOLO-v5s) was sele....

Results

By detecting behaviors and using the scoring model, automatic measurement of students’ behavioral engagement was achieved. Taking one class in the video as an example, the real-time behavioral engagement of each student could be calculated based on behavioral detection results and the scoring model. Figure 6 illustrates the behavioral engagement of two students in this class, displaying their engagement levels at various time points. It was found that, on average, student A .......

Discussion

This study realized the automatic recognition and measurement of behavioral engagement. Compared to previous studies, the behaviors identified were more representative of university classrooms (such as using cell phone), and the assessment of behavioral engagement was presented in figures, making it intuitive to see. This automatic measurement approach enabled teachers to efficiently and clearly assess the engagement of individual students as well as the class as a whole, thereby supporting personalized instruction and i.......

Materials

List of materials used in this article

Name	Company	Catalog Number	Comments
Classroom videos			The video data used in this study are collected from the direct recording platform of Shandong University of Science and Technology (SDUST), which are synchronized videos of real classroom teaching of the course Academic English. 30 videos from 6 classes of non-English majors were collected, about 50 minutes each, to establish a trainable data set of students’ classroom behavior.
IBM SPSS Statistics 26	IBM		SPSS for Windows is a modular software package that integrates data entry, organization, and analysis functions.Its basic functions include data management, statistical analysis, chart analysis, output management, and so on. SPSS is mainly used for consistency analysis and correlation analysis in this article.
Labeling 1.8.6			Labeling is a visual image annotation tool. It is written in Python and utilizes Qt for its graphical interface. Annotations are saved as XML files in the PASCAL VOC format, which is used by ImageNet. Additionally, it supports the YOLO format. It's used for datasets required for object detection networks such as Faster R-CNN, YOLO, and SSD to annotate objects in images.
MMDetection 2.22.0			MMDetection, 2.22.0, MMDetection is an object detection toolbox that includes rich object detection, instance segmentation, and panoramic segmentation methods, as well as related components and modules.