Evaluating University Student Classroom Engagement, Reflective Expression, and Prosocial Intention Using Behavioral Observation and Learning Analytics

Summary

This protocol integrates classroom observation, reflective writing, prosocial-intention measurement, and learning analytics to evaluate university students' behavioral engagement, reflective expression, and socially oriented learning outcomes over a 4-week instructional window. It specifies ethics approval, instrument contents, software workflow, missing-data handling, troubleshooting procedures, and interpretation of representative outcomes, including observed implementation constraints.

Abstract

To address the need for multidimensional assessments of student involvement beyond traditional metrics, this protocol describes a structured approach for evaluating classroom engagement, reflective expression, learning analytics activity, and prosocial intention in undergraduate courses. The protocol combines interval-based behavioral observation, weekly structured reflective writing, de-identified learning-management-system records, and a pre/post prosocial-intention questionnaire. The revised procedures provide a reproducible behavioral coding model, reflective-expression rubric with score examples, software and data-management steps, and explicit troubleshooting guidance for observer disagreement, missing platform records, low reflection submission rates, poor camera visibility, participant attrition, and inconsistent prerequisite knowledge. This protocol should be used when researchers need a low-intrusion, reproducible way to combine live classroom behavior, student self-interpretation, and routine learning management system (LMS) activity during undergraduate instruction. It is less suitable for fully asynchronous courses without observable interaction, settings that prohibit independent observation or record linkage, or studies requiring biometric or emotion-recognition inference. The representative results are reported from the implemented empirical application after verified ethics approval, de-identification of course records, and repository preparation.

Introduction

Evaluating student participation in higher education requires a comprehensive, multi-source approach. Although student engagement is widely conceptualized as a multidimensional construct encompassing observable behavior, cognitive effort, and environmental interaction, current analytical approaches predominantly rely on superficial log data, such as login frequencies and system-recorded actions¹. This overreliance presents a significant challenge for practical classroom-based research. The most critical forms of instructional participation—direct peer interactions, cognitive reflection, and social contributions—often occur entirely ....

Protocol

All methods involving human participants were performed in compliance with the Declaration of Helsinki and the relevant institutional guidelines. For the implemented study, the protocol was approved by the Ethics Committee of Nanchang Institute of Technology before recruitment began (approval no.: NITE778291). Written informed consent was obtained from every participating student. Only de-identified aggregate classroom observation, reflection-score, learning analytics, and questionnaire data were used for analysis; identifiable classroom video, names, institutional IDs, and unredacted reflection text were excluded from public release.

Results

Participant inclusion, data completeness, and score reliability
An initial cohort of 204 students across all class sections was invited to participate, yielding 198 consenting individuals. Fourteen cases were subsequently excluded due to course withdrawal during the protocol window or incomplete data linkage across the observation, reflection, and platform records. Consequently, the final analytic sample comprised 184 students. Participant characteristics, weekly completion rates, and overall data co.......

Discussion

This study proposes a practical protocol that integrates behavioral observation, reflective writing, and learning analytics to comprehensively investigate university students' participation, self-expression, and prosocial awareness within a single instructional window. By capturing different dimensions of engagement, this tri-modal approach addresses recent critiques in higher education research, which argue that time-in-use metrics and sensor-based monitoring are insufficient when evaluated in isolation

Materials

List of materials used in this article

Name	Company	Catalog Number	Comments
Classroom observation coding sheet	In-house	Six binary behavioral indicators	Interval-based coding of on-task attention, verbal participation, peer collaboration, task-focused note-taking/device use, off-task distraction, and help-seeking.
GraphPad Prism	Graphpad	Version 10.2.3	Generation of workflow diagrams, weekly trend plots, distribution plots, and association figures.
Institutional learning management system / course platform	Host university	Existing institutional LMS account system	Collection of de-identified learning analytics.
Microsoft PowerPoint	Microsoft	Microsoft 365 v2402	Generation of workflow diagrams, weekly trend plots, distribution plots, and association figures.
R	R Foundation		ggplot2 3.5.1
Spreadsheet software	Microsoft Corporation	Microsoft Excel 365 v2402	Data entry, cleaning, de-identification checks, and merging of observation/reflection/analytics datasets.
SPSS Statistics	IBM Corp	Version 26.0	Statistical analysis software; Descriptive statistics, repeated-measures comparisons, Cohen's kappa, ICC, Cronbach's alpha, correlations, and regression models.