A Survey and Protocol for Assessing Online Learning Experience, Basic Psychological Needs, and Learning Engagement in University Students

Summary

This protocol provides a reproducible survey and partial least squares structural equation modeling workflow for assessing association pathways among online learning experience, basic psychological needs, and learning engagement in university students, including questionnaire adaptation, screening, measurement assessment, indirect-association testing, robustness checks, and reproducibility records.

Abstract

Online learning is embedded in higher education, but evaluations often rely on access, satisfaction, or completion indicators that do not identify where engagement problems arise. This article presents a reproducible English-language survey and partial least squares structural equation modeling protocol for assessing association pathways among online learning experience, basic psychological needs, and learning engagement in university students. Online learning experience is measured through platform usability, instructional support, interaction quality, and learning flexibility; basic psychological needs through autonomy, competence, and relatedness; and learning engagement through behavioral, cognitive, and emotional engagement. The workflow specifies item-source mapping, questionnaire adaptation, expert review, pilot testing, electronic consent, eligibility screening, response-quality checks, construct scoring, common method bias diagnostics, higher-order construct estimation, measurement-model assessment, structural pathway testing, indirect-association analysis, psychological-need sensitivity analysis, regression-based robustness checks, and reproducibility file locking. Representative results from 386 valid responses demonstrate acceptable reliability, moderate construct correlations, retained measurement quality, a supported indirect association through basic psychological needs, nonsignificant control paths, and uneven dimensional scores. The protocol supports transparent diagnosis of how online learning experience is associated with psychological need satisfaction and engagement while avoiding causal interpretation from cross-sectional survey data.

Introduction

Online learning has become part of routine higher education through learning management systems, video platforms, recorded materials, synchronous sessions, and blended course designs. However, many evaluations still rely on access logs, satisfaction ratings, or completion indicators, which show whether students used an online course but do not clarify whether lower engagement is associated with platform usability, instructional support, interaction quality, learning flexibility, psychological need satisfaction, or a specific engagement dimension¹. Online learning experience should therefore not be reduced to a single satisfaction score. Prior online engagement research emphasizes learner-content, learner-instructor, and learner-learner interaction as important elements of students’ online learning experience². The present protocol measures online learning experience through platform usability, instructional support, interaction quality, and learning flexibility.

Learning engagement is also multidimensional. It includes behavioral participation, cognitive investment, and emotional involvement. Online-course engagement instruments show that engagement cannot be fully represented by visible participation or task completion alone³. Broader educational technology research also shows that behavioral engagement is often easier to observe than cognitive or affective engagement⁴. For this reason, the protocol measures learning engagement through behavioral, cognitive, and emotional engagement rather than collapsing engagement into one undifferentiated score.

Self-determination theory provides the theoretical basis for linking online learning experience with learning engagement. Basic psychological need theory identifies autonomy, competence, and relatedness as central conditions for motivation and adjustment⁵. In online learning, autonomy refers to perceived choice and control, competence to perceived capability in completing learning tasks, and relatedness to felt connection with instructors and classmates. Prior online learning studies using self-determination theory have linked contextual support, need satisfaction, motivation, and engagement^6,7. These findings support examining basic psychological needs as an indirect-association construct between online learning experience and learning engagement, while avoiding causal interpretation because the data are cross-sectional.

A methodological problem in online engagement research is the lack of consistent operational structure. Studies vary in how they define learning conditions, measure engagement, and connect the learning environment to students’ psychological experience⁸. Self-report surveys are also vulnerable to careless responding, patterned answers, post hoc exclusions, missing-data decisions, and common method bias⁹. Therefore, this protocol fixes the English-language questionnaire structure, item-source mapping, scoring rules, electronic consent procedure, eligibility screening, attention-check rule, completion-time threshold, straight-lining criterion, long-string response check, missing-data handling, model specification, and dataset-locking procedure before structural analysis begins.

The conceptual framework specifies online learning experience as the independent construct, basic psychological needs as the indirect-association construct, and learning engagement as the outcome construct. The primary pathway tests whether online learning experience is associated with basic psychological needs, whether basic psychological needs are associated with learning engagement, and whether online learning experience remains directly associated with learning engagement after accounting for basic psychological needs. The protocol is guided by four research questions: whether the structured survey protocol can measure the three constructs with acceptable response quality and construct reliability; whether online learning experience is positively associated with basic psychological needs and learning engagement; whether the association between online learning experience and learning engagement is partly carried through basic psychological needs as an indirect association; and whether psychological-need subdimension analyses and regression-based robustness checks support the same interpretation as the primary PLS-SEM workflow.

This article is positioned as a method-focused contribution rather than a conventional explanatory cross-sectional study. The protocol provides a reproducible workflow for questionnaire adaptation, expert review, pilot testing, survey administration, response screening, construct scoring, measurement-model assessment, structural pathway testing, indirect-association analysis, sensitivity analysis, robustness checking, and reproducibility file locking. Transparent educational research workflows require analytic decisions to be documented before final model interpretation¹⁰. Partial least squares structural equation modeling is used because it allows measurement and structural components to be examined within one workflow, with reflective first-order constructs and analytically derived higher-order construct scores through a disjoint two-stage approach¹¹. Prior work on online student engagement, social presence, learner support, and retention shows that communication, support, connection, and inclusive course design matter beyond simple access or flexibility^12,13. Systematic reviews further indicate that behavioral, cognitive, and emotional engagement may respond differently to online learning conditions¹⁴. Representative results are therefore used only to demonstrate the output of the protocol, not to claim causal effects from cross-sectional survey data¹⁵.

Protocol

The study protocol was reviewed and approved by the Universiti Malaya Research Ethics Committee (Non-Medical) under approval number UM.TNC (P&I)/UMREC_4892 on 7 August 2025. Conduct all procedures in accordance with institutional guidelines for research involving human participants. Obtain electronic informed consent before administering the questionnaire. Use anonymous participant codes during recruitment, screening, scoring, analysis, and file storage. The research tools and software are listed in the Table of Materials.

1. Define the study design and analytical framework

1. Use a cross-sectional quantitative survey design to assess association pathways among online learning experience, basic psychological needs, and learning engagement in university students.
2. Define online learning experience as the independent latent construct, basic psychological needs as the indirect-association latent construct, and learning engagement as the outcome latent construct.
3. Specify the three multidimensional constructs before data collection. Represent online learning experience by platform usability, instructional support, interaction quality, and learning flexibility; basic psychological needs by autonomy need, competence need, and relatedness need; and learning engagement by behavioral engagement, cognitive engagement, and emotional engagement.
4. Use partial least squares structural equation modeling as the primary analytical method to assess measurement quality, higher-order construct scores, structural paths, and the prespecified indirect pathway.
5. Specify the main pathway before analysis: online learning experience → basic psychological needs, basic psychological needs → learning engagement, and online learning experience → learning engagement.
6. Treat all structural paths as association estimates. Do not interpret any pathway as causal because all variables are measured at one time point. Conduct a priori sample-size and power checking before recruitment. Use at least 380 valid responses as the target sample and 350 valid responses as the minimum acceptable post-screening sample.
7. Present the protocol workflow in Figure 1, including recruitment, consent, questionnaire administration, expert review, pilot testing, response screening, dataset locking, measurement assessment, structural testing, indirect-association analysis, robustness checking, and reproducibility-file preparation.
   NOTE: Keep the conceptual model and structural paths unchanged from questionnaire construction through final analysis.

Figure 1: Operational workflow for assessing online learning experience, basic psychological needs, and learning engagement. The figure shows the protocol workflow from recruitment, consent, questionnaire administration, expert review, pilot testing, response screening, and dataset locking to measurement assessment, structural testing, indirect-association analysis, robustness checking, and reproducibility-file preparation. Please click here to view a larger version of this figure.

2. Recruit eligible participants

1. Recruit university students who have participated in at least one online or blended course during the current or most recent academic semester.
2. Include participants who are 18 years of age or above, currently enrolled as undergraduate or postgraduate students, exposed to online or blended learning, able to read the English-language questionnaire, and willing to provide electronic informed consent.
3. Exclude responses when consent is not confirmed, eligibility criteria are not met, the attention-check item is failed, duplicate submission is detected, completion time is below the prespecified threshold, straight-lining is detected, or unresolved eligibility inconsistency is present.
4. Recruit participants through university learning platforms, course communication groups, student email lists, and institutional student channels.
5. Do not collect direct identifiers, including names, student identification numbers, telephone numbers, private email addresses, exact residential addresses, IP addresses, device identifiers, or account identifiers.
6. Record recruitment dates, recruitment channels, submitted responses, exclusion counts by category, retained responses, and final valid sample size in the fieldwork log and screening log.
   PAUSE POINT: Check recruitment progress weekly. Review only response count, recruitment-channel coverage, and preliminary exclusion rate. Do not inspect construct scores, correlations, or pathway estimates during recruitment.

3. Construct the questionnaire

1. Prepare a structured English-language questionnaire with electronic informed consent, eligibility screening, participant background information, substantive scale items, and one instruction-based attention-check item.
2. Use a 5-point Likert response scale for all substantive items: 1 = strongly disagree, 2 = disagree, 3 = neutral, 4 = agree, and 5 = strongly agree.
3. Adapt online learning experience items from prior measures of online learning perception, instructional support, interaction quality, platform usability, and learning flexibility. Adapt basic psychological needs items from established autonomy, competence, and relatedness measures. Adapt learning engagement items from established behavioral, cognitive, and emotional engagement measures.
4. Revise item wording only to fit university online and blended learning contexts. Use clear, non-leading wording and avoid wording that implies high satisfaction, high psychological need satisfaction, or high engagement is expected.
5. Prepare an item-source mapping table before pilot testing. Record item code, construct, dimension, adapted wording, source basis, adaptation note, questionnaire language, and scoring direction for each substantive item.
6. Use 33 substantive Likert-scale items: 12 for online learning experience, nine for basic psychological needs, and 12 for learning engagement. Use three items for each online learning experience dimension, three items for each basic psychological need dimension, and four items for each learning engagement dimension.
7. Place one instruction-based attention-check item after the basic psychological needs block. Use this item only for response-quality screening and exclude it from construct scoring, reliability analysis, validity analysis, PLS-SEM estimation, indirect-association testing, and robustness analysis.
8. Keep the expected questionnaire completion time within 8–12 min during pilot testing. Use Table 1 to document construct domains, dimensions, item codes, number of items, response scale, scoring direction, and adaptation basis. Use Supplementary File 1 to provide the full questionnaire, item-source mapping, variable coding, and scoring rules.
   NOTE: Keep the predictor, indirect association, and outcome items conceptually distinct.

Table 1: Construct domains, dimensions, questionnaire structure, and scoring rules. This table summarizes the questionnaire structure, including construct domains and dimensions, item codes and numbers, example items, adaptation basis, response scale, and scoring rules. The attention-check item is identified as a screening-only item. Please click here to download this Table.

4. Conduct expert review and pilot testing

1. Invite three to five experts in educational psychology, online learning, quantitative methodology, higher education research, or survey design to review the questionnaire.
2. Ask experts to assess item clarity, construct relevance, wording redundancy, response burden, cultural suitability, language clarity, and alignment between each item and its assigned dimension.
3. Record expert-review outcomes in an item-revision log, including original wording, reviewer concern, revision decision, final wording, and reason for retention, revision, or removal.
4. Revise items that are ambiguous, double-barreled, overly similar, culturally inappropriate, weakly aligned with the target dimension, or inconsistent with the English-language survey context.
5. Conduct a pilot test with 30–50 university students who meet the same eligibility criteria as the main study. Ask pilot participants to identify unclear wording, technical display problems, excessive length, repetitive wording, or discomfort caused by any questionnaire item.
6. Record pilot completion time for each participant. Define the formal-study minimum completion-time threshold as the larger value between 3 min and one-third of the pilot median completion time.
7. Calculate preliminary internal consistency for each construct and inspect item-total correlations after pilot testing. Revise or remove weak items only when deletion improves measurement quality and does not weaken construct coverage.
8. Finalize and lock the questionnaire, codebook, item-source mapping, expert-review log, pilot-test record, completion-time threshold, attention-check rule, duplicate-detection rule, and exclusion criteria before formal recruitment.
   PAUSE POINT: Do not begin formal recruitment until all questionnaires, screening, and coding files have been locked.

5. Administer the online questionnaire

1. Build the questionnaire in an online survey system that supports anonymous participation, consent confirmation, eligibility screening, required-response settings, response-time recording, non-identifying duplicate-submission restriction, attention-check placement, and spreadsheet export.
2. Place electronic informed consent before eligibility screening and questionnaire items. Allow participants to proceed only after confirming voluntary participation and present the study purpose in neutral language. State that the study examines students’ perceptions of online learning, psychological need satisfaction, and learning engagement.
3. Set consent, eligibility questions, and all 33 substantive Likert-scale items as required fields. Disable fields for direct identifying information. Do not request names, student numbers, telephone numbers, private email addresses, exact residential addresses, IP addresses, device identifiers, or account identifiers.
4. Allow participants to exit the questionnaire before final submission without penalty. Restrict repeated submissions only through non-identifying survey-platform settings when available. Keep the questionnaire open for 4–6 weeks. Monitor response counts twice per week and document recruitment progress in the fieldwork log.
   NOTE: Do not provide personal score feedback after questionnaire submission.

6. Export and protect the raw data

1. Export all submitted responses in .xlsx and .csv formats immediately after recruitment closes.
2. Save one unchanged .xlsx file as RawData_OnlineLearning_BPN_Engagement_YYYYMMDD.xlsx and one unchanged .csv file as RawData_OnlineLearning_BPN_Engagement_YYYYMMDD.csv.
3. Create a working copy for screening, coding, and analysis. Name it WorkingData_OnlineLearning_BPN_Engagement_YYYYMMDD.xlsx. Assign each submitted response an anonymous participant code in the format P001, P002, P003, and so on.
4. Store the raw dataset in a read-only folder and create a checksum, file-version record, or institutional version-history record before screening begins.
5. Preserve the de-identified raw submitted-record archive and screening metadata in the protected institutional verification folder. Do not include raw excluded records, directly identifiable information, or participant-level raw data in the manuscript or supplementary files submitted for publication.
   PAUSE POINT: Do not begin screening until the raw .xlsx file, raw .csv file, fieldwork log, raw-data version record, and protected raw-data folder have been saved.

7. Screen responses and apply exclusion rules

1. Apply all exclusion rules before examining construct correlations, measurement-model output, structural paths, or indirect-association estimates. Remove responses without consent confirmation or without age, enrollment, online-learning exposure, or questionnaire-language eligibility.
2. Remove duplicate responses according to the prespecified duplicate-detection rule. Identify possible duplicates only by non-identifying survey-platform duplicate flags and anonymous response-pattern information. Retain the earliest complete response when duplicate responses are identified.
3. Remove responses that fail the instruction-based attention-check item or are completed below the formal-study minimum completion-time threshold. Define straight-lining as selecting the same response option for at least 90% of the 33 substantive Likert-scale items. Remove responses that meet this criterion.
4. Flag long-string response patterns when the same response option is selected fo^{r 2}0 or more consecutive substantive Likert-scale items. Remove flagged responses only when they also show implausibly short completion time, failed attention check, straight-lining, or another prespecified quality violation.
5. Flag inconsistent background responses when the participant reports no online or blended learning exposure but reports weekly online learning hours above zero. Remove responses with unresolved eligibility inconsistency.
6. Record every excluded case in a de-identified screening log with anonymous participant code, exclusion category, exclusion reason, screening date, and screening decision.
7. Use Table 2 to document submitted responses, exclusion categories, exclusion counts, retained responses, and final valid sample size. If raw exclusion counts cannot be reconstructed for the representative dataset, state this explicitly.
   NOTE: Exclude cases only according to prespecified consent, eligibility, duplicate, attention-check, completion-time, response-pattern, missingness, or consistency rules. Do not begin coding until the raw dataset, cleaned dataset, screening log, exclusion-rule file, fieldwork log, and dataset-version record have been saved separately and locked.

Table 2: Response screening, exclusion rules, and data-handling decisions. This table summarizes the prespecified screening rules and data-handling decisions for consent, eligibility, duplicate responses, attention check, completion time, response-pattern checks, missingness, and dataset locking. Exclusion rules are applied before examining construct correlations, measurement-model output, structural paths, or indirect-association estimates. Each excluded response is recorded in a de-identified screening log using the anonymous participant code, exclusion category, exclusion reason, screening date, and screening decision. Please click here to download this Table.

8. Handle missing data

1. Prevent missing values in consent, eligibility questions, and the 33 substantive Likert-scale items by setting these fields as required in the survey system. Inspect the cleaned dataset for missing values after screening.
2. Treat missing values in optional background variables as “not reported.” Do not impute optional demographic or background variables.
3. Exclude a background variable from covariate analysis if its missing rate exceeds 10%. Retain a background variable as a covariate only when its missing rate is 10% or lower and the variable is theoretically relevant.
4. Do not impute missing values in the 33 substantive Likert-scale items. If substantive item missingness is detected despite required-response settings, identify whether the problem is caused by survey export, survey logic, or incomplete submission handling before finalizing the cleaned dataset.
   NOTE: Record the missing-data check. Report the number of missing values and the missing rate for participant code, background variables, substantive Likert-scale items, dimension scores, and global construct scores.

9. Code variables and calculate scores

1. Code all substantive Likert-scale items from 1 to 5: 1 = strongly disagree, 2 = disagree, 3 = neutral, 4 = agree, and 5 = strongly agree. Code gender, academic level, year of study, discipline category, weekly online learning hours, and prior online learning experience according to the locked codebook.
2. Inspect all variables for out-of-range values before calculating dimension or global construct scores.
3. Calculate online learning experience dimension scores as item means: platform usability = mean(OLE_PU1–OLE_PU3), instructional support = mean(OLE_IS1–OLE_IS3), interaction quality = mean(OLE_IQ1–OLE_IQ3), and learning flexibility = mean(OLE_LF1–OLE_LF3).
4. Calculate basic psychological needs dimension scores as item means: autonomy need = mean(BPN_AU1–BPN_AU3), competence need = mean(BPN_CO1–BPN_CO3), and relatedness need = mean(BPN_RE1–BPN_RE3).
5. Calculate learning engagement dimension scores as item means: behavioral engagement = mean(LE_BE1–LE_BE4), cognitive engagement = mean(LE_CE1–LE_CE4), and emotional engagement = mean(LE_EE1–LE_EE4).
6. Calculate global construct scores as dimension means: online learning experience = mean (platform usability, instructional support, interaction quality, learning flexibility); basic psychological needs = mean (autonomy need, competence need, relatedness need); and learning engagement = mean (behavioral engagement, cognitive engagement, emotional engagement).
7. Retain item-level data for measurement-model assessment, dimension scores for higher-order construct estimation and subdimension analysis, and global construct scores for descriptive statistics, correlation analysis, structural-model checking, and regression-based robustness analysis.
   NOTE: Do not include the attention-check item in any score, reliability analysis, validity analysis, PLS-SEM estimation, indirect-association analysis, or robustness analysis. Do not standardize original 1–5 scores before descriptive analysis.

10. Control and evaluate common method bias

1. Apply procedural controls before analysis by using anonymous responses, neutral wording, separated construct blocks, one instruction-based attention-check item, required-response settings, and restricted raw-data access.
2. After response screening, use Harman’s single-factor test to examine whether one unrotated factor accounts for a dominant share of item variance. Treat the result as acceptable when the first unrotated factor accounts for less than 50% of total variance.
3. Calculate full collinearity variance inflation factor values for the focal constructs. Treat values below 3.30 as indicating that severe common method bias is unlikely. Record common method bias as a limitation if the first unrotated factor accounts for 50% or more of total variance or if any full collinearity variance inflation factor is 3.30 or higher.
   NOTE: Do not state that common method bias is absent. State only whether the diagnostics indicate severe evidence of a dominant common method factor.

11. Conduct descriptive and preliminary checks

1. Use R 4.4.1 for descriptive statistics, distribution checks, correlation analysis, and regression-based robustness analysis. Calculate frequencies and percentages for categorical participant characteristics.
2. Calculate mean, standard deviation, minimum, maximum, skewness, and kurtosis for substantive items, dimension scores, and global construct scores.
3. Inspect item and score distributions for floor effects, ceiling effects, and severe non-normality. Define a potential floor or ceiling effect as more than 20% of participants selecting the lowest or highest response option for a substantive item.
4. Calculate Pearson correlations among global constructs and dimension scores. Treat correlations as preliminary descriptive evidence only and do not use them as a substitute for the prespecified structural model.
   NOTE: Record floor or ceiling effects as data features. Do not delete cases for distributional reasons unless they also violate prespecified response-quality rules.

12. Estimate the higher-order measurement model

1. Use Smart PLS 4.1 for the primary PLS-SEM analysis and import the cleaned item-level dataset. Specify platform usability, instructional support, interaction quality, learning flexibility, autonomy need, competence need, relatedness need, behavioral engagement, cognitive engagement, and emotional engagement as reflective first-order constructs.
2. Use the disjoint two-stage approach. In the first stage, estimate the reflective first-order constructs and save their latent variable scores.
3. In the second stage, use latent variable scores of platform usability, instructional support, interaction quality, and learning flexibility to represent online learning experience; latent variable scores of autonomy need, competence need, and relatedness need to represent basic psychological needs; and latent variable scores of behavioral engagement, cognitive engagement, and emotional engagement to represent learning engagement.
4. Treat higher-order construct scores as analytically derived representations of multidimensional constructs. Do not describe them as formative unless a formative model is explicitly specified and justified. Save first-stage settings, second-stage settings, latent variable scores, and model-output files in the verification folder.
   NOTE: Use the same higher-order construct procedure in the main structural model, psychological-need subdimension analysis, and regression-based robustness analysis.

13. Assess the measurement model

1. Assess the measurement model before interpreting the structural model. Examine standardized outer loadings for all retained indicators. Retain indicators with loadings of at least 0.70 when possible.
2. Review indicators with loadings between 0.40 and 0.70. Remove an indicator only when deletion improves reliability or convergent validity and does not weaken content coverage. Remove indicators below 0.40 unless a documented theoretical reason supports retention.
3. Assess internal consistency using Cronbach’s alpha and composite reliability. Treat values between 0.70 and 0.95 as acceptable. Assess convergent validity using average variance extracted. Treat values of at least 0.50 as acceptable.
4. Assess discriminant validity using the heterotrait-monotrait ratio. Use 0.85 as the conservative threshold and 0.90 as the maximum acceptable threshold for theoretically adjacent constructs.
5. Record retained indicators, standardized outer loadings, Cronbach’s alpha, composite reliability, average variance extracted, heterotrait-monotrait values, and any item-deletion decisions in the locked measurement-model output.
   NOTE: Do not delete items only to improve statistical values. Remove an item only when statistical weakness and conceptual redundancy are both documented.
   PAUSE POINT: Do not begin structural testing until retained indicators, loading values, reliability values, convergent validity, discriminant validity, codebook entries, and measurement-output values have been checked against the locked analysis files.

14. Test the structural model and mediation pathway

1. Specify three structural paths: online learning experience → basic psychological needs, basic psychological needs → learning engagement, and online learning experience → learning engagement.
2. Include weekly online learning hours and prior online learning experience as prespecified control variables. Treat gender, academic level, year of study, and discipline category as descriptive variables unless included in a prespecified sensitivity model.
3. Examine structural-model collinearity before interpreting path coefficients. Treat variance inflation factor values below 3.30 as indicating that harmful collinearity is unlikely.
4. Estimate standardized path coefficients, t values, p values, and 95% confidence intervals using 5,000 bootstrap resamples with two-tailed testing. Use bias-corrected confidence intervals when available.
5. Estimate the indirect pathway from online learning experience to learning engagement through basic psychological needs. Treat the indirect-association pathway as supported when the bootstrap confidence interval for the indirect effect does not include zero.
6. Record the direct effect, indirect effect, total effect, R² values, adjusted R² values, f² effect sizes, structural-model variance inflation factor values, and control-variable paths in the locked structural-model output. Interpret all structural estimates as association-based pathways rather than causal effects.
   NOTE: Do not remove a prespecified path because it is nonsignificant. Retain the prespecified model unless a coding error or model-specification error is identified.

15. Examine psychological need subdimensions

1. Replace the global basic psychological needs construct with autonomy need, competence need, and relatedness need as parallel indirect-association constructs. Estimate indirect pathways from online learning experience to learning engagement through autonomy need, competence need, and relatedness need separately.
2. Use 5,000 bootstrap resamples for each subdimension pathway. Compare the direction, magnitude, and confidence interval of each subdimension indirect pathway. Treat this analysis as a sensitivity analysis rather than the primary model.
   NOTE: Do not reinterpret the subdimension sensitivity analysis as a new primary model.

16. Conduct regression-based robustness analysis

1. Export the cleaned construct-score dataset and save it as ConstructScores_OnlineLearning_BPN_Engagement_YYYYMMDD.csv. Use R 4.4.1 to regress global basic psychological needs on global online learning experience, weekly online learning hours, and prior online learning experience.
2. Regress global learning engagement on global online learning experience, global basic psychological needs, weekly online learning hours, and prior online learning experience. Estimate the indirect pathway using 5,000 bootstrap resamples.
3. Compare the regression-based indirect pathway with the PLS-SEM indirect pathway. Classify the pathway as robust only when the direction remains consistent and the substantive interpretation does not change. Save the R script, bootstrap settings, and robustness output in the verification folder.
   PAUSE POINT: Do not begin final file locking until the structural model, subdimension analysis, and regression-based robustness analysis have been cross-checked.

17. Handle suboptimal analytical outcomes

1. If Cronbach’s alpha or composite reliability is below 0.70, inspect item-total correlations and standardized outer loadings. If the average variance extracted is below 0.50, inspect retained indicators and identify whether one or more indicators weaken convergent validity.
2. If heterotrait-monotrait values exceed the prespecified threshold, inspect item wording for semantic overlap between theoretically close constructs. Remove a weak item only when it is statistically weak and conceptually redundant. Keeping theoretically adjacent constructs separate when merging them would reduce conceptual clarity.
3. If the indirect pathway is not supported, record the result as an unsupported indirect association. If control variables show nonsignificant associations, record the results without overinterpretation.
   NOTE: Do not revise the theoretical model after analysis to force stronger support.

18. Lock software outputs and reproducibility files

1. Create a final institutional verification folder named Verification_OnlineLearning_BPN_Engagement_YYYYMMDD.
2. Save the raw dataset archive, cleaned dataset, construct-score dataset, codebook, questionnaire, item-source mapping table, expert-review log, pilot-test record, fieldwork log, screening log, exclusion-rule file, completion-time threshold record, Smart PLS outputs, R scripts, R output tables, software-version record, and version-change log in the verification folder.
3. Save Smart PLS 4.1 project files and exported measurement-model, structural-model, bootstrap, and heterotrait-monotrait outputs in a subfolder named SmartPLS_Output. Save R scripts and R output tables in a subfolder named R_Output. Save the questionnaire, item-source mapping, variable coding, and scoring rules as Supplementary File 1.
4. Save final analytic dataset verification, screening summary, missing-data record, common method bias diagnostics, measurement-model results, structural-model results, indirect-association results, subdimension analysis, regression-based robustness record, software settings, and version record as Supplementary File 2.
5. Do not include directly identifiable information, raw excluded records, or participant-level raw data in the manuscript or supplementary files submitted for publication. Lock the final verification folder after consistency checking. Save later corrections as new, dated versions and document the reason in the version-change log.

Results

Final analytic dataset and participant characteristics

The final analytic dataset included 386 university students. Formal recruitment was conducted from 18 August 2025 to 26 September 2025 through university learning platforms, course communication groups, student email lists, and institutional student channels. A total of 432 submitted responses were screened, and 386 responses were retained, giving a valid-response rate of 89.4%. The locked dataset contained 33 substantive Likert-scale items and one instruction-based attention-check item. The attention-check item was used only for response-quality screening and was excluded from construct scoring, reliability analysis, validity analysis, PLS-SEM estimation, indirect-association analysis, and regression-based robustness analysis. The final sample included 223 female students, 154 male students, and nine participants who selected “prefer not to say” or another response option. There were 318 undergraduate students and 68 postgraduate students. Year-of-study distribution included 110 Year 1 students, 102 Year 2 students, 103 Year 3 students, and 71 students in Year 4 or above. Discipline categories included business and management, social sciences, engineering and technology, medicine and health sciences, humanities and arts, and education. Weekly online learning exposure included less than 2 h per week in 41 students, 2–4 h in 116 students, 5–7 h in 137 students, 8–10 h in 62 students, and more than 10 h in 30 students. Prior online learning experience included none in 38 students, less than 6 months in 68 students, 6–12 months in 190 students, and more than 12 months in 90 students. Weekly online learning hours and prior online learning experience were retained as prespecified control variables.

Expert review, pilot testing, and questionnaire verification

Expert review was completed by five experts in educational psychology, online learning, quantitative methodology, higher education research, and survey design. The review identified 13 wording issues, four redundancy issues, and three construct-alignment issues. Fourteen items were revised for clarity, contextual fit, or construct alignment, and no item was removed before pilot testing. Pilot testing was completed by 42 university students who met the main-study eligibility criteria. The median completion time was 9.6 min, and the minimum completion-time threshold was set at 3.2 min. Preliminary Cronbach’s alpha values ranged from 0.721 to 0.814. No item was removed after pilot testing. The final English-language questionnaire included 12 online learning experience items, nine basic psychological needs items, 12 learning engagement items, and one instruction-based attention-check item. Item-source mapping, variable coding, and scoring rules were locked before formal analysis and are provided in Supplementary File 1.

Response screening, missingness, and data-quality checks

After screening, 386 valid responses were retained from 432 submitted responses. Exclusions included four responses without consent confirmation, 17 eligibility exclusions, eight duplicate responses, six attention-check failures, five below-threshold completion-time responses, four straight-lining responses, and two unresolved eligibility inconsistencies. Six long-string patterns were flagged, but these overlapped with other exclusion categories and were not counted as an additional independent exclusion category. Exclusions were recorded in the de-identified screening log. No duplicate participant codes were retained in the locked dataset. No missing values were observed in the 33 substantive Likert-scale items, background variables, dimension-score variables, or global construct-score variables. No imputation was applied. No retained response met the straight-lining rule, no retained response showed a long-string pattern of 20 or more consecutive identical substantive responses, and no unresolved inconsistency was observed between online learning exposure and weekly online learning hours.

Descriptive distribution and common method bias diagnostics

The global online learning experience score was 3.47 ± 0.45. The global basic psychological needs score was 3.40 ± 0.49. The global learning engagement score was 3.42 ± 0.48. Instructional support had the highest mean among online learning experience dimensions, and interaction quality had the lowest mean. Competence need had the highest mean among basic psychological need dimensions, and relatedness need had the lowest mean. Cognitive engagement had the highest mean among learning engagement dimensions, and emotional engagement had the lowest mean. Item-distribution checks did not show severe floor or ceiling effects. The maximum item-level floor response rate was 1.55% for OLE_PU3. The maximum item-level ceiling response rate was 12.95% for OLE_IS2. Both values were below the prespecified 20% threshold. Common method bias diagnostics did not show severe evidence of a dominant common method factor. Harman’s single-factor test showed that the first unrotated factor explained 24.12% of the total variance. Full collinearity diagnostics produced a maximum variance inflation factor of 1.527.

Measurement-model quality

As shown in Table 3, Cronbach’s alpha values ranged from 0.733 for interaction quality to 0.801 for cognitive engagement at the dimension level. The global online learning experience construct showed an alpha of 0.878, basic psychological needs showed an alpha of 0.846, and learning engagement showed an alpha of 0.867. Standardized outer loadings ranged from 0.704 to 0.842 across the 33 substantive items. Composite reliability values ranged from 0.849 to 0.889. Average variance extracted values ranged from 0.653 to 0.702. The maximum heterotrait-monotrait ratio was 0.742. These values met the prespecified measurement-model criteria.

Table 3: Reliability, descriptive statistics, and measurement-model quality. This table reports Cronbach’s alpha, composite reliability, average variance extracted, standardized loading ranges, means, standard deviations, and retention decisions for the first-order dimensions and higher-order constructs. The attention-check item was excluded from reliability, validity, and model estimation. The maximum HTMT value among the first-order constructs was 0.742, below the prespecified conservative threshold of 0.85. The full outer-loading table and HTMT matrix are provided in Supplementary File 2. Abbreviations: AVE = average variance extracted. Please click here to download this Table.

Online learning experience was correlated with basic psychological needs (r = 0.501) and learning engagement (r = 0.497). Basic psychological needs were correlated with learning engagement (r = 0.510).

Structural pathway and indirect-association results

As shown in Figure 2, online learning experience was positively associated with basic psychological needs (β = 0.501, t = 11.259, p < 0.001, 95% CI: 0.414 to 0.588). Basic psychological needs were positively associated with learning engagement (β = 0.349, t = 7.256, p < 0.001, 95% CI: 0.255 to 0.444). Online learning experience remained directly associated with learning engagement (β = 0.324, t = 6.699, p < 0.001, 95% CI: 0.229 to 0.419). The model explained 25.1% of the variance in basic psychological needs and 33.9% of the variance in learning engagement (Table 4). The f² effect size for the online learning experience on basic psychological needs was 0.332. The f² effect sizes for the online learning experience and basic psychological needs on learning engagement were 0.118 and 0.138, respectively.

Figure 2: Structural model with standardized association-based path coefficients. The figure shows the standardized pathways among online learning experience, basic psychological needs, and learning engagement, including the direct path, indirect path, R² values, and prespecified control variables. All paths are interpreted as association-based estimates. Please click here to view a larger version of this figure.

Table 4: Explained variance of endogenous constructs. This table reports the coefficient of determination (R²), adjusted R² values, and predictor variables for each endogenous construct in the structural model. All results are interpreted as association-based estimates because the data were cross-sectional. Abbreviations: OLE_TOTAL = global online learning experience score; BPN_TOTAL = global basic psychological needs score; LE_TOTAL = global learning engagement score. Please click here to download this Table.

Table 5 shows that weekly online learning hours were not significantly associated with basic psychological needs (β = 0.001, p = 0.986) or learning engagement (β = −0.020, p = 0.634).

Table 5: Structural pathway estimates and effect sizes. This table reports standardized structural-pathway estimates, corresponding t values, p values, 95% confidence intervals (CI), f² effect sizes, and pathway decisions for the hypothesized model. Control-variable pathways are also presented. All pathways are interpreted as association-based estimates because the data were cross-sectional. Abbreviations: OLE_TOTAL = global online learning experience score; BPN_TOTAL = global basic psychological needs score; LE_TOTAL = global learning engagement score; WEEKLY_HOURS = weekly online learning hours; PRIOR_EXPERIENCE = prior online learning experience; CI = confidence interval. Please click here to download this Table.

Prior online learning experience was not significantly associated with basic psychological needs (β = −0.005, p = 0.916) or learning engagement (β = 0.013, p = 0.754). The indirect pathway from online learning experience to learning engagement through basic psychological needs was supported. The indirect effect was 0.175, with a 95% bootstrap confidence interval from 0.124 to 0.231. The direct effect of the online learning experience on learning engagement was 0.324, and the total effect was 0.499 (Table 6). This pathway was interpreted as an association-based indirect pathway rather than causal mediation.

Table 6: Direct, indirect, and total association estimates. This table reports the direct, indirect (mediated), and total associations between online learning experience and learning engagement, including bootstrap 95% confidence intervals (CI) and pathway decisions. All effects are interpreted as association-based estimates because the data were cross-sectional. Abbreviations: OLE_TOTAL = global online learning experience score; BPN_TOTAL = global basic psychological needs score; LE_TOTAL = global learning engagement score; CI = confidence interval. Please click here to download this Table.

Psychological-need subdimension sensitivity analysis and robustness analysis

The indirect pathway through autonomy need was supported (indirect effect = 0.087, 95% CI: 0.049 to 0.131). The indirect pathway through competence need was supported and showed the largest indirect effect (indirect effect = 0.148, 95% CI: 0.094 to 0.205). The indirect pathway through relatedness need was also supported (indirect effect = 0.092, 95% CI: 0.056 to 0.132) (Table 7).

Table 7: Sensitivity analysis of basic psychological need subdimensions. This table reports indirect associations through the autonomy, competence, and relatedness subdimensions of basic psychological needs, including bootstrap 95% confidence intervals (CI) and pathway decisions. All effects are interpreted as association-based estimates because the data were cross-sectional. Abbreviations: OLE_TOTAL = global online learning experience score; LE_TOTAL = global learning engagement score; BPN_AU = autonomy need; BPN_CO = competence need; BPN_RE = relatedness need; CI = confidence interval. Please click here to download this Table.

Regression-based robustness analysis produced the same substantive interpretation as the primary PLS-SEM model. Online learning experience remained positively associated with basic psychological needs (β = 0.501, 95% CI: 0.414 to 0.588). Basic psychological needs remained positively associated with learning engagement after adjustment for online learning experience and control variables (β = 0.349, 95% CI: 0.255 to 0.444). The regression-based indirect effect was 0.175, with a 95% bootstrap confidence interval from 0.124 to 0.231. Because the data were cross-sectional, all pathways were interpreted as association-based estimates rather than causal effects (Table 8).

Table 8: Regression-based robustness analysis. This table reports regression-based robustness results for the structural pathways, including adjusted direct and indirect association estimates with bootstrap 95% confidence intervals (CI). All effects are interpreted as association-based estimates because the data were cross-sectional. Abbreviations: OLE_TOTAL = global online learning experience score; BPN_TOTAL = global basic psychological needs score; LE_TOTAL = global learning engagement score; CI = confidence interval. Please click here to download this Table.

Supplementary File 1: Questionnaire, variable coding, and scoring rules. This file provides the English-language questionnaire, eligibility-screening variables, background-variable coding, item wording, item-source mapping, response scale, scoring formulas, variable dictionary, data-handling rules, and consistency checks.Please click here to download this file.

Supplementary File 2: Final analytic dataset verification, analysis settings, and reproducibility records. This file records the locked analytic dataset, fieldwork summary, expert review and pilot-test records, screening summary, missing-data checks, reliability and measurement-model output, structural-model results, sensitivity analysis, robustness analysis, and reproducibility checklist.Please click here to download this file.

Discussion

This protocol provides a reproducible methodological workflow for assessing association pathways among online learning experience, basic psychological needs, and learning engagement in university students. Rather than treating online learning as a single condition, the protocol separates the perceived learning environment into platform usability, instructional support, interaction quality, and learning flexibility. This structure allows researchers to identify which part of the online learning experience is most closely associated with psychological need satisfaction and engagement, instead of relying only on overall satisfaction or course-completion indicators¹⁶.

A central methodological contribution is the prespecification of questionnaire coding, eligibility screening, response-quality checks, construct scoring, dataset locking, and model testing. These steps are important because survey-based educational research is sensitive to inattentive responses, patterned answers, inconsistent eligibility responses, missing-data decisions, and post hoc exclusions¹⁷. By placing these decisions before structural analysis, the protocol makes the analytic pathway reproducible and reduces the risk that screening or scoring rules are changed after results are known.

The protocol also clarifies how the association pathway should be interpreted. Online learning experience is modeled as the independent construct, basic psychological needs as the indirect-association construct, and learning engagement as the outcome construct. This structure is theoretically grounded in self-determination theory, because autonomy, competence, and relatedness provide a mechanism for understanding why some online learning environments are associated with stronger engagement¹⁸. However, because the protocol uses cross-sectional questionnaire data, the pathway must be interpreted as an indirect association rather than causal mediation.

The multidimensional treatment of learning engagement is another strength of the protocol. Behavioral engagement, cognitive engagement, and emotional engagement are measured separately before being summarized as a global engagement construct. This prevents engagement from being reduced to attendance, task completion, or login behavior alone¹⁹. In practical terms, the protocol can distinguish students who complete required tasks from students who are cognitively invested, emotionally involved, and willing to continue participating in online learning activities.

The protocol is also useful for diagnosing weak points in online learning design. For example, a course may provide adequate platform access but still show low relatedness or emotional engagement if communication is delayed, fragmented, or not connected to meaningful learning interaction. This distinction is consistent with online learning models that emphasize social presence, teaching presence, and cognitive presence rather than technology availability alone²⁰. Therefore, the protocol supports targeted improvement of platform design, instructor support, interaction structure, and psychological need satisfaction.

Several limitations should guide interpretation and future use. First, the cross-sectional design does not establish temporal order, so causal claims should not be made²¹. Second, all main variables are self-reported, and common method diagnostics cannot fully remove response-bias concerns²². Third, the model does not directly observe instructor behavior, course design quality, platform analytics, or institutional support. Future studies can extend the protocol through longitudinal measurement, multi-group comparison, learning-analytics integration, or mixed-methods follow-up while retaining the same reproducible screening, scoring, model-testing, and verification logic²³.

Materials

List of materials used in this article

Name	Company	Catalog Number	Comments
Qualtrics online survey platform	Qualtrics	Institutional account; version not applicable	Used to administer the anonymous English-language questionnaire, including consent confirmation, eligibility screening, required-response settings, AC1 attention-check item, response-time recording, and data export.
Microsoft Excel for Microsoft 365	Microsoft Corporation	Microsoft 365 institutional version	Used to store the raw exported dataset, cleaned dataset, codebook, screening log, construct-score file, and verification records in .xlsx and .csv formats.
G*Power	Heinrich Heine University Düsseldorf	G*Power 3.1.9.7	Used for a priori sample-size and power checking before recruitment.
SmartPLS	SmartPLS GmbH	SmartPLS 4.1 Professional	Used for the primary PLS-SEM workflow, including disjoint two-stage higher-order construct estimation, measurement-model assessment, structural-pathway testing, bootstrapping, R², f², and model-output export.
R statistical software	R Foundation / CRAN	R 4.4.1	Used for descriptive statistics, Pearson correlations, common method bias diagnostics, and regression-based robustness analysis.
R package: psych	CRAN	psych 2.4.6	Used for descriptive statistics, reliability checks, and psychometric summaries.
R package: boot	CRAN	boot 1.3-30	Used for bootstrap-based regression robustness checks.
R package: semPlot	CRAN	semPlot 1.1.6	Used only for optional path-diagram cross-checking when R-based visual verification is needed.
OneDrive for Business or SharePoint version history	Microsoft Corporation	Microsoft 365 institutional account	Used to store the verification folder and preserve version history for raw data, cleaned data, codebook, screening log, SmartPLS outputs, R scripts, and supplementary files.
Questionnaire and coding file	Author-prepared study file	Supplementary File 1	Contains the final questionnaire, item-source mapping, variable names, coding rules, dimension-score formulas, global construct-score formulas, and data-handling rules.
Screening and reproducibility record	Author-prepared study file	Supplementary File 2	Contains final dataset verification, screening checks, missing-data record, reliability results, common method bias diagnostics, measurement-model output, structural-model output, indirect-association results, sensitivity analysis, robustness record, and final consistency checklist.
Figure-preparation software	Adobe Illustrator or equivalent vector-editing software	Institutional version	Used to prepare publication-ready workflow and structural-model figures.