Implementing Task-based Learning For Cross-cultural Communication In International Chinese Classrooms

Summary

This pilot protocol describes a cluster-randomized comparison of task-based learning and Presentation-Practice-Production in university-level international Chinese classrooms and provides feasibility-oriented representative outcomes for pragmatic performance, intercultural sensitivity, oral fluency, and speaking anxiety.

Abstract

International Chinese programs increasingly require learners to manage cross-cultural interaction in ways that are not only linguistically accurate but also pragmatically appropriate. This pilot methods article presents a classroom protocol for comparing task-based learning with Presentation-Practice-Production in university-level international Chinese classrooms. The protocol is designed to improve methodological transparency by combining matched instructional content, locked teaching and assessment materials, masked rating, reproducible fluency-accuracy-complexity extraction, and a scripted clustered analysis workflow. Intact classes are assigned at the class level to one of the two instructional conditions, and outcomes are collected at baseline, immediate post-test, and follow-up using an oral discourse completion task, a filmed role-play assessment, questionnaires assessing intercultural sensitivity and foreign language speaking anxiety, and speech-based fluency, accuracy, and complexity indices. In a single-site pilot implementation involving four intact classes and 48 adult learners at or above Chinese Proficiency Test level 3, the workflow produced analyzable classroom, rating, questionnaire, and speech data across all three assessment occasions. Descriptive pilot outputs showed larger observed post-test gains in pragmatic appropriateness, intercultural sensitivity, and words per minute in the task-based condition, together with reduced speaking anxiety. Because the protocol was piloted in a very small clustered sample and remains vulnerable to class- and teacher-level confounding, these findings are intended as feasibility-oriented representative outputs rather than robust evidence of comparative effectiveness. This protocol provides a reproducible framework for future multi-class or multi-site implementation, adaptation, and refinement in international Chinese and related second-language settings.

Introduction

This protocol presents a pilot protocol for implementing and evaluating task-based teaching for cross-cultural communication in university-level international Chinese classrooms, rather than an efficacy trial intended to support strong comparative claims. In international Chinese programs, learners are increasingly expected to do more than produce grammatically acceptable utterances. They are also expected to respond appropriately in socially and culturally sensitive situations, in which successful interaction depends on pragmatic choice, relational positioning, and the ability to adjust language under time pressure^1,2,3. For adult university learners, this remains a persistent instructional challenge because progress in vocabulary and grammar does not necessarily transfer to contextually appropriate performance in spontaneous cross-cultural interaction⁴.

Task-based learning has often been proposed as a suitable approach to this challenge because it places meaning-focused activity at the center of instruction and creates opportunities for learners to mobilize linguistic resources in response to communicative need. By contrast, Presentation-Practice-Production typically moves from explicit presentation to controlled practice and then to constrained production, which can support form-focused learning but may not consistently elicit the contingent pragmatic decisions required in cross-cultural communication^5,6. At the same time, performance in this domain cannot be understood through a single outcome type alone. Pragmatic appropriateness, interactional performance, learner-reported intercultural orientation, speaking-related anxiety, and speech-based fluency, accuracy, and complexity each capture different aspects of how learners participate in cross-cultural communication tasks^7,8.

Despite sustained interest in comparing instructional sequences, prior classroom studies have often remained difficult to interpret and replicate. Task procedures are not always described in sufficient detail; instructional content may differ across conditions; intercultural situations are sometimes only loosely operationalized; and scoring, rater training, and data-processing steps are often underreported. As a result, observed differences may reflect not only instructional sequencing but also variation in task design, teacher delivery, measurement procedures, or analytic choices^9,10. These concerns are especially important in intact-classroom research, where class-level allocation is often the only practical option but also introduces teacher effects, class effects, and contamination risks that can be difficult to disentangle, particularly in a single-site pilot with a very small number of clusters¹¹.

The contribution of the present protocol is therefore methodological. It combines matched communicative scenarios and target expressions across conditions, locked teaching and assessment materials, blinded rubric-based scoring with calibration, reproducible extraction of speech-based fluency-accuracy-complexity features, and a scripted clustered analysis workflow within a single auditable classroom design^12,13. Cross-cultural communication is operationalized through an oral discourse completion task and filmed role-play scenarios that require learners to respond to socially situated interactional problems, while intercultural sensitivity and speaking anxiety are measured through structured questionnaires with fixed scoring rules. This combination is intended to capture complementary aspects of learner performance rather than rely on a single indicator of instructional impact¹⁴.

Because the present implementation is restricted to one institution, adult university learners at Chinese Proficiency Test level 3 or above, and four intact classes, it should be interpreted as a pilot feasibility application of the protocol. Under these conditions, unresolved teacher- and class-level confounding cannot be excluded, and any between-condition differences should be treated as descriptive representative outputs rather than robust evidence of comparative effectiveness¹⁵. Within these limits, the protocol is designed to provide a transparent and reproducible framework for classroom delivery, assessment, rating, transcription, speech-feature extraction, and clustered longitudinal analysis that can be refined in future multi-class or multi-site studies in international Chinese education and related second-language settings¹⁶.

Protocol

The study was conducted in accordance with the regulations of the Ethics Committee of Yunnan College of Business Management under approval number [Approval No. YCBM-25168]. Before enrollment, each prospective participant was provided with a written information sheet explaining the study purpose, classroom procedures, audio-video recording, de-identification, voluntary participation, and the right to withdraw without academic penalty. Written informed consent for both study participation and filming before performing any study-specific procedure was obtained¹⁷. The research tools in the protocol are listed in the Table of Materials.

1. Set up classes, outcomes, and assessment timeline

1. Define the target population and teaching setting
   1. Identify adult, non-native learners of Chinese enrolled in university-level International Chinese classes with an explicit oral-communication objective.
   2. Use intact classes as the unit of allocation. Implement one 6-week cross-cultural communication module within the regular semester timetable, with 1 session per week and 50 min of instruction per session.
   3. In the pilot implementation, include 4 intact classes with 12 learners per class to yield a planned sample of 48 learners.
   4. Use the same lead instructor to teach both conditions with locked lesson plans and standardized timing. Schedule the two instructional arms on different weekdays, keep class rosters fixed throughout the intervention period, and prohibit distribution of arm-specific teaching materials outside the assigned classroom session to reduce contamination between conditions.
2. Define primary, supportive, and secondary outcomes
   1. Define pragmatic performance as the main classroom-performance domain. Assess it using two complementary instruments: an oral discourse completion task (ODCT) and a filmed role-play assessment.
   2. Administer 5 ODCT prompts at each time point. Score the ODCT on a 0-25 composite scale using four prespecified domains: pragmatic appropriateness (0-10), sociocultural alignment (0-5), discourse completeness (0-5), and linguistic control (0-5). Sum the four domain scores to obtain the total ODCT score. Provide the fixed ODCT prompts in Supplementary File 1.
   3. Administer one filmed role-play task per pair or triad at each timepoint. Score the role-play on a separate 0-25 composite scale using four prespecified domains: interactional appropriateness (0-10), responsiveness and repair (0-5), relational stance management (0-5), and linguistic control (0-5). Calculate the final score as the mean of two masked raters. If the two total scores differ by more than 3 points, obtain a third masked rating and calculate the final score as the mean of the two closest totals.
   4. Measure intercultural sensitivity using the Intercultural Sensitivity Scale, administered in a bilingual Chinese-English paper form with 24 items scored on a 5-point Likert scale and a total score range of 24-120.
   5. Measure speaking anxiety using the short-form Foreign Language Classroom Anxiety Scale, administered in the same bilingual Chinese-English format with 8 items scored on a 5-point Likert scale and a total score range of 8-40, where higher scores indicate greater speaking anxiety.
   6. Define supportive speech-production outcomes as fluency-accuracy-complexity indices derived from the role-play recordings, including words per minute, silent pauses of 0.25 s or longer per minute, percentage of T-units containing at least one morpho-syntactic error, and mean clause length.
   7. For the Intercultural Sensitivity Scale, treat the total score as missing if more than 2 items are unanswered. Otherwise, replace missing items with the respondent’s mean score on completed items from that scale. For the short-form Foreign Language Classroom Anxiety Scale, treat the total score as missing if more than 1 item is unanswered. Otherwise, replace the missing item with the respondent’s mean score on completed items from that scale.
   8. Summarize the baseline learner profile by instructional condition using age, gender, native language group, Chinese Proficiency Test level, and prior exposure to Chinese-speaking environments. Report these variables descriptively in Table 1 to document cohort composition rather than claim baseline equivalence in a four-cluster pilot.
3. Fix assessment timepoints and map the study flow
   1. Schedule three assessment timepoints: baseline (T0, within 3 days before the first intervention lesson), immediate post-test (T1, within 5 days after the final lesson), and follow-up (T2, exactly 28 days after T1).
   2. At each time point, administer the ODCT first, the filmed role-play second, and the questionnaires last. Keep this order identical across all classes.
   3. Record screening, consent, class-level allocation, outcome-specific completion, exclusions, attrition, and the final analysis set throughout the study so that the full participant flow can be presented transparently in Figure 1¹⁸.

2. Lock materials, run cluster randomization, and enforce masking

1. Freeze teaching and assessment materials
   1. Store the fixed teaching and assessment materials in a locked study folder before the first intervention lesson.
   2. Set the locked study folder to read-only before the first study lesson and keep it unchanged throughout data collection.
   3. Create a new folder version only when a revision is unavoidable. Record each change, date, reason, and responsible person in a plain-text changelog.
   4. Record the exact software versions used in the workflow, including Praat v6.4.58, ELAN v7.0, Audacity v3.7.7, R v4.5.2, tidyverse v2.0.0, lme4 v1.1-38, emmeans v2.0.1, and lmerTest v3.2-0.
2. Generate class-level allocation with a fixed seed
   1. Assign each intact class a unique identifier (C1-C4) before allocation.
   2. Generate the allocation sequence in R v4.5.2 using the fixed random seed 20260301. Use simple 1:1 random allocation without blocking or stratification because only four intact classes are available in the pilot.
   3. Have the research coordinator generate the allocation sequence. Ensure that this coordinator is not involved in classroom teaching, rating, or scoring of identified materials. Have the same coordinator assign classes after freezing the baseline roster and before the first intervention lesson begins.
   4. Record the date and time of allocation, the seed value, the randomization method, the R version, the name of the sequence generator, the name of the person who assigned the classes, and the output of sessionInfo().
3. Restrict access, minimize contamination, and mask condition labels
   1. Restrict access to the allocation file to the research coordinator and keep the file separate from all de-identified datasets.
   2. Use the same lead instructor across both conditions, but schedule the two arms on different weekdays, distribute printed materials only during the relevant session, and prohibit circulation of arm-specific files outside class.
   3. Instruct learners not to share study materials, screenshots, or role-play prompts with students from other classes until completion of T2 data collection.
   4. Mask condition labels from raters and staff handling de-identified data by using neutral condition codes in filenames, rating sheets, transcripts, and analysis datasets.
   5. Use neutral identifiers (class_id, student_id, pair_id) in manifests, rating sheets, transcripts, and analysis files. Store any linkage file separately in a secure institutional storage location.
      CAUTION: Do not allow raters to access the allocation file, weekly lesson plans, or any raw filename that directly reveals the instructional condition.

3. Recruit intact classes and deliver matched instruction

1. Recruit intact classes and obtain permissions
   1. Obtain permission from course coordinators and the lead instructor before contacting students.
   2. Recruit intact classes that include oral communication as a formal course objective and can complete the full 6-week module within the same academic term.
2. Screen eligibility and collect written consent
   1. Administer a screening and consent form capturing age, gender, native language, Chinese Proficiency Test level, and time spent in Chinese-speaking regions during the prior 2 years.
   2. Include adult, non-native speakers at Chinese Proficiency Test level 3 or above who have not lived in a Chinese-speaking region for more than 3 consecutive months during the prior 2 years.
   3. Exclude learners who do not consent to filming because role-play recording is a required component of the protocol.
3. Build and document baseline characteristics
   1. Assign each consenting participant a student_id and store the linkage file separately from de-identified datasets.
   2. Enter one row per learner in the baseline file with fields for condition, class_id, age, gender, native language group, Chinese Proficiency Test level, and prior exposure to Chinese-speaking environments.
   3. Record baseline characteristics descriptively by instructional condition for later reporting in Table 1. Do not interpret them as evidence that class-level confounding has been removed.
4. Prepare matched syllabi across conditions
   1. Create parallel weekly syllabi for task-based learning and Presentation-Practice-Production using the same communicative scenarios, communicative functions, and target expressions.
   2. Use the same six weekly themes in both arms: classroom disagreement, teacher consultation, peer collaboration, host-family interaction, internship communication, and public-service interaction.
   3. Compile reader-facing versions of the weekly lesson schedule, target-expression mapping, and role-play scenarios in Supplementary File 1.
5. Deliver task-based learning lessons using a fixed task cycle
   1. Introduce the scenario and communicative goal in Chinese for 5 min.
   2. Run a planning phase in small groups for 5 min.
   3. Run the main task phase in pairs or triads for 18 min, with minimal teacher intervention limited to clarification of task instructions.
   4. Conduct a reflection and feedback phase for 20 min, focusing on language choice, interactional strategy, pragmatic appropriateness, and repair.
   5. Use the same realized timing in all six task-based lessons unless a deviation is unavoidable. Record any deviation in the fidelity log.
6. Deliver Presentation-Practice-Production lessons with matched content and timing
   1. Present target expressions and sentence patterns for 16 min using example dialogues or projected materials.
   2. Conduct controlled practice for 16 min, focusing on accurate production of the target forms.
   3. Conduct constrained production for 16 min using the same weekly scenario and target expressions.
   4. Match overall lesson duration, topic coverage, and scenario content to the corresponding task-based lessons. Record all deviations in the fidelity log.
7. Record instructional fidelity after each lesson
   1. After each lesson, log the date, class, scenario label, realized duration of each phase, and any deviation from the planned structure, including any unplanned change in scenario, target expression, or time allocation.
   2. Store fidelity logs in a spreadsheet under the processed-study records.
   3. If a major fidelity deviation occurs, retain the session in the study file, flag it in the fidelity log, and describe the deviation in the final report rather than deleting the session post hoc.
      NOTE: Typical fidelity deviations include shortened task time because of classroom interruptions, temporary equipment repositioning, or omission of one planned feedback step. Log and interpret these events rather than silently removing them.

4. Conduct assessments, record media, and score performance

1. Administer the ODCT at each time point
   1. Present ODCT prompts one at a time on paper using the fixed prompt set.
   2. Allow 30 s of planning per prompt, record one spoken response per prompt, and keep the planning-time setting constant across T0, T1, and T2.
2. Administer the role-play and record interactions
   1. Provide written scenario and role cards and allow 2 min of planning.
   2. Record one 8-min interaction per pair or triad and keep the target interaction duration constant across timepoints.
   3. Define each role-play as an intercultural communication scenario involving a socially situated choice problem, such as disagreement, request mitigation, face management, clarification, or repair under differing expectations.
3. Standardize recording and create a media manifest
   1. Use the same digital video camera and the same external microphone throughout the study.
   2. Record all sessions at 1920 × 1080 resolution, 30 frames/s, and 48 kHz audio sampling. Keep the input-gain setting unchanged after the first recording day.
   3. Place the external microphone at the center of the pair or triad table, approximately 60 cm from the speakers, and document the setup in the study recording checklist.
   4. Save each file under a standardized filename encoding class_id, student_id or pair_id, timepoint, and task type.
   5. Store all recordings in the raw-media folder. Create a manifest listing filenames, file sizes, technical notes, and checksums, and compute SHA-256 checksums for every raw media file.
      NOTE: After confirming that all files are readable and that checksums match the manifest, set the raw-media folder to read-only and conduct all downstream processing on copies stored in the processed-study folder.
4. Train raters and score pragmatic performance under blinding
   1. Recruit two primary raters and one reserve rater, all with experience in Chinese language teaching or language assessment, and ensure that none have access to class allocation.
   2. Provide a calibration set comprising 12 ODCT responses and 6 role-play recordings, balanced across timepoints and condition codes, and select this set before formal scoring begins. Do not modify the calibration set after inspecting outcome patterns.
   3. Compute inter-rater agreement using the intraclass correlation coefficient ICC(2,k) in R v4.5.2. Repeat calibration until ICC reaches at least 0.75, and record the ICC type, calibration-set size, ICC values, and stop decisions in a calibration log file stored under the processed-study records¹⁹.
   4. During formal scoring, assign anonymized files to raters in random order and record scores in a spreadsheet with fields for class_id, student_id, timepoint, task type, rater_id, and score.
   5. If the prespecified reliability threshold is not reached after two recalibration rounds, suspend formal scoring, revise the calibration materials, repeat training, and document the event transparently in the calibration log.
      CAUTION: Do not begin formal scoring until the ICC threshold is reached and documented.
5. Transcribe role-play recordings and compute fluency-accuracy-complexity indices
   1. Transcribe all analyzable role-play recordings from both arms at T0, T1, and T2. Do not perform post hoc sampling after outcome inspection.
   2. Export ELAN annotations as UTF-8 tab-delimited text and store the files in the processed-study transcripts folder.
   3. Apply the same transcription conventions and segmentation rules across all files. Compile reader-facing versions of the rubrics, questionnaire scoring rules, transcription guidelines, and error taxonomy in Supplementary File 2.
   4. Segment each transcript into T-units and clauses. Record segment counts, speaker labels, and any exclusions in the transcript manifest.
   5. Inspect audio timing and compute pause-based measures using Praat v6.4.58, with the silent-pause threshold fixed at 0.25 s.
   6. Compute words per minute using a fixed rule in which repeated lexical items are counted as produced words unless they are immediate self-corrections abandoned before completion. Define speaking time as the total duration of the learner’s voiced output, excluding examiner instructions and non-task setup talk.
   7. Compute silent pauses of 0.25 s or longer per minute, percentage of T-units containing at least one morpho-syntactic error, and mean clause length as total words divided by total clauses.
   8. Write all speaker-level fluency-accuracy-complexity features to one long-format file with one row per learner per timepoint.
   9. If a recording is unusable because of technical failure or irrecoverable speaker overlap, log the reason, retain the file record in the manifest, code the derived features as missing, and carry the exclusion forward into the participant-flow record for Figure 1 and the final Results narrative.
6. Administer questionnaires and assemble the outcome dataset
   1. Administer the Intercultural Sensitivity Scale and the short-form Foreign Language Classroom Anxiety Scale after completion of the oral tasks at each timepoint.
   2. Check questionnaires for missing items before learners leave the room whenever feasible.
   3. If questionnaires remain incomplete, apply the prespecified missing-item rule described in Section 1.2.7. and document the resulting missingness in the analysis log.
   4. Merge ODCT scores, role-play scores, fluency-accuracy-complexity feature files, and questionnaire scores into one long-format dataset with one row per learner per timepoint.

5. Fit models and document analysis outputs

1. Validate identifiers and prepare the analysis-ready dataset
   1. Verify internal consistency of class_id, student_id, pair_id, and timepoint across all sources using scripted checks.
   2. Document exclusions, missingness, unusable recordings, transcript exclusions, and questionnaire-scoring decisions in a log file.
   3. Save the analysis-ready dataset as a CSV file and include a README listing the scripts, parameters, and software versions required to regenerate it.
2. Fit descriptive mixed-effects models for clustered longitudinal outcomes
   1. Because only 4 intact classes are randomized in the pilot, treat the analysis as feasibility-oriented and descriptive rather than confirmatory.
   2. Fit one coherent mixed-effects model for each outcome using R v4.5.2, lme4 v1.1-38, and lmerTest v3.2-0, with fixed effects for group, time, and group × time and random intercepts for class_id and student_id²⁰.
   3. Use the same model form across all outcomes and treat the T0 measurement as one of the repeated observations within the longitudinal model rather than fitting a separate post hoc baseline-adjusted model for selected outcomes.
   4. Report model-estimated marginal means, mean differences, 95% confidence intervals, and exact p values as descriptive model-based summaries only. Interpret nominal significance testing cautiously because the number of clusters is very small.
   5. Apply a false-discovery-rate adjustment only to the family of secondary outcomes and record the exact function name and settings used in the analysis script²¹.
3. Prepare outputs for later reporting
   1. Preserve the baseline dataset structure, variable labels, and denominators in a form that allows learner characteristics to be reported clearly in Table 1.
   2. Preserve the repeated-measures outcome dataset in a form that allows pragmatic outcomes to be summarized across T0, T1, and T2 in Table 2 and supportive speech and questionnaire outcomes to be summarized across the same three occasions in Table 3.
   3. Preserve plotting scripts and output labels in a form that allows pragmatic trajectories to be visualized in Figure 2 and multipanel supportive-outcome profiles to be visualized in Figure 3, with consistent scale naming and uncertainty definitions.
4. Record package versions and regenerate outputs
   1. At the start of analysis, write sessionInfo() and installed package information to version-tracking files.
   2. Load tidyverse v2.0.0 for data handling and emmeans v2.0.1 for estimated marginal means and contrasts, and record the package versions used for the final analysis.
   3. Regenerate all reportable outputs directly from the analysis-ready dataset using the saved scripts. Compile reader-facing workflow documentation, the codebook, and the script inventory in Supplementary File 3.

Results

Study flow and cohort

Recruitment, screening, written consent, class-level allocation, outcome-specific completion, and the final analysis set across baseline (T0), immediate post-test (T1), and follow-up (T2) are summarized in Figure 1. In the pilot implementation, 4 intact classes comprising 48 learners were allocated at the class level, with 2 classes assigned to task-based learning (TBL, n = 24) and 2 classes assigned to Presentation-Practice-...

Discussion

This article presents a pilot classroom protocol for comparing task-based learning and Presentation-Practice-Production in university-level International Chinese classes. Its main contribution is methodological. Rather than offering another broad comparison between two teaching sequences, the protocol specifies how such a comparison can be implemented under classroom conditions while keeping core sources of procedural variation under control. Weekly themes are matched across conditions, target expressions are fixed in ad...

Materials

List of materials used in this article

Name	Company	Catalog Number	Comments
Analysis scripts folder	This study	Version 1.0	Fixed scripts for class randomization, pause detection, dataset construction, model fitting, and plotting.
Audio editing software	Audacity Team	Version 3.7.7	Used for audio inspection and trimming when needed.
Audio monitoring headphones	Audio-Technica	ATH-M20x	Used to monitor playback quality, clipping, and background noise during recording checks.
Classroom recording checklist	This study	Version 1.0	Session-level checklist for camera placement, microphone placement, audio level, storage capacity, room noise, and role-card distribution.
Data dictionary / codebook	This study	Version 1.0	Defines class IDs, learner IDs, timepoints, outcome variables, missingness rules, and derived-feature names.
External microphone	Sony	ECM-LV1	Used as the fixed external microphone for classroom task recording.
Fidelity log template	This study	Version 1.0	Used to record date, class, scenario label, realized duration of each phase, and deviations from the planned structure.
Filmed role-play scoring rubric	This study	Version 1.0	Fixed 0–25 rubric for role-play assessment, including dual masked scoring and adjudication rules.
Intercultural Sensitivity Scale form	This study	Version 1.0	Bilingual English–Chinese questionnaire, 24 items, administered at T0, T1, and T2.
Oral discourse completion task prompts	This study	Version 1.0	Fixed 5-prompt set used at all three assessment occasions.
Phonetic analysis software	Praat	Version 6.4.58	Used for pause detection with the silent-pause threshold fixed at 0.25 s.
Presentation-Practice-Production lesson plans	This study	Version 1.0	Fixed 6-week PPP lesson plans matched to the same themes, target expressions, and communicative functions as the TBL condition.
Raw media storage	SanDisk	SDSSDE60-1T00-G25	External SSD used for encrypted raw-media storage and checksum-verified archiving.
Role-play scenarios and role cards	This study	Version 1.0	Fixed filmed role-play materials used for assessment at T0, T1, and T2.
Statistical computing software	R Project for Statistical Computing	Version 4.5.2	Used for class randomization, clustered longitudinal models, and output generation.
Task-based learning lesson plans	This study	Version 1.0	Fixed 6-week TBL lesson plans using the same weekly themes and target expressions as the PPP condition.
Transcription and annotation software	ELAN	Version 7.0	Used for role-play transcription and time-aligned annotation.
Transcription guidelines	This study	Version 1.0	Fixed conventions for transcription, speaker labeling, segmentation, and exclusion handling.
Tripod / camera support	Manfrotto	MKELES5BK-BH	Used for stable classroom camera placement.
Video camera	Sony	FDR-AX43	Fixed classroom video camera used across all sessions.
Workstation computer	Dell	DC15250	Used for transcription, feature extraction, and statistical analysis.
Short-form Foreign Language Classroom Anxiety Scale form	This study	Version 1.0	Bilingual English–Chinese questionnaire, 8 items, administered at T0, T1, and T2.