Implementing Task-based Learning For Cross-cultural Communication In International Chinese Classrooms

Summary

This pilot protocol describes a cluster-randomized comparison of task-based learning and Presentation-Practice-Production in university-level international Chinese classrooms and provides feasibility-oriented representative outcomes for pragmatic performance, intercultural sensitivity, oral fluency, and speaking anxiety.

Abstract

International Chinese programs increasingly require learners to manage cross-cultural interaction in ways that are not only linguistically accurate but also pragmatically appropriate. This pilot methods article presents a classroom protocol for comparing task-based learning with Presentation-Practice-Production in university-level international Chinese classrooms. The protocol is designed to improve methodological transparency by combining matched instructional content, locked teaching and assessment materials, masked rating, reproducible fluency-accuracy-complexity extraction, and a scripted clustered analysis workflow. Intact classes are assigned at the class level to one of the two instructional conditions, and outcomes are collected at baseline, immediate post-test, and follow-up using an oral discourse completion task, a filmed role-play assessment, questionnaires assessing intercultural sensitivity and foreign language speaking anxiety, and speech-based fluency, accuracy, and complexity indices. In a single-site pilot implementation involving four intact classes and 48 adult learners at or above Chinese Proficiency Test level 3, the workflow produced analyzable classroom, rating, questionnaire, and speech data across all three assessment occasions. Descriptive pilot outputs showed larger observed post-test gains in pragmatic appropriateness, intercultural sensitivity, and words per minute in the task-based condition, together with reduced speaking anxiety. Because the protocol was piloted in a very small clustered sample and remains vulnerable to class- and teacher-level confounding, these findings are intended as feasibility-oriented representative outputs rather than robust evidence of comparative effectiveness. This protocol provides a reproducible framework for future multi-class or multi-site implementation, adaptation, and refinement in international Chinese and related second-language settings.

Introduction

This protocol presents a pilot protocol for implementing and evaluating task-based teaching for cross-cultural communication in university-level international Chinese classrooms, rather than an efficacy trial intended to support strong comparative claims. In international Chinese programs, learners are increasingly expected to do more than produce grammatically acceptable utterances. They are also expected to respond appropriately in socially and culturally sensitive situations, in which successful interaction depends on pragmatic choice, relational positioning, and the ability to adjust language under time pressure^1,2,3. For adult university learners, this remains a persistent instructional challenge because progress in vocabulary and grammar does not necessarily transfer to contextually appropriate performance in spontaneous cross-cultural interaction⁴.

Task-based learning has often been proposed as a suitable approach to this challenge because it places meaning-focused activity at the center of instruction and creates opportunities for learners to mobilize linguistic resources in response to communicative need. By contrast, Presentation-Practice-Production typically moves from explicit presentation to controlled practice and then to constrained production, which can support form-focused learning but may not consistently elicit the contingent pragmatic decisions required in cross-cultural communication^5,6. At the same time, performance in this domain cannot be understood through a single outcome type alone. Pragmatic appropriateness, interactional performance, learner-reported intercultural orientation, speaking-related anxiety, and speech-based fluency, accuracy, and complexity each capture different aspects of how learners participate in cross-cultural communication tasks^7,8.

Despite sustained interest in comparing instructional sequences, prior classroom studies have often remained difficult to interpret and replicate. Task procedures are not always described in sufficient detail; instructional content may differ across conditions; intercultural situations are sometimes only loosely operationalized; and scoring, rater training, and data-processing steps are often underreported. As a result, observed differences may reflect not only instructional sequencing but also variation in task design, teacher delivery, measurement procedures, or analytic choices^9,10. These concerns are especially important in intact-classroom research, where class-level allocation is often the only practical option but also introduces teacher effects, class effects, and contamination risks that can be difficult to disentangle, particularly in a single-site pilot with a very small number of clusters¹¹.

The contribution of the present protocol is therefore methodological. It combines matched communicative scenarios and target expressions across conditions, locked teaching and assessment materials, blinded rubric-based scoring with calibration, reproducible extraction of speech-based fluency-accuracy-complexity features, and a scripted clustered analysis workflow within a single auditable classroom design^12,13. Cross-cultural communication is operationalized through an oral discourse completion task and filmed role-play scenarios that require learners to respond to socially situated interactional problems, while intercultural sensitivity and speaking anxiety are measured through structured questionnaires with fixed scoring rules. This combination is intended to capture complementary aspects of learner performance rather than rely on a single indicator of instructional impact¹⁴.

Because the present implementation is restricted to one institution, adult university learners at Chinese Proficiency Test level 3 or above, and four intact classes, it should be interpreted as a pilot feasibility application of the protocol. Under these conditions, unresolved teacher- and class-level confounding cannot be excluded, and any between-condition differences should be treated as descriptive representative outputs rather than robust evidence of comparative effectiveness¹⁵. Within these limits, the protocol is designed to provide a transparent and reproducible framework for classroom delivery, assessment, rating, transcription, speech-feature extraction, and clustered longitudinal analysis that can be refined in future multi-class or multi-site studies in international Chinese education and related second-language settings¹⁶.

Protocol

The study was conducted in accordance with the regulations of the Ethics Committee of Yunnan College of Business Management under approval number [Approval No. YCBM-25168]. Before enrollment, each prospective participant was provided with a written information sheet explaining the study purpose, classroom procedures, audio-video recording, de-identification, voluntary participation, and the right to withdraw without academic penalty. Written informed consent for both study participation and filming before performing any study-specific procedure was obtained¹⁷. The research tools in the protocol are listed in the Table of Materials.

1. Set up classes, outcomes, and assessment timeline

1. Define the target population and teaching setting
   1. Identify adult, non-native learners of Chinese enrolled in university-level International Chinese classes with an explicit oral-communication objective.
   2. Use intact classes as the unit of allocation. Implement one 6-week cross-cultural communication module within the regular semester timetable, with 1 session per week and 50 min of instruction per session.
   3. In the pilot implementation, include 4 intact classes with 12 learners per class to yield a planned sample of 48 learners.
   4. Use the same lead instructor to teach both conditions with locked lesson plans and standardized timing. Schedule the two instructional arms on different weekdays, keep class rosters fixed throughout the intervention period, and prohibit distribution of arm-specific teaching materials outside the assigned classroom session to reduce contamination between conditions.
2. Define primary, supportive, and secondary outcomes
   1. Define pragmatic performance as the main classroom-performance domain. Assess it using two complementary instruments: an oral discourse completion task (ODCT) and a filmed role-play assessment.
   2. Administer 5 ODCT prompts at each time point. Score the ODCT on a 0-25 composite scale using four prespecified domains: pragmatic appropriateness (0-10), sociocultural alignment (0-5), discourse completeness (0-5), and linguistic control (0-5). Sum the four domain scores to obtain the total ODCT score. Provide the fixed ODCT prompts in Supplementary File 1.
   3. Administer one filmed role-play task per pair or triad at each timepoint. Score the role-play on a separate 0-25 composite scale using four prespecified domains: interactional appropriateness (0-10), responsiveness and repair (0-5), relational stance management (0-5), and linguistic control (0-5). Calculate the final score as the mean of two masked raters. If the two total scores differ by more than 3 points, obtain a third masked rating and calculate the final score as the mean of the two closest totals.
   4. Measure intercultural sensitivity using the Intercultural Sensitivity Scale, administered in a bilingual Chinese-English paper form with 24 items scored on a 5-point Likert scale and a total score range of 24-120.
   5. Measure speaking anxiety using the short-form Foreign Language Classroom Anxiety Scale, administered in the same bilingual Chinese-English format with 8 items scored on a 5-point Likert scale and a total score range of 8-40, where higher scores indicate greater speaking anxiety.
   6. Define supportive speech-production outcomes as fluency-accuracy-complexity indices derived from the role-play recordings, including words per minute, silent pauses of 0.25 s or longer per minute, percentage of T-units containing at least one morpho-syntactic error, and mean clause length.
   7. For the Intercultural Sensitivity Scale, treat the total score as missing if more than 2 items are unanswered. Otherwise, replace missing items with the respondent’s mean score on completed items from that scale. For the short-form Foreign Language Classroom Anxiety Scale, treat the total score as missing if more than 1 item is unanswered. Otherwise, replace the missing item with the respondent’s mean score on completed items from that scale.
   8. Summarize the baseline learner profile by instructional condition using age, gender, native language group, Chinese Proficiency Test level, and prior exposure to Chinese-speaking environments. Report these variables descriptively in Table 1 to document cohort composition rather than claim baseline equivalence in a four-cluster pilot.
3. Fix assessment timepoints and map the study flow
   1. Schedule three assessment timepoints: baseline (T0, within 3 days before the first intervention lesson), immediate post-test (T1, within 5 days after the final lesson), and follow-up (T2, exactly 28 days after T1).
   2. At each time point, administer the ODCT first, the filmed role-play second, and the questionnaires last. Keep this order identical across all classes.
   3. Record screening, consent, class-level allocation, outcome-specific completion, exclusions, attrition, and the final analysis set throughout the study so that the full participant flow can be presented transparently in Figure 1¹⁸.

2. Lock materials, run cluster randomization, and enforce masking

1. Freeze teaching and assessment materials
   1. Store the fixed teaching and assessment materials in a locked study folder before the first intervention lesson.
   2. Set the locked study folder to read-only before the first study lesson and keep it unchanged throughout data collection.
   3. Create a new folder version only when a revision is unavoidable. Record each change, date, reason, and responsible person in a plain-text changelog.
   4. Record the exact software versions used in the workflow, including Praat v6.4.58, ELAN v7.0, Audacity v3.7.7, R v4.5.2, tidyverse v2.0.0, lme4 v1.1-38, emmeans v2.0.1, and lmerTest v3.2-0.
2. Generate class-level allocation with a fixed seed
   1. Assign each intact class a unique identifier (C1-C4) before allocation.
   2. Generate the allocation sequence in R v4.5.2 using the fixed random seed 20260301. Use simple 1:1 random allocation without blocking or stratification because only four intact classes are available in the pilot.
   3. Have the research coordinator generate the allocation sequence. Ensure that this coordinator is not involved in classroom teaching, rating, or scoring of identified materials. Have the same coordinator assign classes after freezing the baseline roster and before the first intervention lesson begins.
   4. Record the date and time of allocation, the seed value, the randomization method, the R version, the name of the sequence generator, the name of the person who assigned the classes, and the output of sessionInfo().
3. Restrict access, minimize contamination, and mask condition labels
   1. Restrict access to the allocation file to the research coordinator and keep the file separate from all de-identified datasets.
   2. Use the same lead instructor across both conditions, but schedule the two arms on different weekdays, distribute printed materials only during the relevant session, and prohibit circulation of arm-specific files outside class.
   3. Instruct learners not to share study materials, screenshots, or role-play prompts with students from other classes until completion of T2 data collection.
   4. Mask condition labels from raters and staff handling de-identified data by using neutral condition codes in filenames, rating sheets, transcripts, and analysis datasets.
   5. Use neutral identifiers (class_id, student_id, pair_id) in manifests, rating sheets, transcripts, and analysis files. Store any linkage file separately in a secure institutional storage location.
      CAUTION: Do not allow raters to access the allocation file, weekly lesson plans, or any raw filename that directly reveals the instructional condition.

3. Recruit intact classes and deliver matched instruction

1. Recruit intact classes and obtain permissions
   1. Obtain permission from course coordinators and the lead instructor before contacting students.
   2. Recruit intact classes that include oral communication as a formal course objective and can complete the full 6-week module within the same academic term.
2. Screen eligibility and collect written consent
   1. Administer a screening and consent form capturing age, gender, native language, Chinese Proficiency Test level, and time spent in Chinese-speaking regions during the prior 2 years.
   2. Include adult, non-native speakers at Chinese Proficiency Test level 3 or above who have not lived in a Chinese-speaking region for more than 3 consecutive months during the prior 2 years.
   3. Exclude learners who do not consent to filming because role-play recording is a required component of the protocol.
3. Build and document baseline characteristics
   1. Assign each consenting participant a student_id and store the linkage file separately from de-identified datasets.
   2. Enter one row per learner in the baseline file with fields for condition, class_id, age, gender, native language group, Chinese Proficiency Test level, and prior exposure to Chinese-speaking environments.
   3. Record baseline characteristics descriptively by instructional condition for later reporting in Table 1. Do not interpret them as evidence that class-level confounding has been removed.
4. Prepare matched syllabi across conditions
   1. Create parallel weekly syllabi for task-based learning and Presentation-Practice-Production using the same communicative scenarios, communicative functions, and target expressions.
   2. Use the same six weekly themes in both arms: classroom disagreement, teacher consultation, peer collaboration, host-family interaction, internship communication, and public-service interaction.
   3. Compile reader-facing versions of the weekly lesson schedule, target-expression mapping, and role-play scenarios in Supplementary File 1.
5. Deliver task-based learning lessons using a fixed task cycle
   1. Introduce the scenario and communicative goal in Chinese for 5 min.
   2. Run a planning phase in small groups for 5 min.
   3. Run the main task phase in pairs or triads for 18 min, with minimal teacher intervention limited to clarification of task instructions.
   4. Conduct a reflection and feedback phase for 20 min, focusing on language choice, interactional strategy, pragmatic appropriateness, and repair.
   5. Use the same realized timing in all six task-based lessons unless a deviation is unavoidable. Record any deviation in the fidelity log.
6. Deliver Presentation-Practice-Production lessons with matched content and timing
   1. Present target expressions and sentence patterns for 16 min using example dialogues or projected materials.
   2. Conduct controlled practice for 16 min, focusing on accurate production of the target forms.
   3. Conduct constrained production for 16 min using the same weekly scenario and target expressions.
   4. Match overall lesson duration, topic coverage, and scenario content to the corresponding task-based lessons. Record all deviations in the fidelity log.
7. Record instructional fidelity after each lesson
   1. After each lesson, log the date, class, scenario label, realized duration of each phase, and any deviation from the planned structure, including any unplanned change in scenario, target expression, or time allocation.
   2. Store fidelity logs in a spreadsheet under the processed-study records.
   3. If a major fidelity deviation occurs, retain the session in the study file, flag it in the fidelity log, and describe the deviation in the final report rather than deleting the session post hoc.
      NOTE: Typical fidelity deviations include shortened task time because of classroom interruptions, temporary equipment repositioning, or omission of one planned feedback step. Log and interpret these events rather than silently removing them.

4. Conduct assessments, record media, and score performance

1. Administer the ODCT at each time point
   1. Present ODCT prompts one at a time on paper using the fixed prompt set.
   2. Allow 30 s of planning per prompt, record one spoken response per prompt, and keep the planning-time setting constant across T0, T1, and T2.
2. Administer the role-play and record interactions
   1. Provide written scenario and role cards and allow 2 min of planning.
   2. Record one 8-min interaction per pair or triad and keep the target interaction duration constant across timepoints.
   3. Define each role-play as an intercultural communication scenario involving a socially situated choice problem, such as disagreement, request mitigation, face management, clarification, or repair under differing expectations.
3. Standardize recording and create a media manifest
   1. Use the same digital video camera and the same external microphone throughout the study.
   2. Record all sessions at 1920 × 1080 resolution, 30 frames/s, and 48 kHz audio sampling. Keep the input-gain setting unchanged after the first recording day.
   3. Place the external microphone at the center of the pair or triad table, approximately 60 cm from the speakers, and document the setup in the study recording checklist.
   4. Save each file under a standardized filename encoding class_id, student_id or pair_id, timepoint, and task type.
   5. Store all recordings in the raw-media folder. Create a manifest listing filenames, file sizes, technical notes, and checksums, and compute SHA-256 checksums for every raw media file.
      NOTE: After confirming that all files are readable and that checksums match the manifest, set the raw-media folder to read-only and conduct all downstream processing on copies stored in the processed-study folder.
4. Train raters and score pragmatic performance under blinding
   1. Recruit two primary raters and one reserve rater, all with experience in Chinese language teaching or language assessment, and ensure that none have access to class allocation.
   2. Provide a calibration set comprising 12 ODCT responses and 6 role-play recordings, balanced across timepoints and condition codes, and select this set before formal scoring begins. Do not modify the calibration set after inspecting outcome patterns.
   3. Compute inter-rater agreement using the intraclass correlation coefficient ICC(2,k) in R v4.5.2. Repeat calibration until ICC reaches at least 0.75, and record the ICC type, calibration-set size, ICC values, and stop decisions in a calibration log file stored under the processed-study records¹⁹.
   4. During formal scoring, assign anonymized files to raters in random order and record scores in a spreadsheet with fields for class_id, student_id, timepoint, task type, rater_id, and score.
   5. If the prespecified reliability threshold is not reached after two recalibration rounds, suspend formal scoring, revise the calibration materials, repeat training, and document the event transparently in the calibration log.
      CAUTION: Do not begin formal scoring until the ICC threshold is reached and documented.
5. Transcribe role-play recordings and compute fluency-accuracy-complexity indices
   1. Transcribe all analyzable role-play recordings from both arms at T0, T1, and T2. Do not perform post hoc sampling after outcome inspection.
   2. Export ELAN annotations as UTF-8 tab-delimited text and store the files in the processed-study transcripts folder.
   3. Apply the same transcription conventions and segmentation rules across all files. Compile reader-facing versions of the rubrics, questionnaire scoring rules, transcription guidelines, and error taxonomy in Supplementary File 2.
   4. Segment each transcript into T-units and clauses. Record segment counts, speaker labels, and any exclusions in the transcript manifest.
   5. Inspect audio timing and compute pause-based measures using Praat v6.4.58, with the silent-pause threshold fixed at 0.25 s.
   6. Compute words per minute using a fixed rule in which repeated lexical items are counted as produced words unless they are immediate self-corrections abandoned before completion. Define speaking time as the total duration of the learner’s voiced output, excluding examiner instructions and non-task setup talk.
   7. Compute silent pauses of 0.25 s or longer per minute, percentage of T-units containing at least one morpho-syntactic error, and mean clause length as total words divided by total clauses.
   8. Write all speaker-level fluency-accuracy-complexity features to one long-format file with one row per learner per timepoint.
   9. If a recording is unusable because of technical failure or irrecoverable speaker overlap, log the reason, retain the file record in the manifest, code the derived features as missing, and carry the exclusion forward into the participant-flow record for Figure 1 and the final Results narrative.
6. Administer questionnaires and assemble the outcome dataset
   1. Administer the Intercultural Sensitivity Scale and the short-form Foreign Language Classroom Anxiety Scale after completion of the oral tasks at each timepoint.
   2. Check questionnaires for missing items before learners leave the room whenever feasible.
   3. If questionnaires remain incomplete, apply the prespecified missing-item rule described in Section 1.2.7. and document the resulting missingness in the analysis log.
   4. Merge ODCT scores, role-play scores, fluency-accuracy-complexity feature files, and questionnaire scores into one long-format dataset with one row per learner per timepoint.

5. Fit models and document analysis outputs

1. Validate identifiers and prepare the analysis-ready dataset
   1. Verify internal consistency of class_id, student_id, pair_id, and timepoint across all sources using scripted checks.
   2. Document exclusions, missingness, unusable recordings, transcript exclusions, and questionnaire-scoring decisions in a log file.
   3. Save the analysis-ready dataset as a CSV file and include a README listing the scripts, parameters, and software versions required to regenerate it.
2. Fit descriptive mixed-effects models for clustered longitudinal outcomes
   1. Because only 4 intact classes are randomized in the pilot, treat the analysis as feasibility-oriented and descriptive rather than confirmatory.
   2. Fit one coherent mixed-effects model for each outcome using R v4.5.2, lme4 v1.1-38, and lmerTest v3.2-0, with fixed effects for group, time, and group × time and random intercepts for class_id and student_id²⁰.
   3. Use the same model form across all outcomes and treat the T0 measurement as one of the repeated observations within the longitudinal model rather than fitting a separate post hoc baseline-adjusted model for selected outcomes.
   4. Report model-estimated marginal means, mean differences, 95% confidence intervals, and exact p values as descriptive model-based summaries only. Interpret nominal significance testing cautiously because the number of clusters is very small.
   5. Apply a false-discovery-rate adjustment only to the family of secondary outcomes and record the exact function name and settings used in the analysis script²¹.
3. Prepare outputs for later reporting
   1. Preserve the baseline dataset structure, variable labels, and denominators in a form that allows learner characteristics to be reported clearly in Table 1.
   2. Preserve the repeated-measures outcome dataset in a form that allows pragmatic outcomes to be summarized across T0, T1, and T2 in Table 2 and supportive speech and questionnaire outcomes to be summarized across the same three occasions in Table 3.
   3. Preserve plotting scripts and output labels in a form that allows pragmatic trajectories to be visualized in Figure 2 and multipanel supportive-outcome profiles to be visualized in Figure 3, with consistent scale naming and uncertainty definitions.
4. Record package versions and regenerate outputs
   1. At the start of analysis, write sessionInfo() and installed package information to version-tracking files.
   2. Load tidyverse v2.0.0 for data handling and emmeans v2.0.1 for estimated marginal means and contrasts, and record the package versions used for the final analysis.
   3. Regenerate all reportable outputs directly from the analysis-ready dataset using the saved scripts. Compile reader-facing workflow documentation, the codebook, and the script inventory in Supplementary File 3.

Results

Study flow and cohort

Recruitment, screening, written consent, class-level allocation, outcome-specific completion, and the final analysis set across baseline (T0), immediate post-test (T1), and follow-up (T2) are summarized in Figure 1. In the pilot implementation, 4 intact classes comprising 48 learners were allocated at the class level, with 2 classes assigned to task-based learning (TBL, n = 24) and 2 classes assigned to Presentation-Practice-Production (PPP, n = 24). At T0, all 48 learners completed the oral discourse completion task (ODCT), questionnaire measures, and the filmed role-play task. At T1, 47 learners completed the ODCT and questionnaires, and 46 analyzable role-play recordings were retained after exclusion of 1 recording because of severe background noise. At T2, 45 learners completed the ODCT and questionnaires, and 44 analyzable role-play recordings remained after excluding 1 additional recording due to irrecoverable speaker overlap. Participant-level de-identified longitudinal data underlying these representative outcomes are provided in Supplementary File 4.

Baseline characteristics

Baseline demographic and language-learning characteristics are presented in Table 1. Because only 4 intact classes were allocated in this pilot, these values are reported to describe cohort composition rather than establish group equivalence. The two conditions were broadly similar in age, gender distribution, Chinese Proficiency Test level, and prior exposure to Chinese-speaking environments, although class-level and teacher-related confounding cannot be ruled out in this design.

Pragmatic performance outcomes

Pragmatic performance outcomes across T0, T1, and T2 are presented in Table 2. For the ODCT composite score (0-25), both groups scored higher at T1 than at T0, with a larger gain in the TBL arm than in the PPP arm (+5.04 vs. +2.50). The TBL group increased from 12.46 ± 3.21 at T0 to 17.50 ± 3.18 at T1 and remained above baseline at 16.92 ± 3.37 at T2. The PPP group increased from 11.98 ± 2.92 at T0 to 14.48 ± 3.63 at T1 and remained above baseline at 13.82 ± 3.70 at T2. This trajectory is shown in Figure 2.

A similar pattern was observed for the filmed role-play assessment. In the TBL arm, the mean role-play score increased from 13.10 ± 2.86 at T0 to 17.84 ± 3.04 at T1 and was 17.10 ± 3.12 at T2. In the PPP arm, the corresponding values were 12.88 ± 2.79, 15.26 ± 3.08, and 14.60 ± 3.17. The increase from T0 to T1 was larger in the TBL arm than in the PPP arm (+4.74 vs. +2.38). Intercultural sensitivity also increased in both groups. In the TBL arm, the total score rose from 48.96 ± 6.19 at T0 to 54.52 ± 6.36 at T1 and remained at 53.60 ± 6.28 at T2. In the PPP arm, the corresponding values were 48.67 ± 7.39, 51.77 ± 8.09, and 50.74 ± 7.96. The increase from T0 to T1 was larger in the TBL arm than in the PPP arm (+5.56 vs. +3.10). Given the pilot design and the small number of clusters, these between-group differences should be interpreted cautiously.

Supportive speech and questionnaire outcomes

Supportive speech and questionnaire outcomes across T0, T1, and T2 are presented in Table 3. Across occasions, both groups showed improvement from baseline to post-test on the supportive outcomes, with the task-based learning (TBL) arm generally showing larger observed changes than the Presentation-Practice-Production (PPP) arm. Words per minute increased in both groups, whereas silent pauses and the percentage of T-units containing at least one morpho-syntactic error decreased. Mean clause length also increased across occasions in both conditions.

Speaking anxiety, indexed using the short-form Foreign Language Classroom Anxiety Scale, decreased from T0 to T1 in both groups and remained below baseline at T2. The reduction was larger in the TBL arm than in the PPP arm. These trajectories are shown in Figure 3.

Figure 1: Study flow and class-level allocation. CONSORT-style diagram summarizing screening, written consent, class-level allocation to task-based learning (TBL) or Presentation-Practice-Production (PPP), and occasion-specific completion of the oral discourse completion task, filmed role-play, questionnaires, and analyzable speech recordings at baseline (T0), immediate post-test (T1), and follow-up (T2), including exclusions and attrition. Please click here to view a larger version of this figure.

Figure 2: Oral discourse completion task performance across occasions. Oral discourse completion task (ODCT) composite scores (0-25) at baseline (T0), immediate post-test (T1), and follow-up (T2) for task-based learning (TBL) and Presentation-Practice-Production (PPP). Points represent group means, and lines connect the three assessment occasions. Please click here to view a larger version of this figure.

Figure 3: Supportive outcome profiles across occasions. Profiles of supportive outcomes for task-based learning (TBL) and Presentation-Practice-Production (PPP) across baseline (T0), immediate post-test (T1), and follow-up (T2), including words per minute, silent pauses of 0.25 s or longer per minute, percentage of T-units containing at least one morpho-syntactic error, mean clause length, and short-form Foreign Language Classroom Anxiety Scale scores. FLCAS-S, short-form Foreign Language Classroom Anxiety Scale. Please click here to view a larger version of this figure.

Characteristic	n = 24 TBL	n = 24 PPP
Age, years, mean ± SD	22.13 ± 1.83	22.38 ± 2.43
Female, n (%)	13 (54.2)	12 (50.0)
Male, n (%)	11 (45.8)	12 (50.0)
Chinese Proficiency Test level 3, n (%)	9 (37.5)	9 (37.5)
Chinese Proficiency Test level 4, n (%)	13 (54.2)	12 (50.0)
Chinese Proficiency Test level 5, n (%)	2 (8.3)	3 (12.5)
Prior residence in a Chinese-speaking environment within the previous 2 years, n (%)	4 (16.7)	5 (20.8)
Years of prior formal Chinese study, mean ± SD	2.4 ± 0.9	2.5 ± 1.0

Table 1: Baseline characteristics by instructional condition. Baseline (T0) demographic and language-learning characteristics of learners in the task-based learning (TBL) and Presentation-Practice-Production (PPP) conditions.

Outcome Measure	Group	n	Mean ± SD	n	Mean ± SD	n	Mean ± SD	Δ	Δ
		(T0)	(T0)	(T1)	(T1)	(T2)	(T2)	(T1-T0)	(T2-T0)
ODCT composite (0–25)	TBL	24	12.46 ± 3.21	24	17.50 ± 3.18	23	16.92 ± 3.37	5.04	4.46
ODCT composite (0–25)	PPP	24	11.98 ± 2.92	23	14.48 ± 3.63	22	13.82 ± 3.70	2.5	1.84
Filmed role-play score (0–25)	TBL	24	13.10 ± 2.86	23	17.84 ± 3.04	22	17.10 ± 3.12	4.74	4
Filmed role-play score (0–25)	PPP	24	12.88 ± 2.79	23	15.26 ± 3.08	22	14.60 ± 3.17	2.38	1.72
Intercultural Sensitivity Scale (24–120)	TBL	24	48.96 ± 6.19	24	54.52 ± 6.36	23	53.60 ± 6.28	5.56	4.64
Intercultural Sensitivity Scale (24–120)	PPP	24	48.67 ± 7.39	23	51.77 ± 8.09	22	50.74 ± 7.96	3.1	2.07

Table 2: Pragmatic performance and intercultural sensitivity across occasions. Oral discourse completion task composite scores, filmed role-play scores, and Intercultural Sensitivity Scale scores at baseline (T0), immediate post-test (T1), and follow-up (T2) for the task-based learning (TBL) and Presentation-Practice-Production (PPP) conditions.

Outcome	Direction	Group	n at T0	T0 mean ± SD	n at T1	T1 mean ± SD	n at T2	T2 mean ± SD	Δ	Δ
									(T1–T0)	(T2–T0)
Words per minute	Higher better	TBL	24	85.0 ± 11.6	23	104.3 ± 13.1	22	100.6 ± 12.8	19.3	15.6
Words per minute	Higher better	PPP	24	85.1 ± 10.9	23	95.0 ± 12.2	22	92.4 ± 11.9	9.9	7.3
Silent pauses ≥0.25 s per minute	Lower better	TBL	24	16.1 ± 3.4	23	11.4 ± 2.9	22	12.1 ± 3.0	−4.7	−4.0
Silent pauses ≥0.25 s per minute	Lower better	PPP	24	15.9 ± 3.1	23	13.5 ± 2.8	22	13.9 ± 2.9	−2.4	−2.0
T-units with ≥1 morpho-syntactic error, %	Lower better	TBL	24	17.0 ± 4.6	23	12.6 ± 3.8	22	13.2 ± 4.0	−4.4	−3.8
T-units with ≥1 morpho-syntactic error, %	Lower better	PPP	24	16.8 ± 4.2	23	14.3 ± 3.9	22	14.8 ± 4.1	−2.5	−2.0
Mean clause length	Higher better	TBL	24	6.80 ± 0.90	23	8.00 ± 1.00	22	7.82 ± 0.98	1.2	1.02
Mean clause length	Higher better	PPP	24	6.70 ± 0.80	23	7.52 ± 0.92	22	7.31 ± 0.89	0.82	0.61
Short-form Foreign Language Classroom Anxiety Scale (8–40)	Lower better	TBL	24	30.0 ± 4.8	24	25.5 ± 4.3	23	26.2 ± 4.4	−4.5	−3.8
Short-form Foreign Language Classroom Anxiety Scale (8–40)	Lower better	PPP	24	30.3 ± 4.6	23	28.8 ± 4.4	22	29.2 ± 4.5	−1.5	−1.1

Table 3: Supportive speech-production and questionnaire outcomes across occasions. Words per minute, silent pauses of 0.25 s or longer per minute, percentage of T-units containing at least one morpho-syntactic error, mean clause length, and short-form Foreign Language Classroom Anxiety Scale scores at baseline (T0), immediate post-test (T1), and follow-up (T2) for the task-based learning (TBL) and Presentation-Practice-Production (PPP) conditions.

Supplementary File 1: Teaching and task materials. Six-week teaching schedule, task-based learning lesson plans, Presentation-Practice-Production lesson plans, scenario-to-target-expression mapping, oral discourse completion task prompts, and role-play scenarios and role cards used in the classroom intervention.Please click here to download this file.

Supplementary File 2: Scoring and coding materials. Oral discourse completion task and role-play scoring rubrics, rater calibration instructions, Intercultural Sensitivity Scale form and scoring rule, short-form Foreign Language Classroom Anxiety Scale form and scoring rule, transcription guidelines, T-unit and clause segmentation rules, and morpho-syntactic error taxonomy.Please click here to download this file.

Supplementary File 3: Reproducibility and analysis materials. Instructional fidelity log template, recording checklist, variable codebook, workflow summary, software and package versions, randomization documentation summary, and script inventory for dataset construction, speech-feature extraction, plotting, and model fitting.Please click here to download this file.

Supplementary File 4: De-identified raw data. Participant-level de-identified longitudinal data across baseline (T0), immediate post-test (T1), and follow-up (T2), including oral discourse completion task scores, filmed role-play scores, Intercultural Sensitivity Scale scores, short-form Foreign Language Classroom Anxiety Scale scores, and supportive speech-production measures.Please click here to download this file.

Discussion

This article presents a pilot classroom protocol for comparing task-based learning and Presentation-Practice-Production in university-level International Chinese classes. Its main contribution is methodological. Rather than offering another broad comparison between two teaching sequences, the protocol specifies how such a comparison can be implemented under classroom conditions while keeping core sources of procedural variation under control. Weekly themes are matched across conditions, target expressions are fixed in advance, teaching and assessment materials are locked before implementation, assessment order is held constant across three occasions, and the workflow from recording to analysis is documented in a form that can be checked and repeated²². In classroom research, these design features are not minor technical additions. They often determine whether a reported between-group difference can be interpreted as an instructional contrast rather than as a by-product of uneven materials, shifting procedures, or undocumented scoring decisions²³.

A second strength of the protocol lies in the way cross-cultural communicative performance is measured. The design does not rely on a single indicator. Instead, it combines oral discourse completion prompts, filmed role-play tasks, learner-reported intercultural sensitivity, speaking-related anxiety, and speech-based production indices. This combination matters because cross-cultural communication is not fully captured by a single format. The oral discourse completion task supports comparability across learners and occasions, whereas filmed role-play provides a more interactional view of how learners respond to socially situated communicative demands. Speech measures such as words per minute, pause frequency, error rate, and mean clause length add further information about how performance changes are distributed across fluency, control, and structural elaboration²⁴. Taken together, these channels make it possible to examine classroom performance from multiple angles without collapsing all change into one score.

The protocol also explicitly outlines several forms of quality control. Class-level allocation is matched to a clustered longitudinal analysis strategy. Rater masking is paired with calibration and a documented reliability threshold. Recording conditions are standardized before data collection, and file integrity is verified before downstream processing begins. Instructional fidelity is logged session by session so that realized timing and deviations from the planned sequence can be traced later²⁵. These safeguards are especially important in educational studies because the interpretability of results can be weakened by small, cumulative departures from protocol even when the broader study design appears sound. In this sense, the present article is intended not only to describe what was taught, but also to show how the surrounding procedures can be stabilized sufficiently for later replication²⁶.

At the same time, the present implementation has clear design limitations. The most important limitation is the very small number of clusters. In cluster-based classroom studies, statistical precision depends heavily on the number of classes rather than on learner count alone. Four intact classes can support a pilot implementation, but they provide only limited protection against class-specific influences and local contextual effects²⁷. This limitation is directly relevant here because classroom responses are shaped not only by lesson sequence, but also by factors such as participation norms, peer dynamics, class atmosphere, and subtle differences in how tasks are taken up. The present workflow reduces some of this variation through matched content, fixed timing, standardized assessment, and common scoring procedures, but it does not remove it. Any between-condition pattern should therefore be interpreted within this constraint.

A related issue concerns teacher and contamination effects. The protocol attempts to reduce contamination by separating class schedules, restricting circulation of arm-specific materials, masking condition labels during rating and analysis, and maintaining a fixed instructional structure. Even so, contamination remains a realistic possibility whenever learners communicate across classes or when one instructor works across more than one class. Likewise, the use of one institutional site and one bounded learner population improves control during pilot implementation, but it narrows external validity²⁸. The present findings therefore should not be generalized automatically beyond adult university learners studying Chinese at or above Chinese Proficiency Test level 3, nor should they be assumed to transfer unchanged to lower-proficiency learners, school settings, or other language programs without further testing.

Measurement limitations should also be acknowledged directly. Oral discourse completion tasks and role-play tasks do not capture the same layer of ability. The former offers tighter elicitation control and more stable comparability, whereas the latter is more sensitive to interactional contingency, partner behavior, and moment-to-moment discourse development²⁹. This difference is not a flaw in either instrument, but it does mean that the two formats should be interpreted as complementary rather than interchangeable. The same caution applies to supportive speech indices. Changes in pausing, fluency, error rate, and clause length can clarify how performance is evolving, but they should not be read as direct substitutes for pragmatic success. Their value lies in contextualizing performance, not in replacing it³⁰.

From a practical perspective, one useful feature of the protocol is that it treats implementation problems as part of the workflow rather than as informal exceptions. Classroom studies rarely proceed without disruption. Lessons may run short, questionnaires may be incomplete, recordings may become unusable, transcripts may require exclusion, and raters may need recalibration before formal scoring can begin. By specifying how such events are to be logged, retained, and reported, the protocol makes it easier for later users to reproduce not only the planned design, but also the way the design responds to routine classroom contingencies³¹. This feature is particularly relevant for video-based and audio-based classroom work, where seemingly minor technical issues can affect both scoring and feature extraction.

The protocol is also intended to be extendable. Future implementations can strengthen inference by increasing the number of intact classes, preregistering the analysis plan, and moving toward multi-site designs while preserving the comparability features defined here: matched content, fixed assessment timing, masked scoring, stable coding rules, and a transparent data-processing pipeline³². The same design backbone can also be adapted beyond International Chinese classrooms by replacing the scenarios, target expressions, and scoring rubrics, while maintaining the same allocation, assessment, de-identification, transcription, and clustered analysis logic³³. In this sense, the value of the present article lies less in claiming a definitive instructional advantage on the basis of a single small pilot than in offering a protocol that can be examined, reused, and refined in future classroom-based work on cross-cultural communication.

Materials

List of materials used in this article

Name	Company	Catalog Number	Comments
Analysis scripts folder	This study	Version 1.0	Fixed scripts for class randomization, pause detection, dataset construction, model fitting, and plotting.
Audio editing software	Audacity Team	Version 3.7.7	Used for audio inspection and trimming when needed.
Audio monitoring headphones	Audio-Technica	ATH-M20x	Used to monitor playback quality, clipping, and background noise during recording checks.
Classroom recording checklist	This study	Version 1.0	Session-level checklist for camera placement, microphone placement, audio level, storage capacity, room noise, and role-card distribution.
Data dictionary / codebook	This study	Version 1.0	Defines class IDs, learner IDs, timepoints, outcome variables, missingness rules, and derived-feature names.
External microphone	Sony	ECM-LV1	Used as the fixed external microphone for classroom task recording.
Fidelity log template	This study	Version 1.0	Used to record date, class, scenario label, realized duration of each phase, and deviations from the planned structure.
Filmed role-play scoring rubric	This study	Version 1.0	Fixed 0–25 rubric for role-play assessment, including dual masked scoring and adjudication rules.
Intercultural Sensitivity Scale form	This study	Version 1.0	Bilingual English–Chinese questionnaire, 24 items, administered at T0, T1, and T2.
Oral discourse completion task prompts	This study	Version 1.0	Fixed 5-prompt set used at all three assessment occasions.
Phonetic analysis software	Praat	Version 6.4.58	Used for pause detection with the silent-pause threshold fixed at 0.25 s.
Presentation-Practice-Production lesson plans	This study	Version 1.0	Fixed 6-week PPP lesson plans matched to the same themes, target expressions, and communicative functions as the TBL condition.
Raw media storage	SanDisk	SDSSDE60-1T00-G25	External SSD used for encrypted raw-media storage and checksum-verified archiving.
Role-play scenarios and role cards	This study	Version 1.0	Fixed filmed role-play materials used for assessment at T0, T1, and T2.
Statistical computing software	R Project for Statistical Computing	Version 4.5.2	Used for class randomization, clustered longitudinal models, and output generation.
Task-based learning lesson plans	This study	Version 1.0	Fixed 6-week TBL lesson plans using the same weekly themes and target expressions as the PPP condition.
Transcription and annotation software	ELAN	Version 7.0	Used for role-play transcription and time-aligned annotation.
Transcription guidelines	This study	Version 1.0	Fixed conventions for transcription, speaker labeling, segmentation, and exclusion handling.
Tripod / camera support	Manfrotto	MKELES5BK-BH	Used for stable classroom camera placement.
Video camera	Sony	FDR-AX43	Fixed classroom video camera used across all sessions.
Workstation computer	Dell	DC15250	Used for transcription, feature extraction, and statistical analysis.
Short-form Foreign Language Classroom Anxiety Scale form	This study	Version 1.0	Bilingual English–Chinese questionnaire, 8 items, administered at T0, T1, and T2.