A Confidence-Guided Multi-Object Tracking Method for Sports Videos Using Jersey Semantic Fusion

Summary

This protocol describes a confidence-guided multi-object tracking workflow for sports videos that integrates jersey color and player number information to improve trajectory association and identity consistency under confidence fluctuations, occlusion, and visually similar athlete appearances.

Abstract

Multi-Object Tracking (MOT) in sports videos provides trajectory information for tactical analysis, player behavior interpretation, and automated statistical analysis. However, sports scenarios frequently involve rapid direction changes, abrupt stops, frequent occlusion, and visually similar teammates, resulting in fluctuations in detection confidence and identity confusion during inter-frame association. To address these challenges, this study presents JerseyTrack, a confidence-guided sports MOT method based on jersey semantic fusion. The workflow adaptively partitions detection boxes into high-, medium-, and low-confidence intervals according to the confidence distribution of each frame. High-confidence detections are associated primarily using motion similarity, whereas medium- and low-confidence detections additionally incorporate jersey color and player number features extracted through color clustering and a lightweight Optical Character Recognition (OCR) model. These semantic features are fused with motion information during supplementary matching to improve trajectory continuity and identity consistency. The method was evaluated on the SportsMOT dataset. JerseyTrack achieved 88.9% Multiple Object Tracking Accuracy (MOTA), 74.3% IDentity F1 Score (IDF1), and 69.9% Higher Order Tracking Accuracy (HOTA), demonstrating improved tracking performance under the evaluated sports tracking setting. This protocol provides a reproducible workflow for integrating confidence-guided association with jersey semantic information to improve multi-object tracking in sports videos.

Introduction

Multi-object tracking (MOT) provides continuous object localization and identity maintenance in video sequences and supports athlete trajectory extraction, tactical analysis, and automated statistical analysis in sports-video applications^1,2,3. Reliable visual information is also related to sport-specific decision-making and performance assessment in sports science, further emphasizing the value of stable video-derived movement data for downstream sports analysis⁴. In sports scenarios, the reliability of tactical data depends on the continuity of track....

Protocol

This study involved secondary analysis of publicly available benchmark video datasets, namely SportsMOT, TeamTrack, SoccerNet Tracking, MOT17, and MOT20. No participants were recruited, no intervention was performed, and no new human-subject data were collected. Ethics committee approval was not required for this study under the applicable institutional requirements of Bangkok Thonburi University.

The following protocol describes the workflow used to reproduce JerseyTrack from data preparation through tracking evaluation. The workflow includes dataset preparation, detector preparation, jersey semantic extraction, confidence-level partitioni....

Results

This section reports the quantitative and qualitative results obtained from the evaluated sports multi-object tracking experiments. The results focus on tracking accuracy, identity preservation, association quality, and robustness under the tested dataset settings. Performance claims are restricted to the evaluated methods, datasets, detector outputs, and evaluation toolkit configuration described in the Protocol and Implementation Setup.

Experimental setup and evaluation design

Discussion

This study evaluated a confidence-guided sports MOT workflow that combines detection-confidence partitioning with jersey semantic cues. Confidence-guided association and semantic feature fusion have been investigated as complementary strategies for improving data association in multi-object tracking^{12,20,23,24,46}. The results show that the complete JerseyTrack.......

Materials

List of materials used in this article

Name	Company	Catalog Number	Comments
CUDA Runtime	NVIDIA	Version 12.8	GPU computing runtime used for detector training, semantic feature extraction, and tracking inference.
EasyOCR	Jaided AI	Version 1.7.2	Optical character recognition toolkit used for player-number recognition.
JerseyTrack source code	Authors	N/A	Custom implementation containing the data preparation, semantic feature extraction, confidence partitioning, tracking, post-processing, and evaluation scripts. Available at: https://doi.org/10.5281/zenodo.21274352
Lightweight CRNN OCR model	Authors	N/A	Custom convolutional recurrent neural network (CRNN) used for player-number recognition.
MOT17 dataset	MOTChallenge	N/A	Public benchmark dataset used for cross-dataset evaluation. Available from: https://motchallenge.net/data/MOT17/
MOT20 dataset	MOTChallenge	N/A	Public benchmark dataset used for cross-dataset evaluation. Available from: https://motchallenge.net/data/MOT20/
motmetrics	motmetrics Developers	Version 1.4.0	Library used for diagnostic multi-object tracking metric calculation.
NVIDIA GPU	NVIDIA	GeForce RTX 5090 D V2	Graphics processing unit used for detector training and tracking inference.
NVIDIA GPU Driver	NVIDIA	Version 610.62	GPU driver used in the experimental environment.
NumPy	NumPy Developers	Version 2.4.4	Numerical computing library used for feature processing and data handling.
OpenCV	OpenCV	Version 4.10.0	Computer vision library used for image loading, cropping, preprocessing, and visualization.
Python	Python Software Foundation	Version 3.12.2	Programming language used throughout the workflow.
PyTorch	PyTorch Foundation	Version 2.11.0+cu128	Deep learning framework used for detector and semantic model implementation.
scikit-learn	scikit-learn Developers	Version 1.9.0	Machine-learning library used for K-means clustering during jersey-color extraction and confidence partitioning.
SoccerNet Tracking dataset	SoccerNet	N/A	Public football tracking benchmark used for cross-dataset evaluation. Available from: https://www.soccer-net.org/tasks/tracking
SportsMOT dataset	SportsMOT Benchmark	N/A	Primary benchmark dataset used for detector preparation and tracking evaluation. Available from: https://deeperaction.github.io/datasets/sportsmot.html
TeamTrack dataset	TeamTrack Benchmark	N/A	Public sports tracking benchmark used for cross-dataset evaluation. Available from: https://atomscott.github.io/TeamTrack/
Torchvision	PyTorch Foundation	Version 0.26.0+cu128	Computer vision library used with PyTorch for image transformation and model support.
TrackEval	TrackEval Developers	Version 1.3.0	Evaluation toolkit used to calculate Multiple Object Tracking Accuracy (MOTA), IDentity F1 Score (IDF1), Higher Order Tracking Accuracy (HOTA), Detection Accuracy (DetA), Association Accuracy (AssA), false positives (FP), false negatives (FN), and identity switches (IDs).
Ultralytics	Ultralytics	Version 8.4.90	Object detection framework used to train the detector and generate athlete detections.