Dual Encoder-Decoder-Encoder with Adversarial Training for Unsupervised Traffic Accident Detection in Surveillance Videos

Summary

This work proposes a dual encoder-decoder-encoder (EDE) model for automated traffic accident detection. Using a two-phase training method, it learns normal driving patterns and identifies anomalies via generative confrontation. The model effectively detects accidents in real-world footage and offers insights into driver behaviors by capturing subtle deviations.

Abstract

To enhance road safety and improve emergency response, traffic incidents should be detected in real-world surveillance footage as quickly as possible. Existing systems largely depend on manual monitoring, which is time-consuming and prone to error. Automated accident detection remains challenging due to the substantial class imbalance: normal driving situations are overrepresented, whereas accidents are rare and diverse. In such cases, traditional computer vision systems often cannot reliably differentiate between normal and abnormal events. This study addresses the problem by developing a deep learning architecture based on a dual encoder-decoder-encoder (EDE) framework. The model uses two shared encoder-decoder pipelines to map image distributions to specified latent distributions in both directions. This framework enables the system to model common traffic behavior patterns and become more sensitive to changes that may indicate dangerous or unusual events. A two-phase training technique is proposed to further improve anomaly detection. In the first phase, the model learns to reconstruct images of normal driving, using reconstruction loss to characterize normal behavior. In the second phase, a generative adversarial mechanism is introduced: reconstructed latent vectors from one EDE are passed to the other, generating synthetic images and latent spaces. This process amplifies differences between real and synthetic outputs, making the system more responsive to subtle signs of potential anomalies. The dual-EDE architecture and adversarial training methodology represent a substantial advance over current methods by modeling both normal and pathological behavior. Experimental results on real-world traffic surveillance datasets demonstrate that the proposed method significantly improves the detection of accidents and unsafe driving behaviors, both in terms of accuracy and robustness.

Introduction

According to the World Health Organization (2023), road traffic injuries are the leading cause of death among children and young adults aged 5-29 years, with approximately 1.3 million fatalities reported globally each year. This alarming statistic underscores the urgent need for automated systems capable of monitoring road traffic¹, detecting anomalies in real time, and reducing delays in emergency response. The integration of artificial intelligence (AI) and the Internet of Things (IoT) into smart city infrastructure has enabled the development of intelligent transportation systems. While closed-circuit television (CCTV)²

Protocol

System

Setup
We deployed the proposed traffic anomaly detection system within a hierarchical and distributed computing framework, leveraging the Intel Tiber Cloud environment. This architecture comprises three tiers -- edge, fog, and cloud -- to ensure low-latency inference, scalable training, and efficient resource allocation across compute nodes.

Edge Tier: Real-time anomaly detection is conducted at the edge using lightweight, GPU-capable embedded devices (e.g., NVIDIA Jetson Nano or equivalent Intel-based platforms with integrated GPUs). These units were co-located with surveilla....

Results

To evaluate the efficacy of the proposed traffic anomaly detection method, we implemented the model on a single video clip and generated visualizations illustrating the system's behavior over time and within the feature space. Although obtained using a simulated EDE pipeline, the outcomes closely reflect the qualitative conclusions that would be expected from an actual model.

The anomaly score timeline depicts the model's frame-by-frame confidence in identifying unusual occurrences (

Discussion

This study presents a deep learning-based traffic anomaly detection system employing an EDE architecture, trained in an unsupervised manner to identify both single- and multi-vehicle accidents in real-world surveillance videos. By modeling typical traffic behavior, the system detects deviations as probable anomalies without requiring labeled anomaly data, thereby addressing scalability and data sparsity challenges in intelligent traffic monitoring. The research advances the field by demonstrating the combined use of late.......

Materials

List of materials used in this article

Name	Company	Catalog Number	Comments
AI City Challenge Track 4 Dataset	AI City Challenge (https://www.aicitychallenge.org)	Track 4, 2021 Release
CUDA Toolkit	NVIDIA Developer	Version 11.3
cuDNN Library	NVIDIA Developer	Compatible with CUDA 11.3
GPU Workstation Cluster (Training)	Amrita School of Computing	—
Local Workstation (Fog Node)	Amrita School of Computing	—
Matplotlib	matplotlib.org	Version 3.3+
NVIDIA Jetson Nano (Edge Device)	NVIDIA	945-13450-0000-100
NVIDIA RTX 3060 GPU (Workstation)	NVIDIA	Varies by manufacturer
NumPy	numpy.org	Version 1.19+
OpenCV	OpenCV.org	Version 4.5+
Pandas	pandas.pydata.org	Version 1.1+
Python	Python Software Foundation	Version 3.8+
PyTorch	PyTorch (https://pytorch.org)	Version 1.10+
Scikit-learn	scikit-learn.org	Version 0.24+
Ubuntu Linux (Operating System)	Canonical Ltd.	Version 20.04 LTS