Design and Implementation of a Field Programmable Gate Array-Based Pedestrian Detection Framework for Autonomous Driving Application

Summary

This research article demonstrates the implementation of a real-time pedestrian detection algorithm on field-programmable gate array hardware, primarily for autonomous driving applications. The algorithm combines the histogram of oriented gradients (HoG) with a support vector machine (SVM) classifier, and results show efficiency in terms of speed, power consumption, and resource utilization.

Abstract

Autonomous driving offers a promising way to tackle the rising number of fatalities from traffic accidents. An autonomous vehicle includes many features, but the ability to detect pedestrians is crucial, challenging, and relevant to various real-time situations like surveillance, tracking people, and monitoring. Accurately identifying pedestrians is difficult because they can appear in different shapes, positions, and postures. They can wear various types of clothing and sometimes be partially hidden or blend in with nearby objects. This paper focuses on the real-time detection of pedestrians for self-driving cars using a popular hardware platform: The field programmable gate array (FPGA), Ultra 96 v2. The study implements a method for pedestrian detection based on a histogram of oriented gradients (HOG) combined with a support vector machine (SVM) classifier to recognize individuals on the FPGA board, leveraging high-level synthesis (HLS) tools. The effectiveness of the system has been tested on both still images and live video. The results show that advanced FPGA boards like the Ultra 96 v2 significantly improve performance metrics. The system operates at a clock frequency of 150 MHz while using less than half of the available resources and consuming around 2.5 W of power. Also, the system reports the pedestrian detection accuracy close to 95% and other efficient metrics for detection evaluation, like precision (78.6%), recall (88.3%), and F1 Score (83.1%). In summary, the developed system can detect pedestrians in real-time and has the potential to significantly improve the development of a smart and safe transportation environment.

Introduction

Urban development and the emergence of smart cities are topics of interest worldwide. All the nations are working towards the development of their cities that can be built in such a way that they can be safe and comfortable for the people living in them^1,2. But at the present situation, it has been observed that as the population grows and road congestion increases, the rate of fatalities resulting from road accidents due to driving negligence and poor visibility issues is rising alarmingly. A promising solution to these issues is the emergence of autonomous vehicles worldwide, which has sparked innovation

Protocol

The implementation procedure used in this research, based on pedestrian detection with HoG + SVM on an FPGA board, leveraging the benefits of high-level synthesis, is illustrated in Figure 2 below.

Figure 2: Design procedure for implementation of pedestrian detection on FPGA board. Phas....

Results

Pedestrian detection implementation on HLS
Figure 4 shows the simulation results on the HLS tool for the pedestrian detection using HoG + SVM. An input image with a pedestrian is fed as the test input to the code, and the output with the detected pedestrians is displayed. There are two sections in the image. The first detection has many bounding boxes around the same pedestrian again and again, and in the second image, the overlapping boxes are removed, and they are supp.......

Discussion

This study effectively implements a real-time pedestrian detection system utilizing the HoG + SVM algorithm on advanced FPGA hardware based on Zynq UltraScale+ MPSoC development board²⁴. The results indicate that the traditional HoG algorithm of human detection¹¹ achieves accuracy close to 95% and utilizes only half of the on-board FPGA resources (LUTs, FFs, BRAM, DSPs), leaving huge capability of including much more processing to do some different tasks. When the entire im.......

Materials

List of materials used in this article

Name	Company	Catalog Number	Comments
Python	Python	Version 3.10
Ultra 96 V2 FPGA Board	Xilinx	Introduced in 2018	Hardware Implementation Platform used for implementing the pedestrian detection algorithm
Vivado	AMD	2019.2	FPGA Programming tool used for programming the Ultra 96 v2 FPGA board with the pedestrian detection algorithm
Vivado HLS	AMD	2019.2	High Level Synthesis Tool used for high level programming of the pedestrian detection code in the paper to export the Intellectual Property (IP)