AI-Driven Intelligent Video Surveillance System for Detecting Vehicles Using Transformer-Based YOLOv8

Abstract

With the rapid growth of urban traffic systems, we have seen a corresponding increase in the need for real-time vehicle monitoring systems that can operate well in difficult to predict environments; regardless of their complexity. Common problems faced by conventional computer vision-based models when working with video data include occluded objects, illumination differences, high volumes of vehicles, and variances in scale between vehicles in different frame locations. To mitigate these shortcomings, we are proposing to develop an AI-based intelligent video surveillance framework incorporating a Transformer-augmented YOLOv8 algorithm and performing multi-object tracking (MOT) via the ByteTrack tracker. The proposed approach incorporates a Transformer-based object detection module at both the YOLOv8 backbone and neck level of the model in order to capture long-range spatial relationships as well as global context with regard to the vehicles in question. As a result of these enhancements, the YOLOv8 object detection results were more robust to illumination differences and crowding effects than would otherwise be the case. In addition to the application of the above Transformer-based methods for object detection, we also applied Non-Maximum Suppression (NMS) and then integrated the object detection results with the ByteTrack tracker in order to provide unique and continuous identification across frames using a lightweight association method. Testing with the UA-DETRAC dataset, which contains 83,791 frames of traffic activity, indicates that our system outperforms the current baseline YOLOv8s algorithm on Precision (0.725), Recall (0.800), F1-Score (0.761), and mAP@0.5 (0.800). Our experimental results demonstrate that the use of transformers to create global context models significantly improve the accuracy of object detections, localize the vehicles more precisely, and track them continuously through multiple video frames; thus making our proposed system appropriate for a wide variety of intelligent transportation applications.