Integrated Audio–Visual Emergency Vehicle Detection for Autonomous Vehicles with Real-Time Response

Abstract

In real traffic, change a lane safely mostly relies on the system’s ability to judge the distance to the car behind and in front of it. Applying rigid rules with specific limits frequently ends up being too restrictive and doesn’t help when making decisions about changing a lane. In this research, I propose a new system that learns from the context and improves it with reinforcement learning that makes it a more accurate and reliable system. To understand the lane change risks in the traffic at the moment I apply ResNet50 with transfer learning and enhance it with LSTM layers. To detect and track cars and also know what they might do I use Mask R-CNN with CNN and LSTM so that all these three things can be done by the model.Since the traffic conditions are always different I also apply an analysis of the weather, speed, acceleration, steering angle, and the road surface conditions as additional inputs to the system.To make the decisions safer I added a Double Deep Q-Network, which was found to be a steadier and faster to train than older reinforcement learning methods in heavy traffic conditions. From the simulation results, we can see that the check of the risks is clearer and more accurate, the decisions are better, and the change of a lane is smoother. So the system is more safe and reliable, and we move one step closer to the smarter transport systems.