Navigating Unsignalized Intersections: Deep RL-Based Decision-Making and Control Framework for Autonomous Vehicles with Pedestrian Integration

Abstract

Unprotected left turns at unsignalized intersections, alongside pedestrians and adversarial vehicles, pose significant challenges for Autonomous Vehicle (AV)s. These challenges stem from the absence of traffic signals or signs, the dynamic nature of the environment shaped by human interactions at crosswalks, and the variability in intersection layouts. This thesis delves into addressing these challenges through the application of a hierarchical Deep Reinforcement Learning (DRL) approach, where the DRL policy governs high-level decision-making (or planning), and low-level Proportional-Integral-Derivative (PID) controllers handle actuation. To evaluate and train DRL policies, it was necessary to create a simulation environment within a high-fidelity environment with realistic behaviors and dynamic vehicle models. To the best of our knowledge, this research marks a pioneering effort in simulating pedestrian interactions within a high-fidelity environment, coexisting alongside adversarial vehicles within the CARLA simulation platform. We have dedicated extensive efforts to the development of this simulation, enabling straightforward customization of parameters such as the number of pedestrians, adversarial vehicles, and reward functions amongst others. This is made available open-source at https://github.com/faizansana/ intersection-carla-gym. The study evaluates five distinct model-free DRL algorithms, namely Deep Q-Learning (DQN), Deep Deterministic Policy Gradient (DDPG), Proximal Policy Optimization (PPO), Recurrent PPO, and Soft Actor Critic (SAC). The primary focus of this work is to conduct a comprehensive comparative analysis of these DRL algorithms within a hierarchical framework to enhance AV decision-making in complex and uncontrolled intersection scenarios. The training code, with its versatile software architecture is made available at https://github.com/faizansana/intersection-driving. Our findings reveal that Recurrent PPO, coupled with a discretized action space, outperforms the other algorithms, displaying the highest success rate and the lowest accident rate for executing unprotected left turns in chaotic intersections. This outcome underscores the potential of Recurrent PPO to navigate such complex traffic scenarios effectively. The thesis concludes by discussing potential extensions of the proposed hierarchical DRL system and outlining promising avenues for future research in the field of autonomous vehicle navigation at challenging and dynamic intersections.