Interactive Data Collection Using CARLA and OpenCDA for Reinforcement Learning

Abstract:  Autonomous vehicles (AVs) have attracted significant research efforts driven by an over $9 billion market value by 2027. Autonomous driving has been considered to be the future solution to reducing crash rates and traffic flow while ensuring drivers’ safety and enabling many possibilities. This project sought to create a virtual driving scenario, by simulating a live road situation, to collect interactive training trajectories for inverse reinforcement learning (RL). To establish this virtual environment, CARLA simulator (http://carla.org/), SUMO (https://www.eclipse.org/sumo/), and OpenCDA (https://opencdadocumentation.readthedocs.io/en/latest/md_files/introduction.html#) were deployed. OpenCDA was used to interface with CARLA and SUMO and enabled real-world scenarios, insimulation communication between vehicles, and a logic flow. SUMO’s python API was utilized to develop XML files for different traffic environments, such as the number of vehicles in the system, their initial locations, initial speed, and final locations during traffic generation. A scenario was established, comprising of a platoon with connected AVs (CAV) driving on the main road and an ego AV(merger) merging onto the main road. A python script was developed that randomized the yaml file’s parameters such that the characteristics of the CAVs in a platoon and the speed of the platoon and ego AV varied during each episode. At the end of each episode, data about the platoon’s characteristics were extracted. This open framework is capable of including additional behaviors of the ego AV for more complicated training and provides flexibility during testing and the recording of the platoon and ego AV’s characteristics. Using a map of a two-lane highway, the parameters for each CAV and the ego AV were randomized for over 50 episodes. Three types of interactions were listed as follows. Merging was defined as the ego AV joining the platoon between two AVs. Joining was defined as the ego AV joining the platoon as a follower at the end. Failure was defined as the ego AV missing the 70 platoon. The table, stored as a CSV file, included the randomized speed of the ego AV and platoon, merging time, and interaction types. Correspondingly, the characteristics of each simulation were recorded. A total of 1,000 simulations were performed to collect data as input into RL training. The project provided a convenient and flexible framework for collecting training data on a specific scenario. The data values retrieved from the CSV file are vital to achieving an increased success rate with inverse reinforced learning.