A Study of Deep Reinforcement Learning in Autonomous Racing Using DeepRacer Car

Reinforcement learning is thought to be a promising branch of machine learning that has the potential to help us develop an Artificial General Intelligence (AGI) machine. Among the machine learning algorithms, primarily, supervised, semi supervised, unsupervised and reinforcement learning, reinforcement learning is different in a sense that it explores the environment without prior knowledge, and determines the optimal action. This study attempts to understand the concept behind reinforcement learning, the mathematics behind it and see it in action by deploying the trained model in Amazon's DeepRacer car. DeepRacer, a 1/18th scaled autonomous car, is the agent which is trained to race autonomously on a track. Optimum race line coordinates were calculated which allowed the agent to follow the fastest possible route on a given track. The agent was then trained using proximal policy optimization (PPO). Performance metrics such as the average reward per episode and cumulative reward were examined to fine tune the model. To further understand the distribution of action spaces, log analyses tools provided by the amazon was used. Based on the log analysis data, any un-used action was removed for efficient training. The trained model was uploaded into the DeepRacer car to test it in a race track outside of simulation.