AI Learns to Park is a series of YouTube videos I released on my Channel about a machine learning project I did in my spare time, where I trained an Agent to learn how to park a car, including parallel parking. I used the Unity engine in combination with the ML-Agents framework.
The AI consists of a deep Neural Network with 3 hidden layers of 128 neurons each. It is trained with the Proximal Policy Optimization (PPO) algorithm, which is a Reinforcement Learning approach.
Basically, the input of the Neural Network are the readings of eight depth sensors, the car’s current speed and position, as well as its relative position to the target. The outputs of the Neural Network are interpreted as engine force, braking force and turning force. These outputs can be seen at the top right corner of the zoomed out camera shots.
The AI starts off with random behaviour, i.e. the Neural Network is initialized with random weights. It then gradually learns to solve the task by reacting to environment feedback accordingly. The environment tells the AI whether it is doing good or bad with positive or negative reward signals. In this project, the AI is rewarded with small positive signals for getting closer to the parking spot, which is outlined in red, and gets a larger reward when it actually reaches the parking spot and stops there. The final reward for reaching the parking spot is dependent on how parallel the car stops in relation to the actual parking position. If the car stops in a 90° angle to the actual parking direction for instance, the AI will only be rewarded a very small amount, relative to the amount it would get for stopping completely parallel to the actual direction. The AI is penalized with a negative reward signal, when it either drives further away from the parking spot or if it crashes into any obstacles.
AI Learns Parallel Parking
A lot of people suggested to try parallel parking in the comments of my frist video. So I created a second video using the same methods, trying different adjustments to the learning algorithm and environment in order to make the agent more generalizing and precise on the parallel parking task. The results were actually quite pleasing.
Two AI Fight for the same Parking Spot
As a fun additional experiment I also unleashed two Agents using the same trained neural network on the same parking lot and let them fight for the same parking spot.
Time of Development
Mid 2019 – Early 2020
