An Adaptive Contention Window using Actor-Critic Reinforcement Learning Algorithm for Vehicular Ad-hoc NETworks

Abstract

The IEEE 802.11p standard’s collision avoidance mechanism is sub-optimal due to the use of a Binary Exponential Backoff (BEB) algorithm in the Medium Access Control (MAC) layer. This algorithm increases the backoff interval upon collision detection to reduce the probability of subsequent collisions. However, it causes a degradation of radio spectrum utilization, leading to bandwidth wastage, especially when managing access to a dense number of stations. An incorrect backoff setting can also increase the likelihood of collisions and decrease network performance. To solve this optimization problem, the proposed model is based on Reinforcement Learning (RL) Actor-Critic (AC) network to adapt the CW value by learning the VANET environment. NS-3 network simulator with the NS3-gym module for VANET is utilized to test and compute the optimal CW values. The results show that the proposed AC and EAC outperform traditional BEB used in the IEEE 802.11p standard as the number of vehicles increases. The throughput is enhanced by 54% and 69% respectively for AC and Enhanced AC (EAC). Similarly, the end-to-end delay for AC is reduced by 19%, and for EAC it’s reduced by 25%.