Deep Q-learning for reducing enhanced distributed channel access collision in IEEE 802.11p of Vehicular Ad Hoc Network

Yongjun Liang, Ismail Fauzi Bin Isnin, Nur Haliza Abdul Wahab, Jiafu Su

Article ID: 9494
Vol 8, Issue 15, 2024


Abstract


The purpose of Vehicular Ad Hoc Network (VANET) is to provide users with better information services through effective communication. For this purpose, IEEE 802.11p proposes a protocol standard based on enhanced distributed channel access (EDCA) contention. In this standard, the backoff algorithm randomly adopts a lower bound of the contention window (CW) that is always fixed at zero. The problem that arises is that in severe network congestion, the backoff process will choose a smaller value to start backoff, thereby increasing conflicts and congestion. The objective of this paper is to solve this unbalanced backoff interval problem in saturation vehicles and this paper proposes a method that is a deep neural network Q-learning-based channel access algorithm (DQL-CSCA), which adjusts backoff with a deep neural network Q-learning algorithm according to vehicle density. Network simulation is conducted using NS3, the proposed algorithm is compared with the CSCA algorithm. The find is that DQL-CSCA can better reduce EDCA collisions.


Keywords


Vehicular Ad Hoc Network; enhanced distributed channel access; packet collision; deep neural network Q-learning

Full Text:

PDF


References


Aliyu, A., Abdullah, A. H., Isnin, I. F., Radzi, R. Z. R. M., Kumar, A., Darwish, T. S., & Joda, U. M. (2020). Road-based multi-metric forwarder evaluation for multipath video streaming in urban vehicular communication. Electronics, 9(10), 1663.

Banda, L., Mzyece, M., and Noel, G. (2012). IP mobility support: Solutions for vehicular networks. IEEE Vehicular Technology Magazine. 7(4), 77-87.

Balador, A., Tavares De Araujo Cesariny Calafate, C. M., Cano, J. C., and Manzoni, P. (2017). “A density-based contention window control scheme for unicast communications in vehicular ad hoc networks.” International Journal of Ad Hoc and Ubiquitous Computing, 24(1-2),65-75.

Ding, Z., Huang, Y., Yuan, H., & Dong, H. (2020). Introduction to reinforcement learning. Deep reinforcement learning: fundamentals, research and applications, 47-123.

Emmanuel, S. and Isnin, I. F. and Mohamad, M. M. (2019) A survey of TDMA-based MAC protocols for vehicular ad hoc networks. International Journal of Engineering and Advanced Technology, 8 (5). pp. 1247-1259. ISSN 2249-8958.

Ernst, D., & Louette, A. (2024). Introduction to reinforcement learning.

Fitah, A., Badri, A., Moughit, M., & Sahel, A. (2018). Performance of DSRC and WIFI for Intelligent Transport Systems in VANET. Procedia Computer Science, 127, 360-368.

Gopinath, A. J., and Nithya, B. (2018). “Mathematical and simulation analysis of contention resolution mechanism for IEEE 802.11 ah networks.” Computer Communications, 124, 87-100.

Gopinath, A. Justin, B. Nithya, Harshit Mogalapalli, and P. Kamalesh Khanna. “Channel Status based Contention Algorithm for Non-safety Applications in IEEE802. 11p Vehicular Network.” Procedia Computer Science 171 (2020): 1479-1488. https://doi.org/10.1016/j.procs.2020.04.158.

Harkat, Y., Amrouche, A., Lamini, E. S., & Kechadi, M. T. (2019). Modeling and performance analysis of the IEEE 802.11 p EDCA mechanism for VANET under saturation traffic conditions and error-prone channel. AEU-International Journal of Electronics and Communications, 101, 33-43.

IEEE Standard for Wireless LAN medium access control (MAC) and physical layer (PHY) specifcations. IEEE Std 802.11-1997, pp. 1–445 (2019). https://doi.org/10.1109/IEEESTD.1997.85951.

Lei, Xiaoying, Xiangjin Chen, and Seung Hyong Rhee. “A hybrid access method for broadcasting of safety messages in IEEE 802.11 p VANETs.” EURASIP Journal on Wireless Communications and Networking 2021, no. 1 (2021): 118. https://doi.org/10.1186/s13638-021-01933-3.

Ma, X., Yang, Y., Li, C., Lu, Y., Zhao, Q., & Jun, Y. (2021). Modeling the interaction between agents in cooperative multi-agent reinforcement learning. arXiv preprint arXiv:2102.06042.

Mahi, M. J. N., Chaki, S., Ahmed, S., Biswas, M., Kaiser, S., Islam, M. S., ... & Whaiduzzaman, M. (2022). A review on VANET research: Perspective of recent emerging technologies. IEEE Access.

RadhaKrishna Karne, D. T. (2021). Review on vanet architecture and applications. Turkish Journal of Computer and Mathematics Education (TURCOMAT), 12(4), 1745-1749.

Rawat, D. B., Popescu, D. C., Yan, G., and Olariu, S. (2011). “Enhancing VANET performance by joint adaptation of transmission power and contention window size”. IEEE Transactions on Parallel and Distributed Systems, 22(9), 1528-1535.

Wang, Y., Shi, J., & Chen, L. (2023). Performance Analysis of IEEE 802.11 p MAC with Considering Capture Effect under Nakagami-m Fading Channel in VANETs. Entropy, 25(2), 218.

Wijesekara, P. A. D. S. N., Sudheera, K. L. K., Sandamali, G. G. N., & Chong, P. H. J. (2023). An Optimization Framework for Data Collection in Software Defined Vehicular Networks. Sensors, 23(3), 1600.

Wu, G., and Xu, P. (2017). “Improving performance by a dynamic adaptive success-collision backoff algorithm for contention-based vehicular network.” IEEE Access, 6, 2496-2505.

Nasir, Q., & Albalt, M. (2009, March). Improved backoff algorithm for IEEE 802.11 networks. In 2009 International Conference on Networking, Sensing and Control (pp. 1-6). IEEE.

Stanica, R., Chaput, E., and Beylot, A. L. (2011, September). “Local density estimation for contention window adaptation in vehicular networks.” In 2011 IEEE 22nd International Symposium on Personal, Indoor and Mobile Radio Communications (pp. 730-734).

Chang, S. W., Cha, J., & Lee, S. S. (2012, February). Adaptive EDCA mechanism for vehicular ad-hoc network. In The International Conference on Information Network 2012 (pp. 379-383). IEEE.

Arora, P. O., and Patel, A. R. (2017, April). “Contention window based data dissemination in VANETs using roadside segmentation.” In 2017 2nd IEEE International Conference for Convergence in Technology (I2CT) (pp. 965-970).

Gattami, A., Bai, Q., & Aggarwal, V. (2021, March). Reinforcement learning for constrained markov decision processes. In International Conference on Artificial Intelligence and Statistics (pp. 2656-2664). PMLR.

Wijesekara, P. A. D. S. N., & Gunawardena, S. (2023, July). A Machine Learning-Aided Network Contention-Aware Link Lifetime-and Delay-Based Hybrid Routing Framework for Software-Defined Vehicular Networks. In Telecom (Vol. 4, No. 3, pp. 393-458). MDPI.




DOI: https://doi.org/10.24294/jipd9494

Refbacks

  • There are currently no refbacks.


Copyright (c) 2024 Yongjun Liang, Ismail Fauzi Bin Isnin, Nur Haliza Abdul Wahab, Jiafu Su

License URL: https://creativecommons.org/licenses/by/4.0/

This site is licensed under a Creative Commons Attribution 4.0 International License.