Equilibrium-Aware Decentralized Cooperative Caching for Internet of Vehicles

10.6919/ICJE.202205_8(5).0023

Yizheng Fan；Honghai Wu

Internet of Vehicles ； Cooperative Caching ； Deep Reinforcement Learning ； Federated Learning ； Attention Mechanism

International Core Journal of Engineering

8卷5期（2022 / 05 / 01）

180 - 193

英文

Since edge caching is expected to solve a series of problems caused by limited spectrum and bandwidth resources, it has also become one of the most potential technologies to break through the bottleneck in the development of the Internet of Vehicles. However, many related studies are lacking in pertinence and difficult to adapt to the dynamic network environment targeting vehicles. Based on the dynamic cooperation among smart vehicles, Edge Base Stations (EBS) and Cloud Data Center (CDC), this paper proposes an equilibrium-aware decentralized cooperative caching for Internet of Vehicles by jointly optimizing caching node selection and content update. In this paper, we model the objective optimization problem as a Markov Decision Process (MDP), use the Deep Deterministic Policy Gradient algorithm to solve the formulated long-term mixed integer linear programming problem, and improve the federated learning training process by considering limited resources as well as vehicle privacy. Among them, an Attention-Weighted Decentralized Cooperative caching (AWDC) is proposed to optimize the caching model for different vehicles and dynamic environments. An attention mechanism is used to control the model weights in the federated learning aggregation step to address the imbalance in the quality of different local models. It assigns different aggregation weights to different quality models to ensure a more accurate model, which can effectively improve the hit rate, reduce the average delay and offload traffic, and improve the fairness among users.

1. Liu L, Chen C, Pei Q, et al. Vehicular Edge Computing and Networking: A Survey. Mobile Networks and Applications. Vol. 16 (2020), p.1145–1168.
   連結：
2. Chen X, Zhang H, Wu C, et al. Optimized Computation Offloading Performance in Virtual Edge Computing Systems Via Deep Reinforcement Learning. IEEE Internet of Things Journal. Vol. 6 (2019) No. 3, P. 4005-4018.
   連結：
3. Lin P, Song Q, Song J, et al. Cooperative Caching and Transmission in CoMP-Integrated Cellular Networks Using Reinforcement Learning. IEEE Transactions on Vehicular Technology. Vol. 69 (2020) No. 5, P. 5508-5520.
   連結：
4. Jiang K, Zhou H, Zeng D, et al. Multi-Agent Reinforcement Learning for Cooperative Edge Caching in Internet of Vehicles. 2020 IEEE 17th International Conference on Mobile Ad Hoc and Sensor Systems (MASS). Online, 2020, P. 455-463.
   連結：
5. Xu X, Gao S, Tao M. Distributed Online Caching for High-Definition Maps in Autonomous Driving Systems. IEEE Wireless Communications Letters. Vol. 10 (2020) No. 7, P. 1390-1394.
   連結：
6. Li T, Sahu AK, Talwalkar A, et al. Federated Learning: Challenges, Methods, and Future Directions. IEEE Signal Processing Magazine. Vol. 37 (2020) No. 3, P. 50-60.
   連結：
7. Ren J, Wang H, Hou T, Zheng S, Federated Learning-Based Computation Offloading Optimization in Edge Computing-Supported Internet of Things. IEEE Access. Vol. 7 (2019), P. 691979201.
   連結：
8. Wang X, Han Y, Wang C, et al. In-Edge AI: Intelligentizing Mobile Edge Computing, Caching and Communication by Federated Learning. IEEE Network. Vol. 33 (2019) No. 5, P. 156-165.
   連結：
9. Breslau L, Pei C, Li F, et al. Web caching and Zipf-like distributions: evidence and implications. IEEE INFOCOM '99. Conference on Computer Communications. Proceedings. Eighteenth Annual Joint Conference of the IEEE Computer and Communications Societies. The Future is Now (Cat. No.99CH36320). New York, 1999, P. 126-1.
   連結：
10. Gao W, Li Q, Zhao B, et al. Social-Aware Multicast in Disruption-Tolerant Networks. IEEE/ACM Transactions on Networking. Vol. 20 (2012) No. 5, P. 1553-1566.
    連結：
11. Luan TH, Shen X, Bai F. Integrity-oriented content transmission in highway vehicular ad hoc networks. 2013 Proceedings IEEE INFOCOM. Turin, 2013, p. 2562–2570.
    連結：
12. Wang XF, Wang CY, Li XH, et al. Federated deep reinforcement learning for Internet of Things with decentralized cooperative edge caching. IEEE Internet of Things Journal, Vol. 7 (2020) No. 10, p. 9441- 9455.
    連結：
13. Liu L, Chen C, Pei Q, et al. Vehicular Edge Computing and Networking: A Survey. Mobile Networks and Applications. Vol. 16 (2020), p. 1145–1168.
    連結：
14. Qiao G, Leng S, Maharjan S, et al. Deep reinforcement learning for cooperative content caching in vehicular edge computing and networks. IEEE Internet Things J. Vol. 7 (2020) No. 1, p. 247–257.
    連結：
15. Lim WYB, Luong NC, Hoang DT, et al. Federated Learning in Mobile Edge Networks: A Comprehensive Survey. IEEE Communications Surveys & Tutorials, Vol. 22 (2020) No. 3, p. 2031-2063.
    連結：
16. Zn A, Gz A, Hui YB. A Review on the Attention Mechanism of Deep Learning. Neurocomputing, Vol. 452 (2021) No. 10, p. 48-62.
    連結：
17. Mills J, Hu J, Min G. Communication-Efficient Federated Learning for Wireless Edge Intelligence in IoT. IEEE Internet of Things Journal, Vol. 7 (2020) No. 7, p. 5986-5994. [19] Information on: http://arxiv.org/abs/1907.0218.
    連結：
18. Li FW, Zhang HB, Wang ZX. V2X Collaborative Caching and Resource Allocation in MEC-Based IoV. Journal on Communications. Vol. 42 (2021) No. 2, P. 26-37.