微信服务号

微信订阅号

2025年8月13日 14:25 星期三
首页 > 过刊浏览>2024年第35卷第3期 >1485-1501. DOI:10.13328/j.cnki.jos.006814
PDF HTML阅读 XML下载 导出引用 引用提醒
基于深度强化学习的WRSN动态时空充电调度
作者:
中图分类号:

TP393

基金项目:

国家自然科学基金(62062047,61662042,61962030)


Dynamic Spatiotemporal Charging Scheduling Based on Deep Reinforcement Learning for WRSN
Author:
  • 摘要
    • | |
  • 访问统计
    • |
  • 参考文献 [45]
    • |
  • 相似文献 [20]
    • | | |
  • 文章评论
    摘要:

    高效的移动充电调度是构建长生命期、可持续运行的无线可充电传感器网络(WRSN)的关键之一.现有基于强化学习的充电策略只考虑了移动充电调度问题的一个维度,即移动充电器(MC)的路径规划,而忽略了充电调度问题中的另一维度,即充电时长调整,因而仍然存在性能限制.提出一种基于深度强化学习的WRSN动态时空充电调度方法(SCSD),建立充电序列调度和充电时长动态调整的深度强化学习模型.针对移动充电调度中离散的充电序列规划和连续的充电时长调整问题,使用DQN为待充电节点优化充电序列,并基于DDPG计算并动态调整序列中待充电节点的充电时长.通过分别从空间和时间两个维度的优化,在避免节点缺电失效的同时,所提出的SCSD可实现充电性能的有效提高.大量仿真实验结果表明,SCSD与现有的几种有代表性的充电方案相比,其充电性能具有明显的优势.

    Abstract:

    Efficient mobile charging scheduling is a key technology to build wireless rechargeable sensor networks (WRSN) which have long life cycle and sustainable operation ability. The existing charging methods based on reinforcement learning only consider the spatial dimension of mobile charging scheduling, i.e., the path planning of mobile chargers (MCs), while leaving out the temporal dimension of the problem, i.e., the adjustment of the charging duration, and thus these methods have suffered some performance limitations. This study proposes a dynamic spatiotemporal charging scheduling scheme based on deep reinforcement learning (SCSD) and establishes a deep reinforcement learning model for dynamic adjustment of charging sequence scheduling and charging duration. In view of the discrete charging sequence planning and continuous charging duration adjustment in mobile charging scheduling, the study uses DQN to optimize the charging sequence for nodes to be charged and calculates and dynamically adjusts the charging duration of the nodes. By optimizing the two dimensions of space and time respectively, the SCSD proposed in this study can effectively improve the charging performance while avoiding the power failure of nodes. Simulation experiments show that SCSD has significant performance advantages over several well-known typical charging schemes.

    参考文献
    [1] Kurs A, Karalis A, Moffatt R, Joannopoulos JD, Fisher P, Soljac?ic? M. Wireless power transfer via strongly coupled magnetic resonances. Science, 2007, 317(5834):83-86.[doi:10.1126/science.1143254]
    [2] Zhang Z, Pang HL, Georgiadis A, Cecati C. Wireless power transfer-An overview. IEEE Transactions on Industrial Electronics, 2019, 66(2):1044-1058.[doi:10.1109/TIE.2018.2835378]
    [3] 胡诚, 汪芸, 王辉. 无线可充电传感器网络中充电规划研究进展. 软件学报, 2016, 27(1):72-95. http://www.jos.org.cn/1000-9825/4883.htm
    Hu C, Wang Y, Wang H. Survey on charging programming in wireless rechargeable sensor networks. Ruan Jian Xue Bao/Journal of Software, 2016, 27(1):72-95 (in Chinese with English abstract). http://www.jos.org.cn/1000-9825/4883.htm
    [4] He SB, Chen JM, Jiang FC, Yau DKY, Xing GL, Sun YX. Energy provisioning in wireless rechargeable sensor networks. IEEE Transactions on Mobile Computing, 2013, 12(10):1931-1942.[doi:10.1109/TMC.2012.161]
    [5] Wang C, Li J, Ye F, Yang YY. Improve charging capability for wireless rechargeable sensor networks using resonant repeaters. In:Proc. of the 35th IEEE Int'l Conf. on Distributed Computing Systems. Columbus:IEEE, 2015. 133-142.
    [6] Cao XB, Xu WZ, Liu XX, Peng J, Liu T. A deep reinforcement learning-based on-demand charging algorithm for wireless rechargeable sensor networks. Ad Hoc Networks, 2021, 110:102278.[doi:10.1016/j.adhoc.2020.102278]
    [7] Yang MY, Liu NB, Zuo L, Feng Y, Liu MH, Gong HG, Liu M. Dynamic charging scheme problem with actor-critic reinforcement learning. IEEE Internet of Things Journal, 2021, 8(1):370-380.[doi:10.1109/JIOT.2020.3005598]
    [8] Li X, Jin M. Charger scheduling optimization framework. In:Proc. of the 18th IEEE Int'l Symp. on Network Computing and Applications. Cambridge:IEEE, 2019. 1-8.
    [9] Wei ZC, Liu F, Lyu Z, Ding X, Shi L, Xia CK. Reinforcement learning for a novel mobile charging strategy in wireless rechargeable sensor networks. In:Proc. of the 13th Int'l Conf. on Wireless Algorithms, Systems, and Applications. Tianjin:Springer, 2018. 485-496.
    [10] Lillicrap TP, Hunt JJ, Pritzel A, Heess N, Erez T, Tassa Y, Silver D, Wierstra D. Continuous control with deep reinforcement learning. In:Proc. of the 4th Int'l Conf. on Learning Representations. San Juan:ICLR, 2016.
    [11] Mnih V, Kavukcuoglu K, Silver D, Graves A, Antonoglou I, Wierstra D, Riedmiller M. Playing atari with deep reinforcement learning. arXiv:1312.5602, 2013.
    [12] Shi Y, Xie LG, Hou YT, Sherali HD. On renewable sensor networks with wireless energy transfer. In:Proc. of the 2011 IEEE INFOCOM. Shanghai:IEEE, 2011. 1350-1358.
    [13] Han GJ, Yang X, Liu L, Zhang WB. A joint energy replenishment and data collection algorithm in wireless rechargeable sensor networks. IEEE Internet of Things Journal, 2018, 5(4):2596-2604.[doi:10.1109/JIOT.2017.2784478]
    [14] Wu PF, Xiao F, Sha C, Huang HP, Sun LJ. Trajectory optimization for UAVs' efficient charging in wireless rechargeable sensor networks. IEEE Transactions on Vehicular Technology, 2020, 69(4):4207-4220.[doi:10.1109/TVT.2020.2969220]
    [15] 魏振春, 孙仁浩, 吕增威, 韩江洪, 石雷, 徐俊逸. 联合充电和数据收集的WCE多目标路径规划算法. 通信学报, 2018, 39(10):22-33.[doi:10.11959/j.issn.1000-436x.2018216]
    Wei ZC, Sun RH, Lyu ZW, Han JH, Shi L, Xu JY. Path planning algorithm for WCE with joint energy replenishment and data collection based on multi-objective optimization. Journal on Communications, 2018, 39(10):22-33 (in Chinese with English abstract).[doi:10.11959/j.issn.1000-436x.2018216]
    [16] Dong Y, Li SY, Bao GJ, Wang CY. An efficient combined charging strategy for large-scale wireless rechargeable sensor networks. IEEE Sensors Journal, 2020, 20(17):10306-10315.[doi:10.1109/JSEN.2020.2990641]
    [17] He L, Kong LH, Gu Y, Pan JP, Zhu T. Evaluating the on-demand mobile charging in wireless sensor networks. IEEE Transactions on Mobile Computing, 2015, 14(9):1861-1875.[doi:10.1109/TMC.2014.2368557]
    [18] Lin C, Yang ZW, Dai HP, Cui LX, Wang L, Wu GW. Minimizing charging delay for directional charging. IEEE/ACM Transactions on Networking, 2021, 29(6):2478-2493.[doi:10.1109/TNET.2021.3095280]
    [19] Sha C, Sun Y, Malekian R. Research on cost-balanced mobile energy replenishment strategy for wireless rechargeable sensor networks. IEEE Transactions on Vehicular Technology, 2020, 69(3):3135-3150.[doi:10.1109/TVT.2019.2962877]
    [20] 水九生, 徐向华. 一种基于多节点充电模型的按需顺带充电方案. 电子学报, 2021, 49(2):346-353.[doi:10.12263/DZXB.20200363]
    Shui JS, Xu XH. An on-demand passer-by charging scheme based on multi-node charging model. Acta Electronica Sinica, 2021, 49(2):346-353 (in Chinese with English abstract).[doi:10.12263/DZXB.20200363]
    [21] Wang YH, Dong Y, Li SY, Huang RY, Shang YH. A new on-demand recharging strategy based on cycle-limitation in a WRSN. Symmetry, 2019, 11(8):1028.[doi:10.3390/sym11081028]
    [22] He L, Fu LK, Zheng LK, Gu Y, Cheng P, Chen JM, Pan JP. ESync:An energy synchronized charging protocol for rechargeable wireless sensor networks. In:Proc. of the 15th ACM Int'l Symp. on Mobile ad Hoc Networking and Computing. Philadelphia:ACM, 2014. 247-256.
    [23] Kan YP, Chang CY, Kuo CH, Roions Research, 1977, 25(1):45-61.
    energy charging mechanism using mobile charger for WRSNs. IEEE Systems Journal, 2022, 16(3):3993-4004.[doi:10.1109/JSYST.2021.3109056]
    [24] Lin C, Guo CY, Dai HP, Wang L, Wu GW. Near optimal charging scheduling for 3-D wireless rechargeable sensor networks with energy constraints. In:Proc. of the 39th IEEE Int'l Conf. on Distributed Computing Systems. Dallas:IEEE, 2019. 624-633.
    [25] Xu WZ, Liang WF, Lin XL, Mao GQ. Efficient scheduling of multiple mobile chargers for wireless sensor networks. IEEE Transactions on Vehicular Technology, 2016, 65(9):7670-7683.[doi:10.1109/TVT.2015.2496971]
    [26] Wang K, Wang L, Lin C, Obaidat MS, Alam M. Prolonging lifetime for wireless rechargeable sensor networks through sleeping and charging scheduling. International Journal of Communication Systems, 2020, 33(8):e4355.[doi:10.1002/dac.4355]
    [27] Liu T, Wu BJ, Wu HY, Peng J. Low-cost collaborative mobile charging for large-scale wireless sensor networks. IEEE Transactions on Mobile Computing, 2017, 16(8):2213-2227.[doi:10.1109/TMC.2016.2616309]
    [28] Lin C, Zhou JZ, Guo CY, Song HB, Wu GW, Obaidat MS. TSCA:A temporal-spatial real-time charging scheduling algorithm for on-demand architecture in wireless rechargeable sensor networks. IEEE Transactions on Mobile Computing, 2018, 17(1):211-224.[doi:10.1109/TMC.2017.2703094]
    [29] Han GJ, Guan HF, Wu JW, Chan S, Shu L, Zhang WB. An uneven cluster-based mobile charging algorithm for wireless rechargeable sensor networks. IEEE Systems Journal, 2019, 13(4):3747-3758.[doi:10.1109/JSYST.2018.2879084]
    [30] Aslam N, Xia KW, Hadi MU. Optimal wireless charging inclusive of intellectual routing based on SARSA learning in renewable wireless sensor networks. IEEE Sensors Journal, 2019, 19(18):8340-8351.[doi:10.1109/JSEN.2019.2918865]
    [31] Soni S, Shrivastava M. Novel wireless charging algorithms to charge mobile wireless sensor network by using reinforcement learning. SN Applied Sciences, 2019, 1(9):1052.[doi:10.1007/s42452-019-1091-2]
    [32] Hu C, Wang Y. Schedulability decision of charging missions in wireless rechargeable sensor networks. In:Proc. of the 11th Annual IEEE Int'l Conf. on Sensing, Communication, and Networking. Singapore:IEEE, 2014. 450-458.
    [33] D'Arienzo M, Iacono M, Marrone S, Nardone R. Estimation of the energy consumption of mobile sensors in WSN environmental monitoring applications. In:Proc. of the 27th Int'l Conf. on Advanced Information Networking and Applications Workshops. Barcelona:IEEE, 2013. 1588-1593.
    [34] Xie LG, Shi Y, Hou YT, Sherali HD. Making sensor networks immortal:An energy-renewal approach with wireless power transfer. IEEE/ACM Transactions on Networking, 2012, 20(6):1748-1761.[doi:10.1109/TNET.2012.2185831]
    [35] Hou YT, Shi Y, Sherali HD. Rate allocation and network lifetime problems for wireless sensor networks. IEEE/ACM Transactions on Networking, 2008, 16(2):321-334.[doi:10.1109/TNET.2007.900407]
    [36] 朱金奇, 冯勇, 孙华志, 刘明, 张兆年. 无线可充电传感器网络中能量饥饿避免的移动充电. 软件学报, 2018, 29(12):3868-3885. http://www.jos.org.cn/1000-9825/5315.htm
    Zhu JQ, Feng Y, Sun HZ, Liu M, Zhang ZN. Energy starvation avoidance mobile charging for wireless rechargeable sensor networks. Ruan Jian Xue Bao/Journal of Software, 2018, 29(12):3868-3885 (in Chinese with English abstract). http://www.jos.org.cn/1000-9825/5315.htm
    [37] Zhao CX, Zhang HJ, Chen FL, Chen SG, Wu CZ, Wang TC. Spatiotemporal charging scheduling in wireless rechargeable sensor networks. Computer Communications, 2020, 152:155-170.[doi:10.1016/j.comcom.2020.01.037]
    [38] Chu M, Li H, Liao XW, Cui SG. Reinforcement learning-based multiaccess control and battery prediction with energy harvesting in IoT systems. IEEE Internet of Things Journal, 2019, 6(2):2009-2020.[doi:10.1109/JIOT.2018.2872440]
    [39] Qiu CR, Hu Y, Chen Y, Zeng B. Deep deterministic policy gradient (DDPG)-based energy harvesting wireless communications. IEEE Internet of Things Journal, 2019, 6(5):8577-8588.[doi:10.1109/JIOT.2019.2921159]
    [40] Panwalkar SS, Iskander W. A survey of scheduling rules. Operat
    引证文献
    网友评论
    网友评论
    分享到微博
    发 布
引用本文

王艺均,冯勇,刘明,刘念伯.基于深度强化学习的WRSN动态时空充电调度.软件学报,2024,35(3):1485-1501

复制
分享
文章指标
  • 点击次数:1508
  • 下载次数: 2643
  • HTML阅读次数: 1211
  • 引用次数: 0
历史
  • 收稿日期:2022-04-13
  • 最后修改日期:2022-05-24
  • 在线发布日期: 2023-05-17
  • 出版日期: 2024-03-06
文章二维码
友情链接:中国科学院软件研究所中国计算机学会中国科学院国家自然科学基金委员会CCF数字图书馆Int'l J of Softw Info计算机系统应用
您是第20541429位访问者
版权所有：中国科学院软件研究所 京ICP备05046678号-3
地址：北京市海淀区中关村南四街4号,邮政编码：100190
电话：010-62562563 传真：010-62562533 Email：jos@iscas.ac.cn
技术支持：北京勤云科技发展有限公司

京公网安备 11040202500063号