Learnable Weighting Mechanism in Model-based Reinforcement Learning

doi:10.13328/j.cnki.jos.006489

微信服务号

微信订阅号

2025-6-30- 0

Home > Archive>Volume 34, Issue 6, 2023 >2765-2775. DOI:10.13328/j.cnki.jos.006489

PDF HTML XML Export Cite reminder

Learnable Weighting Mechanism in Model-based Reinforcement Learning
DOI:
                        10.13328/j.cnki.jos.006489
                    
Author:
                        HUANG Wen-ZhenHUANG Wen-Zhen
School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100049, China;Center for Research on Intelligent System and Engineering (CRISE), Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China
Find this author on CNKI
Find this author on BaiDu
Search for this author on this site
YIN Qi-YueYIN Qi-Yue
School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100049, China;Center for Research on Intelligent System and Engineering (CRISE), Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China
Find this author on CNKI
Find this author on BaiDu
Search for this author on this site
ZHANG Jun-GeZHANG Jun-Ge
School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100049, China;Center for Research on Intelligent System and Engineering (CRISE), Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China
Find this author on CNKI
Find this author on BaiDu
Search for this author on this site
HUANG Kai-QiHUANG Kai-Qi
School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100049, China;Center for Research on Intelligent System and Engineering (CRISE), Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China;Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, Shanghai 200031, China
Find this author on CNKI
Find this author on BaiDu
Search for this author on this site

                    
Affiliation:
Clc Number:TP181
Fund Project:

Article

Figures

Metrics

Reference

Cited by

Materials

Comments

Abstract:

Model-based reinforcement learning methods train a model to simulate the environment by using the collected samples and utilize the imaginary samples generated by the model to optimize the policy, thus they have potential to improve sample efficiency. Nevertheless, due to the shortage of training samples, the environment model is often inaccurate, and the imaginary samples generated by it would be deleterious for the training process. For this reason, a learnable weighting mechanism is proposed which can reduce the negative effect on the training process by weighting the generated samples. The effect of the imaginary samples on the training process is quantified through calculating the difference between the losses on the real samples before and after updating value and policy networks by the imaginary samples. The experimental results show that the reinforcement learning algorithm using the weighting mechanism is superior to existing model-based and model-free algorithms in multiple tasks.

Key words:model-based reinforcement learning;model-bias;meta-learning;reinforcement learning;deep learning

Get Citation

黄文振,尹奇跃,张俊格,黄凯奇.基于模型的强化学习中可学习的样本加权机制.软件学报,2023,34(6):2765-2775

Copy

Article Metrics

Abstract:
PDF:
HTML:
Cited by:

History

Received:April 14,2021
Revised:June 07,2021
Adopted:
Online: October 14,2022
Published: June 06,2023

You are the firstVisitors
Copyright: Institute of Software, Chinese Academy of Sciences Beijing ICP No. 05046678-4
Address：4# South Fourth Street, Zhong Guan Cun, Beijing 100190,Postal Code：100190
Phone：010-62562563 Fax：010-62562533 Email：jos@iscas.ac.cn
Technical Support：Beijing Qinyun Technology Development Co., Ltd.

Beijing Public Network Security No. 11040202500063

微信服务号

微信订阅号

Get Citation

Share

微信扫一扫：分享

Article Metrics

History