複数の上位個体を利用するEvolution Strategyによる深層強化学習におけるパラメータ最適化手法

カテゴリ: 論文誌(論文単位)

グループ名: 【C】電子・情報・システム部門

発行日: 2020/08/01

タイトル(英語): A Parameter Optimization Method for Deep Reinforcement Learning by Evolution Strategy Using Multiple Higher-Ranked Individuals

著者名: 土田　喬皓（千葉工業大学大学院情報科学研究科情報科学専攻），山口　智（千葉工業大学情報科学部情報工学科）

著者名(英語): Takahiro Tsuchida (Graduate School of Information and Computer Science, Chiba Institute of Technology), Satoshi Yamaguchi (Dept. of Computer Science, Chiba Institute of Technology)

キーワード: 強化学習，進化戦略　　reinforcement learning，evolution strategy

要約(英語): As a parameters optimization method for neural networks which is applied to reinforcement learning, Evolution Strategy has been proposed. In this method, neural network parameters are represented by individuals, like ordinary evolutional strategies. While the evolution, a new individual is generated from some distribution that centered a parameter and is weighted according to the order of reward that the neural network corresponding to the individual obtained. However, there are cased that the differences of reward values among the higher order individuals are so few that the updating can not lead to individuals to higher quality. So, in this research, after updating the normal parameters, we select the top individuals who get high rewards and weight them, and propose a method to update the parameters again using those individuals. By focusing on individuals who get a high reward, it is expected to search for a parameter that can obtain a high score earlier than the conventional method. In the experiment, the conventional method and the proposed method are applied to BipedalWalker which is a learning environment of a 2D biped robot in OpenAI Gym, and evaluation is performed and as a result, the proposed method showed better performance than the conventional method.

本誌: 電気学会論文誌C（電子・情報・システム部門誌） Vol.140 No.8 （2020） 特集Ⅰ：社会課題解決に向けた超スマート社会実現技術　特集Ⅱ：国際会議ICESS 2019

本誌掲載ページ: 1019-1027 p

原稿種別: 論文／日本語

電子版へのリンク: https://www.jstage.jst.go.jp/article/ieejeiss/140/8/140_1019/_article/-char/ja/