Auditory stimulus reconstruction with latent diffusion model from MEG signals

カテゴリ: 部門大会

論文No: SS1-1

グループ名: 【C】2023年電気学会電子・情報・システム部門大会

発行日: 2023/08/23

タイトル(英語): Auditory stimulus reconstruction with latent diffusion model from MEG signals

著者名: 仙田 淳（東京大学）,田中 真衣（東京大学）,森 史奈（東京大学）,杉野 正和（東京大学）,大槻 怜央（東京大学）,小谷 潔（東京大学）,神保 泰彦（東京大学）

著者名(英語): Jyun Senda (University of Tokyo),Mai Tanaka (University of Tokyo),Fumina Mori (University of Tokyo),Masato Sugino (University of Tokyo),Reo Otsuki (University of Tokyo),Kiyoshi Kotani (University of Tokyo),Yasuhiko Jimbo (University of Tokyo)

キーワード: 聴覚刺激|刺激再構築|脳磁図|深層学習|拡散モデルブレインデコーディング|audio stimuli|stimulus reconstruction|MEG|deep learning|diffusion modelbrain decoding

要約(日本語): Research to understand the information processing in the brain by modeling the relationship between stimuli and brain activity using deep learning has been conducted. Models that estimate stimuli from brain activity were used to understand the process of stimulus recognition in vision and hearing. In this study, we applied the latent diffusion model, a high quality image generation model, to hearing, and reconstructed spectrograms of auditory stimuli from brain activity measured by MEG. We extracted features from MEG for conditions in predicting stimuli in the model. The diffusion model reconstructed the spectrograms more accurately than CNN. The regions used for feature extraction from MEG were projected onto the cortical surface to confirm the regions involved in auditory recognition.