Evaluating Large Language Model Generated Knowledge Graphs through Intermediate Artifact Analysis

カテゴリ: 研究会(論文単位)

論文No: IS24034

グループ名: 【C】電子・情報・システム部門 情報システム研究会

発行日: 2024/12/02

タイトル(英語): Evaluating Large Language Model Generated Knowledge Graphs through Intermediate Artifact Analysis

著者名: 高 子豪(神奈川大学),秋吉 政徳(神奈川大学)

著者名(英語): Zihao Gao(Kanagawa University),Masanori Akiyoshi(Kanagawa University)

キーワード: 知識グラフ|大規模言語モデル|Knowledge Graph|Large Language Model

要約(日本語): This research proposes a novel metric to evaluate the global quality of knowledge graphs generated by Large Language Models. Using The Tale of the Heike as topic for KG generation, we demonstrate how intermediate artifacts can be extracted from the complete KG using queries designed according to the schema, and we use this sub-graph to compare with a historical record (family tree) for refinement and evaluation in terms of local KG quality.

要約(英語): This research proposes a novel metric to evaluate the global quality of knowledge graphs generated by Large Language Models. Using The Tale of the Heike as topic for KG generation, we demonstrate how intermediate artifacts can be extracted from the complete KG using queries designed according to the schema, and we use this sub-graph to compare with a historical record (family tree) for refinement and evaluation in terms of local KG quality.

本誌: 2024年12月5日-2024年12月6日情報システム研究会

本誌掲載ページ: 29-33 p

原稿種別: 英語

PDFファイルサイズ: 360 Kバイト