A Slot-based Virtual Node Method of Consistent Hashing for Optimization of Index Reconfiguration on Distributed Search

カテゴリ: 論文誌(論文単位)

グループ名: 【C】電子・情報・システム部門

発行日: 2012/10/01

タイトル(英語): A Slot-based Virtual Node Method of Consistent Hashing for Optimization of Index Reconfiguration on Distributed Search

著者名: Yasuhiro Kirihata (Hitachi Solutions, Ltd.), Takashi Onoyama (Hitachi Solutions, Ltd.), Norihisa Komoda (Osaka University)

著者名(英語): Yasuhiro Kirihata (Hitachi Solutions, Ltd.), Takashi Onoyama (Hitachi Solutions, Ltd.), Norihisa Komoda (Osaka University)

キーワード: consistent hashing，distributed search，efficient index reconfiguration，search load balancing

要約(英語): In the distributed search system, the method of mapping and managing documents on segmented indexes is significant to realize load balancing of distributed search process and efficient cluster reconfiguration. Consistent hashing is the advanced method of data mapping which minimizes the network traffic and redundant index data processing in the index reconfiguration. However, if the cluster consists of several thousands of nodes, it requires huge memory resources. Furthermore, it takes a long time to execute the index reconfiguration because of the overhead of many index splitting processes. In this paper, we propose a new method called slot-based virtual node method of consistent hashing to solve the above issues. As the multiple nodes are added or removed, our new method plots or reallocates the virtual nodes on the hash ring space to realize "a bunch of" data migration as far as possible to optimize the index reconfiguration. Slot-based virtual node management saves memory consumption for the mapping information. We actually evaluate memory consumptions of both conventional and our proposed methods to bring out the resource-saving effect. Besides, we estimate lapse times of index reconfiguration processes based on the data processing models to verify the effective reduction of time in our method.

本誌: 電気学会論文誌C（電子・情報・システム部門誌） Vol.132 No.10 （2012） 特集：医療関連電子技術のニュートピック

本誌掲載ページ: 1698-1705 p

原稿種別: 論文／英語

電子版へのリンク: https://www.jstage.jst.go.jp/article/ieejeiss/132/10/132_1698/_article/-char/ja/