Can I read Overlapping statistical word indexing: a new indexing method for Japanese text on EtoBox?

Overlapping statistical word indexing: a new indexing method for Japanese text by Yasushi Ogawa; Toru Matsuda is a scholarly article available to read on EtoBox.

What is Overlapping statistical word indexing: a new indexing method for Japanese text about?

Because word boundaries are not apparently indicated in Asiart languages including Japanese, word indexing cannot simply be applied. Although dictionary-based text segmentation techniques enable word indexing, they have some problems such as dictionary maintenance. N-gram indexing, another conventional indexing method, suffers from increase in index size. This paper proposes a new statistical indexing method. We fist propose a segmentation method for Japanese text which uses statistical information of characters. It needs only a small amount of statistic information and computation, and does not need constant maintenance. We secondly propose a new indexing strategy which extracts some overlapping segments in addition to the segments extracted using the existing strategy. Thus it increases the effectiveness of retrieval.

Author

Yasushi Ogawa; Toru Matsuda

Publisher

ACM Press

Published

1997

Language

EN