共查询到20条相似文献，搜索用时 15 毫秒
1.
Recent years have witnessed considerable advances in information retrieval for European languages other than English. We give an overview of commonly used techniques and we analyze them with respect to their impact on retrieval effectiveness. The techniques considered range from linguistically motivated techniques, such as morphological normalization and compound splitting, to knowledge-free approaches, such as n-gram indexing. Evaluations are carried out against data from the CLEF campaign, covering eight European languages. Our results show that for many of these languages a modicum of linguistic techniques may lead to improvements in retrieval effectiveness, as can the use of language independent techniques. 相似文献
2.
James C. French Allison L. Powell Fredric Gey Natalia Perelman 《Information Retrieval》2002,5(4):323-351
Vocabulary incompatibilities arise when the terms used to index a document collection are largely unknown, or at least not well-known to the users who eventually search the collection. No matter how comprehensive or well-structured the indexing vocabulary, it is of little use if it is not used effectively in query formulation. This paper demonstrates that techniques for mapping user queries into the controlled indexing vocabulary have the potential to radically improve document retrieval performance. We also show how the use of controlled indexing vocabulary can be employed to achieve performance gains for collection selection. Finally, we demonstrate the potential benefit of combining these two techniques in an interactive retrieval environment. Given a user query, our evaluation approach simulates the human user's choice of terms for query augmentation given a list of controlled vocabulary terms suggested by a system. This strategy lets us evaluate interactive strategies without the need for human subjects. 相似文献
3.
Eija Airio 《Information Retrieval》2006,9(3):249-271
The present research studies the impact of decompounding and two different word normalization methods, stemming and lemmatization,
on monolingual and bilingual retrieval. The languages in the monolingual runs are English, Finnish, German and Swedish. The
source language of the bilingual runs is English, and the target languages are Finnish, German and Swedish. In the monolingual
runs, retrieval in a lemmatized compound index gives almost as good results as retrieval in a decompounded index, but in the
bilingual runs differences are found: retrieval in a lemmatized decompounded index performs better than retrieval in a lemmatized
compound index. The reason for the poorer performance of indexes without decompounding in bilingual retrieval is the difference
between the source language and target languages: phrases are used in English, while compounds are used instead of phrases
in Finnish, German and Swedish. No remarkable performance differences could be found between stemming and lemmatization. 相似文献
4.
Jacques Savoy 《Information Retrieval》2007,10(6):509-529
This paper reports on the underlying IR problems encountered when indexing and searching with the Bulgarian language. For
this language we propose a general light stemmer and demonstrate that it can be quite effective, producing significantly better
MAP (around + 34%) than an approach not applying stemming. We implement the GL2 model derived from the Divergence from Randomness paradigm and find its retrieval effectiveness better than other probabilistic, vector-space and language models. The resulting
MAP is found to be about 50% better than the classical tf idf approach. Moreover, increasing the query size enhances the MAP by around 10% (from T to TD). In order to compare the retrieval
effectiveness of our suggested stopword list and the light stemmer developed for the Bulgarian language, we conduct a set
of experiments on another stopword list and also a more complex and aggressive stemmer. Results tend to indicate that there
is no statistically significant difference between these variants and our suggested approach. This paper evaluates other indexing
strategies such as 4-gram indexing and indexing based on the automatic decompounding of compound words. Finally, we analyze
certain queries to discover why we obtained poor results, when indexing Bulgarian documents using the suggested word-based
approach. 相似文献
5.
Word form normalization through lemmatization or stemming is a standard procedure in information retrieval because morphological variation needs to be accounted for and several languages are morphologically non-trivial. Lemmatization is effective but often requires expensive resources. Stemming is also effective in most contexts, generally almost as good as lemmatization and typically much less expensive; besides it also has a query expansion effect. However, in both approaches the idea is to turn many inflectional word forms to a single lemma or stem both in the database index and in queries. This means extra effort in creating database indexes. In this paper we take an opposite approach: we leave the database index un-normalized and enrich the queries to cover for surface form variation of keywords. A potential penalty of the approach would be long queries and slow processing. However, we show that it only matters to cover a negligible number of possible surface forms even in morphologically complex languages to arrive at a performance that is almost as good as that delivered by stemming or lemmatization. Moreover, we show that, at least for typical test collections, it only matters to cover nouns and adjectives in queries. Furthermore, we show that our findings are particularly good for short queries that resemble normal searches of web users. Our approach is called FCG (for Frequent Case (form) Generation). It can be relatively easily implemented for Latin/Greek/Cyrillic alphabet languages by examining their (typically very skewed) nominal form statistics in a small text sample and by creating surface form generators for the 3–9 most frequent forms. We demonstrate the potential of our FCG approach for several languages of varying morphological complexity: Swedish, German, Russian, and Finnish in well-known test collections. Applications include in particular Web IR in languages poor in morphological resources. 相似文献
6.
Due to the great variation of biological names in biomedical text, appropriate tokenization is an important preprocessing step for biomedical information retrieval. Despite its importance, there has been little study on the evaluation of various tokenization strategies for biomedical text. In this work, we conducted a careful, systematic evaluation of a set of tokenization heuristics on all the available TREC biomedical text collections for ad hoc document retrieval, using two representative retrieval methods and a pseudo-relevance feedback method. We also studied the effect of stemming and stop word removal on the retrieval performance. As expected, our experiment results show that tokenization can significantly affect the retrieval accuracy; appropriate tokenization can improve the performance by up to 96%, measured by mean average precision (MAP). In particular, it is shown that different query types require different tokenization heuristics, stemming is effective only for certain queries, and stop word removal in general does not improve the retrieval performance on biomedical text. 相似文献
7.
一种改进的余弦向量度量法文本检索模型 总被引：2，自引：1，他引：1
8.
In this paper, a novel neighborhood based document smoothing model for information retrieval has been proposed. Lexical association between terms is used to provide a context sensitive indexing weight to the document terms, i.e. the term weights are redistributed based on the lexical association with the context words. A generalized retrieval framework has been presented and it has been shown that the vector space model (VSM), divergence from randomness (DFR), Okapi Best Matching 25 (BM25) and the language model (LM) based retrieval frameworks are special cases of this generalized framework. Being proposed in the generalized retrieval framework, the neighborhood based document smoothing model is applicable to all the indexing models that use the term-document frequency scheme. The proposed smoothing model is as efficient as the baseline retrieval frameworks at runtime. Experiments over the TREC datasets show that the neighborhood based document smoothing model consistently improves the retrieval performance of VSM, DFR, BM25 and LM and the improvements are statistically significant. 相似文献
9.
This paper presents four novel techniques for open-vocabulary spoken document retrieval: a method to detect slots that possibly contain a query feature; a method to estimate occurrence probabilities; a technique that we call collection-wide probability re-estimation and a weighting scheme which takes advantage of the fact that long query features are detected more reliably. These four techniques have been evaluated using the TREC-6 spoken document retrieval test collection to determine the improvements in retrieval effectiveness with respect to a baseline retrieval method. Results show that the retrieval effectiveness can be improved considerably despite the large number of speech recognition errors. 相似文献
10.
组配标引是在词表中选择两个及两个以上有形式逻辑关系的词,按照特定规则组成的一组标引词串,用以满足文献多层次、多途径检索的需要。概念组配是文献标引的关键环节。根据参与组配的主题词之间的逻辑关系,概念组配可分为交叉组配、限定组配和联结组配3种基本类型。在实际组配标引工作中,编目员应避免因对新词表不熟悉造成检索词语构成混乱,避免因主题概念转换错误造成粗标、漏标和错标,避免因未遵循专指性标引规则造成切题不当,减少组配标引失误。 相似文献
11.
LSI潜在语义标引方法在情报检索中的应用 总被引：9，自引：2，他引：7
尹锋 《现代图书情报技术》1998,14(4):19-21
介绍了一种基于词相依性的语义结构, 被称为“潜在语义标引”的文献自动标引和检索技术。采用词频统计和奇值分解技术来捕捉文献的语义结构, 得到标引词、提问和文献的向量表示, 检索系统可以预测文献与提问之间的相关度, 达到检索的目的。 相似文献
12.
In Information Retrieval, since it is hard to identify users’ information needs, many approaches have been tried to solve
this problem by expanding initial queries and reweighting the terms in the expanded queries using users’ relevance judgments.
Although relevance feedback is most effective when relevance information about retrieved documents is provided by users, it
is not always available. Another solution is to use correlated terms for query expansion. The main problem with this approach
is how to construct the term-term correlations that can be used effectively to improve retrieval performance. In this study,
we try to construct query concepts that denote users’ information needs from a document space, rather than to reformulate initial queries using the term correlations
and/or users’ relevance feedback. To form query concepts, we extract features from each document, and then cluster the features into primitive concepts that are then used to form
query concepts. Experiments are performed on the Associated Press (AP) dataset taken from the TREC collection. The experimental evaluation
shows that our proposed framework called QCM (Query Concept Method) outperforms baseline probabilistic retrieval model on
TREC retrieval. 相似文献
13.
文献信息自动标引研究 总被引：12，自引：0，他引：12
作者认为我国文献自动标引研究的重点应由分词研究向实际标引研究转移。由此, 作者从标引源的确立、标引词权值的定义和使用、标引词库的构造、自动标引算法的实现等方面阐述了他们在自动标引方面的研究成果, 同时介绍了利用标引词库进行的检索改造。 相似文献
14.
Information retrieval systems operating on free text face difficulties when word forms used in the query and documents do not match. The usual solution is the use of a “stemming component” that reduces related word forms to a common stem. Extensive studies of such components exist for English, but considerably less is known for other languages. Previously, it has been claimed that stemming is essential for highly declensional languages. We report on our experiments on stemming for German, where an additional issue is the handling of compounds, which are formed by concatenating several words. The major contribution of our work that goes beyond its focus on German lies in the investigation of a complete spectrum of approaches, ranging from language-independent to elaborate linguistic methods. The main findings are that stemming is beneficial even when using a simple approach, and that carefully designed decompounding, the splitting of compound words, remarkably boosts performance. All findings are based on a thorough analysis using a large reliable test collection. 相似文献
15.
Christoph Baumgarten 《Information Retrieval》2000,3(3):253-271
This paper describes a probabilistic model for optimum information retrieval in a distributed heterogeneous environment.The model assumes the collection of documents offered by the environment to be partitioned into subcollections. Documents as well as subcollections have to be indexed, where indexing methods using different indexing vocabularies can be employed. A query provided by a user is answered in terms of a ranked list of documents. The model determines a procedure for ranking the documents that stems from the Probability Ranking Principle: For each subcollection, the subcollection's documents are ranked; the resulting ranked lists are combined into a final ranked list of documents, where the ordering is determined by the documents' probabilities of being relevant with respect to the user's query. Various probabilistic ranking methods may be involved in the distributed ranking process. A criterion for effectively limiting the ranking process to a subset of subcollections extends the model.The property that different ranking methods and indexing vocabularies can be used is important when the subcollections are heterogeneous with respect to their content.The model's applicability is experimentally confirmed. When exploiting the degrees of freedom provided by the model, experiments showed evidence that the model even outperforms comparable models for the non-distributed case with respect to retrieval effectiveness. 相似文献
16.
Jacques Savoy 《Information Retrieval》2004,7(1-2):121-148
This paper describes and evaluates different retrieval strategies that are useful for search operations on document collections written in various European languages, namely French, Italian, Spanish and German. We also suggest and evaluate different query translation schemes based on freely available translation resources. In order to cross language barriers, we propose a combined query translation approach that has resulted in interesting retrieval effectiveness. Finally, we suggest a collection merging strategy based on logistic regression that tends to perform better than other merging approaches. 相似文献
17.
Research on cross-language information retrieval (CLIR) has typically been restricted to settings using binary relevance assessments.
In this paper, we present evaluation results for dictionary-based CLIR using graded relevance assessments in a best match
retrieval environment. A text database containing newspaper articles and a related set of 35 search topics were used in the
tests. First, monolingual baseline queries were automatically formed from the topics. Secondly, source language topics (in
English, German, and Swedish) were automatically translated into the target language (Finnish), using structured target queries.
The effectiveness of the translated queries was compared to that of the monolingual queries. Thirdly, pseudo-relevance feedback
was used to expand the original target queries. CLIR performance was evaluated using three relevance thresholds: stringent,
regular, and liberal. When regular or liberal threshold was used, a reasonable performance was achieved. Using stringent threshold,
equally high performance could not be achieved. On all the relevance thresholds the performance of the translated queries
was successfully raised by pseudo-relevance feedback based query expansion. However, the performance of the stringent threshold
in relation to the other thresholds could not be raised by this method. 相似文献
18.
检索词自动扩展词库构建方法的基本思路是：根据语料是否规范化处理进行词库分类建设,优化了系统的检索性能;结合学科类别,对词库语料进行领域划分,引导科技人员对技术领域的准确把握;建设以本体库为基础,将与规范词具有关联性、相似性的语料通过关系表与关联库关联,把科技文献中的关键词组成一个有序的关系网,解决了传统检索系统中检索词无关联的不足;通过对检索词出现频率进行统计分析,进而更新词库,保证本体库、关联库语料的时效性,突破了人工对词库更新管理的受限性。 相似文献
19.
简要介绍网格信息检索的相关概念及服务内容,针对单一索引和查询处理机制所导致的传统信息检索系统各组件必须同步工作的问题,基于OGSA体系架构,构建支持分布式计算的接口模块,实现网格环境下的异步检索。为满足网格检索系统的安全性,引入虚拟组织策略域,通过OGSA的安全层保证在虚拟组织中获取目标资源,从而实现对资源的协调共享和使用。 相似文献
20.
基于多层次概念语义网络结构的中文医学信息语义标引体系和语义检索模型研究 总被引：18，自引：0，他引：18
为了提高中文医学信息检索效率,本文应用语义学研究成果,深入剖析统一医学语言系统(UMLS),从理论上对多层次概念语义网络结构进行了探讨,以此设计了适用于中文医学信息特点的三层概念语义网络结构,并分别确定了各个概念语义网络层次的语义类型和语义关系,进一步完善了医学信息语义网络.以信息检索的认知理论为依据,建立了基于三层概念语义网络结构的中文医学信息语义标引体系和语义检索模型.对扩展检索和语义检索进行统计学Kappa检验,认为两种检索方法的一致性非常显著(p<0.01);与扩展检索中的任何一种方法相比,语义检索方法具有更高的检索效率. 相似文献