共查询到20条相似文献，搜索用时 31 毫秒
1.
Eija Airio 《Information Retrieval》2006,9(3):249-271
The present research studies the impact of decompounding and two different word normalization methods, stemming and lemmatization,
on monolingual and bilingual retrieval. The languages in the monolingual runs are English, Finnish, German and Swedish. The
source language of the bilingual runs is English, and the target languages are Finnish, German and Swedish. In the monolingual
runs, retrieval in a lemmatized compound index gives almost as good results as retrieval in a decompounded index, but in the
bilingual runs differences are found: retrieval in a lemmatized decompounded index performs better than retrieval in a lemmatized
compound index. The reason for the poorer performance of indexes without decompounding in bilingual retrieval is the difference
between the source language and target languages: phrases are used in English, while compounds are used instead of phrases
in Finnish, German and Swedish. No remarkable performance differences could be found between stemming and lemmatization. 相似文献
2.
On knowledge-poor methods for person name matching and lemmatization for highly inflectional languages 总被引：1，自引：1，他引：0
Web person search is one of the most common activities of Internet users. Recently, a vast amount of work on applying various
NLP techniques for person name disambiguation in large web document collections has been reported, where the main focus was
on English and few other major languages. This article reports on knowledge-poor methods for tackling person name matching
and lemmatization in Polish, a highly inflectional language with complex person name declension paradigm. These methods apply
mainly well-established string distance metrics, some new variants thereof, automatically acquired simple suffix-based lemmatization
patterns and some combinations of the aforementioned techniques. Furthermore, we also carried out some initial experiments
on deploying techniques that utilize the context, in which person names appear. Results of numerous experiments are presented.
The evaluation carried out on a data set extracted from a corpus of on-line news articles revealed that achieving lemmatization
accuracy figures greater than 90% seems to be difficult, whereas combining string distance metrics with suffix-based patterns
results in 97.6–99% accuracy for the name matching task. Interestingly, no significant additional gain could be achieved through
integrating some basic techniques, which try to exploit the local context the names appear in. Although our explorations were
focused on Polish, we believe that the work presented in this article constitutes practical guidelines for tackling the same
problem for other highly inflectional languages with similar phenomena.
相似文献
| Marcin SydowEmail: |
3.
Information retrieval systems operating on free text face difficulties when word forms used in the query and documents do not match. The usual solution is the use of a “stemming component” that reduces related word forms to a common stem. Extensive studies of such components exist for English, but considerably less is known for other languages. Previously, it has been claimed that stemming is essential for highly declensional languages. We report on our experiments on stemming for German, where an additional issue is the handling of compounds, which are formed by concatenating several words. The major contribution of our work that goes beyond its focus on German lies in the investigation of a complete spectrum of approaches, ranging from language-independent to elaborate linguistic methods. The main findings are that stemming is beneficial even when using a simple approach, and that carefully designed decompounding, the splitting of compound words, remarkably boosts performance. All findings are based on a thorough analysis using a large reliable test collection. 相似文献
4.
Fernando Martínez-Santiago L. Alfonso Ureña-López Maite Martín-Valdivia 《Information Retrieval》2006,9(1):71-93
A usual strategy to implement CLIR (Cross-Language Information Retrieval) systems is the so-called query translation approach.
The user query is translated for each language present in the multilingual collection in order to compute an independent monolingual
information retrieval process per language. Thus, this approach divides documents according to language. In this way, we obtain
as many different collections as languages. After searching in these corpora and obtaining a result list per language, we
must merge them in order to provide a single list of retrieved articles.
In this paper, we propose an approach to obtain a single list of relevant documents for CLIR systems driven by query translation.
This approach, which we call 2-step RSV (RSV: Retrieval Status Value), is based on the re-indexing of the retrieval documents
according to the query vocabulary, and it performs noticeably better than traditional methods.
The proposed method requires query vocabulary alignment: given a word for a given query, we must know the translation or translations
to the other languages. Because this is not always possible, we have researched on a mixed model. This mixed model is applied
in order to deal with queries with partial word-level alignment. The results prove that even in this scenario, 2-step RSV
performs better than traditional merging methods. 相似文献
5.
Prior-art search in patent retrieval is concerned with finding all existing patents relevant to a patent application. Since patents often appear in different languages, cross-language information retrieval (CLIR) is an essential component of effective patent search. In recent years machine translation (MT) has become the dominant approach to translation in CLIR. Standard MT systems focus on generating proper translations that are morphologically and syntactically correct. Development of effective MT systems of this type requires large training resources and high computational power for training and translation. This is an important issue for patent CLIR where queries are typically very long sometimes taking the form of a full patent application, meaning that query translation using MT systems can be very slow. However, in contrast to MT, the focus for information retrieval (IR) is on the conceptual meaning of the search words regardless of their surface form, or the linguistic structure of the output. Thus much of the complexity of MT is not required for effective CLIR. We present an adapted MT technique specifically designed for CLIR. In this method IR text pre-processing in the form of stop word removal and stemming are applied to the MT training corpus prior to the training phase. Applying this step leads to a significant decrease in the MT computational and training resources requirements. Experimental application of the new approach to the cross language patent retrieval task from CLEF-IP 2010 shows that the new technique to be up to 23 times faster than standard MT for query translations, while maintaining IR effectiveness statistically indistinguishable from standard MT when large training resources are used. Furthermore the new method is significantly better than standard MT when only limited translation training resources are available, which can be a significant issue for translation in specialized domains. The new MT technique also enables patent document translation in a practical amount of time with a resulting significant improvement in the retrieval effectiveness. 相似文献
6.
In this paper, which treats Swedish full text retrieval, the problem of morphological variation of query terms in the document database is studied. The Swedish CLEF 2003 test collection was used, and the effects of combination of indexing strategies with query terms on retrieval effectiveness were studied. Four of the seven tested combinations involved indexing strategies that used normalization, a form of conflation. All of these four combinations employed compound splitting, both during indexing and at query phase. SWETWOL, a morphological analyzer for the Swedish language, was used for normalization and compound splitting. A fifth combination used stemming, while a sixth attempted to group related terms by right hand truncation of query terms. The truncation was performed by a search expert. These six combinations were compared to each other and to a baseline combination, where no attempt was made to counteract the problem of morphological variation of query terms in the document database. Both the truncation combination, the four combinations based on normalization and the stemming combination outperformed the baseline. Truncation had the best performance. The main conclusion of the paper is that truncation, normalization and stemming enhanced retrieval effectiveness in comparison to the baseline. Further, normalization and stemming were not far below truncation. 相似文献
7.
Due to the great variation of biological names in biomedical text, appropriate tokenization is an important preprocessing step for biomedical information retrieval. Despite its importance, there has been little study on the evaluation of various tokenization strategies for biomedical text. In this work, we conducted a careful, systematic evaluation of a set of tokenization heuristics on all the available TREC biomedical text collections for ad hoc document retrieval, using two representative retrieval methods and a pseudo-relevance feedback method. We also studied the effect of stemming and stop word removal on the retrieval performance. As expected, our experiment results show that tokenization can significantly affect the retrieval accuracy; appropriate tokenization can improve the performance by up to 96%, measured by mean average precision (MAP). In particular, it is shown that different query types require different tokenization heuristics, stemming is effective only for certain queries, and stop word removal in general does not improve the retrieval performance on biomedical text. 相似文献
8.
We present a system for multilingual information retrieval that allows users to formulate queries in their preferred language and retrieve relevant information from a collection containing documents in multiple languages. The system is based on a process of document level alignments, where documents of different languages are paired according to their similarity. The resulting mapping allows us to produce a multilingual comparable corpus. Such a corpus has multiple interesting applications. It allows us to build a data structure for query translation in cross-language information retrieval (CLIR). Moreover, we also perform pseudo relevance feedback on the alignments to improve our retrieval results. And finally, multiple retrieval runs can be merged into one unified result list. The resulting system is inexpensive, adaptable to domain-specific collections and new languages and has performed very well at the TREC-7 conference CLIR system comparison. 相似文献
9.
10.
从市场营销数据库中处理大量的数据来查找知识 ,往往耗费巨大 ,而一些附加信息 ,如市场营销领域知识或背景信息 ,通常能指导并限制对有利知识的获取。本文针对市场营销领域知识在数据库知识获取处理中的一些运用技巧作了一些探讨 ,特别是利用市场营销领域知识来简化数据库查找程序 ,充分运用一些描述要查找的信息的假设模型 ,通过查询来证明假设 ,并淘汰一些多余的、相互矛盾的查找规律。对于淘汰下的数据还可以用离群数据挖掘分析和市场营销领域知识进行二次获取知识。 相似文献
11.
12.
Raphaël Hubain Max De Wilde Seth van Hooland 《Cataloging & classification quarterly》2016,54(7):403-417
Ensuring quick and consistent access to large collections of unstructured documents is one of the biggest challenges facing knowledge-intensive organizations. Designing specific vocabularies to index and retrieve documents is often deemed too expensive, full-text search being preferred despite its known limitations. However, the process of creating controlled vocabularies can be partly automated thanks to natural language processing and machine learning techniques. With a case study from the biopharmaceutical industry, we demonstrate how small organizations can use an automated workflow in order to create a controlled vocabulary to index unstructured documents in a semantically meaningful way. 相似文献
13.
潘燕桃 《现代图书情报技术》1996,12(5):10-13
SGML (Standard Generalized Markup Language)是源自印刷出版领域的信息技术。它既是一种标准通用置标语言 ,又是一种文本描述语言。它具有建设和维护数据库、交换和流通数据及制作电子图书等功能。 相似文献
14.
Using Statistical Term Similarity for Sense Disambiguation in Cross-Language Information Retrieval 总被引：2，自引：0，他引：2
Mirna Adriani 《Information Retrieval》2000,2(1):71-82
With the increasing availability of machine-readable bilingual dictionaries, dictionary-based automatic query translation has become a viable approach to Cross-Language Information Retrieval (CLIR). In this approach, resolving term ambiguity is a crucial step. We propose a sense disambiguation technique based on a term-similarity measure for selecting the right translation sense of a query term. In addition, we apply a query expansion technique which is also based on the term similarity measure to improve the effectiveness of the translation queries. The results of our Indonesian to English and English to Indonesian CLIR experiments demonstrate the effectiveness of the sense disambiguation technique. As for the query expansion technique, it is shown to be effective as long as the term ambiguity in the queries has been resolved. In the effort to solve the term ambiguity problem, we discovered that differences in the pattern of word-formation between the two languages render query translations from one language to the other difficult. 相似文献
15.
16.
Jacques Savoy 《Information Retrieval》2004,7(1-2):121-148
This paper describes and evaluates different retrieval strategies that are useful for search operations on document collections written in various European languages, namely French, Italian, Spanish and German. We also suggest and evaluate different query translation schemes based on freely available translation resources. In order to cross language barriers, we propose a combined query translation approach that has resulted in interesting retrieval effectiveness. Finally, we suggest a collection merging strategy based on logistic regression that tends to perform better than other merging approaches. 相似文献
17.
18.
跨语言信息检索理论与应用研究 总被引：5，自引：0，他引：5
随着互联网的全球化发展趋势，跨语言信息检索日益成为信息检索领域中的重要课题，跨语言检索可用一种提问语言检索出用另一种语言书写的信息。文章主要对跨语言信息检索理论应用研究进行了探讨，并对其在专业领域数据库中的应用提出一种思路。 相似文献
19.
The University of South Florida Libraries has supported a campus-wide “Textbook Affordability Project” initiative by creating and maintaining an ebooks database called “Ebooks for the Classroom+.” This database facilitates faculty members’ adoption of ebooks as textbooks through library acquisitions in the hope that students can achieve academic success without paying for expensive textbooks. It is built and run by the library, using data based on vendor-supplied information. This paper discusses the approaches, the challenges, and the future plans for building and updating the database. 相似文献
20.
Points of convergence: seamless long-term access to digital publications and archival records at library and archives Canada 总被引：1，自引：0，他引：1
In 2004, Canada’s national library and national archives merged to form Library and Archives Canada (LAC). LAC has become more than the sum of its parts, creating synergies between library and archives collections and services, realizing efficiencies and satisfying user demands for seamless access to all holdings. LAC has already created and launched Fed Search, an online search tool that provides clients with single-search access to library, archives and online collections. LAC is in the process of building a Trusted Digital Repository that will combine ingest, preservation, management and dissemination services for archives and library collections. 相似文献