Extending WHIRL with background knowledge for improved text classification     

Sarah Zelikovitz  William W. Cohen  Haym Hirsh 《Information Retrieval》2007,10(1):35-67

Intelligent use of the many diverse forms of data available on the Internet requires new tools for managing and manipulating heterogeneous forms of information. This paper uses WHIRL, an extension of relational databases that can manipulate textual data using statistical similarity measures developed by the information retrieval community. We show that although WHIRL is designed for more general similarity-based reasoning tasks, it is competitive with mature systems designed explicitly for inductive classification. In particular, WHIRL is well suited for combining different sources of knowledge in the classification process. We show on a diverse set of tasks that the use of appropriate sets of unlabeled background knowledge often decreases error rates, particularly if the number of examples or the size of the strings in the training set is small. This is especially useful when labeling text is a labor-intensive job and when there is a large amount of information available about a particular problem on the World Wide Web.

Haym HirshEmail:

  相似文献   

马克思主义哲学视角下的语篇语言学     

刘淼 《牡丹江教育学院学报》2011,(1):30-31

语篇语言学又称话语语言学,兴起于20世纪70年代,是现代语言学的重要分支之一,它的诞生与发展,使语言学的研究脱离了形式一逻辑的固有模式,从语言的本质出发去研究语言的基本规律。语言学的这个进步,反映了语言学家们从哲学层面对语言的重新审视,不再把语言看作是一成不变的形式化的内容,而是从语言作为人类的交际手段这一基本属性的角度来研究语言本身。从马克思主义哲学的视角来审视语篇语言学的诞生、研究对象和研究方法,无疑能够更好地探究语篇语言学的哲学基础。  相似文献   

Social media opinion summarization using emotion cognition and convolutional neural networks     

《International Journal of Information Management》2020

Quickly and accurately summarizing representative opinions is a key step for assessing microblog sentiments. The Ortony-Clore-Collins (OCC) model of emotion can offer a rule-based emotion export mechanism. In this paper, we propose an OCC model and a Convolutional Neural Network (CNN) based opinion summarization method for Chinese microblogging systems. We test the proposed method using real world microblog data. We then compare the accuracy of manual sentiment annotation to the accuracy using our OCC-based sentiment classification rule library. Experimental results from analyzing three real-world microblog datasets demonstrate the efficacy of our proposed method. Our study highlights the potential of combining emotion cognition with deep learning in sentiment analysis of social media data.  相似文献   

Authorship attribution based on a probabilistic topic model     

Jacques Savoy 《Information processing & management》2013

This paper describes, evaluates and compares the use of Latent Dirichlet allocation (LDA) as an approach to authorship attribution. Based on this generative probabilistic topic model, we can model each document as a mixture of topic distributions with each topic specifying a distribution over words. Based on author profiles (aggregation of all texts written by the same writer) we suggest computing the distance with a disputed text to determine its possible writer. This distance is based on the difference between the two topic distributions. To evaluate different attribution schemes, we carried out an experiment based on 5408 newspaper articles (Glasgow Herald) written by 20 distinct authors. To complement this experiment, we used 4326 articles extracted from the Italian newspaper La Stampa and written by 20 journalists. This research demonstrates that the LDA-based classification scheme tends to outperform the Delta rule, and the χ² distance, two classical approaches in authorship attribution based on a restricted number of terms. Compared to the Kullback–Leibler divergence, the LDA-based scheme can provide better effectiveness when considering a larger number of terms.  相似文献   

基于粗集的支持向量机文本分类方法研究   总被引：1，自引：0，他引：1  

崔彩霞  王素格 《科技广场》2006,(8):4-6

本文提出了一种基于粗糙集的支持向量机文本分类方法。该方法利用粗糙集的约简理论降低了支持向量的维数,同时保证了分类性能。实验表明该方法能获得较好的分类效果。  相似文献   

一种基于特征编码的Word文档数字水印算法     

方旺盛  舒明磊 《科技广场》2006,(1):4-6

本文基于文本数字水印的特征编码思想,通过设置Word文档中字符的下划线格式以嵌入和检测水印。实验表明,该算法有较强的不可见性,能较好的嵌入和检测水印,从而达到数字水印的一般要求。  相似文献   

A pipelined architecture for distributed text query evaluation   总被引：1，自引：0，他引：1  

Alistair Moffat  William Webber  Justin Zobel  Ricardo Baeza-Yates 《Information Retrieval》2007,10(3):205-231

Two principal query-evaluation methodologies have been described for cluster-based implementation of distributed information retrieval systems: document partitioning and term partitioning. In a document-partitioned system, each of the processors hosts a subset of the documents in the collection, and executes every query against its local sub-collection. In a term-partitioned system, each of the processors hosts a subset of the inverted lists that make up the index of the collection, and serves them to a central machine as they are required for query evaluation. In this paper we introduce a pipelined query-evaluation methodology, based on a term-partitioned index, in which partially evaluated queries are passed amongst the set of processors that host the query terms. This arrangement retains the disk read benefits of term partitioning, but more effectively shares the computational load. We compare the three methodologies experimentally, and show that term distribution is inefficient and scales poorly. The new pipelined approach offers efficient memory utilization and efficient use of disk accesses, but suffers from problems with load balancing between nodes. Until these problems are resolved, document partitioning remains the preferred method. Alistair Moffat was supported by the Australian Research Council, the ARC Special Research Center for Perceptive and Intelligent Machines in Complex Environments, and the NICTA Victoria Laboratory. William Webber and Justin Zobel were supported by the Australian Research Council. Ricardo Baeza-Yates was supported by Grant P01-029-F from Millennium Initiative of Mideplan, Chile; and by the University of Melbourne as a visiting scholar at the time this project was undertaken.  相似文献   

文本分类C#实现*     

刘华 《现代图书情报技术》2007,2(3):43-45

设计并实现一个基于向量空间模型和简单贝叶斯的文本分类系统，系统采用层级多标签的分类策略。详细介绍词语切分统计、终分类器值计算、层级小类校正和兼类判断四个子系统模块。基于向量空间模型分类的第一级大类和层级小类的微平均分别为89.7%和77.8%，简单贝叶斯分别为67.6%和66.5%。  相似文献   

国史知识的语义揭示与组织方法研究     

王颖  张智雄  孙辉  雷枫 《中国图书馆学报》2015,41(4):55-64

深入开展中华人民共和国国史的教育和研究一直是各方高度重视的工作。将国史知识进行语义揭示和组织,对于国史的教育和研究具有重要意义。在参考相关研究成果的基础上,本文提出了“向下挖掘,向上组织”的国史知识语义揭示与组织方法。这一方法以国史本体为基础,在对隐藏于国史资源文本条目中的国史知识对象和相关事实进行语义挖掘和揭示的基础之上,通过国史知识对象的关联,构建国史知识网络,并基于时间、类属、层级及统计等关系,对国史知识内容进行更高层次的多维组织展示,并基于这一思路开发了相应的系统平台,实现了国史知识的揭示、重组和其他创新应用。本文所提出的国史知识语义揭示和组织方法对其他类型知识的开发利用具有参考价值。图7。参考文献12。  相似文献   

Automatic thematic classification of election manifestos     

Suzan Verberne  Eva D’hondt  Antal van den Bosch  Maarten Marx 《Information processing & management》2014

We digitized three years of Dutch election manifestos annotated by the Dutch political scientist Isaac Lipschits. We used these data to train a classifier that can automatically label new, unseen election manifestos with themes. Having the manifestos in a uniform XML format with all paragraphs annotated with their themes has advantages for both electronic publishing of the data and diachronic comparative data analysis. The data that we created will be disclosed to the public through a search interface. This means that it will be possible to query the data and filter them on themes and parties. We optimized the Lipschits classifier on the task of classifying election manifestos using models trained on earlier years. We built a classifier that is suited for classifying election manifestos from 2002 onwards using the data from the 1980s and 1990s. We evaluated the results by having a domain expert manually assess a sample of the classified data. We found that our automatic classifier obtains the same precision as a human classifier on unseen data. Its recall could be improved by extending the set of themes with newly emerged themes. Thus when using old political texts to classify new texts, work is needed to link and expand the set of themes to newer topics.  相似文献