| 一种基于类别分布信息的文本特征选择模型 |
| |
| 引用本文： | 刘海峰,于利军,刘守生.一种基于类别分布信息的文本特征选择模型[J].图书情报工作,2013,57(15):137-141. |
| |
| 作者姓名： | 刘海峰 于利军 刘守生 |
| |
| 作者单位： | 1. 解放军理工大学理学院;
2. 解放军理工大学气象海洋学院 |
| |
| 基金项目： | 本文系国家自然科学基金“直觉模糊聚类理论及其应用”(项目编号:71071161)和江苏省自然科学基金“模糊语言模型研究”(项目编号:BK2012511)研究成果之一。 |
| |
| 摘 要： | TF-IDF是一种常用的文本特征选择方法。基于该模型的特征选择思想,以特征项的类内分布、类间分布信息为依据,通过引入类内分布及类间分布权重因子对模型的TF及IDF部分进行加权,提出一种基于类别分布信息的文本特征选择模型。新模型使得TF部分含有类内文本频数信息,同时IDF部分含有特征项的类间频数信息。随后的文本分类试验表明,平均查全率、查准率分别提高6.4%、7.8%,F1值提高约7%,验证了本研究提出的基于类别分布的文本特征选择模型的有效性。
|
| 关 键 词： | 文本分类 特征选择 TF-IDF 类内分布 类间分布 |
| 收稿时间： | 2013-06-07 |
An Improved TF-IDF Method of Text Feature Selection Based on Category and Frequency |
| |
| Liu Haifeng,Yu Lijun,Liu Shousheng.An Improved TF-IDF Method of Text Feature Selection Based on Category and Frequency[J].Library and Information Service,2013,57(15):137-141. |
| |
| Authors: | Liu Haifeng Yu Lijun Liu Shousheng |
| |
| Institution: | 1. Institute of sciences, PLA university of science and technology, Nanjing 210007;
2. College of meteorology and oceanography, PLA university of science and technology, Nanjing 210022 |
| |
| Abstract: | TF-IDF is a commonly used text feature selection method. Based on the characteristics of the model selection ideas and using the feature within class distribution and the distribution between class information as the foundations, we propose a model of text feature selection based on the category distribution information through the introduction of weighting factor distribution within classes and between classes. The new model makes the TF part contains the within class text frequency information. At the same time, the IDF part contains the between class frequency information. The subsequent text classification experiments proved that the average recall rate, precision rate increased 6.4%, 7.8% respectively. At the same time, the F1 value increased about 7%. We demonstrate the effectiveness of the text feature selection model proposed in this paper. |
| |
| Keywords: | textcategorization feature selection TF-IDF distribution within class distribution between class |
|
| 点击此处可从《图书情报工作》浏览原始摘要信息 |
| 点击此处可从《图书情报工作》下载免费的PDF全文 |
