共查询到17条相似文献，搜索用时 484 毫秒
1.
传统的文本信息处理方法无法表征文本内特征,所以不适用于模糊特征的提取分类提出一种高特征参差性下强收敛性文本的信息处理技术,对文本间和文本内的特征同时进行提取,采用迭代控制的TFIDF对特征进行加权值的计算,最后采用22类文本进行性能测试,结果显示,迭代控制的TFIDF算法能够更加细致地对文本进行分类,对特征进行提取,并且算法收敛速度快,稳定性好,具有很好的应用价值。 相似文献
2.
3.
提出一种基于改进TFIDF算法的海量文本分类识别方法,将特征之间的信息熵与特征内信息熵作为文本分类识别的加权因子,采用神经网络的非线性映射能力实现权值计算和TFIDF算法的模糊化,从而解决文本分类不准确和海量文本的分类问题。采用5个类别文档,每个类别5个文档,3个特征项来进行实际试验验证,结果表明,改进的TFIDF算法能够更好的实现文本识别分类,具有更小的方差特性,对随机文本分布具有更强的鲁棒性,收敛速度更快,具有很好的应用价值。 相似文献
4.
文本自动分类是文本信息处理中的一项基础性工作。将范例推理应用于文本分类中,并利用词语间的词共现信息从文本中抽取主题词和频繁词共现项目集,以及借助聚类算法对范例库进行索引,实现了基于范例推理的文本自动分类系统。实验表明,与基于TFIDF的文本表示方法和最近邻分类算法相比,基于词共现信息的文本表示方法和范例库的聚类索引能有效地改善分类的准确性和效率,从而拓宽了范例推理的应用领域。 相似文献
5.
基于TFIDF和词语关联度的中文关键词提取方法 总被引：1，自引：0，他引：1
关键词提取技术是文本分类、文本聚类、信息检索等技术的基础,在自然语言处理领域有着非常广泛的应用。结合TFIDF关键词抽取方法的特点和中文具有的自然语言词语间相互关联的特性,提出一种基于TFIDF和词语关联度的中文关键词提取方法。该方法通过引入词语关联度,有效避免了单纯采用TFIDF算法产生的偏差。实验结果表明,该方法的平均召回率与传统方法相比得到明显提升。 相似文献
6.
一种基于TFIDF方法的中文关键词抽取算法 总被引：4，自引：1，他引：3
本文在海量智能分词基础之上,提出了一种基于向量空间模型和TFIDF方法的中文关键词抽取算法.该算法在对文本进行自动分词后,用TFIDF方法对文献空间中的每个词进行权重计算,然后根据计算结果抽取出科技文献的关键词.通过自编软件进行的实验测试表明该算法对中文科技文献的关键词自动抽取成效显著. 相似文献
7.
为解决社区问答系统中的问题短文本特征词少、描述信息弱的问题,本文利用维基百科进行特征扩展以辅助中文问题短文本分类。首先通过维基百科概念及链接等信息进行词语相关概念集合抽取,并综合利用链接结构和类别体系信息进行概念间相关度计算。然后以相关概念集合为基础进行特征扩展以补充文本特征语义信息。实验结果表明,本文提出的基于特征扩展的短文本分类算法能有效提高问题短文本分类效果。 相似文献
8.
9.
针对信息安全检测中的检测精度低的问题,提出了基于稀疏距离入侵特征表达的信息安全检测算法。在该算法中,引入稀疏表达对完备词典进行编码,使得学习的稀疏系数可以具有较好的重构特征;其次利用K-SVD算法和支持向量机进行样本分类训练,使得稀疏特征为样本输入;最后利用粒子群算法对多维测试数据进行粒子映射,在满足适应度函数的条件下进行分类迭代寻优。实验表明,该算法具有较好的检测率。 相似文献
10.
神经网络算法是一种非常经典的分类算法,然而神经网络的一个不足之处就是容易陷入过拟合。针对这种不足,正则化神经网路算法与提前终止迭代算法被提了出来。为了进一步研究这两种算法性能的差异,本文通过20个UCI标准数据集上对着这两种方法进行了性能测试。实验显示在分类准确率上正则化神经网路算法要更优秀一些,但是在分类速度上提前终止迭代算法更占优势。 相似文献
11.
用户兴趣本体弥补了基于关键词的用户兴趣模型不能从语义上表达用户兴趣的缺陷,但大多是利用领域本体来构建,很难反映用户多方面和潜在兴趣,并且构建领域本体也是一个难点。本文据此提出一种基于词汇同现的用户兴趣本体构建方法。根据网页浏览记录找到用户兴趣网页集,经过数据处理将其转换成用户兴趣文本集。以TFIDF为指标抽取概念,词汇同现统计提取概念间关系,运用无尺度K-中心点聚类算法对其调整,将有关联用户的本体合并得到多用户本体,该本体能在语义上更全面反映用户兴趣并发现潜在兴趣。 相似文献
12.
In this paper, based on the Smith iteration (Smith, 1968), an inner-outer (IO) iteration algorithm for solving the coupled Lyapunov matrix equations (CLMEs) is presented. First, the IO iteration algorithm for solving the Sylvester matrix equation is proposed, and its convergence is analyzed in detail. Second, the IO iteration algorithm for solving the CLMEs is constructed. By utilizing the latest estimation, a current-estimation-based and two weighted IO iteration algorithms are also given for solving the CLMEs, respectively. Convergence analyses indicate that the iteration solutions generated by these algorithms always converge to the unique solutions to the CLMEs for any initial conditions. Finally, Several numerical examples are provided to show the superiority of the proposed numerical algorithms. 相似文献
13.
自然语言理解心理学在短文本分类中的实证研究 总被引：1，自引：0，他引：1
目前对文本分类研究多数集中在对大规模语料基础上的特征选择或分类器算法的研究。本文是建立在训练样本少且样本长度短的基础上,根据人脑对自然语言理解的心理学原理"人们总是根据已知的最熟悉的、最典型的例子进行判断,只有在该方法不奏效的时候才使用频率这一概念,并且使用的是十分简单的频率"从该角度进行短文本分类的实证研究。以心理学中的"熟悉原理"、"典型原理"等为模型建立特殊词库和典型案例词库,改进了传统文本分类的实验步骤,同时提出了该方法的优势和局限性。 相似文献
14.
《Information processing & management》2020,57(4):102238
In recent years, mainly the functionality of services are described in a short natural text language. Keyword-based searching for web service discovery is not efficient for providing relevant results. When services are clustered according to the similarity, then it reduces search space and due to that search time is also reduced in the web service discovery process. So in the domain of web service clustering, basically topic modeling techniques like Latent Dirichlet Allocation (LDA), Correlated Topic Model (CTM), Hierarchical Dirichlet Processing (HDP), etc. are adopted for dimensionality reduction and feature representation of services in vector space. But as the services are described in the form of short text, so these techniques are not efficient due to lack of occurring words, limited content, etc. In this paper, the performance of web service clustering is evaluated by applying various topic modeling techniques with different clustering algorithms on the crawled dataset from ProgrammableWeb repository. Gibbs Sampling algorithm for Dirichlet Multinomial Mixture (GSDMM) model is proposed as a dimensionality reduction and feature representation of services to overcome the limitations of short text clustering. Results show that GSDMM with K-Means or Agglomerative clustering is outperforming all other methods. The performance of clustering is evaluated based on three extrinsic and two intrinsic evaluation criteria. Dimensionality reduction achieved by GSDMM is 90.88%, 88.84%, and 93.13% on three real-time crawled datasets, which is satisfactory as the performance of clustering is also enhanced by deploying this technique. 相似文献
15.
在文本自动分类中,目前有词频和文档频率统计这两种概率估算方法,采用的估算方法恰当与否会直接影响特征抽取的质量与分类的准确度。本文采用K最近邻算法实现中文文本分类器,在中文平衡与非平衡两种训练语料下进行了训练与分类实验,实验数据表明使用非平衡语料语料时,可以采用基于词频的概率估算方法,使用平衡语料语料时,采用基于文档频率的概率估算方法,能够有效地提取高质量的文本特征,从而提高分类的准确度。 相似文献
16.
在支持向量机和遗传算法的基础上,提出一种新的启发式多层文本分类算法。实验结果证明了该算法的可行性和有效性。文本分类技术是解决大规模文本处理的有效途径。 相似文献
17.
《Information processing & management》2019,56(6):102060
In this era, the proliferating role of social media in our lives has popularized the posting of the short text. The short texts contain limited context with unique characteristics which makes them difficult to handle. Every day billions of short texts are produced in the form of tags, keywords, tweets, phone messages, messenger conversations social network posts, etc. The analysis of these short texts is imperative in the field of text mining and content analysis. The extraction of precise topics from large-scale short text documents is a critical and challenging task. The conventional approaches fail to obtain word co-occurrence patterns in topics due to the sparsity problem in short texts, such as text over the web, social media like Twitter, and news headlines. Therefore, in this paper, the sparsity problem is ameliorated by presenting a novel fuzzy topic modeling (FTM) approach for short text through fuzzy perspective. In this research, the local and global term frequencies are computed through a bag-of-words (BOW) model. To remove the negative impact of high dimensionality on the global term weighting, the principal component analysis is adopted; thereafter the fuzzy c-means algorithm is employed to retrieve the semantically relevant topics from the documents. The experiments are conducted over the three real-world short text datasets: the snippets dataset is in the category of small dataset whereas the other two datasets, Twitter and questions, are the bigger datasets. Experimental results show that the proposed approach discovered the topics more precisely and performed better as compared to other state-of-the-art baseline topic models such as GLTM, CSTM, LTM, LDA, Mix-gram, BTM, SATM, and DREx+LDA. The performance of FTM is also demonstrated in classification, clustering, topic coherence and execution time. FTM classification accuracy is 0.95, 0.94, 0.91, 0.89 and 0.87 on snippets dataset with 50, 75, 100, 125 and 200 number of topics. The classification accuracy of FTM on questions dataset is 0.73, 0.74, 0.70, 0.68 and 0.78 with 50, 75, 100, 125 and 200 number of topics. The classification accuracies of FTM on snippets and questions datasets are higher than state-of-the-art baseline topic models. 相似文献