Automating the Construction of Internet Portals with Machine Learning |
| |
| Authors: | Andrew Kachites McCallum Kamal Nigam Jason Rennie Kristie Seymore |
| |
| Institution: | (1) Just Research and Carnegie Mellon University, The Netherlands;(2) Carnegie Mellon University, The Netherlands;(3) Massachusetts Institute of Technology, USA;(4) Carnegie Mellon University, The Netherlands |
| |
| Abstract: | Domain-specific internet portals are growing in popularity because they gather content from the Web and organize it for easy access, retrieval and search. For example, www.campsearch.com allows complex queries by age, location, cost and specialty over summer camps. This functionality is not possible with general, Web-wide search engines. Unfortunately these portals are difficult and time-consuming to maintain. This paper advocates the use of machine learning techniques to greatly automate the creation and maintenance of domain-specific Internet portals. We describe new research in reinforcement learning, information extraction and text classification that enables efficient spidering, the identification of informative text segments, and the population of topic hierarchies. Using these techniques, we have built a demonstration system: a portal for computer science research papers. It already contains over 50,000 papers and is publicly available at www.cora.justresearch.com. These techniques are widely applicable to portal creation in other domains. |
| |
| Keywords: | spidering crawling reinforcement learning information extraction hidden Markov models text classification naive Bayes expectation-maximization unlabeled data |
| 本文献已被 SpringerLink 等数据库收录！ |
|
