共查询到20条相似文献，搜索用时 15 毫秒
1.
2.
The thesis of this study is to propose an extended methodology for laboratory based Information Retrieval evaluation under incomplete relevance assessments. This new methodology aims to identify potential uncertainty during system comparison that may result from incompleteness. The adoption of this methodology is advantageous, because the detection of epistemic uncertainty – the amount of knowledge (or ignorance) we have about the estimate of a system’s performance – during the evaluation process can guide and direct researchers when evaluating new systems over existing and future test collections. Across a series of experiments we demonstrate how this methodology can lead towards a finer grained analysis of systems. In particular, we show through experimentation how the current practice in Information Retrieval evaluation of using a measurement depth larger than the pooling depth increases uncertainty during system comparison. 相似文献
3.
Jingjing Liu Nicholas J. Belkin Xiangmin Zhang Xiaojun Yuan 《Information processing & management》2013
This paper examines the changes of information searchers’ topic knowledge levels in the process of completing information tasks. Multi-session tasks were used in the study, which enables the convenience of eliciting users’ topic knowledge during their process of completing the whole tasks. The study was a 3-session laboratory experiment with 24 participants, each time working on one subtask in an assigned 3-session general task. The general task was either parallel or dependently structured. Questionnaires were administered before and after each session to elicit users’ perceptions of their knowledge levels, task attributes, and other task features, for both the overall task and the sub-tasks. Our results support the assumption that users’ knowledge generally increases after each search session, but there were exceptions in which a “ceiling” effect was shown. We also found that knowledge was correlated with users’ perceptions of task attributes and accomplishment. In addition, task type was found to affect several aspects of knowledge levels and knowledge change. These findings further our understanding of users’ knowledge in information tasks and are thus helpful for information retrieval research and system design. 相似文献
4.
Understanding users’ navigation on the Web is important towards improving the quality of information and the speed of accessing large-scale Web data sources. Clustering of users’ navigation into sessions has been proposed in order to identify patterns and similarities which are then managed in the context of Web users oriented applications (searching, e-commerce, etc.). This paper deals with the problem of assessing the quality of user session clusters in order to make inferences regarding the users’ navigation behavior. A common model-based clustering algorithm is used to result in clusters of Web users’ sessions. These clusters are validated by using a statistical test, which measures the distances of the clusters’ distributions to infer their dissimilarity and distinguishing level. Furthermore, a visualization method is proposed in order to interpret the relation between clusters. Using real data sets, we illustrate how the proposed analysis can be applied in popular application scenarios to reveal valuable associations among Web users’ navigation sessions. 相似文献
5.
This article analyzes the impact of a computer simulation (business game) on the users’ perceived learning. The theoretical model developed in this paper is derived from collaborative learning and human–computer interaction assumptions. The hypotheses relating to group dynamics and the user–computer interface design with the users’ perceived learning are tested using the business game “FirmReality” on a sample of 173 respondents. Multiple regression and qualitative results show that, in a computer-based simulation context, perceived learning is influenced more by human–computer interaction factors than by group dynamics. Furthermore, results point out a negative effect of interaction with the instructor on individual perceived learning. These results support the possibility to automate the interaction between teacher and students. 相似文献
6.
The presentation of search results on the web has been dominated by the textual form of document representation. On the other hand, the document’s visual aspects such as the layout, colour scheme, or presence of images have been studied in a limited context with regard to their effectiveness of search result presentation. This article presents a comparative evaluation of textual and visual forms of document representation as additional components of document surrogates. A total of 24 people were recruited for our task-based user study. The experimental results suggest that an increased level of document representation available in the search results can facilitate users’ interaction with a search interface. The results also suggest that the two forms of additional representations are likely beneficial to users’ information searching process in different contexts. 相似文献
7.
8.
This study examines the creation of innovation-related knowledge in virtual communities visited mainly by lead users. Such communities enable firms to access a large number of lead users in a cost-efficient way. A propositional framework relates lead users’ characteristics to unique virtual community features to examine their potential impact on the development of valuable innovation knowledge. The authors empirically validate this framework by analyzing online contributions of lead users for mobile service innovation projects. The findings indicate that the value of their contributions stems from their ability to suggest solutions instead of simply describing problems or stating customer needs. Lead users’ technical expertise also makes them particularly well-suited to develop new functionalities, but less so for design and usability improvements. The digital context favors the creation of explicit knowledge that can be easily integrated into the development of new products. Finally, contributions given by lead users in a proactive way contain more novel insights than reactive contributions such as answers to community members’ questions. The findings should help managers stimulate, identify, and improve the use of lead users’ input in virtual communities. 相似文献
9.
The aim of this paper was to analyze users’ behavior during image retrieval exercises. Results revealed that users tend to follow a set search strategy: firstly they input one or two keyword search terms one after another and view the images generated by their initial search and after they navigate their way around the web by using the ‘back to home’ or ‘previous page’ buttons. These results are consistent with existing Web research. Many of the actions recorded revealed that subjects behavior differed depending on if the task set was presented as a closed or open task. In contrast no differences were found for the time subjects took to perform a single action or their use of the AND operator. 相似文献
10.
Many user-centred studies of electronic information resources include a think-aloud element – where users are asked to verbalise their thoughts, interface actions and sometimes their feelings whilst using these resources to help them complete one or more information tasks. These studies are usually conducted with the purpose of identifying usability issues related to the resource(s) used or understanding aspects of users’ information behaviour. However, few of these studies present detailed accounts of how their think-aloud data was collected and analysed or provide detailed reflection on methodological decisions made. In this article, we discuss and reflect on the methodology used when planning and conducting a think-aloud study of lawyers’ interactive information behaviour. Our discussion is framed by Blandford et al.’s PRET A Rapporter (‘ready to report’) framework – a framework that can be used to plan, conduct and describe user-centred studies of electronic information resource use from an information work perspective. 相似文献
11.
When consumers search for health information, a major obstacle is their unfamiliarity with the medical terminology. Even though medical thesauri such as the Medical Subject Headings (MeSH) and related tools (e.g., the MeSH Browser) were created to help consumers find medical term definitions, the lack of direct and explicit integration of these help tools into a health retrieval system prevented them from effectively achieving their objectives. To explore this issue, we conducted an empirical study with two systems: One is a simple interface system supporting query-based searching; the other is an augmented system with two new components supporting MeSH term searching and MeSH tree browsing. A total of 45 subjects were recruited to participate in the study. The results indicated that the augmented system is more effective than the simple system in terms of improving user-perceived topic familiarity and question–answer performance, even though we did not find users spend more time on the augmented system. The two new MeSH help components played a critical role in participants’ health information retrieval and were found to allow them to develop new search strategies. The findings of the study enhanced our understanding of consumers’ search behaviors and shed light on the design of future health information retrieval systems. 相似文献
12.
The study investigated how users’ emotion control and search tasks interact and influence the Web search behavior and performance among experienced Web users. Sixty-seven undergraduate students with substantial Web experience participated in the study. Effects of emotion control and tasks were found significant on the search behavior but not on the search performance. The interaction effect between emotion control and tasks on the search behavior was also significant: effects of users’ emotion control on the search behavior varied depending on search tasks. Profile analyses of search behaviors identified and contrasted the most commonly occurring profiles of search activities in different search tasks. Suggestions were made to improve information literacy programs, and implications for future research were discussed. 相似文献
13.
Mauro Rojas Herrera Edleno Silva de Moura Marco Cristo Thomaz Philippe Silva Altigran Soares da Silva 《Information processing & management》2010
Queries submitted to search engines can be classified according to the user goals into three distinct categories: navigational, informational, and transactional. Such classification may be useful, for instance, as additional information for advertisement selection algorithms and for search engine ranking functions, among other possible applications. This paper presents a study about the impact of using several features extracted from the document collection and query logs on the task of automatically identifying the users’ goals behind their queries. We propose the use of new features not previously reported in literature and study their impact on the quality of the query classification task. Further, we study the impact of each feature on different web collections, showing that the choice of the best set of features may change according to the target collection. 相似文献
14.
Information-seeking is important for lawyers, who have access to many dedicated electronic resources. However there is considerable scope for improving the design of these resources to better support information-seeking. One way of informing design is to use information-seeking models as theoretical lenses to analyse users’ behaviour with existing systems. However many models, including those informed by studying lawyers, analyse information-seeking at a high level of abstraction and are only likely to lead to broad-scoped design insights. We illustrate that one potentially useful (and lower-level) model is Ellis’s – by using it as a lens to analyse and make design suggestions based on the information-seeking behaviour of 27 academic lawyers, who were asked to think aloud whilst using electronic legal resources to find information for their work. We identify similar information-seeking behaviours to those originally found by Ellis and his colleagues in scientific domains, along with several that were not identified in previous studies such as ‘updating’ (which we believe is particularly pertinent to legal information-seeking). We also present a refinement of Ellis’s model based on the identification of several levels that the behaviours were found to operate at and the identification of sets of mutually exclusive subtypes of behaviours. 相似文献
15.
This study investigated whether and how different factors in relation to task, user-perceived knowledge, search process, and system affect users’ search tactic selection. Thirty-one participants, representing the general public with their own tasks, were recruited for this study. Multiple methods were employed to collect data, including pre-questionnaire, verbal protocols, log analysis, diaries, and post-questionnaires. Statistical analysis revealed that seven factors were significantly associated with tactic selection. These factors consist of work task types, search task types, familiarity with topic, search skills, search session length, search phases, and system types. Moreover, the study also discovered, qualitatively, in what ways these factors influence the selection of search tactics. Based on the findings, the authors discuss practical implications for system design to support users’ application of multiple search tactics for each factor. 相似文献
16.
Michael J. Cole Jacek Gwizdka Chang Liu Nicholas J. Belkin Xiangmin Zhang 《Information processing & management》2013
The acquisition of information and the search interaction process is influenced strongly by a person’s use of their knowledge of the domain and the task. In this paper we show that a user’s level of domain knowledge can be inferred from their interactive search behaviors without considering the content of queries or documents. A technique is presented to model a user’s information acquisition process during search using only measurements of eye movement patterns. In a user study (n = 40) of search in the domain of genomics, a representation of the participant’s domain knowledge was constructed using self-ratings of knowledge of genomics-related terms (n = 409). Cognitive effort features associated with reading eye movement patterns were calculated for each reading instance during the search tasks. The results show correlations between the cognitive effort due to reading and an individual’s level of domain knowledge. We construct exploratory regression models that suggest it is possible to build models that can make predictions of the user’s level of knowledge based on real-time measurements of eye movement patterns during a task session. 相似文献
17.
Ann Blandford Anne Adams Simon Attfield George Buchanan Jeremy Gow Stephann Makri Jon Rimmer Claire Warwick 《Information processing & management》2008
The strongest tradition of IR systems evaluation has focused on system effectiveness; more recently, there has been a growing interest in evaluation of Interactive IR systems, balancing system and user-oriented evaluation criteria. In this paper we shift the focus to considering how IR systems, and particularly digital libraries, can be evaluated to assess (and improve) their fit with users’ broader work activities. Taking this focus, we answer a different set of evaluation questions that reveal more about the design of interfaces, user–system interactions and how systems may be deployed in the information working context. The planning and conduct of such evaluation studies share some features with the established methods for conducting IR evaluation studies, but come with a shift in emphasis; for example, a greater range of ethical considerations may be pertinent. We present the PRET A Rapporter framework for structuring user-centred evaluation studies and illustrate its application to three evaluation studies of digital library systems. 相似文献
18.
To resolve some of lexical disagreement problems between queries and FAQs, we propose a reliable FAQ retrieval system using query log clustering. On indexing time, the proposed system clusters the logs of users’ queries into predefined FAQ categories. To increase the precision and the recall rate of clustering, the proposed system adopts a new similarity measure using a machine readable dictionary. On searching time, the proposed system calculates the similarities between users’ queries and each cluster in order to smooth FAQs. By virtue of the cluster-based retrieval technique, the proposed system could partially bridge lexical chasms between queries and FAQs. In addition, the proposed system outperforms the traditional information retrieval systems in FAQ retrieval. 相似文献
19.
To obtain high performances, previous works on FAQ retrieval used high-level knowledge bases or handcrafted rules. However, it is a time and effort consuming job to construct these knowledge bases and rules whenever application domains are changed. To overcome this problem, we propose a high-performance FAQ retrieval system only using users’ query logs as knowledge sources. During indexing time, the proposed system efficiently clusters users’ query logs using classification techniques based on latent semantic analysis. During retrieval time, the proposed system smoothes FAQs using the query log clusters. In the experiment, the proposed system outperformed the conventional information retrieval systems in FAQ retrieval. Based on various experiments, we found that the proposed system could alleviate critical lexical disagreement problems in short document retrieval. In addition, we believe that the proposed system is more practical and reliable than the previous FAQ retrieval systems because it uses only data-driven methods without high-level knowledge sources. 相似文献
20.
The authors of this paper investigate terms of consumers’ diabetes based on a log from the Yahoo!Answers social question and answers (Q&A) forum, ascertain characteristics and relationships among terms related to diabetes from the consumers’ perspective, and reveal users’ diabetes information seeking patterns. In this study, the log analysis method, data coding method, and visualization multiple-dimensional scaling analysis method were used for analysis. The visual analyses were conducted at two levels: terms analysis within a category and category analysis among the categories in the schema. The findings show that the average number of words per question was 128.63, the average number of sentences per question was 8.23, the average number of words per response was 254.83, and the average number of sentences per response was 16.01. There were 12 categories (Cause & Pathophysiology, Sign & Symptom, Diagnosis & Test, Organ & Body Part, Complication & Related Disease, Medication, Treatment, Education & Info Resource, Affect, Social & Culture, Lifestyle, and Nutrient) in the diabetes related schema which emerged from the data coding analysis. The analyses at the two levels show that terms and categories were clustered and patterns were revealed. Future research directions are also included. 相似文献