首页 | 本学科首页   官方微博 | 高级检索  
相似文献
 共查询到20条相似文献，搜索用时 31 毫秒
1.
Pre-adoption expectations often serve as an implicit reference point in users’ evaluation of information systems and are closely associated with their goals of interactions, behaviors, and overall satisfaction. Despite the empirically confirmed impacts, users’ search expectations and their connections to tasks, users, search experiences, and behaviors have been scarcely studied in the context of online information search. To address the gap, we collected 116 sessions from 60 participants in a controlled-lab Web search study and gathered direct feedback on their in-situ expected information gains (e.g., number of useful pages) and expected search efforts (e.g., clicks and dwell time) under each query during search sessions. Our study aims to examine (1) how users’ pre-search experience, task characteristics, and in-session experience affect their current expectations and (2) how user expectations are correlated with search behaviors and satisfaction. Our results with both quantitative and qualitative evidence demonstrate that: (1) user expectation is significantly affected by task characteristics, previous and in-situ search experience; (2) user expectation is closely associated with users’ browsing behaviors and search satisfaction. The knowledge learned about user expectation advances our understanding of users’ search behavioral patterns and their evaluations of interaction experience and will also facilitate the design, implementation, and evaluation of expectation-aware user models, metrics, and information retrieval (IR) systems.  相似文献   

2.
Re-using research resources is essential for advancing knowledge and developing repeatable, empirically solid experiments in scientific fields, including interactive information retrieval (IIR). Despite recent efforts on standardizing research re-use and documentation, how to quantitatively measure the reusability of IIR resources still remains an open challenge. Inspired by the reusability evaluations on Cranfield experiments, our work proactively explores the problem of measuring IIR test collection reusability and makes threefold contributions: (1) constructing a novel usefulness-oriented framework with specific analytical methods for evaluating the reusability of IIR test collections consisting of query sets, document/page sets, and sets of task-document usefulness (tuse); (2) explaining the potential impacts of varying IIR-specific factors (e.g. search tasks, sessions, user characteristics) on test collection reusability; (3) proposing actionable methods for building reusable test collections in IIR and thereby amortizing the true cost of user-oriented evaluations. The Cranfield-inspired reusability assessment framework serves as an initial step towards accurately evaluating the reusability of IIR research resources and measuring the reproducibility of IIR evaluation results. It also demonstrates an innovative approach to integrating the insights from individual heterogeneous user studies with the evaluation techniques developed in standardized ad hoc retrieval experiments, which will facilitate the maturation of IIR fields and eventually benefits both sides of research.  相似文献   

3.
This study aims to explore the relationships between user interaction and digital libraries (DLs) evaluation. User interaction is a multi-dimensional construct and recognized as three dimensions in this study, as user interaction with: information resource; interface; and, tasks. DL evaluation is considered from the user's perspective and defined as users’ perception of DL performance from different perspectives, including the support of DL's interaction design to user interaction (labeled as interaction-design-based (IDB) evaluation), the support of task completion (labeled as task-based evaluation), and a DL's overall performance (labeled as overall evaluation). An experiment with 48 participants was conducted using the China National Knowledge Infrastructure (CNKI (http://cnki.net/), the most widely used digital library in China). Participants searched for four simulated work tasks and one real work task during the experiment, subsequently evaluating their interaction with information resource, interface, and tasks, and DL performance from different perspectives before or after the search. Correlation analysis and stepwise regression analysis were conducted to examine the relationships. The results indicate that a list of factors related to different dimensions of user interaction can significantly predict or be correlated to users’ evaluation of DL performance from different perspectives, including appropriateness, rich and valid links, reasonable page layout, salience of topics, search task difficulty, well-organized web site, easy to learn, accessibility, usefulness, familiarity with task procedure, etc. These factors surface as the most critical criteria for DL evaluation. Based on the results, an integrated DL evaluation framework is developed. The study adds new knowledge about how tasks affect DL evaluation. It has implications for improving the efficiency of DL evaluation and helping DL developers design DLs to better support users’ interaction, task completion, and their overall experience with DLs.  相似文献   

4.
The user experience is an integral component of interactive information retrieval (IIR). However, there is a twofold problem in its measurement. Firstly, while many IIR studies have relied on a single dimension of user feedback, that of satisfaction, experience is a much more complex concept. IIR in general, and exploratory search more specifically, are dynamic, multifaceted experiences that evoke pragmatic and hedonic needs, expectations, and outcomes that are not adequately captured by user satisfaction. Secondly, questionnaires, which are typically the means in which user’s attitudes and perceptions are measured, are not typically subjected to rigorous reliability and validity testing. To address these issues, we administered the multidimensional User Engagement Scale (UES) in an exploratory search environment to assess users’ perceptions of the Perceived Usability (PUs), Aesthetics (AE), Novelty (NO), Felt Involvement (FI), Focused Attention (FA), and Endurability (EN) aspects of the experience. In a typical laboratory-style study, 381 participants performed three relatively complex search tasks using a novel search interface, and responded to the UES immediately upon completion. We used Principal Axis Factor Analysis and Multiple Regression to examine the factor structure of UES items and the relationships amongst factors. Results showed that three of the six sub-scales (PUs, AE, FA) were stable, while NO, FI and EN merged to form a single factor. We discuss recommendations for revising and validating the UES in light of these findings.  相似文献   

5.
An individual's Web search behavior can be influenced by a number of factors, including features and functions of a search engine as well as search education. In contrast to the long-lasting attention to the algorithm and interface dimensions of search, there is a lack of research concerned with the potential effects of user education on search behavior. To address this gap, we ran a three-session field-lab-combined study to examine the effects of user education from two distinct sources – peer advice and cognitive authority (operationalized as video-based student's advice and expert's advice respectively) – on Web search behavior in two different search task scenarios (i.e., factual specific and factual amorphous tasks). We also tested if these behavioral effects persist for a short period of time when the explicit search tips are removed. Using 185 task session data generated by 31 participants in two field and one lab sessions, this study demonstrates that: (1) both peer advice and cognitive authority are effective in stimulating immediate behavioral changes in Web search; (2) the immediate behavioral impact of search advice is broader in factual amorphous task than in factual specific task; (3) framing search tips as the advice from cognitive authority is more likely to generate continuing, short-term effects on Web search behaviors. This research has implications for the design of task-aware user education as well as the study of users’ interactions with IR systems in general.  相似文献   

6.
Interactive query expansion (IQE) (c.f. [Efthimiadis, E. N. (1996). Query expansion. Annual Review of Information Systems and Technology, 31, 121–187]) is a potentially useful technique to help searchers formulate improved query statements, and ultimately retrieve better search results. However, IQE is seldom used in operational settings. Two possible explanations for this are that IQE is generally not integrated into searchers’ established information-seeking behaviors (e.g., examining lists of documents), and it may not be offered at a time in the search when it is needed most (i.e., during the initial query formulation). These challenges can be addressed by coupling IQE more closely with familiar search activities, rather than as a separate functionality that searchers must learn. In this article we introduce and evaluate a variant of IQE known as Real-Time Query Expansion (RTQE). As a searcher enters their query in a text box at the interface, RTQE provides a list of suggested additional query terms, in effect offering query expansion options while the query is formulated. To investigate how the technique is used – and when it may be useful – we conducted a user study comparing three search interfaces: a baseline interface with no query expansion support; an interface that provides expansion options during query entry, and a third interface that provides options after queries have been submitted to a search system. The results show that offering RTQE leads to better quality initial queries, more engagement in the search, and an increase in the uptake of query expansion. However, the results also imply that care must be taken when implementing RTQE interactively. Our findings have broad implications for how IQE should be offered, and form part of our research on the development of techniques to support the increased use of query expansion.  相似文献   

7.
People often search for information in order to learn something new. In recent years, the “search-as-learning” movement has argued that search systems should be better designed to support learning. Current search systems (especially Web search engines) are largely designed and optimized to fulfill simple look-up tasks (e.g., navigational or fact-finding search tasks). However, they provide less support for searchers working on complex tasks that involve learning. Search-as-learning studies have investigated a wide range of research questions. For example, studies have aimed to better understand how characteristics of the individual searcher, the type of search task, and interactive features provided by the system can influence learning outcomes. Learning assessment is a key component in search-as-learning studies. Assessment materials are used to both gauge prior knowledge and measure learning during or after one or more search sessions. In this paper, we provide a systematic review of different types of assessments used in search-as-learning studies to date. The paper makes the following three contributions. First, we review different types of assessments used and discuss their potential benefits and drawbacks. Second, we review assessments used outside of search-as-learning, which may provide insights and opportunities for future research. Third, we provide recommendations for future research. Importantly, we argue that future studies should clearly define learning objectives and develop assessment materials that reliably capture the intended type of learning. For example, assessment materials should test a participant’s ability to engage with specific cognitive processes, which may range from simple (e.g., memorization) to more complex (e.g., critical and creative thinking). Additionally, we argue that future studies should consider two dimensions that are understudied in search-as-learning: long-term retention (i.e., being able to use what was learned in the long term) and transfer of learning (i.e., being able to use what was learned in a novel context).  相似文献   

8.
In order to evaluate the effectiveness of Information Retrieval (IR) systems it is key to collect relevance judgments from human assessors. Crowdsourcing has successfully been used as a method to scale-up the collection of manual relevance judgments, and previous research has investigated the impact of different judgment task design elements (e.g., highlighting query keywords in the document) on judgment quality and efficiency. In this work we investigate the positive and negative impacts of presenting crowd human assessors with more than just the topic and the document to be judged. We deploy different variants of crowdsourced relevance judgment tasks following a between-subjects design in which we present different types of metadata to the human assessor. Specifically, we investigate the effect of human metadata (e.g., what other human assessors think of the current document, as in which relevance level has already been selected by the majority crowd workers), machine metadata (e.g., how IR systems scored this document such as its average position in ranked lists, statistics about the document such as term frequencies). We look at the impact of metadata on judgment quality (i.e., the level of agreement with trained assessors) and cost (i.e., the time it takes for workers to complete the judgments) as well as at how metadata quality positively or negatively impact the collected judgments.  相似文献   

9.
The results from a series of three experiments that used Text Retrieval Conference (TREC) data and TREC search topics are compared. These experiments each involved three novel user interfaces (one per experiment). User interfaces that made it easier for users to view text were found to improve recall in all three experiments. A distinction was found between a cluster of subjects (a majority of whom were search experts) who tended to read fewer documents more carefully (readers, or exclusives) and subjects who skimmed through more documents without reading them as carefully (skimmers, or inclusives). Skimmers were found to have significantly better recall overall. A major outcome from our experiments at TREC and with the TREC data, is that hypertext interfaces to information retrieval (IR) tasks tend to increase recall. Our interpretation of this pattern of results across the three experiments is that increased interaction with the text (more pages viewed) generally improves recall. Findings from one of the experiments indicated that viewing a greater diversity of text on a single screen (i.e., not just more text per se, but more articles available at once) may also improve recall. In an experiment where a traditional (type-in) query interface was contrasted with a condition where queries were marked up on the text, the improvement in recall due to viewing more text was more pronounced with search novices. Our results demonstrate that markup and hypertext interfaces to text retrieval systems can benefit recall and can also benefit novices. The challenge now will be to find modified versions of hypertext interfaces that can improve precision, as well as recall and that can work with users who prefer to use different types of search strategy or have different types of training and experience.  相似文献   

10.
In this paper, we present ViGOR (Video Grouping, Organisation and Recommendation), an exploratory video retrieval system. Exploratory video retrieval tasks are hampered by the lack of semantics associated to video and the overwhelming amount of video items stored in these types of collections (e.g. YouTube, MSN video, etc.). In order to help facilitate these exploratory video search tasks we present a system that utilises two complementary approaches: the first a new search paradigm that allows the semantic grouping of videos and the second the exploitation of past usage history in order to provide video recommendations. We present two types of recommendation techniques adapted to the grouping search paradigm: the first is a global recommendation, which couples the multi-faceted nature of explorative video retrieval tasks with the current user need of information in order to provide recommendations, and second is a local recommendation, which exploits the organisational features of ViGOR in order to provide more localised recommendations based on a specific aspect of the user task. Two user evaluations were carried out in order to (1) validate the new search paradigm provided by ViGOR, characterised by the grouping functionalities and (2) evaluate the usefulness of the proposed recommendation approaches when integrated into ViGOR. The results of our evaluations show (1) that the grouping, organisational and recommendation functionalities can result in an improvement in the users’ search performance without adversely impacting their perceptions of the system and (2) that both recommendation approaches are relevant to the users at different stages of their search, showing the importance of using multi-faceted recommendations for video retrieval systems and also illustrating the many uses of collaborative recommendations for exploratory video search tasks.  相似文献   

11.
Despite a number of studies looking at Web experience and Web searching tactics and behaviours, the specific relationships between experience and cognitive search strategies have not been widely researched. This study investigates how the cognitive search strategies of 80 participants might vary with Web experience as they engaged in two researcher-defined tasks and two participant-defined information seeking tasks. Each of the two researcher-defined tasks and participant-defined tasks included a directed search task and a general-purpose browsing task. While there were almost no significant performance differences between experience levels on any of the four tasks, there were significant differences in the use of cognitive search strategies. Participants with higher levels of Web experience were more likely to use “Parallel player”, “Parallel hub-and-spoke”, “Known address search domain” and “Known address” strategies, whereas participants with lower levels of Web experience were more likely to use “Virtual tourist”, “Link-dependent”, “To-the-point”, “Sequential player”, “Search engine narrowing”, and “Broad first” strategies. The patterns of use and differences between researcher-defined and participant-defined tasks and between directed search tasks and general-purpose browsing tasks are also discussed, although the distribution of search strategies by Web experience were not statistically significant for each individual task.  相似文献   

12.
Increasing people are using mobile applications (apps) for information and activities related to their travel and tourism. Therefore, customer engagement (customer attraction and retention) with mobile travel applications becomes extremely important for travel-related companies. However, there is little empirical research on what factors may drive customers’ engagement of using mobile travel apps. This study proposes a research framework, based on the Stimulus-Organism-Response (S-O-R) model, to explore how and to what extent two types of app attributes (i.e., app design and app performance attributes) stimulate travel apps engagement. Based on a user acceptance testing (UAT testing) of a real travel app development project, the study uses survey data from 804 tourists to validate the proposed research model. The results reveal different effectiveness of app design and app performance attributes, and demonstrate that two app design features (i.e., user interface attractiveness and privacy/security) and three app performance attributes (i.e., compatibility, ease of use, and relative advantages) are important drivers of users’ behavioral engagement of mobile travel apps. Further, psychological engagement and three types of benefit perception (i.e., hedonic benefit, utilitarian benefit, and social benefit) serially mediate the relationships. The results can be used to improve the related mobile app engagement research and the mobile travel app design.  相似文献   

13.
Search log analysis has become a common practice to gain insights into user search behaviour: it helps gain an understanding of user needs and preferences, as well as an insight into how well a system supports such needs. Currently, log analysis is typically focused on low-level user actions, i.e. logged events such as issued queries and clicked results, and often only a selection of such events are logged and analysed. However, types of logged events may differ widely from interface to interface, making comparison between systems difficult. Further, the interpretation of the meaning of and subsequent analysis of a selection of events may lead to conclusions out of context— e.g. the statistics of observed query reformulations may be influenced by the existence of a relevance feedback component. Alternatively, in lab studies user activities can be analysed at a higher level, such as search tactics and strategies, abstracted away from detailed interface implementation. Unfortunately, until now the required manual codings that map logged events to higher-level interpretations have prevented large-scale use of this type of analysis. In this paper, we propose a new method for analysing search logs by (semi-)automatically identifying user search tactics from logged events, allowing large-scale analysis that is comparable across search systems. In addition, as the resulting analysis is at a tactical level we reduce potential issues surrounding the need for interpretation of low-level user actions for log analysis. We validate the efficiency and effectiveness of the proposed tactic identification method using logs of two reference search systems of different natures: a product search system and a video search system. With the identified tactics, we perform a series of novel log analyses in terms of entropy rate of user search tactic sequences, demonstrating how this type of analysis allows comparisons of user search behaviours across systems of different nature and design. This analysis provides insights not achievable with traditional log analysis.  相似文献   

14.
With the increasing provenance of hedonic and social information systems, systems are observed to employ other forms of feedback and design than purely informational in order to increase user engagement and motivation. Three principle classes of motivational design pursuing user engagement have become increasingly established; gamification, quantified-self and social networking. This study investigates how the perceived prominence of these three design classes in users’ use of information system facilitate experiences of affective, informational and social feedback as well as user’s perceived benefits from a system and their continued use intentions. We employ survey data (N = 167) gathered from users of HeiaHeia; an exercise encouragement system that employs features belonging to the three design classes. The results indicate that gamification is positively associated with experiences of affective feedback, quantified-self with experiences of both affective and informational feedback and social networking with experiences of social feedback. Experiences of affective feedback are further strongly associated with user perceived benefits and continued use intentions, whereas experiences of informational feedback are only associated with continued use intentions. Experiences of social feedback had no significant relationship with neither. The findings provide practical insights into how systems can be designed to facilitate different types of feedback that increases users’ engagement, benefits and intentions to continue the use of a system.  相似文献   

15.
  相似文献   

16.
  相似文献   

17.
Recent research in the human computer interaction and information retrieval areas has revealed that search response latency exhibits a clear impact on the user behavior in web search. Such impact is reflected both in users’ subjective perception of the usability of a search engine and in their interaction with the search engine in terms of the number of search results they engage with. However, a similar impact analysis has been missing so far in the context of sponsored search. Since the predominant business model for commercial search engines is advertising via sponsored search results (i.e., search advertisements), understanding how response latency influences the user interaction with the advertisements displayed on the search engine result pages is crucial to increase the revenue of a commercial search engine. To this end, we conduct a large-scale analysis using query logs obtained from a commercial web search. We analyze the short-term and long-term impact of search response latency on the querying and clicking behaviors of users using desktop and mobile devices to access the search engine, as well as the corresponding impact on the revenue of the search engine. This analysis demonstrates the importance of serving sponsored search results with low latency and provides insight into the ad serving policy of commercial search engines to ensure long-term user engagement and search revenue.  相似文献   

18.
[目的/意义]文章旨在探究诸如任务等影响游客社会化搜寻的因素，为旅游机构利用社交服务网站开展营销活动，成功激发游客的出游行为提供决策参考。[方法/过程]本研究基于实验法，考察任务不确定性（确定性任务vs.不确定性任务）、感知可用信息缺失（高vs.低）和时间距离（长vs.短）对游客社会化搜寻动机的影响。[结果/结论]结果显示：1）相对于确定性任务，不确定性任务下，游客的社会化搜寻动机大；2）相对于感知可用信息缺失程度低，感知可用信息缺失程度高下游客的社会化搜寻动机大；3）相对于时间距离长，时间距离短下游客的社会化搜寻动机大；4）时间距离能调节任务不确定性与游客社会化搜寻动机之间的关系，感知时间距离短，任务不确定性对游客社会化搜寻动机无影响；感知时间距离长时，不确定任务下社会化搜寻动机强，确定性任务下社会化搜寻动机弱。  相似文献   

19.
The literature has not fully and adequately explained why contextual (e.g., BERT-based) representations are so successful to improve the effectiveness of some Natural Language Processing tasks, especially Automatic Text Classifications (ATC). In this article, we evince that such representations, when properly tuned to a target domain, produce an extremely separable space that makes the classification task very effective, independently of the classifier employed for solving the ATC task. To demonstrate our hypothesis, we perform a thorough class separability analysis in order to visualize and measure how well BERT-based embeddings separate documents of different classes in comparison with other widely used representation approaches, e.g., TFIDF BoW, static embeddings (e.g., fastText) and zero-shot (non-tuned) contextual embeddings. We also analyze separability in the context of transfer learning and compare BERT-based representations with those obtained from other transformers (e.g., RoBERTa, XLNET). Our experiments covering sixteen datasets in topic and sentiment classification, eight classification methods and three class separability metrics show that the fine-tuned BERT embeddings are highly separable in the corresponding space (e.g., they are 67% more separable than the static embeddings). As a consequence, they allow the simplest classifiers to achieve similar effectiveness as the most complex methods. We also find moderate to high correlations between separability and effectiveness in all experimented scenarios. Overall, our main finding is that more discriminative (i.e., separable) textual representations constitute a critical part of the ATC solutions that, given the current state-of-the-art in classification algorithms, are more prominent than the algorithmic (classifier) method for solving the task.  相似文献   

20.
The purpose of this analysis was to evaluate an existing set of search tasks in terms of their effectiveness as part of a “shared infrastructure” for conducting interactive IR research. Twenty search tasks that varied in their cognitive complexity and domain were assigned to 47 study participants; the 3,101 moves used to complete those tasks were then analyzed in terms of frequency of each type of move and the sequential patterns they formed. The cognitive complexity of the tasks influenced the number of moves used to complete the tasks, with the most complex (i.e., Create) tasks requiring more moves than tasks at other levels of complexity. Across the four domains, the Commerce tasks elicited more search moves per search. When sequences of moves were analyzed, seven patterns were identified; some of these patterns were associated with particular task characteristics. The findings suggest that search tasks can be designed to elicit particular types of search behaviors and, thus, allow researchers to focus attention on particular aspects of IR interactions.  相似文献   

设为首页 | 免责声明 | 关于勤云 | 加入收藏

Copyright©北京勤云科技发展有限公司  京ICP备09084417号