Asking Clarifying Questions: To benefit or to disturb users in Web search? |
| |
| Institution: | 1. Department of Management Science and Engineering, School of Economics and Management, University of Science and Technology Beijing, Beijing 100083, China;2. School of Management and Engineering, Capital University of Economics and Business, Beijing 100070, China;1. Post-Doctoral Research Center, China Central Depository & Clearing Co., Ltd., Beijing, China;2. School of Software, Tsinghua University, Beijing, China;3. School of Information, Renmin University of China, Beijing, China;1. Research Institute for Smart Cities, School of Architecture and Urban Planning, Shenzhen University and Shenzhen Key Laboratory of Spatial Smart Sensing and Services and MNR Technology Innovation Center of Territorial and Spatial Big Data and Guangdong–Hong Kong-Macau Joint Laboratory for Smart Cities, Shenzhen 518060, China;2. Logistics Information Centre, Beijing 100842, China;3. Department of Game Design, Faculty of Arts, Uppsala University, Sweden |
| |
| Abstract: | Modern information-seeking systems are becoming more interactive, mainly through asking Clarifying Questions (CQs) to refine users’ information needs. System-generated CQs may be of different qualities. However, the impact of asking multiple CQs of different qualities in a search session remains underexplored. Given the multi-turn nature of conversational information-seeking sessions, it is critical to understand and measure the impact of CQs of different qualities, when they are posed in various orders. In this paper, we conduct a user study on CQ quality trajectories, i.e., asking CQs of different qualities in chronological order. We aim to investigate to what extent the trajectory of CQs of different qualities affects user search behavior and satisfaction, on both query-level and session-level. Our user study is conducted with 89 participants as search engine users. Participants are asked to complete a set of Web search tasks. We find that the trajectory of CQs does affect the way users interact with Search Engine Result Pages (SERPs), e.g., a preceding high-quality CQ prompts the depth users to interact with SERPs, while a preceding low-quality CQ prevents such interaction. Our study also demonstrates that asking follow-up high-quality CQs improves the low search performance and user satisfaction caused by earlier low-quality CQs. In addition, only showing high-quality CQs while hiding other CQs receives better gains with less effort. That is, always showing all CQs may be risky and low-quality CQs do disturb users. Based on observations from our user study, we further propose a transformer-based model to predict which CQs to ask, to avoid disturbing users. In short, our study provides insights into the effects of trajectory of asking CQs, and our results will be helpful in designing more effective and enjoyable search clarification systems. |
| |
| Keywords: | User study Clarifying Questions Information-seeking systems |
| 本文献已被 ScienceDirect 等数据库收录！ |
|
