| Vol-3854 urn:nbn:de:0074-3854-4 Copyright © 2024 for the individual papers by the papers' authors. Copyright © 2024 for the volume as a collection by its editors. This volume and its papers are published under the Creative Commons License Attribution 4.0 International (CC BY 4.0).

# EMTCIR_UM-CIR 2024
Joint Proceedings of the SIGIR-AP 2024 Workshops EMTCIR 2024 and UM-CIR 2024

### Joint Proceedings of the 1st Workshop on Evaluation Methodologies,Testbeds and Community for Information Access Research (EMTCIR 2024) and the 1st Workshop on User Modelling in Conversational Information Retrieval (UM-CIR 2024)
co-located with the 2nd International ACM SIGIR Conference on Information Retrieval in the Asia Pacific (SIGIR-AP 2024)

### Tokyo, Japan, December 12, 2024.

Edited by

### Praveen Acharya1
Charles L. A. Clarke2
Fabio Crestani3
Xiao Fu4
Gareth J. F. Jones1
Noriko Kando5
Makoto P. Kato6
Aldo Lipani4
Yiqun Liu7

1 Dublin City University, Ireland
2 University of Waterloo, Canada
3 Universitá della Svizzera Italiana, Switzerland
4 University College London, UK
5 National Institute of Informatics, Japan
6 University of Tsukuba, Japan
7 Tsinghua University, China

## Table of Contents

- Preface
  Summary: There were 9 papers submitted for peer-review to these workshops. Out of these, 8 papers were accepted for this volume.

### The 1st Workshop on Evaluation Methodologies,Testbeds and Community for Information Access Research (EMTCIR 2024)

- Evaluating System Responses Based On Overconfidence and Underconfidence
  Tetsuya Sakai
- Can We Use Large Language Models to Fill Relevance Judgment Holes?
  Zahra Abbasiantaeb, Chuan Meng, Leif Azzopardi, Mohammad Aliannejadi
- Searching Unseen Sources for Historical Information: Evaluation Design for the NTCIR-18 SUSHI Pilot Task
  Douglas Oard, Tokinori Suzuki, Emi Ishita, Noriko Kando
- Building Test Collections for Japanese Dense Information Retrieval Technologies and Beyond
  Hideo Joho, Atsushi Keyaki, Yuuki Tachioka, Shuhei Yamamoto
- Understanding Tables in Financial Documents: Shared Tasks for Table Retrieval and Table QA on Japanese Annual Securities Reports
  Yasutomo Kimura, Eisaku Sato, Kazuma Kadowaki, Hokuto Ototake

### The 1st Workshop on User Modelling in Conversational Information Retrieval (UM-CIR 2024)

- Immersive Personalisation for Conversational Information-seeking: The Future of Interactive Information Retrieval (keynote)
  Johanne R. Trippas
- An Evaluation Framework for Conversational Information Retrieval Using User Simulation
  Xiao Fu, Aldo Lipani, Noriko Kando
- Responses to Conversational Information Retrieval Clarifying Questions with User Emotions
  Isin Su Ecevit, Lili Lu, Fabio Crestani
- A Framework for Knowledge Integration in Conversational Information Retrieval
  Praveen Acharya, Noriko Kando, Gareth J.F. Jones

2024-11-27: submitted by Makoto P. Kato, metadata incl. bibliographic data published under Creative Commons CC0
2024-12-01: published on CEUR Workshop Proceedings (CEUR-WS.org, ISSN 1613-0073) |valid HTML5|