OSIRRC 2019
The Open-Source IR Replicability Challenge

Table of Contents

• Preface i-ii
  Ryan Clancy, Nicola Ferro, Claudia Hauff, Jimmy Lin, Tetsuya Sakai, Ze Zhong Wu
• Overview of the 2019 Open-Source IR Replicability Challenge (OSIRRC 2019) 1-7
  Ryan Clancy, Nicola Ferro, Claudia Hauff, Jimmy Lin, Tetsuya Sakai, Ze Zhong Wu

Position Papers

• STELLA: Towards a Framework for the Reproducibility of Online Search Experiments 8-11
  Timo Breuer, Philipp Schaer, Narges Tavalkolpoursaleh, Johann Schaible, Benjamin Wolff, Bernd Müller
• Let's measure run time! Extending the IR replicability infrastructure to include performance aspects 12-16
  Sebastian Hofstätter, Allan Hanbury
• Reproducible IR needs an (IR) (Graph) Query Language 17-20
  Chris Kamphuis, Arjen P. de Vries

Docker Papers

• Entity Retrieval Docker Image for OSIRRC at SIGIR 2019 21-25
  Negar Arabzadeh
• Dockerising Terrier for The Open-Source IR Replicability Challenge (OSIRRC 2019) 26-30
  Arthur Barbosa Câmara, Craig Macdonald
• Dockerizing Automatic Routing Runs for The Open-Source IR Replicability Challenge (OSIRRC 2019) 31-35
  Timo Breuer, Philipp Schaer
• University of Waterloo Docker Images for OSIRRC at SIGIR 2019 36-36
  Ryan Clancy, Zeynep Akkalyoncu Yilmaz, Ze Zhong Wu, Jimmy Lin
• A Docker-Based Replicability Study of a Neural Information Retrieval Model 37-43
  Nicola Ferro, Stefano Marchesin, Alberto Purpura, Gianmaria Silvello
• Dockerizing Indri for OSIRRC 2019 44-46
  Claudia Hauff
• The OldDog Docker Image for OSIRRC at SIGIR 2019 47-49
  Chris Kamphuis, Arjen P. de Vries
• PISA: Performant Indexes and Search for Academia 50-56
  Antonio Mallia, Michał Siedlaczek, Joel Mackenzie, Torsten Suel
• ielab at the Open-Source IR Replicability Challenge 2019 57-61
  Harrisen Scells, Guido Zuccon
• BM25 Pseudo Relevance Feedback Using Anserini at Waseda University 62-63
  Zhaohao Zeng, Tetsuya Sakai