| Vol-2409 urn:nbn:de:0074-2409-1 Copyright © 2019 for the individual papers by the papers' authors. Copyright © 2019 for the volume as a collection by its editors. This volume and its papers are published under the Creative Commons License Attribution 4.0 International (CC BY 4.0).

# OSIRRC 2019
The Open-Source IR Replicability Challenge

### Proceedings of the Open-Source IR Replicability Challenge (OSIRRC 2019)
co-located with 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2019)

### Paris, France, July 25, 2019.

Edited by

### Ryan Clancy *
Nicola Ferro **
Claudia Hauff ***
Jimmy Lin *
Tetsuya Sakai ****
Ze Zhong Wu *

* University of Waterloo, Canada
** University of Padua, Italy
*** Delft University of Technology, The Netherlands
**** Waseda University, Japan

## Table of Contents

- Preface i-ii
  Ryan Clancy, Nicola Ferro, Claudia Hauff, Jimmy Lin, Tetsuya Sakai, Ze Zhong Wu
- Overview of the 2019 Open-Source IR Replicability Challenge (OSIRRC 2019) 1-7
  Ryan Clancy, Nicola Ferro, Claudia Hauff, Jimmy Lin, Tetsuya Sakai, Ze Zhong Wu

### Position Papers

- STELLA: Towards a Framework for the Reproducibility of Online Search Experiments 8-11
  Timo Breuer, Philipp Schaer, Narges Tavalkolpoursaleh, Johann Schaible, Benjamin Wolff, Bernd Müller
- Let's measure run time! Extending the IR replicability infrastructure to include performance aspects 12-16
  Sebastian Hofstätter, Allan Hanbury
- Reproducible IR needs an (IR) (Graph) Query Language 17-20
  Chris Kamphuis, Arjen P. de Vries

### Docker Papers

- Entity Retrieval Docker Image for OSIRRC at SIGIR 2019 21-25
  Negar Arabzadeh
- Dockerising Terrier for The Open-Source IR Replicability Challenge (OSIRRC 2019) 26-30
  Arthur Barbosa Câmara, Craig Macdonald
- Dockerizing Automatic Routing Runs for The Open-Source IR Replicability Challenge (OSIRRC 2019) 31-35
  Timo Breuer, Philipp Schaer
- University of Waterloo Docker Images for OSIRRC at SIGIR 2019 36-36
  Ryan Clancy, Zeynep Akkalyoncu Yilmaz, Ze Zhong Wu, Jimmy Lin
- A Docker-Based Replicability Study of a Neural Information Retrieval Model 37-43
  Nicola Ferro, Stefano Marchesin, Alberto Purpura, Gianmaria Silvello
- Dockerizing Indri for OSIRRC 2019 44-46
  Claudia Hauff
- The OldDog Docker Image for OSIRRC at SIGIR 2019 47-49
  Chris Kamphuis, Arjen P. de Vries
- PISA: Performant Indexes and Search for Academia 50-56
  Antonio Mallia, Michał Siedlaczek, Joel Mackenzie, Torsten Suel
- ielab at the Open-Source IR Replicability Challenge 2019 57-61
  Harrisen Scells, Guido Zuccon
- BM25 Pseudo Relevance Feedback Using Anserini at Waseda University 62-63
  Zhaohao Zeng, Tetsuya Sakai

2019-07-17: submitted by Nicola Ferro, metadata incl. bibliographic data published under Creative Commons CC0
2019-07-19: published on CEUR-WS.org |valid HTML5|