| Vol-3822 urn:nbn:de:0074-3822-6 Copyright © 2024 for the individual papers by the papers' authors. Copyright © 2024 for the volume as a collection by its editors. This volume and its papers are published under the Creative Commons License Attribution 4.0 International (CC BY 4.0).

# SICSA-REALLM 2024
SICSA REALLM Workshop 2024

### Proceedings of the SICSA Workshop on Reasoning, Evaluation and Application of Large Language Models (SICSA-REALLM 2024)

### Aberdeen, United Kingdom, October 17th, 2024.

Edited by

### Kyle Martin
Pedram Salimi
Vihanga Wijayasekara

Robert Gordon University, AIR Research Group, School of Computing, Engineering and Technology (SoCET), Garthdee Road, Aberdeen, Scotland

## Table of Contents

- Preface
  Summary: There were 10 papers submitted for peer-review to this workshop. Out of these, 8 papers were accepted for presentation during the workshop.

### Session 1: LLMs and Applications

- Towards Improving Open-box Hallucination Detection in Large Language Models (LLMs) 1 - 10
  Malavika Suresh, Rahaf Aljundi, Ikechukwu Nkisi-Orji, Nirmalie Wiratunga
- Extended Results for Enhancing Abstract Screening Classification in Evidence-Based Medicine: Incorporating Domain Knowledge into Pre-trained Models 11-18
  Regina Ofori-Boateng, Magaly Aceves-Martins, Nirmalie Wiratunga, Carlos Francisco Moreno-Garcia
- Towards LLM-driven Automated Verification of Best Practices in Digital Public Infrastructures 19 - 23
  Milan Markovic, Somayajulu Sripada, Sujit Kumar Chakrabarti, Raghuram Diddigi

### Session 2: Reasoning with Natural Language

- Formal Dialogue and Large Language Models 24 - 31
  Mark Snaith, Simon Wells
- Evaluating Large Language Models on Qualitative Reasoning Tasks: A Case Study using OpenAI GPT Models 32 - 39
  Najwa AlGhamdi, Kwabena Nuamah, Alan Bundy
- Dual-Task Dialogue Understanding 40 - 46
  Sibgha Anwar, Nirmalie Wiratunga, Mark Snaith

### Session 3: Retrieval-Augmented Generation

- SCaLe-QA: Sri lankan Case Law Embeddings for Legal QA 47 - 55
  Lasal Jayawardena, Nirmalie Wiratunga, Ramitha Abeyratne, Kyle Martin, Ikechukwu Nkisi-Orji, Ruvan Weerasinghe
- Integrating KGs and Ontologies with RAG for Personalised Summarisation in Regulatory Compliance 56 - 61
  Umair Arshad, David Corsar, Ikechukwu Nkisi-Orji

2024-10-29: submitted by Kyle Martin, metadata incl. bibliographic data published under Creative Commons CC0
2024-11-04: published on CEUR Workshop Proceedings (CEUR-WS.org, ISSN 1613-0073) |valid HTML5|