| Vol-3658 urn:nbn:de:0074-3658-1 Copyright © 2024 for the individual papers by the papers' authors. Copyright © 2024 for the volume as a collection by its editors. This volume and its papers are published under the Creative Commons License Attribution 4.0 International (CC BY 4.0).

# MediaEval 2023
Multimedia Benchmark Workshop 2023

### Working Notes Proceedings of the MediaEval 2023 Workshop

### Amsterdam, The Netherlands and Online, 1-2 February 2024.

Edited by

### Steven Hicks (1)
Andreas Lommatzsch (2)
Ali Hürriyetoğlu (3)
Romain Vuillemot (4)
Mihai Gabriel Constant (5)
Vajira Thambawita (1)
Martha Larson (6)

(1) SimulaMet, Norway
(2) Technische Universität Berlin, Germany
(3) Information Technologies Institute - Centre for Research and Technology Hellas, Greece
(4) Ecole Centrale de Lyon, France
(5) University Politehnica of Bucharest, Romania
(6) Radboud University, Netherlands

## Table of Contents

### NewsImages

#### Overview paper:

- The Relation between Texts and Images in News: News Images in MediaEval 2023
  Andreas Lommatzsch, Benjamin Kille, Özlem Özgöbek, Mehdi Elahi, Duc Tien Dang Nguyen

#### Working notes papers:

- Beyond Keywords: ChatGPT's Semantic Understanding for Enhanced Media Search
  Hoang Chau Truong Vinh, Doan Khai Ta, Duc Duy Nguyen, Le Thanh Nguyen, Quang Vinh Nguyen

- Prompt-based Alignment of Headlines and Images Using OpenCLIP
  Lucien Heitz, Yuin Kwan Chan, Hongji Li, Kerui Zeng, Luca Rossetto, Abraham Bernstein

- Multimodal Fusion in NewsImages 2023: Evaluating Translators, Keyphrase Extraction, and CLIP Pre-Training
  Tien-Huy Nguyen, Hoang-Long Nguyen-Huu, Thien-Doanh Le, Huu-Loc Tran, Quoc-Khanh Le-Tran, Hoang-Bach Ngo, Minh-Hung An, Quang-Vinh Dinh

- Ensemble Pre-trained Multimodal Models for Image-text Retrieval in the NewsImages MediaEval 2023
  Taihang Wang, Jianxiang Tian, Xiangrun Li, Xiaoman Xu, Ye Jiang

- Cross-modal Networks, Fine-Tuning, Data Augmentation and Dual Softmax Operation for MediaEval NewsImages 2023
  Antonios Leventakis, Damianos Galanopoulos, Vasileios Mezaris

- Investigating the Performance of the CLIP Model and Concept Matching in Text-Image Retrieval Systems
  Xiaomeng Wang, Mingliang Liang, Martha Larson

- Integrated Multi-stage Contextual Attention Network for Text-Image Matching
  Jiande Sun, Yi Shao, Yawen Chen, Yang Zhang, Tianlin Zhang, Xuan Zhang, Ye Jiang, Jing Li

- NewsImages Fusion: Bridging Textual Context and Visual Content in Media Representation
  Arvind V, Vettri Chezhiyan, Harish J, Dr. Priyadharshini R, Mohanapriya E

- Optimizing Visual Pairings: A CLIP Framework for Precision News Image Rematching
  Pooja Premnath, Venkatasai Ojus Yenumulapalli, Rajalakshmi Sivanaiah, Angel Deborah Suseelan

- Connecting Text and Images in News Articles using VSE++
  Abbhinav Elliah, Mirunalini P, Keerthick V, Haricharan Bharathi, Anirudh Bhaskar, Vithula S

#### Quest for insight papers:

- An Empirical Exploration of Perceived Similarity between News Article Texts and Images
  Lucien Heitz, Abraham Bernstein, Luca Rossetto

### Musti

#### Overview paper:

- The MUSTI challenge@MediaEval 2023 - Multimodal Understanding of Smells in Texts and Images with Zero-shot Evaluation
  Ali Hürriyetoğlu, Inna Novalija, Mathias Zinnen, Vincent Christlein, Pasquale Lisena, Stefano Menini, Marieke van Erp, Raphael Troncy

#### Working notes papers:

- Handle the problem of ample label space by using the Image-guided Feature Extractor on the MUSTI dataset
  Le Ngoc Duc, Le Minh Hung, Dinh Quang Vinh

- MUSTI - Multimodal Understanding of Smells in Texts and Images Using CLIP
  Mirunalini P, Sanjhay V, Rohitram S, Rohith M

- Enhancing Multimodal Language Models with Olfactory Information
  Murathan Kurfalı, Jonas K. Olofsson, Thomas Hörberg

- Multimodal Learning for Image-Text Matching: A Blip-Based Approach
  Dhanya Srinivasan, Subhashree M, Mirunalini P, Jaisakthi S M

- Multimodal and Multilingual Olfactory Matching based on Contrastive Learning
  Sergio Esteban-Romero, Iván Martín-Fernández, Jaime Bellver-Soler, Manuel Gil-Martín, Fernando Fernández-Martínez

- A Text-Image Olfactory Matching Method Based on the Distribution of Real-World Data
  Yi Shao, Yulong Sun, Wenbo Wan, Jing Li, Jiande Sun

### SportsVideo

#### Overview paper:

- SportsVideo: A Multimedia Dataset for Sports Event and Position Detection in Table Tennis and Swimming
  Aymeric Erades, Pierre-Etienne Martin, Romain Vuillemot, Boris Mansencal, Renaud Peteri, Julien Morlier, Stefan Duffner, Jenny Benois-Pineau

#### Working notes papers:

- ConvLSTM for Table Tennis Stroke Classification
  Jansi Rani Sella Veluswami, Ananth Narayanan P, Bhuvan S, Shobith Kumar R

- Two-Stream Network and Attention Mechanism for Sports Video Classification in Table Tennis
  Pengcheng Dong, Hongxin Xie, Fuqiang Zheng, Jiande Sun

- A Hybrid Approach To Stroke Detection In Swimming
  Ankitha Reddy A, Pranav Moorthi, Samyuktaa Sivakumar, Shwetha S, Prabavathy Balasundaram

- Baseline Method for the Sport Task of MediaEval 2023 3D CNNs using Attention Mechanisms for Table Tennis Stoke Detection and Classification.
  Pierre-Etienne Martin

- Automated Recognition of Sports Scores using PyTessaract OCR and CNN: SportsVideo Task at MediaEval 2023
  Bhuvana Jayaraman, Mirnalinee Tt, Harshida Sujatha Palaniraj, Mohith Adluru, Sanjjit Sounderrajan

### Memorability

#### Overview paper:

- Overview of The MediaEval 2023 Predicting Video Memorability Task
  Mihai Gabriel Constantin, Claire-Hélène Demarty, Camilo Fosco, Alba García Seco de Herrera, Sebastian Halder, Graham Healy, Bogdan Ionescu, Ana Matran-Fernandez, Rukiye Savran Kiziltepe, Alan F. Smeaton, Lorin Sweeney

#### Working notes papers:

- Exploring Video Transformers and Automatic Segment Selection for Memorability Prediction
  Iván Martín-Fernández, Sergio Esteban-Romero, Jaime Bellver-Soler, Manuel Gil-Martín, Fernando Fernández-Martínez

- The Impact of Transformers Ensemble on Model Memorability and Generalizability
  Muhammad Mustafa Ali Usmani, Humna Faisal, Muhammad Atif Tahir

- SELAB-HCMUS at MediaEval 2023: A cross-domain and subject-centric approach towards the memorability prediction task
  Minh-Quang Nguyen, Minh-Huy Trinh, Huy-Giap Bui, Khac-Trieu Vo, Minh-Triet Tran, Thien-Phuc Tran, Hai-Dang Nguyen

- AIMultimediaLab at MediaEval 2023: Studying the generalization of media memorability prediction methods.
  Mihai Gabriel Constantin, Bogdan Ionescu

### Medico

#### Overview paper:

- Medico Multimedia Task at MediaEval 2023: Transparent Tracking of Spermatozoa
  Vajira Thambawita, Andrea Storås, Tuan-Luc Huynh, Hai-Dang Nguyen, Minh-Triet Tran, Trung-Nghia Le, Pål Halvorsen, Michael Riegler, Steven Hicks, Thien-Phuc Tran

#### Working notes papers:

- Tracking and Prediction Of Human Spermatozoa Motility Using Yolov8n and Greedy Shape Geometry Technique
  Muhammad Osaid, Abdul Samad, Omer Qureshi, Muhammad Atif Tahir, Muhammad Nouman Durrani

- Optimizing Sperm Detection and Tracking in Fluids with Equalize Class Representation Augmentation
  Trong-Hieu Nguyen-Mau, Quoc-Huy Trinh, Ngoc-Linh Nguyen-Ha, Tuong-Vy Truong-Thuy, Tuan-Anh Yang, Hai-Dang Nguyen, Ngoc-Thao Nguyen, Minh-Triet Tran

- Transparent Tracking of Spermatozoa with YOLOv8
  Bao-Tin Nguyen, Van-Loc Nguyen, Minh-Triet Tran

2024-03-24: submitted by Steven Hicks, metadata incl. bibliographic data published under Creative Commons CC0
2024-04-09: published on CEUR Workshop Proceedings (CEUR-WS.org, ISSN 1613-0073) |valid HTML5|