Can I read An Exploration into the Benefits of the CLIP model for Lifelog Retrieval on EtoBox?

An Exploration into the Benefits of the CLIP model for Lifelog Retrieval by Ly-Duyen Tran; Naushad Alam; Yvette Graham; Linh Khanh Vo; Nghiem Tuong Diep; Binh Nguyen; Liting Zhou; Cathal Gurrin is a scholarly article available to read on EtoBox.

What is An Exploration into the Benefits of the CLIP model for Lifelog Retrieval about?

In this paper, we attempt to fine-tune the CLIP (Contrastive Language-Image Pre-Training) model on the Lifelog Question Answering dataset (LLQA) to investigate retrieval performance of the fine-tuned model over the zero-shot baseline model. We train the model adopting a weight space ensembling approach using a modified loss function to take into account the differences in our dataset (LLQA) when compared with the dataset the CLIP model was originally pretrained on. We further evaluate our fine-tuned model using visual as well as multimodal queries on multiple retrieval tasks, demonstrating improved performance over the zero-shot baseline model. ## CCS CONCEPTS • Computing methodologies → Neural networks; • Information systems → Information retrieval.

Author

Ly-Duyen Tran; Naushad Alam; Yvette Graham; Linh Khanh Vo; Nghiem Tuong Diep; Binh Nguyen; Liting Zhou; Cathal Gurrin

Publisher

ACM

Published

2022

Language

EN