Can I read Image Captioning with Deep Bidirectional LSTMs and Multi-Task Learning on EtoBox?

Image Captioning with Deep Bidirectional LSTMs and Multi-Task Learning by Cheng Wang; Haojin Yang; Christoph Meinel is a Computer Science article available to read on EtoBox.

What is Image Captioning with Deep Bidirectional LSTMs and Multi-Task Learning about?

Generating a novel and descriptive caption of an image is drawing increasing interests in computer vision, natural language processing, and multimedia communities. In this work, we propose an end-to-end trainable deep bidirectional LSTM (Bi-LSTM (Long Short-Term Memory)) model to address the problem. By combining a deep convolutional neural network (CNN) and two separate LSTM networks, our model is capable of learning long-term visual-language interactions by making use of history and future context information at high-level semantic space. We also explore deep multimodal bidirectional models, in which we increase the depth of nonlinearity transition in different ways to learn hierarchical visual-language embeddings. Data augmentation techniques such as multi-crop, multi-scale, and vertical mirror are proposed to prevent overfitting in training deep models. To understand how our models “translate” image to sentence, we visualize and qualitatively analyze the evolution of Bi-LSTM internal states over time. The effectiveness and generality of proposed models are evaluated on four benchmark datasets: Flickr8K, Flickr30K, MSCOCO, and Pascal1K datasets. We demonstrate that Bi-LSTM model

Who reads Image Captioning with Deep Bidirectional LSTMs and Multi-Task Learning?

Author

Cheng Wang; Haojin Yang; Christoph Meinel

Publisher

ACM

Published

2018

Language

EN

Field

Computer Science (Physical Sciences)