Can I read Learning to ignore: rethinking attention in CNNs on EtoBox?

Learning to ignore: rethinking attention in CNNs by Laakom, Firas; Chumachenko, Kateryna; Raitoharju, Jenni; Iosifidis, Alexandros; Gabbouj, Moncef is a scholarly article available to read on EtoBox.

What is Learning to ignore: rethinking attention in CNNs about?

Recently, there has been an increasing interest in applying attention mechanisms in Convolutional Neural Networks (CNNs) to solve computer vision tasks. Most of these methods learn to explicitly identify and highlight relevant parts of the scene and pass the attended image to further layers of the network. In this paper, we argue that such an approach might not be optimal. Arguably, explicitly learning which parts of the image are relevant is typically harder than learning which parts of the image are less relevant and, thus, should be ignored. In fact, in vision domain, there are many easy-to-identify patterns of irrelevant features. For example, image regions close to the borders are less likely to contain useful information for a classification task. Based on this idea, we propose to reformulate the attention mechanism in CNNs to learn to ignore instead of learning to attend. Specifically, we propose to explicitly learn irrelevant information in the scene and suppress it in the produced representation, keeping only important attributes. This implicit attention scheme can be incorporated into any existing attention mechanism. In this work, we validate this idea using two recent a

Author

Laakom, Firas; Chumachenko, Kateryna; Raitoharju, Jenni; Iosifidis, Alexandros; Gabbouj, Moncef

Published

2021

Language

EN