Can I read Attention Guidance Mechanism for Handwritten Mathematical Expression Recognition on EtoBox?

Attention Guidance Mechanism for Handwritten Mathematical Expression Recognition by Liu, Yutian; Ke, Wenjun; Wei, Jianguo is a scholarly article available to read on EtoBox.

What is Attention Guidance Mechanism for Handwritten Mathematical Expression Recognition about?

Handwritten mathematical expression recognition (HMER) is challenging in image-to-text tasks due to the complex layouts of mathematical expressions and suffers from problems including over-parsing and under-parsing. To solve these, previous HMER methods improve the attention mechanism by utilizing historical alignment information. However, this approach has limitations in addressing under-parsing since it cannot correct the erroneous attention on image areas that should be parsed at subsequent decoding steps. This faulty attention causes the attention module to incorporate future context into the current decoding step, thereby confusing the alignment process. To address this issue, we propose an attention guidance mechanism to explicitly suppress attention weights in irrelevant areas and enhance the appropriate ones, thereby inhibiting access to information outside the intended context. Depending on the type of attention guidance, we devise two complementary approaches to refine attention weights: self-guidance that coordinates attention of multiple heads and neighbor-guidance that integrates attention from adjacent time steps. Experiments show that our method outperforms existing

Author

Liu, Yutian; Ke, Wenjun; Wei, Jianguo

Published

2024

Language

EN