Can I read Chinese Text Recognition with A Pre-Trained CLIP-Like Model Through Image-IDS Aligning on EtoBox?

Chinese Text Recognition with A Pre-Trained CLIP-Like Model Through Image-IDS Aligning by Yu, Haiyang; Wang, Xiaocong; Li, Bin; Xue, Xiangyang is a scholarly article available to read on EtoBox.

What is Chinese Text Recognition with A Pre-Trained CLIP-Like Model Through Image-IDS Aligning about?

Scene text recognition has been studied for decades due to its broad applications. However, despite Chinese characters possessing different characteristics from Latin characters, such as complex inner structures and large categories, few methods have been proposed for Chinese Text Recognition (CTR). Particularly, the characteristic of large categories poses challenges in dealing with zero-shot and few-shot Chinese characters. In this paper, inspired by the way humans recognize Chinese texts, we propose a two-stage framework for CTR. Firstly, we pre-train a CLIP-like model through aligning printed character images and Ideographic Description Sequences (IDS). This pre-training stage simulates humans recognizing Chinese characters and obtains the canonical representation of each character. Subsequently, the learned representations are employed to supervise the CTR model, such that traditional single-character recognition can be improved to text-line recognition through image-IDS matching. To evaluate the effectiveness of the proposed method, we conduct extensive experiments on both Chinese character recognition (CCR) and CTR. The experimental results demonstrate that the proposed meth

Author

Yu, Haiyang; Wang, Xiaocong; Li, Bin; Xue, Xiangyang

Published

2023

Language

EN