SuperCaptioning: Image Captioning Using Two-dimensional Word Embedding

Baohua Sun, Lin Yang, Michael Lin, Charles Young, Patrick Dong, Wenhan Zhang, and Jason Dong

Keywords: Image Captioning, Two-dimensional Word Embedding, Transfer Learning

TL;DR: Image captioning using two-dimensional word embedding.

Abstract: Language and vision are processed as two different modal in current work for image captioning. However, recent work on Super Characters method shows the effectiveness of two-dimensional word embedding, which converts text classification problem into image classification problem. In this paper, we propose the SuperCaptioning method, which borrows the idea of two-dimensional word embedding from Super Characters method, and processes the information of language and vision together in one single CNN model. The experimental results on Flickr30k data shows the proposed method gives high quality image captions. An interactive demo is ready to show at the workshop.

Community Implementations: [ 1 code implementation](https://www.catalyzex.com/paper/arxiv:1905.10515/code)