Intermodal Image Representations

Elisabeth Wetzer, Johan Öfverstedt

Keywords: representation learning, image registration

TL;DR: Introducing 2D representations of multimodal images for multimodal image registration using normalized gradient fields, crossmodal mutual information and distribution matching.

Abstract: Representation Learning can be used to generate common image-like representations of images captured by different imaging techniques. These so-called multimodal images can be very different in their appearance, resulting in challenging image alignment tasks. Learned representations have the potential to transform the multimodal task into a - generally simpler - monomodal one, enabling the use of established registration methods. In this ongoing work, we introduce intermodal image representations (IMIRs), which require very little training data, preserve relevant details in the representations and are computationally lightweight.

Submission Number: 20