Siamese Content Loss Networks for Highly Imbalanced Medical Image Segmentation

Brandon Mac, Alan R. Moody, April Khademi

Keywords: Semantic Segmentation, White Matter Hyperintensities, Siamese Networks, Medical Imaging, Magnetic Resonance Imaging, Label Imbalance, Transfer Learning

TL;DR: Utilizing Siamese networks to evaluate segmentation masks during training to improve FCN segmentation performance on highly imbalance medical images.

Track: full conference paper

Paper Type: both

Abstract: Automatic segmentation of white matter hyperintensities (WMHs) in magnetic resonance imaging (MRI) remains highly sought after due to the potential to streamline and alleviate clinical workflows. WMHs are small relative to whole acquired volume, which leads to class imbalance issues, and instability during the training process of many deep learning based solutions. To address this, we propose a method which is robust to effects of class imbalance, through incorporating multi-scale information in the training process. Our method consists of training an encoder-decoder neural network utilizing a Siamese network as an auxiliary loss function. These Siamese networks take in pairs of image pairs, input images masked with ground truth labels, and input images masked with predictions, and computes multi-resolution feature vector representations and provides gradient feedback in the form of a L2 norm. We leverage transfer learning in our Siamese network, and present positive results without need to further train. It was found these methods are more robust for training segmentation neural networks and provide greater generalizability. Our method was cross-validated on multi-center data, yielding significant overall agreement with manual annotations.

Presentation Upload Agreement: I agree that my presentation material (videos and slides) will be made public.