# Review — Uncertainty-Aware Self-ensembling Model for Semi-supervised 3D Left Atrium Segmentation

## UA+MT, **Semi-Supervised Segmentation Using Teacher-Student Paradigm**

--

Uncertainty-Aware Self-ensembling Model for Semi-supervised 3D Left Atrium Segmentation, UA+MT, by The Chinese University of Hong Kong,2019 MICCAI, Over 300 Citations(Sik-Ho Tsang @ Medium)

Medical Imaging, Medical Image Analysis, Semi-Supervised Learning, Image Segmentation, V-Net

- A novel
**uncertainty-aware semi-supervised framework**is proposed for**left atrium segmentation****from 3D MR images**. - The framework consists of a student model and a teacher model, and
**the student model learns from the teacher model**by minimizing a**segmentation loss**and a**consistency loss**with respect to the targets of the teacher model.

# Outline

**Semi-Supervised Segmentation****Uncertainty-Aware Mean Teacher Framework (UA-MT)****Experimental Results**

**1. Semi-Supervised Segmentation**

## 1.1. Definitions

- Let say we have the
**3D data**, where the training set consists ofand*N*labeled data, called*M*unlabeled dataand*DL*respectively:*DU*

- where
of the size*xi**H*×*W*×*D*is the**input volume**and*yi*∈ {0, 1}*H*×*W*×*D*is the**ground-truth annotations**.

## 1.2. Loss Functions

- The goal of
**semi-supervised segmentation framework**is to minimize the following**combined objective function**:

- where
denotes the*Ls***supervised loss (e.g., cross-entropy loss)**to evaluate the**quality of the network output on labeled inputs**, and represents the*Lc***unsupervised consistency loss**for measuring the**consistency between the prediction of the teacher model and the student model**for the**same input**.*xi*under different perturbations- Here,
denotes the*f*(·)**segmentation neural network**;**(**and*θ’*,*ξ’*)**(**represents the*θ*,*ξ*)**weights and different perturbation operations**(e.g., adding noise to input and network Dropout) of the**teacher**and**student**models. is an ramp-up weighting coefficient that controls the*λ***trade-off between the supervised and unsupervised loss**:

**At the beginning**, when the model is not well trained,such that the above loss function*λ*is small**mainly depends on supervised loss**.**As the training continues**(where*t*is the training step),such that loss function is*λ*becomes larger**a combination of supervised loss and consistency loss.**- The
**teacher’s weights**as an*θ’***exponential moving average (EMA) of the student’s weights**to ensemble the information in different training step:*θ*

- where
*α*is the EMA decay.

# 2. Uncertainty-Aware Mean Teacher Framework (UA-MT)

## 2.1. Uncertainty Estimation

on the*T*stochastic forward passes**teacher**model under random Dropout and input Gaussian noise for each input volume. Therefore, for**each voxel**in the input, we obtain**a set of softmax probability vector**:

- The
**predictive entropy**is:

- where
is the*pct***probability of the**.*c*-th class in the*t*-th time prediction - The uncertainty is estimated in voxel level and the
**uncertainty of the whole volume**is:*U*

## 2.2. Uncertainty-Aware Consistency Loss

- The
**uncertainty-aware consistency loss**as the*Lc***voxel-level mean squared error (MSE) loss of the teacher and student models**only for the most certainty predictions:

- where
*I*(·) is the indicator function;and*f*’*v*are the*fv***predictions of teacher model and student model at the**, respectively.*v*-th voxel is the*uv***estimated uncertainty**; and*U*at the*v*-th voxelis a*H***threshold**to select the most certain targets.

With the proposed uncertainty-aware consistency loss in the training procedure,

both the student and teacher can learn more reliable knowledge, which can thenreduce the overall uncertainty of the model.

## 2.3. Model Architecture

**V-Net**is used as the network**backbone**. The short residual connection is removed in each convolution block, and a**joint cross-entropy loss and dice loss**are used.- To adapt the V-Net as a Bayesian network to estimate the uncertainty,
**two****Dropout****layers**with Dropout rate 0.5 are**added after the L-Stage 5 layer and R-Stage 1 layer**of the V-Net.

# 3. Experimental Results

## 3.1. Dataset

**Atrial Segmentation Challenge dataset**is used. It provides**100 3D gadolinium-enhanced MR imaging scans (GE-MRIs)**and**LA segmentation mask**for training and validation.- These scans have an isotropic resolution of 0.625×0.625×0.625mm³. The 100 scans are split into
**80 scans**for**training**and**20 scans**for**evaluation**. All the scans were**cropped centering at the heart region**for better comparison of the segmentation performance of different methods.

## 3.2. SOTA Comparisons

- The above table shows the segmentation performance of V-Net trained with
**only the labeled data (the first two rows)**and the proposed semi-supervised method (UA-MT) on the testing dataset. - The
**fully supervised****V-Net****upper bound (3rd to 4th rows)**. - Compared with the Vanilla V-Net, adding Dropout (Bayesian V-Net) improves the segmentation performance, and achieves an average Dice of 86.03% and Jaccard of 76.06% with only the labeled training data.

By utilizing the unlabeled data, the semi-supervised framework furtherimproves the segmentation by 4.15% Jaccradand2.85% Dice.

- Compared with the self-training method, the
**DAN and ASDNet improve by 0.60% and 0.98% Dice, respectively**, showing the effect of adversarial learning in semi-supervised learning.**The ASDNet is better than DAN**, since it selects the trustworthy region of unlabeled data for training the segmentation network. - The self-ensembling-based methods
**TCSE achieve slightly better performance than ASDNet**, demonstrating that perturbation-based consistency loss is helpful for the semi-supervised segmentation problem.

Notably,

the proposed method (UA-MT) achieves the best performanceover the state-of-the-art semi-supervised methods, except that the ASD performance is comparable with ASDNet.

## 3.3. Analyses

The proposed uncertainty-aware method

outperforms both the MT model and MT-Dice model.

- Compared with the supervised method, the proposed results have
**higher overlap ratio**with the**ground truth (the second row)**and produce**less false positives (the first row)**. - As shown in (d), the network estimates
**high uncertainty near the boundary and ambiguous regions of great vessels**.

## Reference

[2019 MICCAI] [UA+MT]

Uncertainty-Aware Self-ensembling Model for Semi-supervised 3D Left Atrium Segmentation

## Biomedical Image Semi-Supervised Learning

**2019** [UA+MT]