Review — Mean Teachers are Better Role Models: Weight-Averaged Consistency Targets Improve Semi-Supervised Deep Learning Results

Mean Teacher, Teacher Student Approach, for Semi-Supervised Learning

Teacher Student Approach (Image from Pixabay)

Outline

Conceptual Idea of Applying Noise & Ensembling

2. Mean Teacher

Mean Teacher

2.1. Applying Noise

2.2. Model Ensembling Using EMA

2.3. Consistency Cost

3. Experimental Results

3.1. SVHN & CIAFR-10

Error rate percentage on SVHN over 10 runs (4 runs when using all labels).
Error rate percentage on CIFAR-10 over 10 runs (4 runs when using all labels).

3.2. SVHN with Extra Unlabeled Data

Error percentage over 10 runs on SVHN with extra unlabeled training data

3.3. Analysis of the Training Curves

Smoothened classification cost (top) and classification error (bottom) of Mean Teacher and baseline Π model on SVHN over the first 100000 training steps

3.4. Mean Teacher with ResNet on CIFAR-10 and ImageNet

Error rate percentage of ResNet Mean Teacher compared to the state of the art

PhD, Researcher. I share what I've learnt and done. :) My LinkedIn: https://www.linkedin.com/in/sh-tsang/, My Paper Reading List: https://bit.ly/33TDhxG