# Brief Review — Fast AutoAugment

**The First Place in AutoCV Competition of NeurIPS 2019 AutoDL Challenge**

4 min readOct 21, 2022

@ Medium)

Fast AutoAugment, by UNIST, Kakao Brain, and Université de Montréal

FAA2019 NeurIPS, Over 300 Citations(

Data Augmentation, Image Classification

**AutoAugment****(****AA****)**automatically**searches for augmentation policies**but requires**thousands of GPU hours**, as tabulated above.**Fast AutoAugment (FAA)**is proposed, which finds effective augmentation policies via a**more efficient search strategy**based on**density matching**.

# Outline

**Fast AutoAugment (FAA)****Results**

**1. Fast AutoAugment (FAA)**

## 1.1. Search Space

- Let
be*O***a set of augmentation**on the input**image****space**. Each operation*X**O*has**two parameters**: the**calling probability**and the*p***magnitude**. (Some operations (e.g. invert, flip) do not use the magnitude.)*λ* - Let
be*S***the set of sub-policies**where**a sub-policy**consists of*τ*∈*S*, where each operation is applied to an input image sequentially with the probability*N*consecutive operations*p*as follows:

**The output of sub-policy**can be described by a composition of operations as:*τ*(*x*)

- Where:

- The
**above figure**shows a**specific example**of augmented images by.*τ* - Thus, the
**final policy**is a*T***collection of**and*NT*sub-policies*T*(*D*)**a set of augmented images of dataset**:*D*transformed by every sub-policies*τ*∈*T*

## 1.2. Search Strategy

- For
**exploration**, the proposed method**splits the train dataset**.*Dtrain*into*K*-folds, which consists of two datasets*D*(*k*)*M*and*D*(*k*)*A* - Then,
**model parameter**. After training, the algorithm*θ*is trained in parallel on each*D*(*k*)*M***evaluates**.*B*bundles of augmentation policies on*DA* denotes the*L*(*θ|T*(*D*))**loss on augmented dataset**. In practice, the*T*(*D*)**categorical cross-entropy loss**is minimized.- These B candidate policies {
*T*1, …,*TB*} are**explored via Bayesian optimization**. The**Expected Improvement (EI) criterion [18]**is employed*B*efficiently.

- During the exploration process, the proposed algorithm
**does not train model parameter**The*θ*from scratch again.**top-**obtained from each*N*policies*K*-fold are**appended to an augmentation list**.*T*

- where
*R*() is the accuracy. - Finally, the whole training set is trained using
*T**.

# 2. Results

FAA significantly improves the performances of the baseline and Cutoutfor any network while achievingcomparable performances to those ofAAand PBA.

- On reduced CIFAR-10, FAA only takes 3.5 GPU-hours for the policy search.
- By considering the worst case,
**PyramidNet****+****ShakeDrop****requires 780 GPU-hours**which is even**less than the computation time of****AA****(5000 GPU-hours).**

The proposed FAA contributed to **win the first place in AutoCV competition of NeurIPS 2019 AutoDL challenge.**

## Reference

[2019 NeurIPS] [FAA]

Fast AutoAugment

## 1.1. Image Classification

**1989 **… **2019 **[FAA] … **2022 **[ConvNeXt] [PVTv2] [ViT-G] [AS-MLP]