# Reading: IGCV2 — Interleaved Structured Sparse Convolution (Image Classification)

## Outperforms Light Weight Model Like IGCNet / IGCV1 and MobileNetV1, Also Outperforms or On Par With Xception, DenseNet, ResNet, WRN, RiR, FractalNet

In this story, **IGCV2 (Interleaved Structured Sparse Convolution)**, by Sun Yat-Sen University, Guangdong Key Laboratory of Information Security Technology, Microsoft Research, Hefei University of Technology, and University of Central Florida, is briefly presented. In this paper:

- IGCNet / IGCV1 is improved as IGCV2 with more generalized form.
- Thus, a lighter model is obtained, the redundancy is further eliminated and the storage and time cost are saved.

This is a paper in **2018 CVPR **with over **50 citations**. (Sik-Ho Tsang @ Medium)

- In IGCNet / IGCV1, as shown above, it is
**split into primary group convolutions and secondary group convolutions.** - And there are
**permutation before and after secondary group convolutions.** **Primary group convolutions are the 3×3 spatial convolutions.****Secondary group convolutions are the 1×1 spatial convolutions.**

The above operations can be further generalized. This process can be repeated more times.

## 1.2. Xception

- The Xception block consists of a 1×1 convolution layer followed by a channel-wise convolution layer.

The 1×1 convolution layer can be sparse as well, i.e. group convolutions.

**2. IGCV2: Network Architecture**

**W1, W2, W3 (denoted as solid arrows) are sparse block matrices corresponding to group convolutions.****P1 and P2 (denoted as dashed arrows) are permutation matrices.**- The resulting composed kernel W3P2W2P1W1 is ensured to satisfy the
which guarantees that*complementary condition***for each output channel, there exists one and only one path connecting the output channel to each input channel.** - The bold line connecting gray feature maps shows such a path.
**Mathematically, IGCV2 is formulated as below:**

- More detailed architecture is as shown below:

*x*×(3×3, 1) means a 3×3 channel-wise convolution with the channel number being*x*.*L*and*K*are the hyper-parameters of IGCV2. [*L*-1,*x*; (1×1,*K*)] denotes the (*L*-1) group 1×1 convolutions with each branch containing*K*channels.- For IGCV2 (
*Cx*),*L*= 3. - For IGCV2*(
*Cx*),*K*= 8,*L*= RoundUp(log*K*(*x*))+1.

**3. Experimental Results**

## 3.1. Comparison with Xception

- The results over 20-layer networks with various widths are shown above.
- The channel number at the first convolutional layer in Xception as 35.
- We can see that
**the network with fewer number of parameters, performs better than****Xception****, which shows the powerfulness of IGCV2 block.**

## 3.2. Comparison with IGCV1

- IGCV2 outperforms IGCV1 with fewer number of parameters using models with various depths.

## 3.3. SOTA Comparison

- e.g.: DenseNet-BC(k = 12) with more number of parameters achieves lower classification error on CIFAR-100 and CIFAR-10 compared with IGCV2* (C416).
- We can also see that IGCV2 outperforms or on par with many other networks with smaller model size such as FractalNet, ResNet, RiR, ResNet34, WRN.

## 3.4. Comparison with MobileNet on ImageNet

- The above result demonstrates that IGCV2 is effective as well on large scale image dataset, outperforms MobileNetV1.

# 4. **IGCV3: Network Architecture & Preliminary Results**

- IGCV3 is formed by combining the bottleneck with IGCV2.
- Each 1×1 group convolution contains 2 (
*g*1 =*g*2 = 8) branches. - In the constructed networks, there is a skip connection for each block except the downsampling blocks, and two IGCV3 blocks correspond to one block MobileNetV2.
- IGCV3 is on par with MobileNetV2 with similar number of parameters.

IGCV3 is published in 2018 BMVC with more explanations and results. Hope I can review it in the coming future!

This is the 13th story in this month!

## Reference

[2018 CVPR] [IGCV2]

IGCV2: Interleaved Structured Sparse Convolutional Neural Networks

## Image Classification

[LeNet] [AlexNet] [Maxout] [NIN] [ZFNet] [VGGNet] [Highway] [SPPNet] [PReLU-Net] [STN] [DeepImage] [SqueezeNet] [GoogLeNet / Inception-v1] [BN-Inception / Inception-v2] [Inception-v3] [Inception-v4] [Xception] [MobileNetV1] [ResNet] [Pre-Activation ResNet] [RiR] [RoR] [Stochastic Depth] [WRN] [ResNet-38] [Shake-Shake] [FractalNet] [Trimps-Soushen] [PolyNet] [ResNeXt] [DenseNet] [PyramidNet] [DRN] [DPN] [Residual Attention Network] [DMRNet / DFN-MR] [IGCNet / IGCV1] [Deep Roots] [MSDNet] [ShuffleNet V1] [SENet] [NASNet] [MobileNetV2] [CondenseNet] [IGCV2]