Reading: Zhang ICIP’19 — Dual-Input CNN-Based Interpolation Scheme (HEVC Inter Prediction)
In this story, “Advanced CNN Based Motion Compensation Fractional Interpolation” (Zhang ICIP’19), by Shanghai Jiao Tong University, and University of Missouri-Kansas City, is briefly presented. In this paper:
- A dual-input CNN-Based Interpolation Scheme is designed where the inputs are the prediction and residual parts of reference blocks.
- Dual-Input CNN Network Architecture
- Experimental Results
1. Dual-Input CNN Network Architecture
- In order to extract information from both prediction and residual parts of the relative reference block, separate convolution layers are used to handle them.
- After several convolution layers, the feature maps of prediction and residual channels are concatenated to form the input of following layers.
- The network is a fully convolutional network, and ReLU is adopted as the activation function. The convolutions use 64 filters with size 3×3.
- The last layer used to combine previous feature map and generate output contains a single 3×3 filter.
- Residual learning strategy is adopted.
- Standard Euclidean loss is used:
- The training data can be derived at the decoder side of HEVC directly.
- Training data: two 4K sequences TrafficFlow, CampfireParty, and one test sequence of HEVC BlowingBubbles.
- HM-16.7 is used under low delay P configuration.
- Only half-pel (yellow) positions are predicted using CNN as shown below:
- where integer-pels (blue) are available at both encoder and decoder.
2. Experimental Results
- The proposed CNN obtains 0.9% BD-rate reduction against HEVC while Zhang VCIP’17 can only obtain 0.4% BD-rate reduction.
3.2. RD Curves
- The above RD curves show that the proposed CNN is more efficient at high bitrate condition than at low bitrate condition.
This is the 21st story in this month!
[2019 ICIP] [Zhang ICIP’19]
Advanced CNN Based Motion Compensation Fractional Interpolation
Codec Inter Prediction
H.264 [DRNFRUC & DRNWCMC]
HEVC [CNNIF] [Zhang VCIP’17] [NNIP] [Ibrahim ISM’18] [VC-LAPGAN] [VI-CNN] [FRUC+DVRF][FRUC+DVRF+VECNN] [RSR] [Zhao ISCAS’18 & TCSVT’19] [Ma ISCAS’19] [Zhang ICIP’19] [ES] [CNN-SR & CNN-UniSR & CNN-BiSR] [DeepFrame] [U+DVPN] [Multi-Scale CNN]