Reading: GVTCNN — Group Variational Transformation Convolutional Neural Network (HEVC Inter)

Outperforms CNNIF, 1.9% BD-rate Reduction Obtained Under LDP Configuration

interpolate the sub-pel pixels If based on the integer-pel pixels Il

In this story, Group Variational Transformation Convolutional Neural Network (GVTCNN), by Peking University, is briefly presented. I read this because I work on video coding research. In this paper:

  • GVTCNN is used to interpolate the sub-pel pixels based on the integer-pel pixels for inter coding.

This is a paper in 2018 DCC. (Sik-Ho Tsang @ Medium)

Outline

  1. GVTCNN: Network Architecture
  2. Sample Collection
  3. Experimental Results

1. GVTCNN: Network Architecture

GVTCNN: Network Architecture
  • The integer-position sample Il is the input of the network.
  • 3×3 kernel size for all the convolutional layers with PReLU used.
  • Standard MSE loss function is used.

1.1. Shared Feature Map Extraction Component

  • A feature map with 48 channels is initially generated from the integer-position sample, followed by 8 convolution layers with 10 channels which are lightweight and cost less to save the learnt parameters.
  • The 10-th layer later derives a 48-channel feature map.
  • After 9 convolutional layers with 3×3 kernel size, the receptive field of each point in the shared feature map is 19×19.

1.2. Group Variational Transformation

  • The group variational transformation is further performed over the shared feature maps with a specific convolutional layer for each sub-pixel sample.
  • Different residual maps are then generated and we obtain the final inferred sub-pixel position samples by adding the residual maps to the integer-position sample.

2. Sample Collection

Training data generation for GVTCNN
  • The training data is first blurred with a Gaussian filter to simulate the correlations between the integer-position sample and sub-pixel position samples.
  • Sub-pixel position samples are later sampled from the blurred image.
  • As for the input integer-position sample generation, an intermediate integer sample is previously down-sampled from the raw image. Then, the intermediate down-sampled version is coded by HEVC.
  • Two networks are trained separately for 3 half-pixel position samples and 12 quarter-pixel position samples to better generate the samples at different sub-pixel levels, called GVTCNN-H and GVTCNN-Q respectively.
  • For GVTCNN-H, 200 training images and 200 testing images in the set BSDS500 are used to train it. 3×3 Gaussian kernels with random standard deviations in the range [0.5, 0.6] are used for blurring.
  • For GVTCNN-Q, 10 YUV sequences at size 1024×768 and 1920×1080 are used to extract 89 high resolution frames to synthesize training data. Standard deviations of Gaussian kernels ranges from [0.7, 0.8].
  • A model is trained for each QP.

3. Experimental Results

BD-Rate (%) on HEVC Test Sequences
  • HM-16.15 is used under LDP configuration.
  • 1.9% BD-rate reduction is obtained.
BD-Rate (%) on HEVC Test Sequences
  • CNNIF only interpolates half-pel pixels, thus GVTCNN-H is used for fair comparison.
  • GVTCNN-H outperforms CNNIF with 2.4% BD-rate reduction.
BD-Rate (%) on HEVC Test Sequences
  • By training a model separately for each sub-pel position, the BD-rate reduction is still similar to the one using GVT. (Not so clear to me about the network architecture here.)

This is the 26th story in this month!

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store