Reading: ES— An Extended Skip Strategy for Inter Prediction (HEVC Inter)

Using SepConv, Outperforms FRUC+DVRF, Average of 4.4% BD-Rate Gain Compared to Conventional HEVC

In this story, An Extended Skip Strategy for Inter Prediction (Tao VCIP’19), by Huazhong Univ. of Sci. & Tech. Wuhan, and Hangzhou Dianzi University, is briefly described. I read this because I work on video coding research.

In this paper, by interpolating a new reference frame using SepConv, an extended skip (ES) mode is introduced at frame level and CTU level to increase the coding efficiency. Finally, it outperforms FRUC+DVRF that I’ve just read about in this morning.

This is a paper in 2019 VCIP. (Sik-Ho Tsang @ Medium)

Outline

  1. Hierarchical B Structure in HEVC
  2. Proposed Approach: Frame-Level ES and CTU-Level ES
  3. Experimental Results

1. Hierarchical B Structure in HEVC

Hierarchical B Structure in HEVC
  • When Hierarchical B Structure is used in HEVC, i.e. Random Access Configuration. The coding order is based on the temporal layer (TL). The frames with lower TL are coded first.
  • In this case, I0 and B8 are coded first (TL=0), then B4 (TL=1), then B2 and B6 (TL=2), and finally B1, B3, B5 and B7 (TL=3).
  • With this arrangement, frames with lower TL can be acted as reference frames for frames with higher TL so as to have efficient compression.

2. Proposed Approach: Frame-Level ES and CTU-Level ES

2.1. Generating Additional Reference Frame Using SepConv

Generating Additional Reference Frame Using SepConv
  • First, SepConv is used to generate an additional reference frame FARP.
  • (If interested, please read my story about SepConv. That’s also why I read AdaConv and SepConv, lol.)

2.2. Frame-Level Extended Skip (ES) Mode

Frame-Level Extended Skip (ES) Mode
  • If the frame-level ES mode is chosen as the optimal mode, the FARP is directly employed as the optimal prediction frame and the current frame can be skipped.
  • In this case, only an ES flag is required to be signalled in the bitstream when the ES mode is enabled, which brings considerable bitrate saving.
  • Otherwise, the CTU-level ES mode will be sequentially conducted to further achieve the optimal coding performance.

2.3. CTU-Level Extended Skip (ES) Mode

CTU-Level Extended Skip (ES) Mode
  • For each 64×64 CTUs in the current frame, an ES flag is signalled in the bitstreams to indicate whether the ES mode is chosen.
  • If ES flag is true, the co-located block in the virtual reference frame is treated as the reconstruction block.
  • Thus, there is no bit consumption for motion information. The bit expense of ES flags accounts for the majority of the overall bitrate of the current block.
  • For this part, it is similar to FRUC+DVRF.

3. Experimental Results

  • HM-16.6 is used.
  • 2 seconds of video are encoded for each sequence using RA configuration, i.e. hierarchical B structure.
  • When ES mode is applied only to TL=3 frames, ES provides on average 2.7% BD rate gain on HEVC sequences, and up to 5.6% gain is achieved on BQSquare.
  • And ES outperforms FRUC+DVRF [11].
  • When ES mode is applied only to TL=3 and TL = 2 frames, 4.4% coding gain is achieved, and up to 10.7% coding gain is obtained for BQSquare.
  • And ES outperforms FRUC+DVRF [11].

During the days of coronavirus, let me have a challenge of writing 30 stories again for this month ..? Is it good? This is the 13th story in this month. Thanks for visiting my story..

PhD, Researcher. I share what I've learnt and done. :) My LinkedIn: https://www.linkedin.com/in/sh-tsang/, My Paper Reading List: https://bit.ly/33TDhxG

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

4 Interesting Data Science Project Ideas

Cracking Business Case Interviews for Data Scientists — Part 2

Machine Learning/Data Science take home challenge based Question and Solutions

Data Visualization with Pandas and Seaborn 🎓

Defining ‘Done’ in Data Science

The Polls Were Wrong — Again

Establishing a culture of Analytics

How I built a simple Fake News detector on Amazon SageMaker

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Sik-Ho Tsang

Sik-Ho Tsang

PhD, Researcher. I share what I've learnt and done. :) My LinkedIn: https://www.linkedin.com/in/sh-tsang/, My Paper Reading List: https://bit.ly/33TDhxG

More from Medium

Should We Unify all ML Frameworks?

BERT — Bidirectional Encoder Representation from Transformers

Review — fastText: Enriching Word Vectors with Subword Information

Paper Review: Parameter Prediction for Unseen Deep Architectures