Reading: ES— An Extended Skip Strategy for Inter Prediction (HEVC Inter)
Using SepConv, Outperforms FRUC+DVRF, Average of 4.4% BD-Rate Gain Compared to Conventional HEVC
In this story, An Extended Skip Strategy for Inter Prediction (Tao VCIP’19), by Huazhong Univ. of Sci. & Tech. Wuhan, and Hangzhou Dianzi University, is briefly described. I read this because I work on video coding research.
In this paper, by interpolating a new reference frame using SepConv, an extended skip (ES) mode is introduced at frame level and CTU level to increase the coding efficiency. Finally, it outperforms FRUC+DVRF that I’ve just read about in this morning.
This is a paper in 2019 VCIP. (Sik-Ho Tsang @ Medium)
Outline
- Hierarchical B Structure in HEVC
- Proposed Approach: Frame-Level ES and CTU-Level ES
- Experimental Results
1. Hierarchical B Structure in HEVC
- When Hierarchical B Structure is used in HEVC, i.e. Random Access Configuration. The coding order is based on the temporal layer (TL). The frames with lower TL are coded first.
- In this case, I0 and B8 are coded first (TL=0), then B4 (TL=1), then B2 and B6 (TL=2), and finally B1, B3, B5 and B7 (TL=3).
- With this arrangement, frames with lower TL can be acted as reference frames for frames with higher TL so as to have efficient compression.
2. Proposed Approach: Frame-Level ES and CTU-Level ES
2.1. Generating Additional Reference Frame Using SepConv
- First, SepConv is used to generate an additional reference frame FARP.
- (If interested, please read my story about SepConv. That’s also why I read AdaConv and SepConv, lol.)
2.2. Frame-Level Extended Skip (ES) Mode
- If the frame-level ES mode is chosen as the optimal mode, the FARP is directly employed as the optimal prediction frame and the current frame can be skipped.
- In this case, only an ES flag is required to be signalled in the bitstream when the ES mode is enabled, which brings considerable bitrate saving.
- Otherwise, the CTU-level ES mode will be sequentially conducted to further achieve the optimal coding performance.
2.3. CTU-Level Extended Skip (ES) Mode
- For each 64×64 CTUs in the current frame, an ES flag is signalled in the bitstreams to indicate whether the ES mode is chosen.
- If ES flag is true, the co-located block in the virtual reference frame is treated as the reconstruction block.
- Thus, there is no bit consumption for motion information. The bit expense of ES flags accounts for the majority of the overall bitrate of the current block.
- For this part, it is similar to FRUC+DVRF.
3. Experimental Results
- HM-16.6 is used.
- 2 seconds of video are encoded for each sequence using RA configuration, i.e. hierarchical B structure.
- When ES mode is applied only to TL=3 frames, ES provides on average 2.7% BD rate gain on HEVC sequences, and up to 5.6% gain is achieved on BQSquare.
- And ES outperforms FRUC+DVRF [11].
- When ES mode is applied only to TL=3 and TL = 2 frames, 4.4% coding gain is achieved, and up to 10.7% coding gain is obtained for BQSquare.
- And ES outperforms FRUC+DVRF [11].
During the days of coronavirus, let me have a challenge of writing 30 stories again for this month ..? Is it good? This is the 13th story in this month. Thanks for visiting my story..
Reference
[2019 VCIP] [ES]
An Extended Skip Strategy for Inter Prediction
Codec Inter Prediction
HEVC [Zhang VCIP’17] [NNIP] [Ibrahim ISM’18] [FRUC+DVRF] [Zhao ISCAS’18 & TCSVT’19] [Ma ISCAS’19] [ES]
