Reading: ETH-CNN & ETH-LSTM — Reducing Complexity of HEVC (Fast HEVC Intra & Inter Prediction)

39.76% to 59.74%, and 43.14% to 64.07% Time Reduction with Only 1.722% and 1.483% BD-Rate Increase for LDB & RA Configurations Respectively, Outperforms Liu TIP’16 and Li ICME’17

In this story, ETH-CNN & ETH-LSTM, by Beihang University, and Imperial College London, is presented. I read this because I work on video coding research. This paper extends the conference paper Li ICME’17, which involves the LSTM in the network architecture, which is also a first attempt to use LSTM for predicting CU partition in HEVC. This is a paper in 2018 TIP. (Sik-Ho Tsang @ Medium)

Outline

  1. CPH-Inter Database

1. CPH-Inter Database

  • CPH-Intra Database has been proposed in Li ICME’17.

2. ETH-CNN Network Architecture

Image for post
Image for post
ETH-CNN Network Architecture

2.1. Network Architecture

  • Preprocessing layers: The raw CTU is preprocessed by mean removal and down-sampling in three parallel branches B1 to B3, corresponding to three levels of HCPM.
Image for post
Image for post
  • where H is the cross entropy between ground-truth and predicted labels.

2.2. Bi-Threshold Decision Scheme

  • For better tradeoff between complexity and performance, bi-threshold decision scheme is used.

3. ETH-LSTM Network Architecture

Image for post
Image for post
ETH-LSTM Network Architecture
  • The input to ETH-LSTM is the residue of each CTU.

4. Experimental Results

4.1. BD-Rate Under AI Configuration

Image for post
Image for post
BD-Rate Under AI Configuration on CPH-Intra Test Set
  • Using ETH-CNN, 1.386% BD-rate reduction is obtained with 64.01% to 70.52% time reduction.
Image for post
Image for post
BD-Rate Under AI Configuration on HEVC Test Set
  • Using ETH-CNN, 2.247% BD-rate reduction is obtained with 56.92% to 66.47% time reduction.

4.2. BD-Rate Under LDP Configuration

Image for post
Image for post
BD-Rate Under LDP Configuration on HEVC Test Set
  • Using ETH-LSTM, 1.495% BD-rate reduction is obtained with 43.84% to 62.94% time reduction.

4.3. BD-Rate Under LDB & RA Configurations

Image for post
Image for post
BD-Rate Under LDB & RA Configurations on HEVC Test Set
  • Again, using ETH-LSTM obtains the lowest BD-rate increase with large amount of time reduction.

4.4. Ablation Study

Image for post
Image for post
Ablation Study
  • For AI, ETH-CNN outperforms Liu TIP’16 and Li ICME’17.

4.5. Running Time

Image for post
Image for post
(a) AI configuration. (b) LDP configuration.
  • Both ETH-CNN & ETH-LSTM consume less than 1% of the time required by the original HM.

Since this is a TIP transaction paper, there are still a lot of details and results skipped here. Please feel free to read the paper if interested.

During the days of coronavirus, A challenge of writing 30/35/40/45 stories again for this month has been accomplished. This is the 45th story in this month..!! Let me challenge 50 stories… or take a rest and watch Netflix first?? Thanks for visiting my story..

Written by

PhD, Researcher. I share what I've learnt and done. :) My LinkedIn: https://www.linkedin.com/in/sh-tsang/, My Paper Reading List: https://bit.ly/33TDhxG

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store