Reading: Bonnineau ICASSP’20 — VVC & SR for Efficient Delivery (VVC Intra)

Using SRFBN for upsampling, 29% BD-Rate Saving

4 min readJul 9, 2020

In this story, Versatile Video Coding and Super-Resolution for Efficiency Delivery of 8K Video with 4K Backward-Compatibility (Bonnineau ICASSP’20), by IRT b<>com, TDF, and Univ Rennes, is briefly presented. In this paper:

By using Super-Resolution with Feedback Network (SRFBN), there is visual quality gain over simulcast, especially on bit-rates lower than 30Mb/s with average gain of 0.77dB, 0.015, and 7.97 for PSNR, SSIM, and VMAF, respectively.
and it outperforms the Lanczos filter in average by 29% of BD-rate savings.

This is a paper in 2020 ICASSP. Authors have put the article to arXiv. (Sik-Ho Tsang @ Medium)

Outline

Simulcast
Spatial Scalability
Proposed Pre and Post-Processing
Experimental Results

1. Simulcast

One simple way to deliver both 4K and 8K videos is simulcast.
By using Simulcast, the 8K video is encoded.
Also, this 8K video is downsized to 4K then encoded.
Thus, the 8K and 4K videos are encoded independently and then transmitted.
However, the content of 8K and 4K are the same, with only the size different. The redundancy is not fully exploited which makes the bitrate high.
The bit-rate required by HEVC for 8K applications in 60Hz and 120Hz (temporally scalable) is around 80Mbps.
For satellite transmission with DVBS2X, bandwidth in the range 70–80Mb/s can be reached.
For terrestrial transmission, such bandwidth requirements prevent the deployment of 4K and 8K simultaneously, as practical DVB-T2 channels offer bandwidth in the range 30–40Mb/s over an 8MHz channel.

2. Spatial Scalability

There is scalable coding provided to encode the 4K video as base layer (BL) and the 8K video as enhancement layer (EL).
When encoding the 8K video, the information at 4K video (BL) is utilized so that the bitrate for 8K video is much lower compared with simulcast.
However, due to codec compatibility issues caused by a late integration of the HEVC scalable extension, spatial-scalability is not much present in the current broadcast ecosystem.

3. Proposed Pre and Post-Processing

In this approach, the 8K is downsized to 4K then encoded.
After transmitting the 4K video to the decoder side, the 4K video is decoded. Also, this 4K video is upsampled by deep learning approach to reconstruct the 8K video.
By this mean, only 4K video is transmitted, but with the support of 8K video service.
To provide high quality 8K video, Super-Resolution with Feedback Network (SRFBN) is used for upsample (super resolution)

4. Experimental Results

4.1. Settings

VVC: The proposed Pre and Post-Processing
HEVC: Simulcast
SHVC: Spatial Scalability
All Intra configuration is used with QPs of 17, 22, 27, 32 and 37.

4.2. SHVC vs Simulcast

**SHVC (Spatial Scalability) Against HEVC (Simulcast)**

SHVC achieves 13.81%, 13.11%, and 12.17% of average BD-rate savings over simulcast for PSNR, SSIM, and VMAF, respectively.
But SHVC is not popular in industry as mentioned.

4.3. Coding Performance at VVC

SRFBN enables average gain over Lanczos up to 29.02%, 14.40%, and 28.53% of BD-rate savings regarding PSNR, SSIM and VMAF metrics, respectively.
Regarding PSNR, SRFBNand Lanczos have better performance over simulcast until approximately 100Mb/s and 30Mb/s, respectively.
Regarding SSIM and VMAF, SRFBN is more efficient than simulcast for all the presented bit-rate range.

**BD-metric results per bit-rate interval (Mb/s) for pre/post-processing with VVC compared to simulcast with VVC**

Three bit-rate intervals are considered: lower than 30Mb/s, from 30Mb/s to 80Mb/s, more than 80Mb/s.
The performance gap between SRFBN and Lanczos is more significant for the most complex sequences.

This is the 3rd story in this month.

Reference

[2020 ICASSP] [Bonnineau ICASSP’20]
Versatile Video Coding and Super-Resolution for Efficiency Delivery of 8K Video with 4K Backward-Compatibility

Codec Intra Prediction

JPEG [MS-ROI] [Baig JVICU’17]
HEVC [Xu VCIP’17] [Song VCIP’17] [Li VCIP’17] [Puri EUSIPCO’17] [IPCNN] [IPFCN] [HybridNN, Li ICIP’18] [Liu MMM’18] [CNNAC] [Li TCSVT’18] [Spatial RNN] [PS-RNN] [AP-CNN] [MIP] [Wang VCIP’19] [IntraNN] [CNNAC TCSVT’19] [CNN-CR] [CNNMC Yokoyama ICCE’20] [PNNS] [CNNCP]
VVC [CNNIF & CNNMC] [Brand PCS’19] [Bonnineau ICASSP’20]