Review — LIU4K: A Comprehensive Benchmark for Single Image Compression Artifact Reduction (Codec Filtering)
LIU4K, Large-Scale Ideal Ultra high-definition 4K, A New 4K Image Dataset is Proposed
In this story, A Comprehensive Benchmark for Single Image Compression Artifact Reduction, (LIU4K), by Peking University, and University of Science and Technology of China, is reviewed. In this paper:
- LIU4K: Large-Scale Ideal Ultra high-definition 4K, a new dataset, is proposed.
- A summary of state-of-the-art single image filtering approaches are also reviewed and evaluated using both full reference and no reference metric for objective and subjective quality measurement respectively.
This is a paper in 2020 TIP where TIP has a high impact factor of 9.34. (Sik-Ho Tsang @ Medium)
- LIU4K: Large-Scale Ideal Ultra high-definition 4K
- Summary of State-Of-The-Art Single Image Filtering Approaches
1. LIU4K: Large-Scale Ideal Ultra high-definition 4K
1.1. High-resolution definition
- Compared to previous datasets, the resolution of the images in LIU4K is 2848×4288, offering abundant materials for testing and evaluating the performance on 4K/8K display devices.
- The training, testing, and validation images include 1,500, 200, and 80 4K images, which is much more than in previous datasets.
1.3. Diversified and complex signals
- LIU4K achieves the best results in terms of entropy-driven non-reference metrics (ENTROPY), which demonstrates its signal diversity and complexity.
1.4. High visual quality
- LIU4K wins in general purpose non-reference metrics (ENIQA and NIQE).
2. Summary of State-Of-The-Art Single Image Filtering Approaches
- There are four categories: filter-based methods, probabilistic prior-based methods, deep learning-based JPEG artifacts removal methods (starting from 2015 ARCNN), and deep learning-based loop filter methods (starting from 2017 VRCNN).
- Most previous works are based on these two standards: JPEG and HEVC.
- Deep learning approaches are more likely to focus in this story.
2.1. Deep Learning-Based JPEG Artifacts Removal
- The works fall into two main streams: better network architectures (2–1) and better utilization of DCT domain information (2–2).
- Better utilization of DCT domain information: D³, DMCNN, MWCNN (Wavelet), etc.
- Besides network improvement, some works try to embed traditional priors or constraints into deep networks.
- Below shows PSNR that improved across the years.
2.2. Deep Learning-Based Loop Filter Methods (HEVC)
- Deep-learning based loop filters focus more on handling the degradation caused by variable-size partitions and utilizing side information from codecs (1), e.g.: VRCNN, RHCNN, and MLSDRN.
- There are still a lot of works summarized in the paper.
This paper summarizes and introduces a lot of approaches, it is very good for those who starts to work on the filtering research.
- For JPEG artifact reduction, the models are trained on the training of LIU4K.
- For loop filters, the models are trained on the training sets of both BSD500 and LIU4K. During the training phase, 80 additional 4K images are used as LIU4K validation set.
- ARCNN is modified/enhanced for better convergence.
- DMCNN obtains better performance in terms of PSNR and SSIM.
- But it cannot obtain good results for NIQE and BRISQUE, which are no-reference metrics.
- Mean Opinion Scores (MOS) are also obtained. The scores are given by 40 participants.
- It is observed that, DMCNN, MemNet, and PRN achieve overall superior visual quality than other methods.
- It is observed that DMCNN achieves the overall best visual quality.
- The summarization and experiments are in very details. I’m just here to introduce there is a paper summarizing the JPEG and HEVC filtering works.
I believe there will be research works making use of this new LIU4K dataset in the coming future.
JPEG [ARCNN] [RED-Net] [DnCNN] [Li ICME’17] [MemNet] [MWCNN] [CAR-DRN] [LIU4K]
JPEG-HDR [Han VCIP’20]
HEVC [Lin DCC’16] [IFCNN] [VRCNN] [DCAD] [MMS-net] [DRN] [Lee ICCE’18] [DS-CNN] [CNNF] [RHCNN] [VRCNN-ext] [S-CNN & C-CNN] [MLSDRN] [ARTN] [Double-Input CNN] [CNNIF & CNNMC] [B-DRRN] [Residual-VRN] [Liu PCS’19] [DIA_Net] [RRCNN] [QE-CNN] [MRRN] [Jia TIP’19] [EDCNN] [VRCNN-BN] [MACNN] [Yue VCIP’20] [SEFCNN] [LIU4K]
AVS3 [Lin PCS’19] [CNNLF]
VVC [AResNet] [Lu CVPRW’19] [Wang APSIPA ASC’19] [ADCNN] [PRN] [DRCNN] [Zhang ICME’20] [MGNLF] [RCAN+PRN+] [Nasiri VCIP’20]