Reading: Baig JVICU’17 — Multiple Hypothesis Colorization and Its Application to Image Compression (JPEG)

Outperforms JPEG By a Large Margin

Bit Savings by Chroma Colorization
  • First, while colorization for artistic purposes simply involves predicting plausible chroma, colorization for compression requires generating output colors that are as close as possible to the ground truth with only grayscale image as input.
  • Second, many objects in the real world exhibit multiple possible colors. Thus, in order to disambiguate the colorization problem some additional information must be stored to reproduce the true colors with good accuracy.
  • Finally, the proposed approach outperforms JPEG color coding by a large margin as shown above.


  1. Proposed Framework
  2. Experimental Results

1. Proposed Framework

Proposed Framework
  • The proposed network architecture predicts K color hypotheses per pixel by inputting the grayscale image.
  • This is achieved by means of a CNN whose single trunk splits at a certain depth into K distinct branches, each outputting a 2-channel color output per pixel. All layers, both in the trunk as well as in the K branches, are fully convolutional.
K-Branch Variants on CIFAR100
  • K=5 has the best performance.
  • The network is based on ResNet, which is shown in the appendix of the paper.

1.1. Branch Index

  • In order to be decodable, a simple solution is to store an index of the branch that best approximates the ground truth color for every pixel.
  • Then at the decoding time, the grayscale version of the image goes through the network to produce the K color outputs and use the stored branch indices to select the appropriate branch for every pixel.

1.2. Superpixels for Branch Index

Superpixels for Branch Index
  • Oracle one is treated as the ideal case.
  • Two ways are considered to define regions for compression:
  • (1) a subdivision of the image into a grid of fixed-size square cells, and
  • (2) a segmentation of the grayscale photo into superpixels using traditional bottom-up segmentation methods.

1.3. Global Correction

  • The global correction corrects for a similarity transform (scale and translation) between the ground truth color and the estimate.
  • In a compression scenario, These 4 global parameters (2 for each color channel) are simply stored to produce more accurate color reconstructions.

2. Experimental Results

2.1. RD Curves

RD Curves
  • The proposed one using superpixels (Red) outperforms JPEG (Blue).

2.2. Visualizations

  • With low cost superpixel encoding, the reconstructed colors are close to the original one.
SOTA Comparison
  • From the above figure, we can see that the bytes used by the proposed approach are much fewer than [12] and [13], but without any obvious quality degradation.



Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store