Quantization table design revisited for image/video coding

TLDR: This work proposes an efficient statistical-model-based algorithm using the Laplacian model to design quantization tables for JPEG encoding and shows that a quantization table can be optimized in a way that the resulting distortion complies with certain behavior.

Abstract: Quantization table design is revisited for image/video coding where soft decision quantization (SDQ) is considered. Unlike conventional approaches where quantization table design is bundled with a specific encoding method, we assume optimal SDQ encoding and design a quantization table for the purpose of reconstruction. Under this assumption, we model transform coefficients across different frequencies as independently distributed random sources and apply the Shannon lower bound to approximate the rate distortion function of each source. We then show that a quantization table can be optimized in a way that the resulting distortion complies with certain behavior. Lastly, guided by this new theoretical result, we propose an efficient statistical-model-based algorithm using the Laplacian model to design quantization tables for JPEG encoding. Compared with the state of the art, the proposed algorithm provides an average 0.5 dB gain in PSNR with computational complexity reduced by a factor of more than 2000 when SDQ is off, and a 0.1 dB performance gain with 85% of the complexity reduced when SDQ is on.