Objective Quality Assessment for Image Retargeting

Chih-Chung Hsu Chia-Wen Lin Yuming Fang Weisi Lin

NTHU NVLab, Taiwan
m121754@gmail.com

NTHU NVLab, Taiwan
cwlin@ee.nthu.edu.tw

NTU, Singapore
leo.fangyuming@gmail.com

NTU, Singapore
wslin@ntu.edu.sg

 

 

Image retargeting techniques aim to obtain retargeted images with different sizes or aspect ratios for various display screens. Various content-aware image retargeting algorithms have been proposed recently. However, there is still no effective objective metric for visual quality assessment of retargeted images. In this paper, we propose a novel full-reference objective metric for assessing visual quality of a retargeted image based on perceptual geometric distortion and information loss. The proposed metric measures the geometric distortion of retargeted images based on the local variance of SIFT flow vector fields. Furthermore, a visual saliency map is derived to characterize human perception of the geometric distortion. Besides, the information loss in a retargeted image, which is estimated based on the saliency map, is also taken into account in the proposed metric. Subjective tests are conducted to evaluate the performance of the proposed metric. Experimental results show the good consistency between the proposed objective metric and the subjective rankings.

Code Updated (2015/1/5)!

Menu

  1. Data & code
  2. Algorithm
  3. Results


Data & code

Code
  1. Our metric code (in Matlab)
  2. Other metrics
  1. Image retargeting code
Manuscript (Available soon)
  1. Manuscript (PDF)

 


Algorithm


Which one is better?

SIFT Flow estimation: dense correspondence estimation using method in [1].
Saliency detection: saleicny map estimation using method in [2].

PGD measurement: flow field analysis for geometric distortion with human visual system.
SLR measurement: information loss estimation by calculating the ratio between the saliency map of the original image and that of retargeted one.



Experimental Results
Rank correlation comparison for PGD, SLR, and both.


 

Rank correlation comparison for the proposed metric and others.


 

Visualized Results

Fig.4 (a) The original image, and the visualized perceptual distortion maps obtained from (b) multi-operator method, (c) seam carving, (d) Shift-map, and (e) warping method. Their overall quality indices are qtotal = 0.88, 0.42, 0.8, and 0.65, respectively.



Reference

[1] C. Liu, J. Yuen, and A. Torralba, “Dense correspondence across scenes and its applications,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 5, pp. 978-994, May 2011.
[2] Y. Fang, W. Lin, Z. Chen, and C.-W. Lin, “Saliency detection in the compressed domain for adaptive image retargeting,” IEEE Trans. Image Process. vol. 21, no. 9, pp. 3888-3901, Sept. 2012.