Submitted to IEEE Trans. Circuits and Systems for Video Technology


Multiscale Saliency Detection
Using Random Walk with Restart

Supplementary Materials


Jun-Seong Kim     Jae-Young Sim     Chang-Su Kim
junssi153@korea.ac.kr     jysim@unist.ac.kr     changsukim@korea.ac.kr
Korea University     UNIST     Korea University


Abstract

In this work, we propose a graph-based multiscale saliency detection algorithm, by modeling eye movements as a random walk on a graph. The proposed algorithm first extracts intensity, color, and compactness features from an input image. It then constructs a fully connected graph by employing image blocks as the nodes. It assigns a high edge weight, if the two connected nodes have dissimilar intensity and color features and if the ending node is more compact than the starting node. Then, the proposed algorithm computes the stationary distribution of the Markov chain on the graph as the saliency map. However, the performance of the saliency detection depends on the relative block size in an image. To provide a more reliable saliency map, we develop a coarse-to-fine refinement technique for multiscale saliency maps based on the random walk with restart (RWR). Specifically, we use the saliency map at a coarse scale as the restarting distribution of RWR at a fine scale. Experimental results demonstrate that the proposed algorithm detects visual saliency more precisely and more reliably than conventional algorithms. Moreover, the proposed algorithm can be efficiently used in the applications of proto-object extraction and image retargeting.




Comparison with the Conventional Algorithms

1. MSRA dataset [1] (150 images)
Input image STB [2] SR [3] PQFT [4] LDS [1] FT [5] GC [6] CA [7] STSD [8] GBVS [9] RW [10,11] Proposed
 
2. MIT dataset [12] (30 images)
Input image STB [2] SR [3] PQFT [4] LDS [1] FT [5] GC [6] CA [7] STSD [8] GBVS [9] RW [10,11] Proposed
 
3. FIFA dataset [13] (10 images)
Input image STB [2] SR [3] PQFT [4] LDS [1] FT [5] GC [6] CA [7] STSD [8] GBVS [9] RW [10,11] Proposed
 
4. TORONTO dataset [14] (10 images)
Input image STB [2] SR [3] PQFT [4] LDS [1] FT [5] GC [6] CA [7] STSD [8] GBVS [9] RW [10,11] Proposed

References

[1] T. Liu, J. Sun, N.-N. Zheng, X. Tang, and H.-Y. Shum, "Learning to detect a salient object," in Proc. IEEE CVPR, pp. 1–8.
[2] D. Walther and C. Koch, "Modeling attention to salient proto-objects," Neural Netw, vol. 19, pp. 1395–1407, 2006.
[3] X. Hou and L. Zhang, "Saliency detection: A spectral residual approach," in Proc. IEEE CVPR, pp. 1–8.
[4] C. Guo, Q. Ma, and L. Zhang, "Spatio-temporal saliency using phase spectrum of quaternion Fourier transform," in Proc. IEEE CVPR, 2008, pp. 1–8.
[5] R. Achanta, S. Hemami, F. Estrada, and S. S¨usstrunk, "Frequency-tuned salient region detection," in Proc. IEEE CVPR, 2009, pp. 1597–1604.
[6] M.-M. Cheng, G.-X. Zhang, N. J. Mitra, X. Huang, and S.-M. Hu, "Global contrast based salient region detection," in Proc. IEEE CVPR, 2011, pp. 409–416.
[7] S. Goferman, L. Zelnik-Manor, and A. Tal, "Context-aware saliency detection," in Proc. IEEE CVPR, 2010, pp. 2376–2383.
[8] W. Kim, C. Jung, and C. Kim, "Spatiotemporal saliency detection and its applications in static and dynamic scenes," IEEE Trans. Circuits Syst. Video Technol., vol. 21, no. 4, pp. 446–456, Apr. 2011.
[9] J. Harel, C. Koch, and P. Perona, "Graph-based visual saliency," in Proc. Adv. Neural Inf. Process. Syst, 2006, pp. 545–552.
[10] V. Gopalakrishnan, Y. Hu, and D. Rajan, "Random walks on graphs to model saliency in images," in Proc. IEEE CVPR, 2009, pp. 1698–1705.
[11] V. Gopalakrishnan, Y. Hu, and D. Rajan, "Random walks on graphs for salient object detection in images," IEEE Trans. Image Process., vol. 19, no. 12, pp. 3232–3242, Dec. 2010.
[12] T. Judd, K. Ehinger, F. Durand, and A. Torralba, "Learning to predict where humans look," in IEEE International Conference on Computer Vision (ICCV), 2009.
[13] M. Cerf, E. P. Frady, and C. Koch, "Faces and text attract gaze independent of the task: Experimental data and computer model," J. Vision, vol. 9, no. 12, pp. 1–15, Nov. 2009.
[14] N. D. B. Bruce and J. K. Tsotsos, "Saliency based on information maximization," in Proc. Adv. Neural Inf. Process. Syst, 2005.