Saliency Aggregation: A Data-driven Approach

Long Mai, Yuzhen Niu, and Feng Liu

Computer Science Department, Portland State University

Saliency aggregation. Individual saliency methods, such as GC [6], FT [2], and CA [12], often complement each other. Saliency aggregation can effectively combine their results and perform better than each of them.

Abstract

A variety of methods have been developed for visual saliency analysis. These methods often complement each other. This paper addresses the problem of aggregating various saliency analysis methods such that the aggregation result outperforms each individual one. We have two major observations. First, different methods perform differently in saliency analysis. Second, the performance of a saliency analysis method varies with individual images. Our idea is to use data-driven approaches to saliency aggregation that appropriately consider the performance gaps among individual methods and the performance dependence of each method on individual images. This paper discusses various data-driven approaches and finds that the image-dependent aggregation method works best. Specifically, our method uses a Conditional Random Field (CRF) framework for saliency aggregation that not only models the contribution from individual saliency map but also the interaction between neighboring pixels. To account for the dependence of aggregation on an individual image, our approach selects a subset of images similar to the input image from a training data set and trains the CRF aggregation model only using this subset instead of the whole training set. Our experiments on public saliency benchmarks show that our aggregation method outperforms each individual saliency method and is robust with the selection of aggregated methods.

Paper

Long Mai, Yuzhen Niu, and Feng Liu. Saliency Aggregation: A Data-driven Approach.  IEEE CVPR 2013. PDF Related Paper Long Mai and Feng Liu. Comparing Salient Object Detection Results without Ground Truth.  ECCV 2014. Yuzhen Niu, Yujie Geng, Xueqing Li, and Feng Liu. Leveraging Stereopsis for Saliency Analysis.  IEEE CVPR 2012. Project  PDF

Experiment Data Link

This dataset includes the original FT saliency benchmark and our results. Specifically, it contains 1000 images (image_<ID>.jpg), 1000 ground-truth saliency mask (mask_<ID>.jpg), and the saliency maps from our image-dependent saliency aggregation method (CRF_GIST_map_<ID>), with <ID> range from 1 to 1000. The original FT dataset can be obtained following the instructions on http://ivrgwww.epfl.ch/supplementary_material/RK_CVPR09/