DeepSaliency: Multi-Task Deep Neural Network Model for Salient Object Detection

Abstract

A key problem in salient object detection is how to effectively model the semantic properties of salient objects in a data-driven manner. In this paper, we propose a multi-task deep saliency model based on a fully convolutional neural network (FCNN) with global input (whole raw images) and global output (whole saliency maps). In principle, the proposed saliency model takes a data-driven strategy for encoding the underlying saliency prior information, and then sets up a multi-task learning scheme for exploring the intrinsic correlations between saliency detection and semantic image segmentation. Through collaborative feature learning from such two correlated tasks, the shared fully convolutional layers produce effective features for object perception. Moreover, it is capable of capturing the semantic information on salient objects across different levels using the fully convolutional layers, which investigate the feature-sharing properties of salient object detection with great feature redundancy reduction. Finally, we present a graph Laplacian regularized nonlinear regression model for saliency refinement. Experimental results demonstrate the effectiveness of our approach in comparison with the state-of-the-art approaches

Fig. 2: Architecture of the proposed fully convolutional neural network for training. The FCNN scheme carries out the task of saliency detection in conjunction with the task of object class segmentation, which share a convolution part with 15 layers. The segmentation task seeks for the intrinsic object semantic information of the image, while the saliency task aims to find the salient objects. For the saliency detection task, we only use the saliency-related network for testing.

Experiments:

Fig. 3: Precision-recall curves of different saliency detection methods on 8 benchmark datasets. Overall, the proposed approach performs well with higher precision in the case of a fixed recall.

Fig. 4: Comparison of average F-measure using adaptive threshold (aveF), maximum F-measure of average precision recall curve (maxF), AUC scores and MAE scores (smaller better). Our approach (OurLO) achieves the best performance in all these metrics. The results from the last three columns are directly quoted from their original papers. Since using THUS for training, the saliency results of OurBR are null.

Fig. 5: Qualitative comparison of different approaches on several challenging samples with ground truth (GT). Clearly, our approach obtains more visually feasible saliency detection results than the comparison approaches.