Learning color and locality cues for moving object detection and segmentation

Feng Liu and Michael Gleicher

Computer Sciences Department, University of Wisconsin-Madison

Abstract
This paper presents an algorithm for automatically detecting and segmenting a moving object from a monocular video. Detecting and segmenting a moving object from a video with limited object motion is challenging. Since existing automatic algorithms rely on motion to detect the moving object, they cannot work well when the object motion is sparse and insufficient. In this paper, we present an unsupervised algorithm to learn object color and locality cues from the sparse motion information. We first detect key frames with reliable motion cues and then estimate moving sub-objects based on these motion cues using a Markov Random Field (MRF) framework. From these sub-objects, we learn an appearance model as a color Gaussian Mixture Model. To avoid the false classification of background pixels with similar color to the moving objects, the locations of these sub-objects are propagated to neighboring frames as locality cues. Finally, robust moving object segmentation is achieved by combining these learned color and locality cues with motion cues in a MRF framework. Experiments on videos with a variety of object and camera motion demonstrate the effectiveness of this algorithm.
A working example of our algorithm. Step 1: Extract key frames (a) with reliable motion cues (b); Step 2: Estimate moving sub-objects (c) in key frames using motion cues; Step 3: Learn color and locality cues from these moving sub-objects. (d) shows the likelihood of each pixel belonging to the moving object based on the color and motion cues. Step 4: Segment the moving object using color, locality and motion cues. The final results are shown in (e).
Paper
Feng Liu and Michael Gleicher. Learning color and locality cues for moving object detection and segmentation. 
IEEE CVPR 2009, Miami Beach, Florida, USA, June, 2009. pp. 320-327. PDF
Demo
download it here
watch it on Youtube or below
Search engine friendly content