Research‎ > ‎

Metric Learning Driven Multi-Task Structured Output Optimization for Robust Keypoint Tracking

Abstract:

As an important and challenging problem in computer vision and graphics, keypoint-based object tracking is typically formulated in a spatio-temporal statistical learning framework. However, most existing keypoint trackers are incapable of effectively modeling and balancing the following three aspects in a simultaneous manner: temporal model coherence across frames, spatial model consistency within frames, and discriminative feature construction. To address this issue, we propose a robust keypoint tracker based on spatio-temporal multi-task structured output optimization driven by discriminative metric learning. Consequently, temporal model coherence is characterized by multi-task structured keypoint model learning over several adjacent frames, while spatial model consistency is modeled by solving a geometric verification based structured learning problem. Discriminative feature construction is enabled by metric learning to ensure the intra-class compactness and inter-class separability. Finally, the above three modules are simultaneously optimized in a joint learning scheme. Experimental results have demonstrated the effectiveness of our tracker.

Figure 1: Illustration of our tracking approach


Figure 2: Experimental results

Publication:

    Liming Zhao, Xi Li, Jun Xiao, Fei Wu, and Yueting Zhuang. "Metric Learning Driven Multi-Task Structured Output Optimization for Robust Keypoint Tracking." In Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence (AAAI), 2015. [paper] [supp] [slides]
(Full Oral Presentation, Acceptance Rate: 238/1991=11.95%)
Corresponding Author: Prof. Xi Li

@inproceedings{SMM_tracker,
  title = {Metric Learning Driven Multi-Task Structured Output Optimization for Robust Keypoint Tracking},
  author = {Liming Zhao and Xi Li and Jun Xiao and Fei Wu and Yueting Zhuang},
  conference = {Proceedings of the 29th {AAAI} Conference on Artificial Intelligence},
  year = {2015},
  pages = {3864-3870},
  month = {Jan},
}

Demo Video:

YouTube

Dataset:

chart
keyboard
food
book
(The following 5 sequences are from SamHare)
barbapapa
comic
map
paper
phone

Executable File:

SMM_Tracker.zip  (Executable binary code is available for Windows.)