Metric Learning Driven Multi-Task Structured Output Optimization for Robust Keypoint Tracking

Abstract:

As an important and challenging problem in computer vision and graphics, keypoint-based object tracking is typically formulated in a spatio-temporal statistical learning framework. However, most existing keypoint trackers are incapable of effectively modeling and balancing the following three aspects in a simultaneous manner: temporal model coherence across frames, spatial model consistency within frames, and discriminative feature construction. To address this issue, we propose a robust keypoint tracker based on spatio-temporal multi-task structured output optimization driven by discriminative metric learning. Consequently, temporal model coherence is characterized by multi-task structured keypoint model learning over several adjacent frames, while spatial model consistency is modeled by solving a geometric verification based structured learning problem. Discriminative feature construction is enabled by metric learning to ensure the intra-class compactness and inter-class separability. Finally, the above three modules are simultaneously optimized in a joint learning scheme. Experimental results have demonstrated the effectiveness of our tracker.

Publication:

Liming Zhao, Xi Li, Jun Xiao, Fei Wu, and Yueting Zhuang. "Metric Learning Driven Multi-Task Structured Output Optimization for Robust Keypoint Tracking." In Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence (AAAI), 2015. [paper] [supp] [slides] [dataset (google drive)] [dataset (baidu disk)]

(Full Oral Presentation, Acceptance Rate: 238/1991=11.95%)

Corresponding Author: Prof. Xi Li

@inproceedings{SMM_tracker, title = {Metric Learning Driven Multi-Task Structured Output Optimization for Robust Keypoint Tracking}, author = {Liming Zhao and Xi Li and Jun Xiao and Fei Wu and Yueting Zhuang}, conference = {Proceedings of the 29th {AAAI} Conference on Artificial Intelligence}, year = {2015}, pages = {3864-3870}, month = {Jan}, }

Figure 1: Illustration of our tracking approach