Distance Metric Learning Using Dropout: A Structured Regularization Approach

• Published in: Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining
• Main Authors: Qian, Qi, Hu, Juhua, Jin, Rong, Pei, Jian, Zhu, Shenghuo
• Format: Text
• Language: unknown
• Published: UW Tacoma Digital Commons 2014
• Subjects: distance metric learning; dropout; DML
• Online Access: https://digitalcommons.tacoma.uw.edu/tech_pub/205; https://doi.org/10.1145/2623330.2623678

Distance metric learning (DML) aims to learn a distance metric better than Euclidean distance. It has been successfully applied to various tasks, e.g., classification, clustering and information retrieval. Many DML algorithms suffer from the over-fitting problem because of a large number of parameters to be determined in DML. In this paper, we exploit the dropout technique, which has been successfully applied in deep learning to alleviate the over-fitting problem, for DML. Different from the previous studies that only apply dropout to training data, we apply dropout to both the learned metrics and the training data. We illustrate that application of dropout to DML is essentially equivalent to matrix norm based regularization. Compared with the standard regularization scheme in DML, dropout is advantageous in simulating the structured regularizers which have shown consistently better performance than non structured regularizers. We verify, both empirically and theoretically, that dropout is effective in regulating the learned metric to avoid the over-fitting problem. Last, we examine the idea of wrapping the dropout technique in the state-of-art DML methods and observe that the dropout technique can significantly improve the performance of the original DML methods.