Metric Learning with Adaptive Density Discrimination
Distance metric learning (DML) approaches learn a transformation to a representation space where distance is in correspondence with a predefined notion of similarity. While such models offer a number of compelling benefits, it has been difficult for these to compete with modern classification algori...
Main Authors: | , , , |
---|---|
Format: | Text |
Language: | unknown |
Published: |
2015
|
Subjects: | |
Online Access: | http://arxiv.org/abs/1511.05939 |
id |
ftarxivpreprints:oai:arXiv.org:1511.05939 |
---|---|
record_format |
openpolar |
spelling |
ftarxivpreprints:oai:arXiv.org:1511.05939 2023-09-05T13:19:06+02:00 Metric Learning with Adaptive Density Discrimination Rippel, Oren Paluri, Manohar Dollar, Piotr Bourdev, Lubomir 2015-11-18 http://arxiv.org/abs/1511.05939 unknown http://arxiv.org/abs/1511.05939 Statistics - Machine Learning Computer Science - Machine Learning text 2015 ftarxivpreprints 2023-08-16T13:49:37Z Distance metric learning (DML) approaches learn a transformation to a representation space where distance is in correspondence with a predefined notion of similarity. While such models offer a number of compelling benefits, it has been difficult for these to compete with modern classification algorithms in performance and even in feature extraction. In this work, we propose a novel approach explicitly designed to address a number of subtle yet important issues which have stymied earlier DML algorithms. It maintains an explicit model of the distributions of the different classes in representation space. It then employs this knowledge to adaptively assess similarity, and achieve local discrimination by penalizing class distribution overlap. We demonstrate the effectiveness of this idea on several tasks. Our approach achieves state-of-the-art classification results on a number of fine-grained visual recognition datasets, surpassing the standard softmax classifier and outperforming triplet loss by a relative margin of 30-40%. In terms of computational performance, it alleviates training inefficiencies in the traditional triplet loss, reaching the same error in 5-30 times fewer iterations. Beyond classification, we further validate the saliency of the learnt representations via their attribute concentration and hierarchy recovery properties, achieving 10-25% relative gains on the softmax classifier and 25-50% on triplet loss in these tasks. Comment: ICLR 2016 Text DML ArXiv.org (Cornell University Library) |
institution |
Open Polar |
collection |
ArXiv.org (Cornell University Library) |
op_collection_id |
ftarxivpreprints |
language |
unknown |
topic |
Statistics - Machine Learning Computer Science - Machine Learning |
spellingShingle |
Statistics - Machine Learning Computer Science - Machine Learning Rippel, Oren Paluri, Manohar Dollar, Piotr Bourdev, Lubomir Metric Learning with Adaptive Density Discrimination |
topic_facet |
Statistics - Machine Learning Computer Science - Machine Learning |
description |
Distance metric learning (DML) approaches learn a transformation to a representation space where distance is in correspondence with a predefined notion of similarity. While such models offer a number of compelling benefits, it has been difficult for these to compete with modern classification algorithms in performance and even in feature extraction. In this work, we propose a novel approach explicitly designed to address a number of subtle yet important issues which have stymied earlier DML algorithms. It maintains an explicit model of the distributions of the different classes in representation space. It then employs this knowledge to adaptively assess similarity, and achieve local discrimination by penalizing class distribution overlap. We demonstrate the effectiveness of this idea on several tasks. Our approach achieves state-of-the-art classification results on a number of fine-grained visual recognition datasets, surpassing the standard softmax classifier and outperforming triplet loss by a relative margin of 30-40%. In terms of computational performance, it alleviates training inefficiencies in the traditional triplet loss, reaching the same error in 5-30 times fewer iterations. Beyond classification, we further validate the saliency of the learnt representations via their attribute concentration and hierarchy recovery properties, achieving 10-25% relative gains on the softmax classifier and 25-50% on triplet loss in these tasks. Comment: ICLR 2016 |
format |
Text |
author |
Rippel, Oren Paluri, Manohar Dollar, Piotr Bourdev, Lubomir |
author_facet |
Rippel, Oren Paluri, Manohar Dollar, Piotr Bourdev, Lubomir |
author_sort |
Rippel, Oren |
title |
Metric Learning with Adaptive Density Discrimination |
title_short |
Metric Learning with Adaptive Density Discrimination |
title_full |
Metric Learning with Adaptive Density Discrimination |
title_fullStr |
Metric Learning with Adaptive Density Discrimination |
title_full_unstemmed |
Metric Learning with Adaptive Density Discrimination |
title_sort |
metric learning with adaptive density discrimination |
publishDate |
2015 |
url |
http://arxiv.org/abs/1511.05939 |
genre |
DML |
genre_facet |
DML |
op_relation |
http://arxiv.org/abs/1511.05939 |
_version_ |
1776199910777946112 |