Revisiting Training Strategies and Generalization Performance in Deep Metric Learning

Deep Metric Learning (DML) is arguably one of the most influential lines of research for learning visual similarities with many proposed approaches every year. Although the field benefits from the rapid progress, the divergence in training protocols, architectures, and parameter choices make an unbi...

Full description

Bibliographic Details
Main Authors: Roth, Karsten, Milbich, Timo, Sinha, Samarth, Gupta, Prateek, Ommer, Björn, Cohen, Joseph Paul
Format: Text
Language:unknown
Published: 2020
Subjects:
Computer Science - Computer Vision and Pattern Recognition
DML
Online Access:http://arxiv.org/abs/2002.08473
id ftarxivpreprints:oai:arXiv.org:2002.08473
record_format openpolar
spelling ftarxivpreprints:oai:arXiv.org:2002.08473 2023-09-05T13:19:04+02:00 Revisiting Training Strategies and Generalization Performance in Deep Metric Learning Roth, Karsten Milbich, Timo Sinha, Samarth Gupta, Prateek Ommer, Björn Cohen, Joseph Paul 2020-02-19 http://arxiv.org/abs/2002.08473 unknown http://arxiv.org/abs/2002.08473 Computer Science - Computer Vision and Pattern Recognition text 2020 ftarxivpreprints 2023-08-16T15:44:48Z Deep Metric Learning (DML) is arguably one of the most influential lines of research for learning visual similarities with many proposed approaches every year. Although the field benefits from the rapid progress, the divergence in training protocols, architectures, and parameter choices make an unbiased comparison difficult. To provide a consistent reference point, we revisit the most widely used DML objective functions and conduct a study of the crucial parameter choices as well as the commonly neglected mini-batch sampling process. Under consistent comparison, DML objectives show much higher saturation than indicated by literature. Further based on our analysis, we uncover a correlation between the embedding space density and compression to the generalization performance of DML models. Exploiting these insights, we propose a simple, yet effective, training regularization to reliably boost the performance of ranking-based DML models on various standard benchmark datasets. Code and a publicly accessible WandB-repo are available at https://github.com/Confusezius/Revisiting_Deep_Metric_Learning_PyTorch. Comment: ICML 2020. Main paper 8.25 pages, 26 pages total Text DML ArXiv.org (Cornell University Library)
institution Open Polar
collection ArXiv.org (Cornell University Library)
op_collection_id ftarxivpreprints
language unknown
topic Computer Science - Computer Vision and Pattern Recognition
spellingShingle Computer Science - Computer Vision and Pattern Recognition
Roth, Karsten
Milbich, Timo
Sinha, Samarth
Gupta, Prateek
Ommer, Björn
Cohen, Joseph Paul
Revisiting Training Strategies and Generalization Performance in Deep Metric Learning
topic_facet Computer Science - Computer Vision and Pattern Recognition
description Deep Metric Learning (DML) is arguably one of the most influential lines of research for learning visual similarities with many proposed approaches every year. Although the field benefits from the rapid progress, the divergence in training protocols, architectures, and parameter choices make an unbiased comparison difficult. To provide a consistent reference point, we revisit the most widely used DML objective functions and conduct a study of the crucial parameter choices as well as the commonly neglected mini-batch sampling process. Under consistent comparison, DML objectives show much higher saturation than indicated by literature. Further based on our analysis, we uncover a correlation between the embedding space density and compression to the generalization performance of DML models. Exploiting these insights, we propose a simple, yet effective, training regularization to reliably boost the performance of ranking-based DML models on various standard benchmark datasets. Code and a publicly accessible WandB-repo are available at https://github.com/Confusezius/Revisiting_Deep_Metric_Learning_PyTorch. Comment: ICML 2020. Main paper 8.25 pages, 26 pages total
format Text
author Roth, Karsten
Milbich, Timo
Sinha, Samarth
Gupta, Prateek
Ommer, Björn
Cohen, Joseph Paul
author_facet Roth, Karsten
Milbich, Timo
Sinha, Samarth
Gupta, Prateek
Ommer, Björn
Cohen, Joseph Paul
author_sort Roth, Karsten
title Revisiting Training Strategies and Generalization Performance in Deep Metric Learning
title_short Revisiting Training Strategies and Generalization Performance in Deep Metric Learning
title_full Revisiting Training Strategies and Generalization Performance in Deep Metric Learning
title_fullStr Revisiting Training Strategies and Generalization Performance in Deep Metric Learning
title_full_unstemmed Revisiting Training Strategies and Generalization Performance in Deep Metric Learning
title_sort revisiting training strategies and generalization performance in deep metric learning
publishDate 2020
url http://arxiv.org/abs/2002.08473
genre DML
genre_facet DML
op_relation http://arxiv.org/abs/2002.08473
_version_ 1776199881697787904

Similar Items