Self-Paced Learning with Adaptive Deep Visual Embeddings
Selecting the most appropriate data examples to present a deep neural network (DNN) at different stages of training is an unsolved challenge. Though practitioners typically ignore this problem, a non-trivial data scheduling method may result in a significant improvement in both convergence and gener...
Main Authors: | , |
---|---|
Format: | Report |
Language: | unknown |
Published: |
arXiv
2018
|
Subjects: | |
Online Access: | https://dx.doi.org/10.48550/arxiv.1807.09200 https://arxiv.org/abs/1807.09200 |
id |
ftdatacite:10.48550/arxiv.1807.09200 |
---|---|
record_format |
openpolar |
spelling |
ftdatacite:10.48550/arxiv.1807.09200 2023-05-15T16:02:00+02:00 Self-Paced Learning with Adaptive Deep Visual Embeddings Thangarasa, Vithursan Taylor, Graham W. 2018 https://dx.doi.org/10.48550/arxiv.1807.09200 https://arxiv.org/abs/1807.09200 unknown arXiv arXiv.org perpetual, non-exclusive license http://arxiv.org/licenses/nonexclusive-distrib/1.0/ Machine Learning cs.LG Computer Vision and Pattern Recognition cs.CV Machine Learning stat.ML FOS Computer and information sciences Preprint Article article CreativeWork 2018 ftdatacite https://doi.org/10.48550/arxiv.1807.09200 2022-04-01T09:19:22Z Selecting the most appropriate data examples to present a deep neural network (DNN) at different stages of training is an unsolved challenge. Though practitioners typically ignore this problem, a non-trivial data scheduling method may result in a significant improvement in both convergence and generalization performance. In this paper, we introduce Self-Paced Learning with Adaptive Deep Visual Embeddings (SPL-ADVisE), a novel end-to-end training protocol that unites self-paced learning (SPL) and deep metric learning (DML). We leverage the Magnet Loss to train an embedding convolutional neural network (CNN) to learn a salient representation space. The student CNN classifier dynamically selects similar instance-level training examples to form a mini-batch, where the easiness from the cross-entropy loss and the true diverseness of examples from the learned metric space serve as sample importance priors. To demonstrate the effectiveness of SPL-ADVisE, we use deep CNN architectures for the task of supervised image classification on several coarse- and fine-grained visual recognition datasets. Results show that, across all datasets, the proposed method converges faster and reaches a higher final accuracy than other SPL variants, particularly on fine-grained classes. : Published as a conference paper at BMVC 2018 Report DML DataCite Metadata Store (German National Library of Science and Technology) |
institution |
Open Polar |
collection |
DataCite Metadata Store (German National Library of Science and Technology) |
op_collection_id |
ftdatacite |
language |
unknown |
topic |
Machine Learning cs.LG Computer Vision and Pattern Recognition cs.CV Machine Learning stat.ML FOS Computer and information sciences |
spellingShingle |
Machine Learning cs.LG Computer Vision and Pattern Recognition cs.CV Machine Learning stat.ML FOS Computer and information sciences Thangarasa, Vithursan Taylor, Graham W. Self-Paced Learning with Adaptive Deep Visual Embeddings |
topic_facet |
Machine Learning cs.LG Computer Vision and Pattern Recognition cs.CV Machine Learning stat.ML FOS Computer and information sciences |
description |
Selecting the most appropriate data examples to present a deep neural network (DNN) at different stages of training is an unsolved challenge. Though practitioners typically ignore this problem, a non-trivial data scheduling method may result in a significant improvement in both convergence and generalization performance. In this paper, we introduce Self-Paced Learning with Adaptive Deep Visual Embeddings (SPL-ADVisE), a novel end-to-end training protocol that unites self-paced learning (SPL) and deep metric learning (DML). We leverage the Magnet Loss to train an embedding convolutional neural network (CNN) to learn a salient representation space. The student CNN classifier dynamically selects similar instance-level training examples to form a mini-batch, where the easiness from the cross-entropy loss and the true diverseness of examples from the learned metric space serve as sample importance priors. To demonstrate the effectiveness of SPL-ADVisE, we use deep CNN architectures for the task of supervised image classification on several coarse- and fine-grained visual recognition datasets. Results show that, across all datasets, the proposed method converges faster and reaches a higher final accuracy than other SPL variants, particularly on fine-grained classes. : Published as a conference paper at BMVC 2018 |
format |
Report |
author |
Thangarasa, Vithursan Taylor, Graham W. |
author_facet |
Thangarasa, Vithursan Taylor, Graham W. |
author_sort |
Thangarasa, Vithursan |
title |
Self-Paced Learning with Adaptive Deep Visual Embeddings |
title_short |
Self-Paced Learning with Adaptive Deep Visual Embeddings |
title_full |
Self-Paced Learning with Adaptive Deep Visual Embeddings |
title_fullStr |
Self-Paced Learning with Adaptive Deep Visual Embeddings |
title_full_unstemmed |
Self-Paced Learning with Adaptive Deep Visual Embeddings |
title_sort |
self-paced learning with adaptive deep visual embeddings |
publisher |
arXiv |
publishDate |
2018 |
url |
https://dx.doi.org/10.48550/arxiv.1807.09200 https://arxiv.org/abs/1807.09200 |
genre |
DML |
genre_facet |
DML |
op_rights |
arXiv.org perpetual, non-exclusive license http://arxiv.org/licenses/nonexclusive-distrib/1.0/ |
op_doi |
https://doi.org/10.48550/arxiv.1807.09200 |
_version_ |
1766397650181029888 |