Deep Multimodal Distance Metric Learning Using Click Constraints for Image Ranking
© 2013 IEEE. How do we retrieve images accurately? Also, how do we rank a group of images precisely and efficiently for specific queries? These problems are critical for researchers and engineers to generate a novel image searching engine. First, it is important to obtain an appropriate description...
Main Authors: | , , , |
---|---|
Format: | Article in Journal/Newspaper |
Language: | unknown |
Published: |
2017
|
Subjects: | |
Online Access: | http://hdl.handle.net/10453/123820 |
id |
ftunivtsydney:oai:opus.lib.uts.edu.au:10453/123820 |
---|---|
record_format |
openpolar |
spelling |
ftunivtsydney:oai:opus.lib.uts.edu.au:10453/123820 2023-05-15T16:01:48+02:00 Deep Multimodal Distance Metric Learning Using Click Constraints for Image Ranking Yu, J Yang, X Gao, F Tao, D 2017-12-01 application/pdf http://hdl.handle.net/10453/123820 unknown http://purl.org/au-research/grants/arc/FT130101457 http://purl.org/au-research/grants/arc/DP140102164 http://purl.org/au-research/grants/arc/LE140100061 IEEE Transactions on Cybernetics 10.1109/TCYB.2016.2591583 IEEE Transactions on Cybernetics, 2017, 47 (12), pp. 4014 - 4024 2168-2267 http://hdl.handle.net/10453/123820 Artificial Intelligence & Image Processing Journal Article 2017 ftunivtsydney 2022-03-13T13:45:15Z © 2013 IEEE. How do we retrieve images accurately? Also, how do we rank a group of images precisely and efficiently for specific queries? These problems are critical for researchers and engineers to generate a novel image searching engine. First, it is important to obtain an appropriate description that effectively represent the images. In this paper, multimodal features are considered for describing images. The images unique properties are reflected by visual features, which are correlated to each other. However, semantic gaps always exist between images visual features and semantics. Therefore, we utilize click feature to reduce the semantic gap. The second key issue is learning an appropriate distance metric to combine these multimodal features. This paper develops a novel deep multimodal distance metric learning (Deep-MDML) method. A structured ranking model is adopted to utilize both visual and click features in distance metric learning (DML). Specifically, images and their related ranking results are first collected to form the training set. Multimodal features, including click and visual features, are collected with these images. Next, a group of autoencoders is applied to obtain initially a distance metric in different visual spaces, and an MDML method is used to assign optimal weights for different modalities. Next, we conduct alternating optimization to train the ranking model, which is used for the ranking of new queries with click features. Compared with existing image ranking methods, the proposed method adopts a new ranking model to use multimodal features, including click features and visual features in DML. We operated experiments to analyze the proposed Deep-MDML in two benchmark data sets, and the results validate the effects of the method. Article in Journal/Newspaper DML University of Technology Sydney: OPUS - Open Publications of UTS Scholars |
institution |
Open Polar |
collection |
University of Technology Sydney: OPUS - Open Publications of UTS Scholars |
op_collection_id |
ftunivtsydney |
language |
unknown |
topic |
Artificial Intelligence & Image Processing |
spellingShingle |
Artificial Intelligence & Image Processing Yu, J Yang, X Gao, F Tao, D Deep Multimodal Distance Metric Learning Using Click Constraints for Image Ranking |
topic_facet |
Artificial Intelligence & Image Processing |
description |
© 2013 IEEE. How do we retrieve images accurately? Also, how do we rank a group of images precisely and efficiently for specific queries? These problems are critical for researchers and engineers to generate a novel image searching engine. First, it is important to obtain an appropriate description that effectively represent the images. In this paper, multimodal features are considered for describing images. The images unique properties are reflected by visual features, which are correlated to each other. However, semantic gaps always exist between images visual features and semantics. Therefore, we utilize click feature to reduce the semantic gap. The second key issue is learning an appropriate distance metric to combine these multimodal features. This paper develops a novel deep multimodal distance metric learning (Deep-MDML) method. A structured ranking model is adopted to utilize both visual and click features in distance metric learning (DML). Specifically, images and their related ranking results are first collected to form the training set. Multimodal features, including click and visual features, are collected with these images. Next, a group of autoencoders is applied to obtain initially a distance metric in different visual spaces, and an MDML method is used to assign optimal weights for different modalities. Next, we conduct alternating optimization to train the ranking model, which is used for the ranking of new queries with click features. Compared with existing image ranking methods, the proposed method adopts a new ranking model to use multimodal features, including click features and visual features in DML. We operated experiments to analyze the proposed Deep-MDML in two benchmark data sets, and the results validate the effects of the method. |
format |
Article in Journal/Newspaper |
author |
Yu, J Yang, X Gao, F Tao, D |
author_facet |
Yu, J Yang, X Gao, F Tao, D |
author_sort |
Yu, J |
title |
Deep Multimodal Distance Metric Learning Using Click Constraints for Image Ranking |
title_short |
Deep Multimodal Distance Metric Learning Using Click Constraints for Image Ranking |
title_full |
Deep Multimodal Distance Metric Learning Using Click Constraints for Image Ranking |
title_fullStr |
Deep Multimodal Distance Metric Learning Using Click Constraints for Image Ranking |
title_full_unstemmed |
Deep Multimodal Distance Metric Learning Using Click Constraints for Image Ranking |
title_sort |
deep multimodal distance metric learning using click constraints for image ranking |
publishDate |
2017 |
url |
http://hdl.handle.net/10453/123820 |
genre |
DML |
genre_facet |
DML |
op_relation |
http://purl.org/au-research/grants/arc/FT130101457 http://purl.org/au-research/grants/arc/DP140102164 http://purl.org/au-research/grants/arc/LE140100061 IEEE Transactions on Cybernetics 10.1109/TCYB.2016.2591583 IEEE Transactions on Cybernetics, 2017, 47 (12), pp. 4014 - 4024 2168-2267 http://hdl.handle.net/10453/123820 |
_version_ |
1766397521172627456 |