Zero-shot fashion products clustering on social image streams

• Main Authors: Poveda Pena, Jonatan, Tous Liesa, Rubén
• Other Authors: Universitat Politècnica de Catalunya. Departament d'Arquitectura de Computadors, Universitat Politècnica de Catalunya. CAP - Grup de Computació d'Altes Prestacions
• Format: Conference Object
• Language: English
• Published: Springer 2019
• Subjects: Àrees temàtiques de la UPC::Informàtica::Intel·ligència artificial::Aprenentatge automàtic; Àrees temàtiques de la UPC::Enginyeria de la telecomunicació::Processament del senyal::Processament de la imatge i del senyal vídeo; Machine learning; Computer vision; Online social networks; Deep metric learning; Zero-shot learning; Clustering; Supervised learning; Aprenentatge automàtic; Visió per ordinador; Xarxes socials en línia; DML
• Online Access: http://hdl.handle.net/2117/348727; https://doi.org/10.1007/978-3-030-37599-7_63

Computer Vision methods have been proposed to solve the problem of matching photographs containing some products from users in social media to products in retail catalogues. This is challenging due to the quality of the photographies, difficulties in dealing with garments and their category taxonomy. A N-Shot Learning approach is required as retail catalogues may contain hundreds of different products for which, in many cases, only one image is provided. This framework can be solved by means of Deep Metric Learning (DML) techniques, in which a metric to discriminate similar than dissimilar samples is learnt. The performance of different authors tackling this problem varies a lot but even if they perform reasonably well, the set of elements they need to return in order to include the exact product is large. As after the query there is a person curating the results, it is important to return the smallest set of elements possible, being ideally just to return only one: the related product. This paper proposes to solve the image-to-product image matching problem through a product retrieval system using DML and Zero-short Learning, focusing on garments, and applying some of the last advances on clustering techniques. This work is partially supported by the Spanish Ministry of Economy and Competi-tivity under contract TIN2015-65316-P and by the SGR programme (2014-SGR-1051and 2017-SGR-962) of the Catalan Government. Peer Reviewed Postprint (author's final draft)