A cross-modal deep metric learning model for disease diagnosis based on chest x-ray images
The emergence of unknown diseases is often with few or no samples available. Zero-shot learning and few-shot learning have promising applications in medical image analysis. In this paper, we propose a Cross-Modal Deep Metric Learning Generalized Zero-Shot Learning (CM-DML-GZSL) model. The proposed n...
Published in: | Multimedia Tools and Applications |
---|---|
Main Authors: | , , , |
Format: | Text |
Language: | English |
Published: |
Springer US
2023
|
Subjects: | |
Online Access: | http://www.ncbi.nlm.nih.gov/pmc/articles/PMC10015533/ https://doi.org/10.1007/s11042-023-14790-7 |
id |
ftpubmed:oai:pubmedcentral.nih.gov:10015533 |
---|---|
record_format |
openpolar |
spelling |
ftpubmed:oai:pubmedcentral.nih.gov:10015533 2023-05-15T16:02:02+02:00 A cross-modal deep metric learning model for disease diagnosis based on chest x-ray images Jin, Yufei Lu, Huijuan Li, Zhao Wang, Yanbin 2023-03-15 http://www.ncbi.nlm.nih.gov/pmc/articles/PMC10015533/ https://doi.org/10.1007/s11042-023-14790-7 en eng Springer US http://www.ncbi.nlm.nih.gov/pmc/articles/PMC10015533/ http://dx.doi.org/10.1007/s11042-023-14790-7 © The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature 2023, Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law. This article is made available via the PMC Open Access Subset for unrestricted research re-use and secondary analysis in any form or by any means with acknowledgement of the original source. These permissions are granted for the duration of the World Health Organization (WHO) declaration of COVID-19 as a global pandemic. Multimed Tools Appl Article Text 2023 ftpubmed https://doi.org/10.1007/s11042-023-14790-7 2023-03-19T02:05:53Z The emergence of unknown diseases is often with few or no samples available. Zero-shot learning and few-shot learning have promising applications in medical image analysis. In this paper, we propose a Cross-Modal Deep Metric Learning Generalized Zero-Shot Learning (CM-DML-GZSL) model. The proposed network consists of a visual feature extractor, a fixed semantic feature extractor, and a deep regression module. The network belongs to a two-stream network for multiple modalities. In a multi-label setting, each sample contains a small number of positive labels and a large number of negative labels on average. This positive-negative imbalance dominates the optimization procedure and may prevent the establishment of an effective correspondence between visual features and semantic vectors during training, resulting in a low degree of accuracy. A novel weighted focused Euclidean distance metric loss is introduced in this regard. This loss not only can dynamically increase the weight of hard samples and decrease the weight of simple samples, but it can also promote the connection between samples and semantic vectors corresponding to their positive labels, which helps mitigate bias in predicting unseen classes in the generalized zero-shot learning setting. The weighted focused Euclidean distance metric loss function can dynamically adjust sample weights, enabling zero-shot multi-label learning for chest X-ray diagnosis, as experimental results on large publicly available datasets demonstrate. Text DML PubMed Central (PMC) Multimedia Tools and Applications |
institution |
Open Polar |
collection |
PubMed Central (PMC) |
op_collection_id |
ftpubmed |
language |
English |
topic |
Article |
spellingShingle |
Article Jin, Yufei Lu, Huijuan Li, Zhao Wang, Yanbin A cross-modal deep metric learning model for disease diagnosis based on chest x-ray images |
topic_facet |
Article |
description |
The emergence of unknown diseases is often with few or no samples available. Zero-shot learning and few-shot learning have promising applications in medical image analysis. In this paper, we propose a Cross-Modal Deep Metric Learning Generalized Zero-Shot Learning (CM-DML-GZSL) model. The proposed network consists of a visual feature extractor, a fixed semantic feature extractor, and a deep regression module. The network belongs to a two-stream network for multiple modalities. In a multi-label setting, each sample contains a small number of positive labels and a large number of negative labels on average. This positive-negative imbalance dominates the optimization procedure and may prevent the establishment of an effective correspondence between visual features and semantic vectors during training, resulting in a low degree of accuracy. A novel weighted focused Euclidean distance metric loss is introduced in this regard. This loss not only can dynamically increase the weight of hard samples and decrease the weight of simple samples, but it can also promote the connection between samples and semantic vectors corresponding to their positive labels, which helps mitigate bias in predicting unseen classes in the generalized zero-shot learning setting. The weighted focused Euclidean distance metric loss function can dynamically adjust sample weights, enabling zero-shot multi-label learning for chest X-ray diagnosis, as experimental results on large publicly available datasets demonstrate. |
format |
Text |
author |
Jin, Yufei Lu, Huijuan Li, Zhao Wang, Yanbin |
author_facet |
Jin, Yufei Lu, Huijuan Li, Zhao Wang, Yanbin |
author_sort |
Jin, Yufei |
title |
A cross-modal deep metric learning model for disease diagnosis based on chest x-ray images |
title_short |
A cross-modal deep metric learning model for disease diagnosis based on chest x-ray images |
title_full |
A cross-modal deep metric learning model for disease diagnosis based on chest x-ray images |
title_fullStr |
A cross-modal deep metric learning model for disease diagnosis based on chest x-ray images |
title_full_unstemmed |
A cross-modal deep metric learning model for disease diagnosis based on chest x-ray images |
title_sort |
cross-modal deep metric learning model for disease diagnosis based on chest x-ray images |
publisher |
Springer US |
publishDate |
2023 |
url |
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC10015533/ https://doi.org/10.1007/s11042-023-14790-7 |
genre |
DML |
genre_facet |
DML |
op_source |
Multimed Tools Appl |
op_relation |
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC10015533/ http://dx.doi.org/10.1007/s11042-023-14790-7 |
op_rights |
© The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature 2023, Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law. This article is made available via the PMC Open Access Subset for unrestricted research re-use and secondary analysis in any form or by any means with acknowledgement of the original source. These permissions are granted for the duration of the World Health Organization (WHO) declaration of COVID-19 as a global pandemic. |
op_doi |
https://doi.org/10.1007/s11042-023-14790-7 |
container_title |
Multimedia Tools and Applications |
_version_ |
1766397667912450048 |