Unsupervised Anomaly Detection via Deep Metric Learning with End-to-End Optimization
We investigate unsupervised anomaly detection for high-dimensional data and introduce a deep metric learning (DML) based framework. In particular, we learn a distance metric through a deep neural network. Through this metric, we project the data into the metric space that better separates the anomal...
Main Authors: | , |
---|---|
Format: | Text |
Language: | unknown |
Published: |
2020
|
Subjects: | |
Online Access: | http://arxiv.org/abs/2005.05865 |
id |
ftarxivpreprints:oai:arXiv.org:2005.05865 |
---|---|
record_format |
openpolar |
spelling |
ftarxivpreprints:oai:arXiv.org:2005.05865 2023-09-05T13:19:05+02:00 Unsupervised Anomaly Detection via Deep Metric Learning with End-to-End Optimization Yilmaz, Selim F. Kozat, Suleyman S. 2020-05-12 http://arxiv.org/abs/2005.05865 unknown http://arxiv.org/abs/2005.05865 Computer Science - Machine Learning Statistics - Machine Learning text 2020 ftarxivpreprints 2023-08-16T15:52:44Z We investigate unsupervised anomaly detection for high-dimensional data and introduce a deep metric learning (DML) based framework. In particular, we learn a distance metric through a deep neural network. Through this metric, we project the data into the metric space that better separates the anomalies from the normal data and reduces the effect of the curse of dimensionality for high-dimensional data. We present a novel data distillation method through self-supervision to remedy the conventional practice of assuming all data as normal. We also employ the hard mining technique from the DML literature. We show these components improve the performance of our model and significantly reduce the running time. Through an extensive set of experiments on the 14 real-world datasets, our method demonstrates significant performance gains compared to the state-of-the-art unsupervised anomaly detection methods, e.g., an absolute improvement between 4.44% and 11.74% on the average over the 14 datasets. Furthermore, we share the source code of our method on Github to facilitate further research. Comment: 11 pages, 3 figures Text DML ArXiv.org (Cornell University Library) |
institution |
Open Polar |
collection |
ArXiv.org (Cornell University Library) |
op_collection_id |
ftarxivpreprints |
language |
unknown |
topic |
Computer Science - Machine Learning Statistics - Machine Learning |
spellingShingle |
Computer Science - Machine Learning Statistics - Machine Learning Yilmaz, Selim F. Kozat, Suleyman S. Unsupervised Anomaly Detection via Deep Metric Learning with End-to-End Optimization |
topic_facet |
Computer Science - Machine Learning Statistics - Machine Learning |
description |
We investigate unsupervised anomaly detection for high-dimensional data and introduce a deep metric learning (DML) based framework. In particular, we learn a distance metric through a deep neural network. Through this metric, we project the data into the metric space that better separates the anomalies from the normal data and reduces the effect of the curse of dimensionality for high-dimensional data. We present a novel data distillation method through self-supervision to remedy the conventional practice of assuming all data as normal. We also employ the hard mining technique from the DML literature. We show these components improve the performance of our model and significantly reduce the running time. Through an extensive set of experiments on the 14 real-world datasets, our method demonstrates significant performance gains compared to the state-of-the-art unsupervised anomaly detection methods, e.g., an absolute improvement between 4.44% and 11.74% on the average over the 14 datasets. Furthermore, we share the source code of our method on Github to facilitate further research. Comment: 11 pages, 3 figures |
format |
Text |
author |
Yilmaz, Selim F. Kozat, Suleyman S. |
author_facet |
Yilmaz, Selim F. Kozat, Suleyman S. |
author_sort |
Yilmaz, Selim F. |
title |
Unsupervised Anomaly Detection via Deep Metric Learning with End-to-End Optimization |
title_short |
Unsupervised Anomaly Detection via Deep Metric Learning with End-to-End Optimization |
title_full |
Unsupervised Anomaly Detection via Deep Metric Learning with End-to-End Optimization |
title_fullStr |
Unsupervised Anomaly Detection via Deep Metric Learning with End-to-End Optimization |
title_full_unstemmed |
Unsupervised Anomaly Detection via Deep Metric Learning with End-to-End Optimization |
title_sort |
unsupervised anomaly detection via deep metric learning with end-to-end optimization |
publishDate |
2020 |
url |
http://arxiv.org/abs/2005.05865 |
genre |
DML |
genre_facet |
DML |
op_relation |
http://arxiv.org/abs/2005.05865 |
_version_ |
1776199907247390720 |