Overcoming the curse of dimensionality with Laplacian regularization in semi-supervised learning
International audience As annotations of data can be scarce in large-scale practical problems, leveraging unlabelled examples is one of the most important aspects of machine learning. This is the aim of semi-supervised learning. To benefit from the access to unlabelled data, it is natural to diffuse...
Main Authors: | , , , |
---|---|
Other Authors: | , , , , , , , , , , , |
Format: | Conference Object |
Language: | English |
Published: |
HAL CCSD
2021
|
Subjects: | |
Online Access: | https://hal.archives-ouvertes.fr/hal-03454809 https://hal.archives-ouvertes.fr/hal-03454809/document https://hal.archives-ouvertes.fr/hal-03454809/file/main.pdf |
id |
ftunivnantes:oai:HAL:hal-03454809v1 |
---|---|
record_format |
openpolar |
spelling |
ftunivnantes:oai:HAL:hal-03454809v1 2023-05-15T14:27:31+02:00 Overcoming the curse of dimensionality with Laplacian regularization in semi-supervised learning Cabannes, Vivien Pillaud-Vivien, Loucas Bach, Francis Rudi, Alessandro Statistical Machine Learning and Parsimony (SIERRA) Département d'informatique - ENS Paris (DI-ENS) École normale supérieure - Paris (ENS-PSL) Université Paris sciences et lettres (PSL)-Université Paris sciences et lettres (PSL)-Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-École normale supérieure - Paris (ENS-PSL) Université Paris sciences et lettres (PSL)-Université Paris sciences et lettres (PSL)-Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-Centre National de la Recherche Scientifique (CNRS)-Inria de Paris Institut National de Recherche en Informatique et en Automatique (Inria) École des Ponts ParisTech (ENPC) Ecole Polytechnique Fédérale de Lausanne (EPFL) This work was funded in part by the French government under management of Agence Nationale de la Recherche as part of the “Investissements d’avenir” program, reference ANR-19-P3IA-0001 (PRAIRIE 3IA Institute). We also acknowledge support of the European Research Council (grants SEQUOIA 724063 and REAL 94790). ANR-19-P3IA-0001,PRAIRIE,PaRis Artificial Intelligence Research InstitutE(2019) European Project: 724063,ERC-2016-COG,SEQUOIA(2017) European Project: 0947904(2010) Online, Unknown Region 2021-12-06 https://hal.archives-ouvertes.fr/hal-03454809 https://hal.archives-ouvertes.fr/hal-03454809/document https://hal.archives-ouvertes.fr/hal-03454809/file/main.pdf en eng HAL CCSD info:eu-repo/grantAgreement//724063/EU/Robust algorithms for learning from modern data/SEQUOIA info:eu-repo/grantAgreement//0947904/EU/Collaborative Research: Origin of the Alexander Terrane in the Arctic Realm?/ hal-03454809 https://hal.archives-ouvertes.fr/hal-03454809 https://hal.archives-ouvertes.fr/hal-03454809/document https://hal.archives-ouvertes.fr/hal-03454809/file/main.pdf info:eu-repo/semantics/OpenAccess NeurIPS 2021 - Thirty-fifth conference on Neural Information Processing Systems (NeurIPS) https://hal.archives-ouvertes.fr/hal-03454809 NeurIPS 2021 - Thirty-fifth conference on Neural Information Processing Systems (NeurIPS), Dec 2021, Online, Unknown Region [STAT.ML]Statistics [stat]/Machine Learning [stat.ML] [INFO.INFO-LG]Computer Science [cs]/Machine Learning [cs.LG] [MATH.MATH-ST]Mathematics [math]/Statistics [math.ST] info:eu-repo/semantics/conferenceObject Conference papers 2021 ftunivnantes 2022-06-12T06:03:33Z International audience As annotations of data can be scarce in large-scale practical problems, leveraging unlabelled examples is one of the most important aspects of machine learning. This is the aim of semi-supervised learning. To benefit from the access to unlabelled data, it is natural to diffuse smoothly knowledge of labelled data to unlabelled one. This induces to the use of Laplacian regularization. Yet, current implementations of Laplacian regularization suffer from several drawbacks, notably the well-known curse of dimensionality. In this paper, we provide a statistical analysis to overcome those issues, and unveil a large body of spectral filtering methods that exhibit desirable behaviors. They are implemented through (reproducing) kernel methods, for which we provide realistic computational guidelines in order to make our method usable with large amounts of data. Conference Object Arctic Université de Nantes: HAL-UNIV-NANTES |
institution |
Open Polar |
collection |
Université de Nantes: HAL-UNIV-NANTES |
op_collection_id |
ftunivnantes |
language |
English |
topic |
[STAT.ML]Statistics [stat]/Machine Learning [stat.ML] [INFO.INFO-LG]Computer Science [cs]/Machine Learning [cs.LG] [MATH.MATH-ST]Mathematics [math]/Statistics [math.ST] |
spellingShingle |
[STAT.ML]Statistics [stat]/Machine Learning [stat.ML] [INFO.INFO-LG]Computer Science [cs]/Machine Learning [cs.LG] [MATH.MATH-ST]Mathematics [math]/Statistics [math.ST] Cabannes, Vivien Pillaud-Vivien, Loucas Bach, Francis Rudi, Alessandro Overcoming the curse of dimensionality with Laplacian regularization in semi-supervised learning |
topic_facet |
[STAT.ML]Statistics [stat]/Machine Learning [stat.ML] [INFO.INFO-LG]Computer Science [cs]/Machine Learning [cs.LG] [MATH.MATH-ST]Mathematics [math]/Statistics [math.ST] |
description |
International audience As annotations of data can be scarce in large-scale practical problems, leveraging unlabelled examples is one of the most important aspects of machine learning. This is the aim of semi-supervised learning. To benefit from the access to unlabelled data, it is natural to diffuse smoothly knowledge of labelled data to unlabelled one. This induces to the use of Laplacian regularization. Yet, current implementations of Laplacian regularization suffer from several drawbacks, notably the well-known curse of dimensionality. In this paper, we provide a statistical analysis to overcome those issues, and unveil a large body of spectral filtering methods that exhibit desirable behaviors. They are implemented through (reproducing) kernel methods, for which we provide realistic computational guidelines in order to make our method usable with large amounts of data. |
author2 |
Statistical Machine Learning and Parsimony (SIERRA) Département d'informatique - ENS Paris (DI-ENS) École normale supérieure - Paris (ENS-PSL) Université Paris sciences et lettres (PSL)-Université Paris sciences et lettres (PSL)-Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-École normale supérieure - Paris (ENS-PSL) Université Paris sciences et lettres (PSL)-Université Paris sciences et lettres (PSL)-Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-Centre National de la Recherche Scientifique (CNRS)-Inria de Paris Institut National de Recherche en Informatique et en Automatique (Inria) École des Ponts ParisTech (ENPC) Ecole Polytechnique Fédérale de Lausanne (EPFL) This work was funded in part by the French government under management of Agence Nationale de la Recherche as part of the “Investissements d’avenir” program, reference ANR-19-P3IA-0001 (PRAIRIE 3IA Institute). We also acknowledge support of the European Research Council (grants SEQUOIA 724063 and REAL 94790). ANR-19-P3IA-0001,PRAIRIE,PaRis Artificial Intelligence Research InstitutE(2019) European Project: 724063,ERC-2016-COG,SEQUOIA(2017) European Project: 0947904(2010) |
format |
Conference Object |
author |
Cabannes, Vivien Pillaud-Vivien, Loucas Bach, Francis Rudi, Alessandro |
author_facet |
Cabannes, Vivien Pillaud-Vivien, Loucas Bach, Francis Rudi, Alessandro |
author_sort |
Cabannes, Vivien |
title |
Overcoming the curse of dimensionality with Laplacian regularization in semi-supervised learning |
title_short |
Overcoming the curse of dimensionality with Laplacian regularization in semi-supervised learning |
title_full |
Overcoming the curse of dimensionality with Laplacian regularization in semi-supervised learning |
title_fullStr |
Overcoming the curse of dimensionality with Laplacian regularization in semi-supervised learning |
title_full_unstemmed |
Overcoming the curse of dimensionality with Laplacian regularization in semi-supervised learning |
title_sort |
overcoming the curse of dimensionality with laplacian regularization in semi-supervised learning |
publisher |
HAL CCSD |
publishDate |
2021 |
url |
https://hal.archives-ouvertes.fr/hal-03454809 https://hal.archives-ouvertes.fr/hal-03454809/document https://hal.archives-ouvertes.fr/hal-03454809/file/main.pdf |
op_coverage |
Online, Unknown Region |
genre |
Arctic |
genre_facet |
Arctic |
op_source |
NeurIPS 2021 - Thirty-fifth conference on Neural Information Processing Systems (NeurIPS) https://hal.archives-ouvertes.fr/hal-03454809 NeurIPS 2021 - Thirty-fifth conference on Neural Information Processing Systems (NeurIPS), Dec 2021, Online, Unknown Region |
op_relation |
info:eu-repo/grantAgreement//724063/EU/Robust algorithms for learning from modern data/SEQUOIA info:eu-repo/grantAgreement//0947904/EU/Collaborative Research: Origin of the Alexander Terrane in the Arctic Realm?/ hal-03454809 https://hal.archives-ouvertes.fr/hal-03454809 https://hal.archives-ouvertes.fr/hal-03454809/document https://hal.archives-ouvertes.fr/hal-03454809/file/main.pdf |
op_rights |
info:eu-repo/semantics/OpenAccess |
_version_ |
1766301279536021504 |