The IceProd framework: Distributed data processing for the IceCube neutrino observatory
IceCube is a one-gigaton instrument located at the geographic South Pole, designed to detect cosmic neutrinos, identify the particle nature of dark matter, and study high-energy neutrinos themselves. Simulation of the IceCube detector and processing of data require a significant amount of computatio...
Main Authors: | , , , , , , , |
---|---|
Format: | Article in Journal/Newspaper |
Language: | English |
Published: |
2014
|
Subjects: | |
Online Access: | http://hdl.handle.net/2013/ULB-DIPOT:oai:dipot.ulb.ac.be:2013/207489 https://dipot.ulb.ac.be/dspace/bitstream/2013/207489/1/Elsevier_191116.pdf |
id |
ftunivbruxelles:oai:dipot.ulb.ac.be:2013/207489 |
---|---|
record_format |
openpolar |
spelling |
ftunivbruxelles:oai:dipot.ulb.ac.be:2013/207489 2023-05-15T18:22:34+02:00 The IceProd framework: Distributed data processing for the IceCube neutrino observatory Aartsen, M. G. Hanson, Kael Heereman von Zuydtwyck, David Meures, Thomas O'Murchadha, Aongus Pinat, Elisa Aguilar Sanchez, Juan Antonio Toscano, Simona 2014 1 full-text file(s): application/pdf http://hdl.handle.net/2013/ULB-DIPOT:oai:dipot.ulb.ac.be:2013/207489 https://dipot.ulb.ac.be/dspace/bitstream/2013/207489/1/Elsevier_191116.pdf en eng uri/info:doi/10.1016/j.jpdc.2014.08.001 uri/info:pii/S0743731514001439 uri/info:scp/84915776438 https://dipot.ulb.ac.be/dspace/bitstream/2013/207489/1/Elsevier_191116.pdf http://hdl.handle.net/2013/ULB-DIPOT:oai:dipot.ulb.ac.be:2013/207489 1 full-text file(s): info:eu-repo/semantics/restrictedAccess Journal of parallel and distributed computing, 75 Physique des particules élémentaires Astrophysique Data management Distributed computing Grid computing Monitoring info:eu-repo/semantics/article info:ulb-repo/semantics/articlePeerReview info:ulb-repo/semantics/openurl/article 2014 ftunivbruxelles 2022-06-12T21:10:12Z IceCube is a one-gigaton instrument located at the geographic South Pole, designed to detect cosmic neutrinos, identify the particle nature of dark matter, and study high-energy neutrinos themselves. Simulation of the IceCube detector and processing of data require a significant amount of computational resources. This paper presents the first detailed description of IceProd, a lightweight distributed management system designed to meet these requirements. It is driven by a central database in order to manage mass production of simulations and analysis of data produced by the IceCube detector. IceProd runs as a separate layer on top of other middleware and can take advantage of a variety of computing resources, including grids and batch systems such as CREAM, HTCondor, and PBS. This is accomplished by a set of dedicated daemons that process job submission in a coordinated fashion through the use of middleware plugins that serve to abstract the details of job submission and job management from the framework. 0 SCOPUS: ar.j info:eu-repo/semantics/published Article in Journal/Newspaper South pole DI-fusion : dépôt institutionnel de l'Université libre de Bruxelles (ULB) South Pole |
institution |
Open Polar |
collection |
DI-fusion : dépôt institutionnel de l'Université libre de Bruxelles (ULB) |
op_collection_id |
ftunivbruxelles |
language |
English |
topic |
Physique des particules élémentaires Astrophysique Data management Distributed computing Grid computing Monitoring |
spellingShingle |
Physique des particules élémentaires Astrophysique Data management Distributed computing Grid computing Monitoring Aartsen, M. G. Hanson, Kael Heereman von Zuydtwyck, David Meures, Thomas O'Murchadha, Aongus Pinat, Elisa Aguilar Sanchez, Juan Antonio Toscano, Simona The IceProd framework: Distributed data processing for the IceCube neutrino observatory |
topic_facet |
Physique des particules élémentaires Astrophysique Data management Distributed computing Grid computing Monitoring |
description |
IceCube is a one-gigaton instrument located at the geographic South Pole, designed to detect cosmic neutrinos, identify the particle nature of dark matter, and study high-energy neutrinos themselves. Simulation of the IceCube detector and processing of data require a significant amount of computational resources. This paper presents the first detailed description of IceProd, a lightweight distributed management system designed to meet these requirements. It is driven by a central database in order to manage mass production of simulations and analysis of data produced by the IceCube detector. IceProd runs as a separate layer on top of other middleware and can take advantage of a variety of computing resources, including grids and batch systems such as CREAM, HTCondor, and PBS. This is accomplished by a set of dedicated daemons that process job submission in a coordinated fashion through the use of middleware plugins that serve to abstract the details of job submission and job management from the framework. 0 SCOPUS: ar.j info:eu-repo/semantics/published |
format |
Article in Journal/Newspaper |
author |
Aartsen, M. G. Hanson, Kael Heereman von Zuydtwyck, David Meures, Thomas O'Murchadha, Aongus Pinat, Elisa Aguilar Sanchez, Juan Antonio Toscano, Simona |
author_facet |
Aartsen, M. G. Hanson, Kael Heereman von Zuydtwyck, David Meures, Thomas O'Murchadha, Aongus Pinat, Elisa Aguilar Sanchez, Juan Antonio Toscano, Simona |
author_sort |
Aartsen, M. G. |
title |
The IceProd framework: Distributed data processing for the IceCube neutrino observatory |
title_short |
The IceProd framework: Distributed data processing for the IceCube neutrino observatory |
title_full |
The IceProd framework: Distributed data processing for the IceCube neutrino observatory |
title_fullStr |
The IceProd framework: Distributed data processing for the IceCube neutrino observatory |
title_full_unstemmed |
The IceProd framework: Distributed data processing for the IceCube neutrino observatory |
title_sort |
iceprod framework: distributed data processing for the icecube neutrino observatory |
publishDate |
2014 |
url |
http://hdl.handle.net/2013/ULB-DIPOT:oai:dipot.ulb.ac.be:2013/207489 https://dipot.ulb.ac.be/dspace/bitstream/2013/207489/1/Elsevier_191116.pdf |
geographic |
South Pole |
geographic_facet |
South Pole |
genre |
South pole |
genre_facet |
South pole |
op_source |
Journal of parallel and distributed computing, 75 |
op_relation |
uri/info:doi/10.1016/j.jpdc.2014.08.001 uri/info:pii/S0743731514001439 uri/info:scp/84915776438 https://dipot.ulb.ac.be/dspace/bitstream/2013/207489/1/Elsevier_191116.pdf http://hdl.handle.net/2013/ULB-DIPOT:oai:dipot.ulb.ac.be:2013/207489 |
op_rights |
1 full-text file(s): info:eu-repo/semantics/restrictedAccess |
_version_ |
1766201984398917632 |