The IceProd framework: distributed data processing for the IceCube neutrino observatory
IceCube is a one-gigaton instrument located at the geographic South Pole, designed to detect cosmic neutrinos, iden- tify the particle nature of dark matter, and study high-energy neutrinos themselves. Simulation of the IceCube detector and processing of data require a significant amount of computat...
Published in: | Journal of Parallel and Distributed Computing |
---|---|
Main Authors: | , |
Format: | Article in Journal/Newspaper |
Language: | unknown |
Published: |
Elsevier
2016
|
Subjects: | |
Online Access: | https://doi.org/10.1016/j.jpdc.2014.08.001 https://ora.ox.ac.uk/objects/uuid:3ffcaa9a-9c37-4665-a774-a7f5634dc155 |
id |
ftuloxford:oai:ora.ox.ac.uk:uuid:3ffcaa9a-9c37-4665-a774-a7f5634dc155 |
---|---|
record_format |
openpolar |
spelling |
ftuloxford:oai:ora.ox.ac.uk:uuid:3ffcaa9a-9c37-4665-a774-a7f5634dc155 2024-10-06T13:52:51+00:00 The IceProd framework: distributed data processing for the IceCube neutrino observatory Aartsen, MG Sarkar, S 2016-07-28 https://doi.org/10.1016/j.jpdc.2014.08.001 https://ora.ox.ac.uk/objects/uuid:3ffcaa9a-9c37-4665-a774-a7f5634dc155 unknown Elsevier doi:10.1016/j.jpdc.2014.08.001 https://ora.ox.ac.uk/objects/uuid:3ffcaa9a-9c37-4665-a774-a7f5634dc155 https://doi.org/10.1016/j.jpdc.2014.08.001 info:eu-repo/semantics/openAccess Journal article 2016 ftuloxford https://doi.org/10.1016/j.jpdc.2014.08.001 2024-09-06T07:47:32Z IceCube is a one-gigaton instrument located at the geographic South Pole, designed to detect cosmic neutrinos, iden- tify the particle nature of dark matter, and study high-energy neutrinos themselves. Simulation of the IceCube detector and processing of data require a significant amount of computational resources. IceProd is a distributed management system based on Python, XML-RPC and GridFTP. It is driven by a central database in order to coordinate and admin- ister production of simulations and processing of data produced by the IceCube detector. IceProd runs as a separate layer on top of other middleware and can take advantage of a variety of computing resources, including grids and batch systems such as CREAM, Condor, and PBS. This is accomplished by a set of dedicated daemons that process job submission in a coordinated fashion through the use of middleware plugins that serve to abstract the details of job submission and job management from the framework. Article in Journal/Newspaper South pole ORA - Oxford University Research Archive South Pole Journal of Parallel and Distributed Computing 75 198 211 |
institution |
Open Polar |
collection |
ORA - Oxford University Research Archive |
op_collection_id |
ftuloxford |
language |
unknown |
description |
IceCube is a one-gigaton instrument located at the geographic South Pole, designed to detect cosmic neutrinos, iden- tify the particle nature of dark matter, and study high-energy neutrinos themselves. Simulation of the IceCube detector and processing of data require a significant amount of computational resources. IceProd is a distributed management system based on Python, XML-RPC and GridFTP. It is driven by a central database in order to coordinate and admin- ister production of simulations and processing of data produced by the IceCube detector. IceProd runs as a separate layer on top of other middleware and can take advantage of a variety of computing resources, including grids and batch systems such as CREAM, Condor, and PBS. This is accomplished by a set of dedicated daemons that process job submission in a coordinated fashion through the use of middleware plugins that serve to abstract the details of job submission and job management from the framework. |
format |
Article in Journal/Newspaper |
author |
Aartsen, MG Sarkar, S |
spellingShingle |
Aartsen, MG Sarkar, S The IceProd framework: distributed data processing for the IceCube neutrino observatory |
author_facet |
Aartsen, MG Sarkar, S |
author_sort |
Aartsen, MG |
title |
The IceProd framework: distributed data processing for the IceCube neutrino observatory |
title_short |
The IceProd framework: distributed data processing for the IceCube neutrino observatory |
title_full |
The IceProd framework: distributed data processing for the IceCube neutrino observatory |
title_fullStr |
The IceProd framework: distributed data processing for the IceCube neutrino observatory |
title_full_unstemmed |
The IceProd framework: distributed data processing for the IceCube neutrino observatory |
title_sort |
iceprod framework: distributed data processing for the icecube neutrino observatory |
publisher |
Elsevier |
publishDate |
2016 |
url |
https://doi.org/10.1016/j.jpdc.2014.08.001 https://ora.ox.ac.uk/objects/uuid:3ffcaa9a-9c37-4665-a774-a7f5634dc155 |
geographic |
South Pole |
geographic_facet |
South Pole |
genre |
South pole |
genre_facet |
South pole |
op_relation |
doi:10.1016/j.jpdc.2014.08.001 https://ora.ox.ac.uk/objects/uuid:3ffcaa9a-9c37-4665-a774-a7f5634dc155 https://doi.org/10.1016/j.jpdc.2014.08.001 |
op_rights |
info:eu-repo/semantics/openAccess |
op_doi |
https://doi.org/10.1016/j.jpdc.2014.08.001 |
container_title |
Journal of Parallel and Distributed Computing |
container_volume |
75 |
container_start_page |
198 |
op_container_end_page |
211 |
_version_ |
1812181445077106688 |