On the reliability of acoustic annotation and automatic detections of Antarctic blue whale calls under different acoustic conditions
Evaluation of the performance of computer-based algorithms to automatically detect mammalian vocalizations often relies on comparisons between detector outputs and a reference data set, generally obtained by manual annotation of acoustic recordings. To explore the reproducibility of these manual ann...
Main Authors: | , , , , |
---|---|
Format: | Conference Object |
Language: | unknown |
Published: |
2018
|
Subjects: | |
Online Access: | https://epic.awi.de/id/eprint/48164/ https://hdl.handle.net/10013/epic.b3ff4f68-c00a-4fbe-9362-c9abaf565f8c |
id |
ftawi:oai:epic.awi.de:48164 |
---|---|
record_format |
openpolar |
spelling |
ftawi:oai:epic.awi.de:48164 2023-05-15T13:45:22+02:00 On the reliability of acoustic annotation and automatic detections of Antarctic blue whale calls under different acoustic conditions Leroy, Emmanuelle Thomisch, Karolin Royer, J.-Y. Boebel, Olaf van Opzeeland, Ilse 2018 https://epic.awi.de/id/eprint/48164/ https://hdl.handle.net/10013/epic.b3ff4f68-c00a-4fbe-9362-c9abaf565f8c unknown Leroy, E. , Thomisch, K. orcid:0000-0002-7144-8369 , Royer, J. Y. , Boebel, O. orcid:0000-0002-2259-0035 and van Opzeeland, I. (2018) On the reliability of acoustic annotation and automatic detections of Antarctic blue whale calls under different acoustic conditions , DCLDE, Paris, 5 May 2018 - 9 May 2018 . hdl:10013/epic.b3ff4f68-c00a-4fbe-9362-c9abaf565f8c EPIC3DCLDE, Paris, 2018-05-05-2018-05-09 Conference notRev 2018 ftawi 2022-10-02T23:12:36Z Evaluation of the performance of computer-based algorithms to automatically detect mammalian vocalizations often relies on comparisons between detector outputs and a reference data set, generally obtained by manual annotation of acoustic recordings. To explore the reproducibility of these manual annotations, we investigate inter- and intra-analyst variability in manually annotated Antarctic blue whale (ABW) Z-calls by two analysts using acoustic data from two ocean basins with different call abundance and background noise. Manual annotations exhibited strong inter- and intra-analyst variability, with less than 50% agreement between analysts. This variability is mainly caused by the difficulty of reliably and reproducibly distinguishing single calls in an ABW chorus, which consists of overlapping distant calls. Furthermore, the performance of two automated detectors, one based on spectrogram correlation and one on subspace-detection strategy, was evaluated by comparing detector output to a “conservative” manually annotated reference data set, comprising only annotations that matched between analysts. This study highlights the need for a standardized approach for human annotations and automatic detections, including a quantitative description of their performance, to improve the comparability of acoustic data, which is particularly relevant in the context of collaborative approaches collecting and analyzing large passive acoustic data sets. Conference Object Antarc* Antarctic Blue whale Alfred Wegener Institute for Polar- and Marine Research (AWI): ePIC (electronic Publication Information Center) Antarctic |
institution |
Open Polar |
collection |
Alfred Wegener Institute for Polar- and Marine Research (AWI): ePIC (electronic Publication Information Center) |
op_collection_id |
ftawi |
language |
unknown |
description |
Evaluation of the performance of computer-based algorithms to automatically detect mammalian vocalizations often relies on comparisons between detector outputs and a reference data set, generally obtained by manual annotation of acoustic recordings. To explore the reproducibility of these manual annotations, we investigate inter- and intra-analyst variability in manually annotated Antarctic blue whale (ABW) Z-calls by two analysts using acoustic data from two ocean basins with different call abundance and background noise. Manual annotations exhibited strong inter- and intra-analyst variability, with less than 50% agreement between analysts. This variability is mainly caused by the difficulty of reliably and reproducibly distinguishing single calls in an ABW chorus, which consists of overlapping distant calls. Furthermore, the performance of two automated detectors, one based on spectrogram correlation and one on subspace-detection strategy, was evaluated by comparing detector output to a “conservative” manually annotated reference data set, comprising only annotations that matched between analysts. This study highlights the need for a standardized approach for human annotations and automatic detections, including a quantitative description of their performance, to improve the comparability of acoustic data, which is particularly relevant in the context of collaborative approaches collecting and analyzing large passive acoustic data sets. |
format |
Conference Object |
author |
Leroy, Emmanuelle Thomisch, Karolin Royer, J.-Y. Boebel, Olaf van Opzeeland, Ilse |
spellingShingle |
Leroy, Emmanuelle Thomisch, Karolin Royer, J.-Y. Boebel, Olaf van Opzeeland, Ilse On the reliability of acoustic annotation and automatic detections of Antarctic blue whale calls under different acoustic conditions |
author_facet |
Leroy, Emmanuelle Thomisch, Karolin Royer, J.-Y. Boebel, Olaf van Opzeeland, Ilse |
author_sort |
Leroy, Emmanuelle |
title |
On the reliability of acoustic annotation and automatic detections of Antarctic blue whale calls under different acoustic conditions |
title_short |
On the reliability of acoustic annotation and automatic detections of Antarctic blue whale calls under different acoustic conditions |
title_full |
On the reliability of acoustic annotation and automatic detections of Antarctic blue whale calls under different acoustic conditions |
title_fullStr |
On the reliability of acoustic annotation and automatic detections of Antarctic blue whale calls under different acoustic conditions |
title_full_unstemmed |
On the reliability of acoustic annotation and automatic detections of Antarctic blue whale calls under different acoustic conditions |
title_sort |
on the reliability of acoustic annotation and automatic detections of antarctic blue whale calls under different acoustic conditions |
publishDate |
2018 |
url |
https://epic.awi.de/id/eprint/48164/ https://hdl.handle.net/10013/epic.b3ff4f68-c00a-4fbe-9362-c9abaf565f8c |
geographic |
Antarctic |
geographic_facet |
Antarctic |
genre |
Antarc* Antarctic Blue whale |
genre_facet |
Antarc* Antarctic Blue whale |
op_source |
EPIC3DCLDE, Paris, 2018-05-05-2018-05-09 |
op_relation |
Leroy, E. , Thomisch, K. orcid:0000-0002-7144-8369 , Royer, J. Y. , Boebel, O. orcid:0000-0002-2259-0035 and van Opzeeland, I. (2018) On the reliability of acoustic annotation and automatic detections of Antarctic blue whale calls under different acoustic conditions , DCLDE, Paris, 5 May 2018 - 9 May 2018 . hdl:10013/epic.b3ff4f68-c00a-4fbe-9362-c9abaf565f8c |
_version_ |
1766223106496528384 |