Improve automatic detection of animal call sequences with temporal context
Many animals rely on long-form communication, in the form of songs, for vital functions such as mate attraction and territorial defence. We explored the prospect of improving automatic recognition performance by using the temporal context inherent in song. The ability to accurately detect sequences...
Published in: | Journal of The Royal Society Interface |
---|---|
Main Authors: | , , , , , , , , , , |
Format: | Text |
Language: | English |
Published: |
The Royal Society
2021
|
Subjects: | |
Online Access: | http://www.ncbi.nlm.nih.gov/pmc/articles/PMC8292017/ http://www.ncbi.nlm.nih.gov/pubmed/34283944 https://doi.org/10.1098/rsif.2021.0297 |
id |
ftpubmed:oai:pubmedcentral.nih.gov:8292017 |
---|---|
record_format |
openpolar |
spelling |
ftpubmed:oai:pubmedcentral.nih.gov:8292017 2023-05-15T15:36:38+02:00 Improve automatic detection of animal call sequences with temporal context Madhusudhana, Shyam Shiu, Yu Klinck, Holger Fleishman, Erica Liu, Xiaobai Nosal, Eva-Marie Helble, Tyler Cholewiak, Danielle Gillespie, Douglas Širović, Ana Roch, Marie A. 2021-07-21 http://www.ncbi.nlm.nih.gov/pmc/articles/PMC8292017/ http://www.ncbi.nlm.nih.gov/pubmed/34283944 https://doi.org/10.1098/rsif.2021.0297 en eng The Royal Society http://www.ncbi.nlm.nih.gov/pmc/articles/PMC8292017/ http://www.ncbi.nlm.nih.gov/pubmed/34283944 http://dx.doi.org/10.1098/rsif.2021.0297 © 2021 The Authors. https://creativecommons.org/licenses/by/4.0/Published by the Royal Society under the terms of the Creative Commons Attribution License http://creativecommons.org/licenses/by/4.0/ (https://creativecommons.org/licenses/by/4.0/) , which permits unrestricted use, provided the original author and source are credited. CC-BY J R Soc Interface Life Sciences–Mathematics interface Text 2021 ftpubmed https://doi.org/10.1098/rsif.2021.0297 2021-07-25T00:46:35Z Many animals rely on long-form communication, in the form of songs, for vital functions such as mate attraction and territorial defence. We explored the prospect of improving automatic recognition performance by using the temporal context inherent in song. The ability to accurately detect sequences of calls has implications for conservation and biological studies. We show that the performance of a convolutional neural network (CNN), designed to detect song notes (calls) in short-duration audio segments, can be improved by combining it with a recurrent network designed to process sequences of learned representations from the CNN on a longer time scale. The combined system of independently trained CNN and long short-term memory (LSTM) network models exploits the temporal patterns between song notes. We demonstrate the technique using recordings of fin whale (Balaenoptera physalus) songs, which comprise patterned sequences of characteristic notes. We evaluated several variants of the CNN + LSTM network. Relative to the baseline CNN model, the CNN + LSTM models reduced performance variance, offering a 9–17% increase in area under the precision–recall curve and a 9–18% increase in peak F1-scores. These results show that the inclusion of temporal information may offer a valuable pathway for improving the automatic recognition and transcription of wildlife recordings. Text Balaenoptera physalus Fin whale PubMed Central (PMC) Journal of The Royal Society Interface 18 180 20210297 |
institution |
Open Polar |
collection |
PubMed Central (PMC) |
op_collection_id |
ftpubmed |
language |
English |
topic |
Life Sciences–Mathematics interface |
spellingShingle |
Life Sciences–Mathematics interface Madhusudhana, Shyam Shiu, Yu Klinck, Holger Fleishman, Erica Liu, Xiaobai Nosal, Eva-Marie Helble, Tyler Cholewiak, Danielle Gillespie, Douglas Širović, Ana Roch, Marie A. Improve automatic detection of animal call sequences with temporal context |
topic_facet |
Life Sciences–Mathematics interface |
description |
Many animals rely on long-form communication, in the form of songs, for vital functions such as mate attraction and territorial defence. We explored the prospect of improving automatic recognition performance by using the temporal context inherent in song. The ability to accurately detect sequences of calls has implications for conservation and biological studies. We show that the performance of a convolutional neural network (CNN), designed to detect song notes (calls) in short-duration audio segments, can be improved by combining it with a recurrent network designed to process sequences of learned representations from the CNN on a longer time scale. The combined system of independently trained CNN and long short-term memory (LSTM) network models exploits the temporal patterns between song notes. We demonstrate the technique using recordings of fin whale (Balaenoptera physalus) songs, which comprise patterned sequences of characteristic notes. We evaluated several variants of the CNN + LSTM network. Relative to the baseline CNN model, the CNN + LSTM models reduced performance variance, offering a 9–17% increase in area under the precision–recall curve and a 9–18% increase in peak F1-scores. These results show that the inclusion of temporal information may offer a valuable pathway for improving the automatic recognition and transcription of wildlife recordings. |
format |
Text |
author |
Madhusudhana, Shyam Shiu, Yu Klinck, Holger Fleishman, Erica Liu, Xiaobai Nosal, Eva-Marie Helble, Tyler Cholewiak, Danielle Gillespie, Douglas Širović, Ana Roch, Marie A. |
author_facet |
Madhusudhana, Shyam Shiu, Yu Klinck, Holger Fleishman, Erica Liu, Xiaobai Nosal, Eva-Marie Helble, Tyler Cholewiak, Danielle Gillespie, Douglas Širović, Ana Roch, Marie A. |
author_sort |
Madhusudhana, Shyam |
title |
Improve automatic detection of animal call sequences with temporal context |
title_short |
Improve automatic detection of animal call sequences with temporal context |
title_full |
Improve automatic detection of animal call sequences with temporal context |
title_fullStr |
Improve automatic detection of animal call sequences with temporal context |
title_full_unstemmed |
Improve automatic detection of animal call sequences with temporal context |
title_sort |
improve automatic detection of animal call sequences with temporal context |
publisher |
The Royal Society |
publishDate |
2021 |
url |
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC8292017/ http://www.ncbi.nlm.nih.gov/pubmed/34283944 https://doi.org/10.1098/rsif.2021.0297 |
genre |
Balaenoptera physalus Fin whale |
genre_facet |
Balaenoptera physalus Fin whale |
op_source |
J R Soc Interface |
op_relation |
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC8292017/ http://www.ncbi.nlm.nih.gov/pubmed/34283944 http://dx.doi.org/10.1098/rsif.2021.0297 |
op_rights |
© 2021 The Authors. https://creativecommons.org/licenses/by/4.0/Published by the Royal Society under the terms of the Creative Commons Attribution License http://creativecommons.org/licenses/by/4.0/ (https://creativecommons.org/licenses/by/4.0/) , which permits unrestricted use, provided the original author and source are credited. |
op_rightsnorm |
CC-BY |
op_doi |
https://doi.org/10.1098/rsif.2021.0297 |
container_title |
Journal of The Royal Society Interface |
container_volume |
18 |
container_issue |
180 |
container_start_page |
20210297 |
_version_ |
1766367008253804544 |