Detection and tracking of belugas, kayaks and motorized boats in drone video using deep learning
Aerial imagery surveys are commonly used in marine mammal research to determine population size, distribution and habitat use. Analysis of aerial photos involves hours of manually identifying individuals present in each image and converting raw counts into useable biological statistics. Our research...
Published in: | Journal of Unmanned Vehicle Systems |
---|---|
Main Authors: | , , , |
Format: | Article in Journal/Newspaper |
Language: | English |
Published: |
Canadian Science Publishing
2022
|
Subjects: | |
Online Access: | http://dx.doi.org/10.1139/juvs-2021-0024 https://cdnsciencepub.com/doi/full-xml/10.1139/juvs-2021-0024 https://cdnsciencepub.com/doi/pdf/10.1139/juvs-2021-0024 |
id |
crcansciencepubl:10.1139/juvs-2021-0024 |
---|---|
record_format |
openpolar |
spelling |
crcansciencepubl:10.1139/juvs-2021-0024 2024-04-28T08:14:29+00:00 Detection and tracking of belugas, kayaks and motorized boats in drone video using deep learning Harasyn, Madison L. Chan, Wayne S. Ausen, Emma L. Barber, David G. 2022 http://dx.doi.org/10.1139/juvs-2021-0024 https://cdnsciencepub.com/doi/full-xml/10.1139/juvs-2021-0024 https://cdnsciencepub.com/doi/pdf/10.1139/juvs-2021-0024 en eng Canadian Science Publishing http://www.nrcresearchpress.com/page/about/CorporateTextAndDataMining Drone Systems and Applications volume 10, issue 1, page 77-96 ISSN 2564-4939 journal-article 2022 crcansciencepubl https://doi.org/10.1139/juvs-2021-0024 2024-04-02T06:55:54Z Aerial imagery surveys are commonly used in marine mammal research to determine population size, distribution and habitat use. Analysis of aerial photos involves hours of manually identifying individuals present in each image and converting raw counts into useable biological statistics. Our research proposes the use of deep learning algorithms to increase the efficiency of the marine mammal research workflow. To test the feasibility of this proposal, the existing YOLOv4 convolutional neural network model was trained to detect belugas, kayaks and motorized boats in oblique drone imagery, collected from a stationary tethered system. Automated computer-based object detection achieved the following precision and recall, respectively, for each class: beluga = 74%/72%; boat = 97%/99%; and kayak = 96%/96%. We then tested the performance of computer vision tracking of belugas and occupied watercraft in drone videos using the DeepSORT tracking algorithm, which achieved a multiple-object tracking accuracy (MOTA) ranging from 37% to 88% and multiple object tracking precision (MOTP) between 63% and 86%. Results from this research indicate that deep learning technology can detect and track features more consistently than human annotators, allowing for larger datasets to be processed within a fraction of the time while avoiding discrepancies introduced by labeling fatigue or multiple human annotators. Article in Journal/Newspaper Beluga Beluga* Canadian Science Publishing Journal of Unmanned Vehicle Systems |
institution |
Open Polar |
collection |
Canadian Science Publishing |
op_collection_id |
crcansciencepubl |
language |
English |
description |
Aerial imagery surveys are commonly used in marine mammal research to determine population size, distribution and habitat use. Analysis of aerial photos involves hours of manually identifying individuals present in each image and converting raw counts into useable biological statistics. Our research proposes the use of deep learning algorithms to increase the efficiency of the marine mammal research workflow. To test the feasibility of this proposal, the existing YOLOv4 convolutional neural network model was trained to detect belugas, kayaks and motorized boats in oblique drone imagery, collected from a stationary tethered system. Automated computer-based object detection achieved the following precision and recall, respectively, for each class: beluga = 74%/72%; boat = 97%/99%; and kayak = 96%/96%. We then tested the performance of computer vision tracking of belugas and occupied watercraft in drone videos using the DeepSORT tracking algorithm, which achieved a multiple-object tracking accuracy (MOTA) ranging from 37% to 88% and multiple object tracking precision (MOTP) between 63% and 86%. Results from this research indicate that deep learning technology can detect and track features more consistently than human annotators, allowing for larger datasets to be processed within a fraction of the time while avoiding discrepancies introduced by labeling fatigue or multiple human annotators. |
format |
Article in Journal/Newspaper |
author |
Harasyn, Madison L. Chan, Wayne S. Ausen, Emma L. Barber, David G. |
spellingShingle |
Harasyn, Madison L. Chan, Wayne S. Ausen, Emma L. Barber, David G. Detection and tracking of belugas, kayaks and motorized boats in drone video using deep learning |
author_facet |
Harasyn, Madison L. Chan, Wayne S. Ausen, Emma L. Barber, David G. |
author_sort |
Harasyn, Madison L. |
title |
Detection and tracking of belugas, kayaks and motorized boats in drone video using deep learning |
title_short |
Detection and tracking of belugas, kayaks and motorized boats in drone video using deep learning |
title_full |
Detection and tracking of belugas, kayaks and motorized boats in drone video using deep learning |
title_fullStr |
Detection and tracking of belugas, kayaks and motorized boats in drone video using deep learning |
title_full_unstemmed |
Detection and tracking of belugas, kayaks and motorized boats in drone video using deep learning |
title_sort |
detection and tracking of belugas, kayaks and motorized boats in drone video using deep learning |
publisher |
Canadian Science Publishing |
publishDate |
2022 |
url |
http://dx.doi.org/10.1139/juvs-2021-0024 https://cdnsciencepub.com/doi/full-xml/10.1139/juvs-2021-0024 https://cdnsciencepub.com/doi/pdf/10.1139/juvs-2021-0024 |
genre |
Beluga Beluga* |
genre_facet |
Beluga Beluga* |
op_source |
Drone Systems and Applications volume 10, issue 1, page 77-96 ISSN 2564-4939 |
op_rights |
http://www.nrcresearchpress.com/page/about/CorporateTextAndDataMining |
op_doi |
https://doi.org/10.1139/juvs-2021-0024 |
container_title |
Journal of Unmanned Vehicle Systems |
_version_ |
1797580522131554304 |