Navigating an Ocean of Video Data: Deep Learning for Humpback Whale Classification in YouTube Videos
Image analysis technologies empowered by artificial intelligence (AI) have proved images and videos to be an opportune source of data to learn about humpback whale (Megaptera novaeangliae) population sizes and dynamics. With the advent of social media, platforms such as YouTube present an abundance...
Main Author: | |
---|---|
Format: | Text |
Language: | unknown |
Published: |
2022
|
Subjects: | |
Online Access: | http://arxiv.org/abs/2212.00822 |
id |
ftarxivpreprints:oai:arXiv.org:2212.00822 |
---|---|
record_format |
openpolar |
spelling |
ftarxivpreprints:oai:arXiv.org:2212.00822 2023-09-05T13:20:04+02:00 Navigating an Ocean of Video Data: Deep Learning for Humpback Whale Classification in YouTube Videos Ramirez, Michelle 2022-12-01 http://arxiv.org/abs/2212.00822 unknown http://arxiv.org/abs/2212.00822 Computer Science - Computer Vision and Pattern Recognition Quantitative Biology - Quantitative Methods text 2022 ftarxivpreprints 2023-08-16T17:25:32Z Image analysis technologies empowered by artificial intelligence (AI) have proved images and videos to be an opportune source of data to learn about humpback whale (Megaptera novaeangliae) population sizes and dynamics. With the advent of social media, platforms such as YouTube present an abundance of video data across spatiotemporal contexts documenting humpback whale encounters from users worldwide. In our work, we focus on automating the classification of YouTube videos as relevant or irrelevant based on whether they document a true humpback whale encounter or not via deep learning. We use a CNN-RNN architecture pretrained on the ImageNet dataset for classification of YouTube videos as relevant or irrelevant. We achieve an average 85.7% accuracy, and 84.7% (irrelevant)/ 86.6% (relevant) F1 scores using five-fold cross validation for evaluation on the dataset. We show that deep learning can be used as a time-efficient step to make social media a viable source of image and video data for biodiversity assessments. Text Humpback Whale Megaptera novaeangliae ArXiv.org (Cornell University Library) |
institution |
Open Polar |
collection |
ArXiv.org (Cornell University Library) |
op_collection_id |
ftarxivpreprints |
language |
unknown |
topic |
Computer Science - Computer Vision and Pattern Recognition Quantitative Biology - Quantitative Methods |
spellingShingle |
Computer Science - Computer Vision and Pattern Recognition Quantitative Biology - Quantitative Methods Ramirez, Michelle Navigating an Ocean of Video Data: Deep Learning for Humpback Whale Classification in YouTube Videos |
topic_facet |
Computer Science - Computer Vision and Pattern Recognition Quantitative Biology - Quantitative Methods |
description |
Image analysis technologies empowered by artificial intelligence (AI) have proved images and videos to be an opportune source of data to learn about humpback whale (Megaptera novaeangliae) population sizes and dynamics. With the advent of social media, platforms such as YouTube present an abundance of video data across spatiotemporal contexts documenting humpback whale encounters from users worldwide. In our work, we focus on automating the classification of YouTube videos as relevant or irrelevant based on whether they document a true humpback whale encounter or not via deep learning. We use a CNN-RNN architecture pretrained on the ImageNet dataset for classification of YouTube videos as relevant or irrelevant. We achieve an average 85.7% accuracy, and 84.7% (irrelevant)/ 86.6% (relevant) F1 scores using five-fold cross validation for evaluation on the dataset. We show that deep learning can be used as a time-efficient step to make social media a viable source of image and video data for biodiversity assessments. |
format |
Text |
author |
Ramirez, Michelle |
author_facet |
Ramirez, Michelle |
author_sort |
Ramirez, Michelle |
title |
Navigating an Ocean of Video Data: Deep Learning for Humpback Whale Classification in YouTube Videos |
title_short |
Navigating an Ocean of Video Data: Deep Learning for Humpback Whale Classification in YouTube Videos |
title_full |
Navigating an Ocean of Video Data: Deep Learning for Humpback Whale Classification in YouTube Videos |
title_fullStr |
Navigating an Ocean of Video Data: Deep Learning for Humpback Whale Classification in YouTube Videos |
title_full_unstemmed |
Navigating an Ocean of Video Data: Deep Learning for Humpback Whale Classification in YouTube Videos |
title_sort |
navigating an ocean of video data: deep learning for humpback whale classification in youtube videos |
publishDate |
2022 |
url |
http://arxiv.org/abs/2212.00822 |
genre |
Humpback Whale Megaptera novaeangliae |
genre_facet |
Humpback Whale Megaptera novaeangliae |
op_relation |
http://arxiv.org/abs/2212.00822 |
_version_ |
1776200806686523392 |