Evaluating Thermal and Color Sensors for Automating Detection of Penguins and Pinnipeds in Images Collected with an Unoccupied Aerial System

• Published in: Drones
• Main Authors: Jefferson T. Hinke, Louise M. Giuseffi, Victoria R. Hermanson, Samuel M. Woodman, Douglas J. Krause
• Format: Text
• Language: English
• Published: Multidisciplinary Digital Publishing Institute 2022
• Subjects: automated detection; Antarctica; drone; census; image analysis; Antarctic; Cape Shirreff; Livingston Island; Shirreff; Antarc*; Antarctic Fur Seals; Arctocephalus gazella
• Online Access: https://doi.org/10.3390/drones6090255

Estimating seabird and pinniped abundance is central to wildlife management and ecosystem monitoring in Antarctica. Unoccupied aerial systems (UAS) can collect images to support monitoring, but manual image analysis is often impractical. Automating target detection using deep learning techniques may improve data acquisition, but different image sensors may affect target detectability and model performance. We compared the performance of automated detection models based on infrared (IR) or color (RGB) images and tested whether IR images, or training data that included annotations of non-target features, improved model performance. For this assessment, we collected paired IR and RGB images of nesting penguins (Pygoscelis spp.) and aggregations of Antarctic fur seals (Arctocephalus gazella) with a small UAS at Cape Shirreff, Livingston Island (60.79 °W, 62.46 °S). We trained seven independent classification models using the Video and Image Analytics for Marine Environments (VIAME) software and created an open-access R tool, vvipr, to standardize the assessment of VIAME-based model performance. We found that the IR images and the addition of non-target annotations had no clear benefits for model performance given the available data. Nonetheless, the generally high performance of the penguin models provided encouraging results for further improving automated image analysis from UAS surveys.