Low-power, Continuous Remote Behavioral Localization with Event Cameras

• Main Authors: Hamann, Friedhelm, Ghosh, Suman, Martinez, Ignacio Juarez, Hart, Tom, Kacelnik, Alex, Gallego, Guillermo
• Format: Text
• Language: unknown
• Published: 2023
• Subjects: Computer Science - Computer Vision and Pattern Recognition; Computer Science - Artificial Intelligence; Antarc*; Antarctica
• Online Access: http://arxiv.org/abs/2312.03799

Researchers in natural science need reliable methods for quantifying animal behavior. Recently, numerous computer vision methods emerged to automate the process. However, observing wild species at remote locations remains a challenging task due to difficult lighting conditions and constraints on power supply and data storage. Event cameras offer unique advantages for battery-dependent remote monitoring due to their low power consumption and high dynamic range capabilities. We use this novel sensor to quantify a behavior in Chinstrap penguins called ecstatic display. We formulate the problem as a temporal action detection task, determining the start and end times of the behavior. For this purpose, we recorded a colony of breeding penguins in Antarctica during several weeks and labeled event data on 16 nests. The developed method consists of a generator of candidate time intervals (proposals) and a classifier of the actions within them. The experiments show that the event cameras' natural response to motion is effective for continuous behavior monitoring and detection, reaching a mean average precision (mAP) of 58% (which increases to 63% in good weather conditions). The results also demonstrate the robustness against various lighting conditions contained in the challenging dataset. The low-power capabilities of the event camera allows to record three times longer than with a conventional camera. This work pioneers the use of event cameras for remote wildlife observation, opening new interdisciplinary opportunities. https://tub-rip.github.io/eventpenguins/ Comment: 13 pages, 7 figures, 11 tables, Project page: https://tub-rip.github.io/eventpenguins/