Deep learning to extract the meteorological by-catch of wildlife cameras

• Main Authors: Alison, Jamie, Payne, Stephanie, Alexander, Jake, Bjorkman, Anne, Clark, Vincent Ralph, Gwate, Onalenna, Huntsaar, Maria, Iseli, Evelin, Lenoir, Jonathan, Mann, Hjalte Mads Rosenstand, Steenhuisen, Sandy‐lynn, Høye, Toke Thomas
• Other Authors: Department of Ecoscience Aarhus, Aarhus University Aarhus, Afromontane Research Unit Bloemfontein (ARU), University of the Free State South Africa (UFS), Department of Plant Sciences Qwaqwa Campus, Institute for Integrative Biology Zürich (IBZ), Eidgenössische Technische Hochschule - Swiss Federal Institute of Technology Zürich (ETH Zürich), Department of biological & environmental sciences, Göteborgs Universitet = University of Gothenburg (GU), Gothenburg Global Biodiversity Centre, Geography Department Qwaqwa Campus, Department of Arctic Biology, The University Centre in Svalbard (UNIS), Department of Arctic and Marine Biology, University of Tromsø (UiT), Ecologie et Dynamique des Systèmes Anthropisés - UMR CNRS 7058 UPJV (EDYSAN), Université de Picardie Jules Verne (UPJV)-Centre National de la Recherche Scientifique (CNRS), Arctic Research Centre Aarhus (ARC)
• Format: Article in Journal/Newspaper
• Language: English
• Published: CCSD 2024
• Subjects: [SDE]Environmental Sciences; Svalbard; Norway
• Online Access: https://u-picardie.hal.science/hal-04517824; https://u-picardie.hal.science/hal-04517824v1/document; https://u-picardie.hal.science/hal-04517824v1/file/Alison_al_preprint_GCB.pdf; https://doi.org/10.1101/2023.09.25.558780

International audience Microclimate – proximal climatic variation at scales of meters and minutes – can exacerbate or mitigate the impacts of climate change on biodiversity. However, most microclimate studies are temperature-centric, and do not consider meteorological factors such as sunshine, hail and snow. Meanwhile, remote cameras have become a primary tool to monitor wild plants and animals, even at micro-scales, with deep learning tools to rapidly convert images into ecological data. However, deep learning applications for wildlife imagery have focused exclusively on living subjects. Here, we identify an overlooked opportunity to extract latent, yet ecologically relevant, meteorological information. We produce an annotated image dataset of micrometeorological conditions across 49 wildlife cameras in South Africa’s Maloti-Drakensberg and the Swiss Alps. We train ensemble deep learning models to classify conditions as overcast, sunshine, hail or snow. Our best model achieves 91.7% accuracy on test cameras not seen during training. Furthermore, we show how effective accuracy is raised to 96% by disregarding 14.1% of classifications where ensemble member models did not reach a consensus. Unlike previous studies we test model performance in remote and novel locations, distinguishing overcast and sunny conditions in Svalbard, Norway with 79.3% accuracy (93.9% consensus accuracy). Our model rapidly classifies sunshine, snow and hail in almost 2 million unlabeled images. Resulting micrometeorological data illustrated common seasonal patterns of summer hailstorms and autumn snowfalls across mountains in the northern and southern hemispheres. However, daily patterns of sunshine and shade diverged between sites, impacting daily temperature cycles. Crucially, we leverage micrometeorological data to demonstrate that (1) experimental warming using open-top chambers shortens early snow events in autumn, and (2) image-derived sunshine marginally outperforms sensor-derived temperature when predicting bumblebee foraging. ...