Deep learning to extract the meteorological by-catch of wildlife cameras

• Main Authors: Alison, Jamie, Payne, Stephanie, Alexander, Jake, id_orcid:0 000-0003-2226-7913, Bjorkman, Anne D., Clark, Vincent Ralph, Gwate, Onalenna, Huntsaar, Maria, Iseli, Evelin, Lenoir, Jonathan, Mann, Hjalte Mads Rosenstand, Steenhuisen, Sandy-Lynn, Høye, Toke Thomas
• Format: Article in Journal/Newspaper
• Language: English
• Published: Wiley-Blackwell 2024
• Subjects: alpine ecology; automated monitoring; bees; micrometeorology; proximal sensing; snow melt; time-lapse photography; Trifolium pratense; Svalbard
• Online Access: https://hdl.handle.net/20.500.11850/648127; https://doi.org/10.3929/ethz-b-000648127

Microclimate—proximal climatic variation at scales of metres and minutes—can exacerbate or mitigate the impacts of climate change on biodiversity. However, most microclimate studies are temperature centric, and do not consider meteorological factors such as sunshine, hail and snow. Meanwhile, remote cameras have become a primary tool to monitor wild plants and animals, even at micro-scales, and deep learning tools rapidly convert images into ecological data. However, deep learning applications for wildlife imagery have focused exclusively on living subjects. Here, we identify an overlooked opportunity to extract latent, ecologically relevant meteorological information. We produce an annotated image dataset of micrometeorological conditions across 49 wildlife cameras in South Africa's Maloti-Drakensberg and the Swiss Alps. We train ensemble deep learning models to classify conditions as overcast, sunshine, hail or snow. We achieve 91.7% accuracy on test cameras not seen during training. Furthermore, we show how effective accuracy is raised to 96% by disregarding 14.1% of classifications where ensemble member models did not reach a consensus. For two-class weather classification (overcast vs. sunshine) in a novel location in Svalbard, Norway, we achieve 79.3% accuracy (93.9% consensus accuracy), outperforming a benchmark model from the computer vision literature (75.5% accuracy). Our model rapidly classifies sunshine, snow and hail in almost 2 million unlabelled images. Resulting micrometeorological data illustrated common seasonal patterns of summer hailstorms and autumn snowfalls across mountains in the northern and southern hemispheres. However, daily patterns of sunshine and shade diverged between sites, impacting daily temperature cycles. Crucially, we leverage micrometeorological data to demonstrate that (1) experimental warming using open-top chambers shortens early snow events in autumn, and (2) image-derived sunshine marginally outperforms sensor-derived temperature when predicting bumblebee foraging. ...