Temporally-consistent 3D Reconstruction of Birds ...

• Main Authors: Hägerlind, Johannes, Hentati-Sundberg, Jonas, Wandt, Bastian
• Format: Article in Journal/Newspaper
• Language: unknown
• Published: arXiv 2024
• Subjects: Computer Vision and Pattern Recognition cs.CV; FOS: Computer and information sciences; Common Murre
• Online Access: https://dx.doi.org/10.48550/arxiv.2408.13629; https://arxiv.org/abs/2408.13629

This paper deals with 3D reconstruction of seabirds which recently came into focus of environmental scientists as valuable bio-indicators for environmental change. Such 3D information is beneficial for analyzing the bird's behavior and physiological shape, for example by tracking motion, shape, and appearance changes. From a computer vision perspective birds are especially challenging due to their rapid and oftentimes non-rigid motions. We propose an approach to reconstruct the 3D pose and shape from monocular videos of a specific breed of seabird - the common murre. Our approach comprises a full pipeline of detection, tracking, segmentation, and temporally consistent 3D reconstruction. Additionally, we propose a temporal loss that extends current single-image 3D bird pose estimators to the temporal domain. Moreover, we provide a real-world dataset of 10000 frames of video observations on average capture nine birds simultaneously, comprising a large variety of motions and interactions, including a smaller ...