Multi-Sensor 3D Data Visualization in Virtual Reality for Planetary Science and Mission Operations

Ferrous Ward, Cody Paige, Don Derek Haddad, Jess Todd, Jennifier Heldmann, Darlene Lim, Dava Newman, Ariel Ekblaw, ICES506: Human Exploration Beyond Low Earth Orbit: Missions and Technologies The 52nd International Conference on Environmental Systems was held in Calgary, Canada, on 16 July 2023 thro...

Full description

Bibliographic Details
Main Authors: Ward, Ferrous, Paige, Cody, Haddad, Don Derek, Todd, Jess, Heldmann, Jennifier, Lim, Darlene, Ekblaw, Ariel, Newman, Dava
Format: Conference Object
Language:English
Published: 2023 International Conference on Environmental Systems 2023
Subjects:
Online Access:https://hdl.handle.net/2346/94734
Description
Summary:Ferrous Ward, Cody Paige, Don Derek Haddad, Jess Todd, Jennifier Heldmann, Darlene Lim, Dava Newman, Ariel Ekblaw, ICES506: Human Exploration Beyond Low Earth Orbit: Missions and Technologies The 52nd International Conference on Environmental Systems was held in Calgary, Canada, on 16 July 2023 through 20 July 2023. The MIT Resource Exploration and Science of our Cosmic Environment (RESOURCE) project aims to bridge the operations gap between current uses of VR as a training tool and future uses as an immersive interface for mission support operations and science. We describe the development of "digital twin" field sites within a VR environment from several low cost and commercially available sensors for photography, LiDAR, combination LiDAR-RGB, and photospectrometry. Three-dimensional point-cloud datasets of rocky outcrops were taken from Marblehead, Massachusetts and Svalbard, Norway. These sites were selected for the presence of features across multiple scales and fidelity as analog sites for a terrestrial space environment. They contain large landscape features like hills, depressions or permafrost features (>>1m scale), boulders (~1m scale), small rocks (~10cm), and small scale geologic features such as veining or fracturing (<1cm). These data were collected using a Velodyne LiDAR, Sony digital photography camera, DJI aerial drone, and Hamamatsu mini-spectrometer. Depth models were reconstructed with Open3D and AliceVision/Meshroom. These models were aligned and exported to a VR-capable file format. Geologic data was manually checked for key feature presence, and the point clouds were compared for distance accuracy and feature matching with the photogrammetric model in CloudCompare. Finally, the aligned models and point clouds were rendered in the Unreal 5 VR game engine. As part of this work, we discuss the data collection techniques for dense LiDAR point clouds, as well as handheld and aerial drone photogrammetry. We also discuss considerations for data resolution and ...