Multimodal Deep Learning for Heterogeneous GNSS-R Data Fusion and Ocean Wind Speed Retrieval

The comprehensiveness of the raw input data and the effectiveness of feature engineering are two key factors affecting the performance of machine learning. To improve the data comprehensiveness for Global Navigation Satellite System Reflectometry (GNSS-R) ocean wind speed retrieval, this article int...

Full description

Bibliographic Details
Published in:IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
Main Authors: Chu, Xiaohan, He, Jie, Song, Hongqing, Qi, Yue, Sun, Yueqiang, Bai, Weihua, Li, Wei, Wu, Qiwu
Format: Report
Language:English
Published: 2020
Subjects:
Feature extraction
Wind speed
Data models
Sea surface
Sea measurements
Machine learning
Delay-Doppler map (DDM)
GNSS-reflectometry (GNSS-R)
multimodal deep learning
ocean surface wind retrieval
SEA-ICE
SCATTERING
Sea ice
Online Access:http://ir.nssc.ac.cn/handle/122/7580
https://doi.org/10.1109/JSTARS.2020.3010879
Description
Summary:The comprehensiveness of the raw input data and the effectiveness of feature engineering are two key factors affecting the performance of machine learning. To improve the data comprehensiveness for Global Navigation Satellite System Reflectometry (GNSS-R) ocean wind speed retrieval, this article introduces a new input data structure, which is composed of Delay-Doppler maps (DDM) and all satellite receiver status (SRS) parameters. Then, to overcome the difficulty of handcrafted feature engineering and effectively fusion the information of DDM and SRS, we presented a heterogeneous multimodal deep learning (HMDL) method to retrieve the wind speed according to the heterogeneity of the input data. The proposed model is verified by the performance evaluation of realistic data sets obtained from TDS-1. The new input data structure improves the prediction accuracy at 13.5% to 30.7% on mean absolute error (MAE) at 10.6% to 29.5% on the root mean square error (RMSE). The HMDL improves the prediction accuracy at 7.7% on MAE and 7.1% on RMSE. The whole proposed solution improves the prediction accuracy at 36.3% on MAE and 36.8% on RMSE, comparing with the traditional neural network-based solution. The results clearly show that both the introduction of the new input data structure and HMDL effectively improve the accuracy and robustness of GNSS-R wind speed retrieval.

Similar Items