A deep learning-based method to identify and count pelagic and mesopelagic fishes from trawl camera images

• Published in: ICES Journal of Marine Science
• Main Authors: Allken, Vaneeda, Rosen, Shale, Handegard, Nils Olav, Malde, Ketil
• Other Authors: Demer, David, Research Council of Norway, Norwegian Ministry of Trade, Industry and Fisheries
• Format: Article in Journal/Newspaper
• Language: English
• Published: Oxford University Press (OUP) 2021
• Subjects: Norwegian Sea
• Online Access: http://dx.doi.org/10.1093/icesjms/fsab227; https://academic.oup.com/icesjms/article-pdf/78/10/3780/41772702/fsab227.pdf

Abstract Fish counts and species information can be obtained from images taken within trawls, which enables trawl surveys to operate without extracting fish from their habitat, yields distribution data at fine scale for better interpretation of acoustic results, and can detect fish that are not retained in the catch due to mesh selection. To automate the process of image-based fish detection and identification, we trained a deep learning algorithm (RetinaNet) on images collected from the trawl-mounted Deep Vision camera system. In this study, we focused on the detection of blue whiting, Atlantic herring, Atlantic mackerel, and mesopelagic fishes from images collected in the Norwegian sea. To address the need for large amounts of annotated data to train these models, we used a combination of real and synthetic images, and obtained a mean average precision of 0.845 on a test set of 918 images. Regression models were used to compare predicted fish counts, which were derived from RetinaNet classification of fish in the individual image frames, with catch data collected at 20 trawl stations. We have automatically detected and counted fish from individual images, related these counts to the trawl catches, and discussed how to use this in regular trawl surveys.