End-to-End Segmentation and Classification of Zooplankton Using Shadowgraphy and Convolutional Neural Networks

Abstract

Highlights What are the main findings?1. Four CNN-based classification algorithms were successfully implemented for categorizing zooplankton with >95% accuracy using over 70k images. 2. Niched networks are a tractable, modular and efficient avenue for specific taxonomic or functional classification. What are the implications of the main findings?1. This segmentation and classification algorithm can potentially be used to study zooplankton interactions and various functional traits. 2. Its high accuracy when given out-of-distribution data shows that the algorithm can be used in diverse ecosystems. Abstract With in situ imaging systems becoming more common, precise, and economically viable, use of these systems has grown dramatically, including both automated classification and biomass estimations. However, a rather large and overlooked portion of these efforts is reliable detection and classification of these organisms as they pass through the imaging device. This paper focuses on the development of an end-to-end classification CNN-based algorithm for marine zooplankton using the in situ Ichthyoplankton Imaging System (ISIIS-DPI) from Bellamare (La Jolla, CA, USA). Our novel approach considers many issues with automated segmentation and classification, including over-segmentation, noise segmentation, and organism size input. This allows for classifications in diverse water types, demonstrated by the comparison of three datasets created in conjunction with this project, each with very different water properties and zooplankton communities (Florida Gulf coast; Trondheimsfjord, Norway; Sargasso Sea). Our segmented image dataset contains 70,624 regions of interest (ROIs) across four organism classes-Chaetognath, Crustacean, Gelatinous, and Larvacean-with two classes dedicated to detritus. Four common network architectures-Resnet, Xception, GoogleNet, and Darknet-are trained on this dataset, with final test accuracies in the range of 95.94-96.09%. Following this initial training, a secondary level of classification is introduced. The base Gelatinous class is further divided into six groups.