Automated detection and classification of southern African Roman seabream using mask R-CNN

The availability of relatively cheap, high-resolution digital cameras has led to an exponential increase in the capture of natural environments and their inhabitants. Video-based surveys are particularly useful in the underwater domain where observation by humans can be expensive, dangerous, inaccessible, or destructive to the natural environment. However, a large majority of marine data has never gone through analysis by human experts – a process that is slow, expensive, and not scalable. We test a Mask R-CNN object detection framework for the automated localisation, classification, counting and tracking of fish in unconstrained underwater environments. We present a novel, labelled image dataset of roman seabream ( Chrysoblephus laticeps ), a fish species endemic to Southern Africa, to train and validate the accuracy of our model. The Mask R-CNN model accurately detected and classified roman seabream on the training dataset (mAP50 = 80.29%), validation dataset (mAP50 = 80.35%), as well as on previously unseen footage (test dataset) (mAP50 = 81.45%). The fact that the model performs well on previously unseen data suggests that it is capable of generalising to new streams of data not included in this research.