A multitask joint framework for real-time person search

Person searches generally involve three important parts: person detection, feature extraction and identity comparison. However, a person search integrating detection, extraction and comparison has the two following drawbacks. First, the accuracy of detection will affect the accuracy of comparison. Second, it is difficult to achieve real-time results in real-world applications. To solve these problems, we propose a multitask joint framework for real-time person search (MJF) that optimizes person detection, feature extraction and identity comparison. For the person detection module, we propose the YOLOv5-GS model, which is trained with a person dataset. YOLOv5-GS combines the advantages of the Ghostnet and the squeeze-and-excitation block and improves the speed of person detection. For the feature extraction module, we design a model adaptation architecture, which can select different networks according to the number of people. It can balance the relationship between accuracy and speed. For identity comparison, we propose a 3D pooled table and a matching strategy to improve identification accuracy. On the condition of 1920 × 1080-resolution video and a 200-ID table, the IR and the FPS achieved by our method reach 82.69% and 25.14, respectively. Therefore, the MJF can achieve real-time person search.