Multi-View and Multi-Scale Fine-Grained Vehicle Classification with Channel Convolution Feature Fusion

TLDR: In this article, a fine-grained vehicle classification using a multi-branch convolutional neural network (CNN) with multiple views of the vehicle was proposed to solve the vehicle's make classification task.

Abstract: A computer vision solution applied to an automatic toll collection (ATC) with a subscription/membership is proposed in this paper. In this application, a unique identifier (ID) is related to a concrete vehicle and a membership. A camera system is put in place to verify that for each transaction the vehicle and the ID correspond with the actual membership data. The visual system extracts different vehicle characteristics including license plate number, make, model, color, number of axles, etc. The system then compares the extracted characteristics with those found in the membership. We focus on solving the vehicle's make classification task. We propose a fine-grained vehicle classification that exploits the multi-camera composition of the system by feeding a multi-branch convolutional neural network (CNN) with multiple views of the vehicle. Each branch of the network uses a cascade approach to localize the vehicle and its most salient regions, as well as extracting multi-scale features per view. The extracted features are late fused using a convolutional approach and used to classify the vehicle's make. Our network learns to extract discriminant features from different views and regions of interest and to fuse them in the best possible way to improve classification performance. The presented evaluations show that the proposed multi-view network architecture significantly improves the vehicle's make classification performance when compared to single view approaches.