Cumulative link models for deep ordinal classification

This paper proposes a deep convolutional neural network model for ordinal regression by considering a family of probabilistic ordinal link functions in the output layer. The link functions are those used for cumulative link models, which are traditional statistical linear models based on projecting each pattern into a 1-dimensional space. A set of ordered thresholds splits this space into the different classes of the problem. In our case, the projections are estimated by a non-linear deep neural network. To further improve the results, we combine these ordinal models with a loss function that takes into account the distance between the categories, based on the weighted Kappa index. Three different link functions are studied in the experimental study, and the results are contrasted with statistical analysis. The experiments run over two different ordinal classification problems and the statistical tests confirm that these models improve the results of a nominal model and outperform other robust proposals considered in the literature.

This paper proposes a sparse representation layer in the feature extraction stage of a convolutional neural network (CNN). Our goal is to add sparse transforms to a target network to improve its performance without introducing an extra calculation burden. First, the proposed method was achieved by inserting the sparse representation layers into a target network’s shallow layers, and the network was trained end-to-end using a supervised learning algorithm. Second, In the forward pass the network captured the features through the convolutional layers and sparse representation layers accomplished with wavelet and shearlet transforms. Thirdly, in the backward pass the weights of the learned kernels of the network were updated through a back-propagated error, while the sparse representation layers were fixed and did not require updating. The proposed method was verified on five datasets with the task of image classification: FOOD-101, CIFAR10/100, DTD, Brodatz and ImageNet. The experimental results show that the proposed method leads to higher recognition accuracy in image classification, and the additional computational cost is relatively small compared to the baseline CNN model.

A convolutional neural network with sparse representation

Unimodal regularisation based on beta distribution for deep ordinal regression

https://archive-ouverte.unige.ch/unige:165022/ATTACHMENT01

Lingling Fang

Papers

Image classification with an RGB-channel nonsubsampled contourlet transform and a convolutional neural network