The explosion of digital photos has posed a challenge for storage and transmission bandwidth. The thesis briefly discusses my work on efficient image set compression. The lossless compression of near-duplicate image collection is carried out using multiple steps, where each step is computationally demanding. My task is to make them work faster without compromising on compression efficiency. In this pursuit, we propose approaches for efficient clustering, fast direction oriented motion estimation algorithm, and an image reordering scheme with minimum predictive costs for better compression. The preliminary results for the proposed approach are promising. We also aim to extend our approach to hyperspectral and medical image set compression.

Efficient Image Set Compression

Vehicle trajectory prediction is a keystone for the application of the internet of vehicles (IoV). With the help of deep learning and big data, it is possible to understand the between-vehicle interaction pattern hidden in the complex traffic environment. In this paper, we propose a novel spatial-temporal dynamic attention network for vehicle trajectory prediction, which can comprehensively capture temporal and social patterns in a hierarchical manner. The social relation between vehicles is captured at each timestamp and thus retains the dynamic variation of interaction. The temporal correlation in terms of individual motion state as well as social interaction is captured by different sequential models. Furthermore, a driving intention-specific feature fusion mechanism is proposed such that the extracted temporal and social features can be integrated adaptively for the maneuver-based multi-modal trajectory prediction. Experimental results on two real-world datasets show that compared with the state-of-the-art algorithms, our proposal achieves comparable prediction performance for short-term prediction, however, works much better for long-term prediction. Additionally, various ablation analysis is provided to evaluate the effectiveness of our proposed network components. The code will be available at https://xbchen82.github.io/resource/.

Intention-Aware Vehicle Trajectory Prediction Based on Spatial-Temporal Dynamic Attention Network for Internet of Vehicles

A3N: Attention-based adversarial autoencoder network for detecting anomalies in video sequence

Joint estimation of human body in point cloud is a key step for tracking human movements. In this work, we present a geometric method to achieve detection of the joints from a single-frame point cloud captured using a Time-of-Flight (ToF) camera. Three-dimensional (3D) human silhouette, as global feature of the single-frame point cloud, is extracted based on the pre-processed data, the angle and aspect ratio of the silhouette are subsequently utilized to perform pose recognition, and then 14 joints of human body are derived via geometric features of 3D silhouette. To verify this method, we test on an in-house captured 3D dataset containing 1200-frame depth images, which can be categorized into four different poses (upright, raising hands, parallel arms, and akimbo). Furthermore, we test on a subset of the G3D dataset. By hand-labelling the joints of each human body as the ground truth for validation and benchmarks, the average normalized error of our geometric method is less than 5.8 cm. When the distance threshold from the ground truth is 10 cm, the results demonstrate that our proposed method delivers improved performance with an average accuracy in the range of 90%.

/pdf/3d-joints-estimation-of-the-human-body-in-single-frame-point-3p250o8avd.pdf

3D Joints Estimation of the Human Body in Single-Frame Point Cloud

Vehicle trajectory prediction is a keystone for the application of the internet of vehicles (IoV). With the help of deep learning and big data, it is possible to understand the between-vehicle interaction pattern hidden in the complex traffic environment. In this paper, we propose a novel spatial-temporal dynamic attention network for vehicle trajectory prediction, which can comprehensively capture temporal and social patterns in a hierarchical manner. The social relation between vehicles is captured at each timestamp and thus retains the dynamic variation of interaction. The temporal correlation in terms of individual motion state as well as social interaction is captured by different sequential models. Furthermore, a driving intention-specific feature fusion mechanism is proposed such that the extracted temporal and social features can be integrated adaptively for the maneuver-based multi-modal trajectory prediction. Experimental results on two real-world datasets show that compared with the state-of-the-art algorithms, our proposal achieves comparable prediction performance for short-term prediction, however, works much better for long-term prediction. Additionally, various ablation analysis is provided to evaluate the effectiveness of our proposed network components. The code will be available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://xbchen82.github.io/resource/</uri> . 

Feature coding has been recently considered to facilitate intelligent video analysis for urban computing. Instead of raw videos, extracted features in the front-end are encoded and transmitted to the back-end for further processing. In this article, we present a lossless key-point sequence compression approach for efficient feature coding. The essence of this predict-and-encode strategy is to eliminate the spatial and temporal redundancies of key points in videos. Multiple prediction modes with an adaptive mode selection method are proposed to handle key-point sequences with various structures and motion. Experimental results validate the effectiveness of the proposed scheme on four types of widely used key-point sequences in video analysis.

Key-Point Sequence Lossless Compression for Intelligent Video Analysis

Motion estimation is one of the most crucial and time-consuming component of video compression methods. However, much research has been done to improve computational complexity at the expense of the loss in performance of matching of blocks. A novel block matching algorithm named efficient direction-oriented search is proposed. For this, the proposed algorithm firstly aims to dynamically switch between search regions based on the location of minimum distortion error. The search region dimension is also made adaptive for faster convergence. Then the computational complexity is reduced by using a proposed horizontal, vertical wings diamond search pattern and, two ± 45 ∘ inclined hexagon-shaped direction-oriented search patterns. For further speed-up in the search process, partial distortion calculations are employed. A method for optimal threshold value selection based on the distortion statistics for different partial distortion calculations is presented. The performance of the proposed algorithm is evaluated for different video sequences containing: slow, medium, fast, and directional motion content. The experimental results indicate that significant improvement in speed-up can be achieved while maintaining the better peak signal-to-noise-ratio performance. For directional motion video sequences, the proposed method even outperforms the full search algorithm with a significantly lower computational cost.

Efficient direction-oriented search algorithm for block motion estimation

Clustering and categorization of similar images using SIFT and SURF require a high computational cost. In this paper, a simple approach to reduce the cardinality of keypoint set and prune the dimension of SIFT and SURF feature descriptors for efficient image clustering is proposed. For this purpose, sparsely spaced (uniformly distributed) important keypoints are chosen. In addition, multiple reduced dimensional variants of SIFT and SURF descriptors are presented. Moreover, clustering time complexity is also improved by proposed contextual bag-of-features approach for partitioned keypoint set. The F-measure statistic is used to evaluate clustering performance on a California-ND dataset containing near-duplicate images. Clustering accuracy of the proposed pruned SIFT and SURF is found to be at par with traditional SIFT and SURF with a significant reduction in computational cost.

Pruning SIFT & SURF for Efficient Clustering of Near-duplicate Images

The huge amount of data in surveillance video coding demands high compression rates with lower computational requirements for efficient storage and archival. The motion estimation is a very time-consuming process in the traditional video coding framework, and hence reducing computational complexity is a pressing task, especially for surveillance videos. The presence of significant background proportion in surveillance videos makes its special case for coding. The existing surveillance video coding methods propose separate search mechanisms for background and foreground regions. However, they still suffer from misclassification and inefficient search strategies since it does not consider the inherent motion characteristics of the foreground regions. In this paper, a background-foreground-boundary aware block matching algorithm is proposed to exploit special characteristics of the surveillance videos. A novel three-step framework is proposed for boundary aware block matching process. For this, firstly, the blocks are categorized into three classes, namely, background, foreground, and boundary blocks. Secondly, the motion search is performed by employing different search strategies for each class. The zero-motion vector-based search is employed for background blocks. Whereas, to exploit fast and directional motion characteristics of the boundary and foreground blocks, the eight rotating uni-wing diamond search patterns are proposed. Thirdly, the speed-up is achieved through the novel region-based sub-sampled structure. The experimental results demonstrate that two to four times speed-up over existing methods can be achieved through this scheme while maintaining better matching accuracy.

Tushar Shankar Shinde

Papers

Key-Point Sequence Lossless Compression for Intelligent Video Analysis

Efficient Image Set Compression

Efficient direction-oriented search algorithm for block motion estimation

Pruning SIFT & SURF for Efficient Clustering of Near-duplicate Images

Background foreground boundary aware efficient motion search for surveillance videos