Due to the workforce shortage caused by the declining birth rate and aging population, robotics is one of the solutions to replace humans and overcome this urgent problem. This paper introduces a deep learning-based object detection algorithm for empty-dish recycling robots to automatically recycle dishes in restaurants and canteens, etc. In detail, a lightweight object detection model YOLO-GD (Ghost Net and Depthwise convolution) is proposed for detecting dishes in images such as cups, chopsticks, bowls, towels, etc., and an image processing-based catch point calculation is designed for extracting the catch point coordinates of the different-type dishes. The coordinates are used to recycle the target dishes by controlling the robot arm. Jetson Nano is equipped on the robot as a computer module, and the YOLO-GD model is also quantized by TensorRT for improving the performance. The experimental results demonstrate that the YOLO-GD model is only 1/5 size of the state-of-the-art model YOLOv4, and the mAP of YOLO-GD achieves 97.38%, 3.41% higher than YOLOv4. After quantization, the YOLO-GD model decreases the inference time per image from 207.92 ms to 32.75 ms, and the mAP is 97.42%, which is slightly higher than the model without quantization. Through the proposed image processing method, the catch points of various types of dishes are effectively extracted. The functions of empty-dish recycling are realized and will lead to further development toward practical use.

/pdf/yolo-gd-a-deep-learning-based-object-detection-algorithm-for-1j3ntct6.pdf

YOLO-GD: A Deep Learning-Based Object Detection Algorithm for Empty-Dish Recycling Robots

Abstract Deep neural networks (DNNs) have made significant achievements in a wide variety of domains. For the deep learning tasks, multiple excellent hardware platforms provide efficient solutions, including graphics processing units (GPUs), central processing units (CPUs), field programmable gate arrays (FPGAs), and application-specific integrated circuit (ASIC). Nonetheless, CPUs outperform other solutions including GPUs in many cases for the inference workload of DNNs with the support of various techniques, such as the high-performance libraries being the basic building blocks for DNNs. Thus, CPUs have been a preferred choice for DNN inference applications, particularly in the low-latency demand scenarios. However, the DNN inference efficiency remains a critical issue, especially when low latency is required under conditions with limited hardware resources, such as embedded systems. At the same time, the hardware features have not been fully exploited for DNNs and there is much room for improvement. To this end, this paper conducts a series of experiments to make a thorough study for the inference workload of prominent state-of-the-art DNN architectures on a single-instruction-multiple-data (SIMD) CPU platform, as well as with widely applicable scopes for multiple hardware platforms. The study goes into depth in DNNs: the CPU kernel-instruction level performance characteristics of DNNs including branches, branch prediction misses, cache misses, etc, and the underlying convolutional computing mechanism at the SIMD level; The thorough layer-wise time consumption details with potential time-cost bottlenecks; And the exhaustive dynamic activation sparsity with exact details on the redundancy of DNNs. The research provides researchers with comprehensive and insightful details, as well as crucial target areas for optimising and improving the efficiency of DNNs at both the hardware and software levels. 

/pdf/an-architecture-level-analysis-on-deep-learning-models-for-5jthl0q2.pdf

An architecture-level analysis on deep learning models for low-impact computations

https://onlinelibrary.wiley.com/doi/pdfdirect/10.1049/cit2.12104

The early Japanese books reorganization by combining image processing and deep learning

Deep-learning-based applications bring impressive results to graph machine learning and are widely used in fields such as autonomous driving and language translations. Nevertheless, the tremendous capacity of convolutional neural networks makes it difficult for them to be implemented on resource-constrained devices. Channel pruning provides a promising solution to compress networks by removing a redundant calculation. Existing pruning methods measure the importance of each filter and discard the less important ones until reaching a fixed compression target. However, the static approach limits the pruning effect. Thus, we propose a dynamic channel-pruning method that dynamically identifies and removes less important filters based on a redundancy analysis of its feature maps. Experimental results show that 77.10% of floating-point operations per second (FLOPs) and 91.72% of the parameters are reduced on VGG16BN with only a 0.54% accuracy drop. Furthermore, the compressed models were implemented on the field-programmable gate array (FPGA) and a significant speed-up was observed.

/pdf/feature-map-analysis-based-dynamic-cnn-pruning-and-the-8b3xix2r.pdf

Feature Map Analysis-Based Dynamic CNN Pruning and the Acceleration on FPGAs

The pooling function is vital for deep neural networks (DNNs). The operation is to generalize the representation of feature maps and progressively cut down the spatial size of feature maps to optimize the computing consumption of the network. Furthermore, the function is also the basis for the computer vision attention mechanism. However, as a matter of fact, pooling is a down-sampling operation, which makes the feature-map representation approximately to small translations with the summary statistic of adjacent pixels. As a result, the function inevitably leads to information loss more or less. In this article, we propose a fused max-average pooling (FMAPooling) operation as well as an improved channel attention mechanism (FMAttn) by utilizing the two pooling functions to enhance the feature representation for DNNs. Basically, the methods are to enhance multiple-level features extracted by max pooling and average pooling respectively. The effectiveness of the proposals is verified with VGG, ResNet, and MobileNetV2 architectures on CIFAR10/100 and ImageNet100. According to the experimental results, the FMAPooling brings up to 1.63% accuracy improvement compared with the baseline model; the FMAttn achieves up to 2.21% accuracy improvement compared with the previous channel attention mechanism. Furthermore, the proposals are extensible and could be embedded into various DNN models easily, or take the place of certain structures of DNNs. The computation burden introduced by the proposals is negligible.

Enhanced mechanisms of pooling and channel attention for deep learning feature maps

To accelerate the practical applications of artificial intelligence, this paper proposes a high efficient layer-wise refined pruning method for deep neural networks at the software level and accelerates the inference process at the hardware level on a field-programmable gate array (FPGA). The refined pruning operation is based on the channel-wise importance indexes of each layer and the layer-wise input sparsity of convolutional layers. The method utilizes the characteristics of the native networks without introducing any extra workloads to the training phase. In addition, the operation is easy to be extended to various state-of-the-art deep neural networks. The effectiveness of the method is verified on ResNet architecture and VGG networks in terms of dataset CIFAR10, CIFAR100, and ImageNet100. Experimental results show that in terms of ResNet50 on CIFAR10 and ResNet101 on CIFAR100, more than 85% of parameters and Floating-Point Operations are pruned with only 0.35% and 0.40% accuracy loss, respectively. As for the VGG network, 87.05% of parameters and 75.78% of Floating-Point Operations are pruned with only 0.74% accuracy loss for VGG13BN on CIFAR10. Furthermore, we accelerate the networks at the hardware level on the FPGA platform by utilizing the tool Vitis AI. For two threads mode in FPGA, the throughput/fps of the pruned VGG13BN and ResNet101 achieves 151.99 fps and 124.31 fps, respectively, and the pruned networks achieve about 4.3× and 1.8× speed up for VGG13BN and ResNet101, respectively, compared with the original networks on FPGA.

https://downloads.hindawi.com/journals/cin/2022/8039281.pdf

Optimizing the Deep Neural Networks by Layer-Wise Refined Pruning and the Acceleration on FPGA

Oracle bone inscriptions (OBIs) are a kind of hieroglyph, used about 3,600 years ago for divination and the recording of events. The characters on these OBIs are of great interest because they are precursors to the modern Chinese characters widely used across Asia today. However, as the OBIs were only discovered in 1899, there are currently few documents to describe them. Hence, recognizing and unlocking the meaning of OBIs helps to understand the ancient history of China, the evaluation of Chinese characters, and more. Currently, deep learning has made great progress and brought about a revolution in the research field of recognition, and shows good potential to meet the challenges of OBIs recognition. Due to the scarcity of resources, many OBIs contain only a small number of instances, which causes dataset imbalance and limits the accuracy of recognition. This paper attempts to provide a suite of OBIs recognition methods comprising an original OBIs dataset creation, dynamic dataset augmentation, and a novel deep learning-based recognition method. To this end, we create an original OBIs dataset and propose a modified Generative Adversarial Network for augmenting the original OBIs dataset. The augmented data is then dynamically selected for training the deep learning model considering the data imbalance problem. A novel model called C-A Net is proposed for OBIs recognition. The results of evaluation experiments show that the dynamical dataset augmentation can effectively locate a suitable training dataset for the deep learning model and solve the problem of imbalanced OBIs distribution. In addition, the recognition accuracy of C-A Net is 91.10%, which is higher than that of eight state-of-the-art models, and thus effectively suppresses the occurrence of overfitting. We also present an original OBIs dataset named OBI125, which is currently the only rubbing-type OBIs dataset that is open to the public. The code is available at http://www.ihpc.se.ritsumei.ac.jp/obidataset.html.

Dynamic Dataset Augmentation for Deep Learning-based Oracle Bone Inscriptions Recognition

The world is facing a shrinking workforce by the sagging birth rate and an aging population. Robot techniques are one of the best solutions for taking place of humans and overcoming this emergency issue. This paper introduces a deep learning-based empty-dish recycling robot for realizing the automatic empty-dish recycling after breakfast, dinner, or lunch in a restaurant, canteen, or cafeteria. A deep learning model–You Only Look Once (YOLO)–is equipped for dish detection such as cups, bowls, chopsticks, towels et al., and catch points are calculated for controlling the robot arm to recycle the target dishes. Finally, the YOLOv4 model is quantized by TensorRT and deployed on Jetson Nano. The real-time dish detection YOLO is focused on this paper, the experimental results show that after the YOLO model quantization, the detection time of a single image is increased from 3.93s to 0.44s, with more than 96.00% high accuracy on Precision, Recall, and F1 values. The functions of empty-dish recycling are realized, which will lead to further development toward practical use.

Lin Meng

Papers

Optimizing the Deep Neural Networks by Layer-Wise Refined Pruning and the Acceleration on FPGA

An architecture-level analysis on deep learning models for low-impact computations

Dynamic Dataset Augmentation for Deep Learning-based Oracle Bone Inscriptions Recognition

Deep Learning-based Real-time Object Detection for Empty-Dish Recycling Robot

The early Japanese books reorganization by combining image processing and deep learning