The invention relates to a cascaded convolutional neural network-based quick detection method of an irregular-object grasping pose of a robot. Firstly, a cascaded-type two-stage convolutional-neural-network model of a position-attitude rough-to-fine form is constructed, in a first stage, a region-based fully convolutional network (R-FCN) is adopted to realize grasping positioning and rough estimation of a grasping angle, and in a second stage, accurate calculation of the grasping angle is realized through constructing a new Angle-Net model; and then current scene images containing to-be-grasped objects are collected to be used as original on-site image samples to be used for training, the two-stage convolutional-neural-network model is trained by means of a transfer learning mechanism, then each collected monocular color image is input to the cascaded-type two-stage convolutional-neural-network model in online running, and finally, an end executor of the robot is driven by an obtainedgrasping position and attitude for object grasping control. According to the method, grasping detection accuracy is high, detection speed of the irregular-object grasping pose of the robot is effectively increased, and real-time performance of running of a grasping attitude detection algorithm is improved.

Cascaded convolutional neural network-based quick detection method of irregular-object grasping pose of robot

The invention discloses a mechanical arm control method and device based on deep learning. The mechanical arm control method includes the steps of collecting pictures of a grabbed object; according tothe collected pictures, calculating an optimal grab process; according to the optimal grab process, controlling a mechanical arm to grab the object. According to the mechanical arm control method anddevice based on deep learning, by collecting the plane and deep three-dimensional pictures, features are extracted and output through a convolutional neural network, and a rectangular grab frame is used for quantitatively showing grab strategies, wherein grab information includes the spatial location, coordinates and posture of the object, the size of the portion cooperating with grab, and the like; meanwhile, all alternative rectangular frames are compared, scored and sorted to select an optimal item, two neural training networks with different sizes are designed, the smaller network is usedfor first-round screening, the larger network is used for sorting the alternative grab strategies, and the robustness and accuracy of mechanical arm grab tasks are improved.

Mechanical arm control method and device based on deep learning

The invention discloses a robot target recognition method and device and a robot. The method comprises the steps that a video stream collected by an image acquisition device of the robot is received;a video image in the video stream is input into an image recognition model; the target area of a specified target in the video image is marked, and the position information of the target area is acquired; and according to the position information of the target area, the robot is controlled to move to a specified destination until the robot moves to the specified destination, so as to carry out preset processing or operation on the specified target. According to the technical scheme, the robot can process or operate the specified target, so that a work mode can be targetedly or purposefully selected; the robot target recognition method and device and the robot are universal; and the user experience is enhanced.

Robot target recognition method and device and robot

The invention discloses an intelligent grabbing system based on deep learning. The system comprises a multi-degree-of-freedom mechanical arm, an end effector, a stereoscopic vision device, a WIFI relay module and an industrial control computer. The invention also discloses an intelligent grabbing method based on deep learning. A neural network model of deep learning is trained and integrated withimage processing technology in order to recognize the position and the material type of a target object in a three-dimensional space. Then, the mechanical arm is controlled to grab the target object.With the system and the method, recognition rate and positioning accuracy of the target object are greatly improved. Therefore, the system and the method are suitable for an object grabbing task in acomplicated situation.

Intelligent grabbing system and method based on deep learning

The invention provides a summoning device and method of a floor sweeping robot. The device comprises a GPU processing unit module, a deep camera module and a microphone module; the method comprises the steps that the floor sweeping robot moves to the periphery of a summoner through voice recognition and character recognition to perform following movement or cleaning operation. According to the device and the method, the deep camera module and the microphone are utilized to perform deep image and voice extraction respectively, comparison is conducted on body shape characteristics of the summoner through data processing of the GPU to find out the location of the summoner so that the floor sweeping robot can move to the periphery of the summoner to perform following movement or cleaning operation, character recognition algorithms are diversified, the method for deducing the height of a character has the advantages of being simple and low in computation intensity, and the summoning response of the floor sweeping robot is timely.

Summoning device and method of floor sweeping robot

The invention discloses an intelligent service robot and a working method. According to the intelligent service robot and the working method, deep learning and a robot entity are combined. Before the robot is used, a deep learning network is trained through mass data at first, and thus objects can be recognized; and then the robot entity is combined. A current environment image is acquired through a camera at first, the positions and class labels of detection targets can be obtained through the deep learning network, then depth map information is fused, and accordingly the mutual relation of the objects and the robot in space can be obtained. In the tracking process, the robot can dynamically adjust the tracking speed according to the distance. In the grabbing process of the robot, the mutual relation of the objects and the robot can be subjected to projection transformation into a coordinate system of a mechanical arm through an algorithm, and hence the mechanical arm can be guided for grabbing.

Intelligent service robot and working method

The invention discloses a blind guiding system based on deep learning and a working method. The blind guiding system based on deep learning comprises an image acquiring device, a positioning device, a suspended object detecting device and a vibrating and sound feedback device, wherein the image acquiring device, the positioning device, the suspended object detecting device and the vibrating and sound feedback device are connected with a central processing unit; the image acquiring device, the positioning device and the suspended object detecting device acquire current information and transmit the current information to the central processing unit; the central processing unit generates a virtual sidewalk for the bind through sensor data information by using a deep learning image processing method; feedback is implemented by an earphone sound feedback device and a vibrating feedback device; and the blind is guided in real time. In addition, by a socializing function of a walking stick, the blind can integrate into society and go outside.

Blind guiding system based on deep learning and working method

The invention provides a smart wheelchair based on an eyeball control screen, which aims to widen the eye control scope. The smart wheel chair comprises an electrically propelled wheelchair, an information acquisition device fixed on the electrically propelled wheelchair, an information processing device, an information display device and a first bracket and a second bracket for fixing the three devices. The information acquisition device is used for shooting a user's face image; the information processing device is used for extracting the information of the eyes and judging the eyes' open orclose and pupil movement state. The judged eye movement state is converted into a command for controlling the cursor to move and clicking the displayed content of the electronic display screen while being converted into a command for controlling the movement of the electrically propelled wheelchair, and the user controls the starting, braking and left-right rotation of the electrically propelled wheelchair through effective blinking and pupil rotation, when the electrically propelled wheelchair is stationary, the user controls the mouse to click and move on the electronic display screen through the effective blinking and pupil rotation, so that the user can communicate with the outside through the eyes.

Smart wheelchair based on eyeball control screen

As an effective technique to achieve the implementation of deep neural networks in edge devices, model quantization has been successfully applied in many practical applications. No matter the methods of quantization aware training (QAT) or post-training quantization (PTQ), they all depend on the target bit-widths. When the precision of quantization is adjusted, it is necessary to fine-tune the quantized model or minimize the quantization noise, which brings inconvenience in practical applications. In this work, we propose a method to train a model for all quantization that supports diverse bit-widths (e.g., form 8-bit to 1-bit) to satisfy the online quantization bit-width adjustment. It is hot-swappable that can provide specific quantization strategies for different candidates through multiscale quantization. We use wavelet decomposition and reconstruction to increase the diversity of weights, thus significantly improving the performance of each quantization candidate, especially at ultra-low bit-widths (e.g., 3-bit, 2-bit, and 1-bit). Experimental results on ImageNet and COCO show that our method can achieve accuracy comparable performance to dedicated models trained at the same precision.

Huang Zhongjian

Papers

Intelligent service robot and working method

Blind guiding system based on deep learning and working method

Smart wheelchair based on eyeball control screen

One Model for All Quantization: A Quantized Network Supporting Hot-Swap Bit-Width Adjustment.