Jialin Chen

Bio: Jialin Chen is an academic researcher from Northwestern Polytechnical University. The author has contributed to research in topics: Visual servoing & Inverse kinematics. The author has an hindex of 4, co-authored 4 publications receiving 55 citations.

An Adaptive Strategy Selection Method With Reinforcement Learning for Robotic Soccer Games

Abstract: Robotic soccer games, which have become popular, require timely and precise decision-making in a dynamic environment. To address the problems of complexity in a critical situation, policy improvement in robotic soccer games must occur. This paper proposes an adaptive decision-making method that uses reinforcement learning (RL), and the decision-making system for a robotic soccer game is composed of two subsystems. The first subsystem in the architecture for the proposed method criticizes the situation, and the second subsystem implements decision-making policy. Inspired by the support vector machine (SVM), a situation classification method, which is called an improved SVM, embeds a decision tree structure and simultaneously addresses the problems of a large scale and multiple classifications. When a variety of situations that are collected in the field are classified and congregated into the tree structure, the problem of local strategy selection for each individual class of situations over time is regarded as a RL problem and is solved using a Q-learning method. The results of simulations and experiments demonstrate that the proposed method allows satisfactory decision-making.

Collision Avoidance for Redundant Robots in Position-Based Visual Servoing

Abstract: To tackle the problem on trajectory planning or the design of control law, this paper introduces a visual servoing system for a manipulator with redundant joints that the trajectory of the manipulator approaching the target is determined spontaneously by the visual control law. The proposed method resolves joint solution for visual servoing and obstacle avoidance. The work comprises of two procedures, feature extraction for position-based visual servoing (PBVS) and collision avoidance within the working envelope. In the PBVS control, the target pose must be reconstructed with respect to the robot and this results in a Cartesian motion-planning problem. Once the geometric relationship between the target and the end effector is determined, a secure inverse kinematics method incorporating trajectory planning is used to solve the solution of the redundant manipulator by the virtual repulsive torque method. Therefore, the links of the manipulator can always maintain a safe distance from obstacles while approaching the target smoothly. The proposed method is verified with its applicability in experiments using an eye-in-hand manipulator with seven joints. For reusability and extensibility, the system has been coded and constructed in the framework of the Robot Operating System so as that the developed algorithms can be disseminated to different platforms.

A learning approach to image-based visual servoing with a bagging method of velocity calculations

Abstract: Visual servoing allows accurate control of positioning and motion relative to a stationary or moving target using vision and is the subject of many studies. Most servoing gains for image-based visual servoing methods are selected heuristically or empirically so accuracy is affected. This study uses reinforcement learning to adaptively tune the proportional servoing gain over sequences of image based visual servoing, instead of using a constant gain. The system is used to control hovering and tracking for a stationary or slowly moving target. An image Jacobian matrix with four dimensions is constructed because a quad-rotor drone features under-actuated dynamics. The Moore-Penrose pseudo inverse method is usually used to calculate the inverse image Jacobian matrix, but this study uses a bagging approach to calculate the inverse kinematics . The desired velocity is obtained from time-varying image errors, which gives greater robustness. An adaptive method to calculate the servoing gain method is proposed, whereby the selection of an appropriate servo gain over time is regarded as a reinforcement learning problem. The proposed visual servoing control system is implemented and tested experimentally using a quad-rotor drone system. The experimental results demonstrate that the proposed method is more robust and converges faster than other methods.

A Sample Aggregation Approach to Experiences Replay of Dyna-Q Learning

Abstract: In a complex environment, the learning efficiency of reinforcement learning methods always decreases due to large-scale or continuous spaces problems, which can cause the well-known curse of dimensionality. To deal with this problem and enhance learning efficiency, this paper introduces an aggregation method by using framework of sample aggregation based on Chinese restaurant process (CRP), named FSA-CRP, to cluster experiential samples, which is represented by quadruples of the current state, action, next state, and the obtained reward. In addition, the proposed algorithm applies a similarity estimation method, the MinHash method, to calculate the similarity between samples. Moreover, to improve the learning efficiency, the experience sharing Dyna learning algorithm based on samples/clusters prediction method is proposed. While an agent learns the value function of the current state, it acquires clustering results, the value functions of the sample merge with the original as the updated value function of the cluster. In indirect learning (planning) for the Dyna-Q, a learning agent looks for the most likely branches of the constructed FSA-CRP model to raise up learning efficiency. The most likely branches will be selected by an improved action/sample selection algorithm. The algorithm applies the probability that the sample appears in the cluster to select simulated experiences for indirect learning. To verify the validity and applicability of the proposed method, experiments are conducted on a simulated maze and a cart-pole system. The results demonstrate that the proposed method can effectively accelerate the learning process.

BehaveSense: Continuous authentication for security-sensitive mobile apps using behavioral biometrics

Abstract: With the emergence of smartphones as an essential part of our daily lives, continuous authentication becomes an urgent need which could efficiently protect user security and privacy. However, only a small percentage of apps contain sensitive data. To save energy and protect user security, we propose BehaveSense, an accurate and efficient continuous authentication method for security-sensitive mobile apps using touch-based behavioral biometrics. By exploring four different types of touch operations, we train the owner model using One-Class SVM (OCSVM) and isolation forest (iForest), and calculate the accuracy of each type with the model. Afterwards, we calculate the confidence level of each type using the Bayesian theorem. Finally, we obtain the accuracy of a touch operation sequence with an improved expectedprob algorithm. To validate the effectiveness of the proposed method, we conduct a series of experiments. We collect the WeChat app data of 45 volunteers during two weeks. Experimental results show that our method can recognize user identity efficiently. Specifically, our method achieves average accuracy of approaching 95.85% for touch operation sequence, when considering 9 touch operations. Our method is very promising to authenticate user.

A Fuzzy Adaptive Approach to Decoupled Visual Servoing for a Wheeled Mobile Robot

Abstract: To address the performance bottleneck for image-based visual servoing (IBVS), it is necessary to have appropriate servoing control laws, increased accuracy for image feature detection, and minimal approximation errors. This article proposes a fuzzy adaptive method for decoupled IBVS that allows the efficient control of a wheeled mobile robot (WMR). To address the under-actuated dynamics of the WMR, a decoupled controller is used and translation and rotation are decoupled by using two independent servoing gains, instead of the single servoing gain that is used for traditional IBVS. To reduce the effect of image noise, this article develops an improved bagging method for the decoupled controller that calculates the inverse kinematics and does not use the Moore–Penrose pseudoinverse method. To improve convergence, improved Q-learning is used to adaptively adjust the mixture parameter for the image Jacobian matrix (IQ-IBVS). This allows the mixture parameter can be adjusted while the robot moves under the influence of servo control. A fuzzy method is used to tune the learning rate for the IQ-IBVS method, which ensures effective learning. The results of simulation and experiments show that the proposed method performs better than other methods, in terms of convergence.

A Distributed Multi-Agent Dynamic Area Coverage Algorithm Based on Reinforcement Learning

Abstract: Dynamic area coverage is widely used in military and civil fields. Improving coverage efficiency is an important research direction for multi-agent dynamic area coverage. In this paper, we focus on the non-optimal coverage problem of free dynamic area coverage algorithms. We propose a distributed dynamic area coverage algorithm based on reinforcement learning and a $\gamma $ -information map. The $\gamma $ -information map can transform the continuous dynamic coverage process into a discrete $\gamma $ point traversal process, while ensuring no-hole coverage. When agent communication covers the whole target area, agents can obtain the global optimal coverage strategy by learning the whole dynamic coverage process. In the event that communication does not cover the whole target area, agents can obtain a local optimal coverage strategy; in addition, agents can use the proposed algorithm to obtain a global optimal coverage path through off-line planning. Simulation results demonstrate that the required time for area coverage with the proposed algorithm is close to the optimal value, and the performance of the proposed algorithm is significantly better than the distributed anti-flocking Algorithms for dynamic area coverage.

Statistical time features for global corrosion assessment in a truss bridge from vibration signals

Abstract: Truss-type structures are commonly used for configuring civil structures such as bridges, roof supports, cranes, among others. However, they are susceptible to suffer different types of damage such as cracks, loosened bolts, and corrosion, being the latter one of the most common and aggressive ones. In this paper, it is explored the application of statistical time features (STFs) extracted from raw vibration signals of truss-type bridges under dynamic excitation for assessing their health condition. Then, the Kruskal-Wallis method (KWM) is used for determining the most discriminating STFs and a feature reduction criterion is applied to select the most useful ones for assessing the structure's condition. The selected STF is employed for configuring a decision tree and thus determining the condition of the structure automatically. The effectiveness of the proposal is verified under three levels of corrosion damage (i.e., incipient, moderate and severe), which are artificially generated. For the analysis, the tri-axial vibrations are explored. Results show that the proposal can identify damage from an incipient damage condition. An accuracy of 100% is reached using only a sensor placed on the structure.

Collision Avoidance for Redundant Robots in Position-Based Visual Servoing

Abstract: To tackle the problem on trajectory planning or the design of control law, this paper introduces a visual servoing system for a manipulator with redundant joints that the trajectory of the manipulator approaching the target is determined spontaneously by the visual control law. The proposed method resolves joint solution for visual servoing and obstacle avoidance. The work comprises of two procedures, feature extraction for position-based visual servoing (PBVS) and collision avoidance within the working envelope. In the PBVS control, the target pose must be reconstructed with respect to the robot and this results in a Cartesian motion-planning problem. Once the geometric relationship between the target and the end effector is determined, a secure inverse kinematics method incorporating trajectory planning is used to solve the solution of the redundant manipulator by the virtual repulsive torque method. Therefore, the links of the manipulator can always maintain a safe distance from obstacles while approaching the target smoothly. The proposed method is verified with its applicability in experiments using an eye-in-hand manipulator with seven joints. For reusability and extensibility, the system has been coded and constructed in the framework of the Robot Operating System so as that the developed algorithms can be disseminated to different platforms.