Showing papers in "Robotics and Computer-integrated Manufacturing in 2023"
TL;DR: In this paper , a hybrid robust convolutional autoencoder (HRCAE) was proposed for unsupervised anomaly detection of machine tools under noises, which can effectively fuse multi-sensor information and enhance network robustness by training in parallel.
Abstract: • A new FDD loss function to suppress the noises is designed. • Construct the PCDF module to enhance the robustness of the network. • The unsupervised anomaly detection of machine tools under noises. • The proposed HRCAE performs effective in the CNC machine tool. Anomaly detection of machine tools plays a vital role in the machinery industry to sustain efficient operation and avoid catastrophic failures. Compared to traditional machine learning and signal processing methods, deep learning has greater adaptive capability and end-to-end convenience. However, challenges still exist in recent research in anomaly detection of machine tools based on deep learning despite the marvelous endeavors so far, such as the necessity of labeled data for model training and insufficient consideration of noise effects. During machine operation, labeled data is often difficult to obtain; the collected data contains varying degrees of noise disturbances. To address the above challenges, this paper develops a hybrid robust convolutional autoencoder (HRCAE) for unsupervised anomaly detection of machine tools under noises. A parallel convolutional distribution fitting (PCDF) module is constructed, which can effectively fuse multi-sensor information and enhance network robustness by training in parallel to better fit the data distribution with unsupervised learning. A fused directional distance (FDD) loss function is designed to comprehensively consider the distance and angle differences among the data, which can effectively suppress the influence of noises and further improve the model robustness. The proposed method is validated by real computer numerical control (CNC) machine tool data, obtaining better performance of unsupervised anomaly detection under different noises compared to other popular unsupervised improved autoencoder methods.
53 citations
TL;DR: In this paper , the authors present a comprehensive and systematic review on the robotic milling of complex components, elaborating the history and state-of-the-art on stiffness, dynamics, posture planning, chatter, and compensation.
Abstract: • The challenges faced by milling robots in machining operations are systematically reviewed. • The performance indexes commonly used to evaluate machining systems are highlighted. • The differences between regenerative and modal coupling chatter are compared. • Robotic processing stability is analyzed from two aspects of posture and cutting condition optimization. • Future trends of milling robots and problems to be solved are prospected. Milling refers to a class of material processing methods that relies on a high-speed rotating milling cutter removing extra material to get desired shapes and features. Being distinct from conventional material removal techniques, like CNC machining, robotic milling has the benefits of lower cost, higher flexibility, better adaptability, etc. Therefore, robotic milling has attracted a large amount of researchers’ interest and become an important material-removing way in the machining of complex parts. Trying to present a comprehensive and systematic review on the robotic milling of complex components, this paper elaborates the history and state of the art on stiffness, dynamics, posture planning, chatter, and compensation in the robotic milling process. Furthermore, future potential research topics about robotic milling are also discussed.
19 citations
TL;DR: In this article , a deep learning-based automatic defect detection solution, You Only Look Once (YOLO)-attention, based on YOLOv4, which achieves both fast and accurate defect detection for wire and arc additive manufacturing (WAAM), is presented.
Abstract: Wire and arc additive manufacturing (WAAM) is an emerging manufacturing technology that is widely used in different manufacturing industries. To achieve fully automated production, WAAM requires a dependable, efficient, and automatic defect detection system. Although machine learning is dominant in the object detection domain, classic algorithms have defect detection difficulty in WAAM due to complex defect types and noisy detection environments. This paper presents a deep learning-based novel automatic defect detection solution, you only look once (YOLO)-attention, based on YOLOv4, which achieves both fast and accurate defect detection for WAAM. YOLO-attention makes improvements on three existing object detection models: the channel-wise attention mechanism, multiple spatial pyramid pooling, and exponential moving average. The evaluation on the WAAM defect dataset shows that our model obtains a 94.5 mean average precision (mAP) with at least 42 frames per second. This method has been applied to additive manufacturing of single-pass, multi-pass deposition and parts. It demonstrates its feasibility in practical industrial applications and has potential as a vision-based methodology that can be implemented in real-time defect detection systems.
19 citations
TL;DR: In this paper , a deep learning approach is successfully developed for the online detection of defects, and the defects are effectively controlled by closed-loop adjustment of process parameters, which is never achievable in any conventional methods of composite fabrication.
Abstract: Real-time defect detection and closed-loop adjustment of additive manufacturing (AM) are essential to ensure the quality of as-fabricated products, especially for carbon fiber reinforced polymer (CFRP) composites via AM. Machine learning is typically limited to the application of online monitoring of AM systems due to a lack of accurate and accessible databases. In this work, a system is developed for real-time identification of defective regions, and closed-loop adjustment of process parameters for robot-based CFRP AM is validated. The main novelty is the development of a deep learning model for defect detection, classification, and evaluation in real-time with high accuracy. The proposed method is able to identify two types of CFRP defects (i.e., misalignment and abrasion). The combined deep learning with geometric analysis of the level of misalignment is applied to quantify the severity of individual defects. A deep learning approach is successfully developed for the online detection of defects, and the defects are effectively controlled by closed-loop adjustment of process parameters, which is never achievable in any conventional methods of composite fabrication.
14 citations
TL;DR: In this article , a thorough literature review of the use of machine learning techniques in the context of human-robot collaboration is presented, where 45 key papers were selected and analyzed, and a clustering of works based on the type of collaborative tasks, evaluation metrics and cognitive variables modelled is proposed.
Abstract: Technological progress increasingly envisions the use of robots interacting with people in everyday life. Human–robot collaboration (HRC) is the approach that explores the interaction between a human and a robot, during the completion of a common objective, at the cognitive and physical level. In HRC works, a cognitive model is typically built, which collects inputs from the environment and from the user, elaborates and translates these into information that can be used by the robot itself. Machine learning is a recent approach to build the cognitive model and behavioural block, with high potential in HRC. Consequently, this paper proposes a thorough literature review of the use of machine learning techniques in the context of human–robot collaboration. 45 key papers were selected and analysed, and a clustering of works based on the type of collaborative tasks, evaluation metrics and cognitive variables modelled is proposed. Then, a deep analysis on different families of machine learning algorithms and their properties, along with the sensing modalities used, is carried out. Among the observations, it is outlined the importance of the machine learning algorithms to incorporate time dependencies. The salient features of these works are then cross-analysed to show trends in HRC and give guidelines for future works, comparing them with other aspects of HRC not appeared in the review.
14 citations
TL;DR: In this paper , a theoretical modeling approach based on the hierarchical structure of CNC systems is proposed, and an edge intelligence-driven digital twin architecture for CNC system is proposed.
Abstract: • Propose a theoretical modeling approach based on the hierarchical structure of CNC systems. • Proposes an edge intelligence-driven digital twin architecture for CNC system. • Proposes a generic digital twin modeling method. • Proposes a novel task partitioning method to improve system throughput. • Proposes an adaptive model selection algorithm . In recent years, digital twin (DT) technology has gradually become the primary way to achieve the intelligence of CNC systems. However, with the development of next-generation information technologies such as artificial intelligence (AI) and its wide application in CNC systems, the limitation of computing power and network resources has become one of the urgent problems that must be solved by the DT of CNC systems. To address these problems, a theoretical modeling method for CNC systems based on its hierarchical structure is proposed first, and the edge intelligence (EI) technology is introduced to support the deployment of DT models. Meanwhile, a model partitioning method and a model selection algorithm are proposed to support real-time model response in the model deployment process. In addition, an application case of EI-driven DT of CNC system is given to diagnose and predict the tool wear during machining processes.
14 citations
TL;DR: In this paper , the authors conducted a state-of-the-art survey of AR-assisted digital twins from the perspective of different sectors of the industrial field, covering a total of 118 selected publications, including product design, robotic-related works, cyber physical interaction, and human ergonomics.
Abstract: The combination of Augmented Reality (AR) and Digital Twin (DT) has begun to show its potential nowadays, leading to a growing research interest in both academia and industry. Especially under the current human-centric trend, AR embraces the potential to integrate operators into the new generation of Human Cyber–Physical System (HCPS), in which DT is a pillar component. Some review articles have focused on this topic and discussed the benefits of combining AR and DT, but all of them are limited to a specific domain. To fill the gap, this research conducts a state-of-the-art survey (till 17-July-2022) from the AR-assisted DT perspective across different sectors of the industrial field, covering a total of 118 selected publications. Firstly, application scenarios and functions of AR-assisted DT are summarized by following the engineering lifecycle, among which production process, service design, and Human–Machine Interaction (HMI) are hot topics. Then, improvements specifically brought by AR are analyzed according to three dimensions, namely virtual twin, hybrid twin, and cognitive twin, respectively. Finally, challenges and future perspectives of AR-assisted DT for futuristic human-centric industry transformation are proposed, including promoting product design, robotic-related works, cyber–physical interaction, and human ergonomics.
12 citations
TL;DR: The Proactive Human-Robot Collaboration (HRC) as discussed by the authors is a collaborative human-robot symbiotic relation with a 5C intelligence, from Connection, Coordination, Cyber, Cognition to Coevolution, and finally embracing mutual-cognitive, predictable, and self-organising intelligent capabilities.
Abstract: Human–Robot Collaboration (HRC) has a pivotal role in smart manufacturing for strict requirements of human-centricity, sustainability, and resilience. However, existing HRC development mainly undertakes either a human-dominant or robot-dominant manner, where human and robotic agents reactively perform operations by following pre-defined instructions, thus far from an efficient integration of robotic automation and human cognition. The stiff human–robot relations fail to be qualified for complex manufacturing tasks and cannot ease the physical and psychological load of human operators. In response to these realistic needs, this paper presents our arguments on the obvious trend, concept, systematic architecture, and enabling technologies of Proactive HRC, serving as a prospective vision and research topic for future work in the human-centric smart manufacturing era. Human–robot symbiotic relation is evolving with a 5C intelligence — from Connection, Coordination, Cyber, Cognition to Coevolution, and finally embracing mutual-cognitive, predictable, and self-organising intelligent capabilities, i.e., the Proactive HRC. With proactive robot control, multiple human and robotic agents collaboratively operate manufacturing tasks, considering each others’ operation needs, desired resources, and qualified complementary capabilities. This paper also highlights current challenges and future research directions, which deserve more research efforts for real-world applications of Proactive HRC. It is hoped that this work can attract more open discussions and provide useful insights to both academic and industrial practitioners in their exploration of human–robot flexible production.
12 citations
TL;DR: In this paper , an anomaly detection and dynamic scheduling framework based on digital twin (DT) is proposed for flexible job shop, where a multi-level production process monitoring model is proposed to detect anomaly and an improved grey wolf optimization algorithm is introduced to solve the scheduling problem.
Abstract: Scheduling scheme is one of the critical factors affecting the production efficiency. In the actual production, anomalies will lead to scheduling deviation and influence scheme execution, which makes the traditional job shop scheduling methods are not sufficient to meet the needs of real-time and accuracy. By introducing digital twin (DT), further convergence between physical and virtual space can be achieved, which enormously reinforces real-time performance of job shop scheduling. For flexible job shop, an anomaly detection and dynamic scheduling framework based on DT is proposed in this paper. Previously, a multi-level production process monitoring model is proposed to detect anomaly. Then, a real-time optimization strategy of scheduling scheme based on rolling window mechanism is explored to enforce dynamic scheduling optimization. Finally, the improved grey wolf optimization algorithm is introduced to solve the scheduling problem. Under this framework, it is possible to monitor the deviation between the actual processing state and the planned processing state in real time and effectively reduce the deviation. An equipment manufacturing job shop is taken as a case study to illustrate the effectiveness and advantages of the proposed framework.
12 citations
TL;DR: In this paper , both Deep Q-Networks (DQN) and Dueling DQN (DDQN)-based algorithms for scheduling of decentralized robot services are proposed.
Abstract: • Both Deep Q-Networks (DQN) and Dueling Deep Q-Networks (DDQN)-based algorithms for scheduling of decentralized robot services are proposed. • Performance of different algorithms, including DQN, DDQN, and other three benchmark algorithms, indicates that DDQN performs the best with respect to each indicator. • Effects of different combinations of weight coefficients and influencing degrees of different indicators on the overall scheduling objective are analyzed. Cloud manufacturing is a service-oriented manufacturing model that offers manufacturing resources as cloud services. Robots are an important type of manufacturing resources. In cloud manufacturng, large-scale distributed robots are encapsulated into cloud services and provided to consumers in an on-demand manner. How to effectively and efficiently manage and schedule decentralized robot services in cloud manufacturing to achieve on-demand provisioning is a challenging issue. During the past few years, Deep Reinforcement Learning (DRL) has become very popular and successfully been applied to many different areas such as games, robotics, and manufacturing. DRL also holds tremendous potential for solving scheduling issues in cloud manufacturing. To this end, this paper is devoted to exploring effective approaches for scheduling of decentralized robot manufacturing services in cloud manufacturing with DRL. Specifically, both Deep Q-Networks (DQN) and Dueling Deep Q-Networks (DDQN)-based scheduling algorithms are proposed. Performance of different algorithms, including DQN, DDQN, and other three benchmark algorithms, indicates that DDQN performs the best with respect to each indicator. Effects of different combinations of weight coefficients and influencing degrees of different indicators on the overall scheduling objective are analyzed. Results indicate that the DDQN-based scheduling algorithm is able to generate scheduling solutions efficiently.
10 citations
TL;DR: Wang et al. as discussed by the authors proposed a mutual-cognitive safe HRI approach including worker visual augmentation, robot velocity control, Digital Twin-enabled motion preview and collision detection, and deep reinforcement learning-based robot collision avoidance motion planning in the Augmented Reality-assisted manner.
Abstract: With the emergence of Industry 5.0, the human-centric manufacturing paradigm requires manufacturing equipment (robots, etc.) interactively assist human workers to deal with dynamic and complex production tasks. To achieve symbiotic human–robot interaction (HRI), the safety issue serves as a prerequisite foundation. Regarding the growing individualized demand of manufacturing tasks, the conventional rule-based safe HRI measures could not well address the safety requirements due to inflexibility and lacking synergy. To fill the gap, this work proposes a mutual-cognitive safe HRI approach including worker visual augmentation, robot velocity control, Digital Twin-enabled motion preview and collision detection , and Deep Reinforcement Learning-based robot collision avoidance motion planning in the Augmented Reality-assisted manner. Finally, the feasibility of the system design and the performance of the proposed approach are validated by establishing and executing the prototype HRI system in a practical scene. • Introduced an AR-assisted system architecture for mutual-cognitive safe HRI. • Implemented distance-based robot velocity control and area-based workers’ visual aids functions. • Developed the robot DT-enabled motion preview for workers to enhance safe cognition and for robots to detect collision. • Proposed a curriculum learning-based DRL motion planning policy for collision avoidance.
TL;DR: In this paper , a review of the recent advancement of robot-assisted polishing is presented, which is intended as a roadmap for researchers or engineers who are interested in robot assisted polishing or machining.
Abstract: Nowadays, precision components have been extensively used in various industries such as biomedicine, photonics, and optics. To achieve nanometric surface roughness, high-precision polishing is mandatory. Compared with the conventional polishing techniques, the robot-assisted polishing has the advantages of flexible working ranges and low cost. Therefore, it has been more widely deployed in demanding polishing tasks. However, few review articles were found on this topic to provide a general progress on the development of the robot-assisted polishing technology. This paper reviews the recent advancement of the robot-assisted polishing. Firstly, the integration of robots with various polishing techniques is overviewed, followed by the introduction of the research status of constant force control in robot-assisted polishing. Then, the errors in typical robot-assisted polishing systems are analyzed, together with the corresponding compensation methods. At last, the future trends in the development of robot-assisted polishing are discussed. This review is intended as a roadmap for researchers or engineers who are interested in robot-assisted polishing or machining.
TL;DR: In this article , a real-time digital twin flexible job shop scheduling (R-DTFJSS) method with edge computing is proposed to solve the problem of abnormal disturbances in real-world manufacturing.
Abstract: • A new R-DTFJSS idea was developed to realise high-efficiency production. • Edge computing-based R-DTFJSS framework can enhance the data processing ability. • A variable time window-based RS strategy was used to achieve operation allocation. • An IHA-based solve method was designed to reduce the computational complexity. Production scheduling is the central link between enterprise production and operation management and is also the key to realising efficient, high-quality and sustainable production. However, in real-world manufacturing, the frequent occurrence of abnormal disturbance leads to the deviation of scheduling, which affects the accuracy and reliability of scheduling execution. The traditional dynamic scheduling methods (TDSMs) cannot solve this problem effectively. This paper presents a real-time digital twin flexible job shop scheduling (R-DTFJSS) method with edge computing to address the issue. Firstly, an overall framework of R-DTFJSS is proposed to realise real-time scheduling (RS) through real-time interaction between physical workshop (PW) and virtual workshop (VW). Secondly, the implementation process of R-DTFJSS is designed to realise real-time operation allocation. Then, to obtain the optimal RS result, an improved Hungarian algorithm (IHA) is adopted. Finally, a case simulation from an industrial case of a cooperative enterprise is described and analysed to verify the effectiveness of the proposed R-DTFJSS method. The results show that compared with the TDSMs, the R-DTFJSS method can effectively deal with unexpected and frequent abnormal disturbances in the production process.
TL;DR: In this paper , the relationship between the grinding force and the grinding depth in the robotic abrasive belt grinding is analyzed in detail, the robot machining pose error model considering the deformation of the grinding head is established, and the Inconel 718 alloy machining experiment of the robotic ABS grinding is designed.
Abstract: Robotic abrasive belt grinding has been successfully applied to the grinding and polishing of aerospace parts. However, due to the flexible characteristics of robotic abrasive belt grinding and the time-varying characteristics of the polishing contact force, as well as the plastic and difficult-to-machine material properties of Inconel 718 alloy, it is very difficult to control the actual removal depth and force of the polished surface, which brings great challenges to robot automatic polishing. Therefore, the relationship between the grinding force and the grinding depth in the robotic abrasive belt grinding is analyzed in detail, the robot machining pose error model considering the deformation of the grinding head is established, and the Inconel 718 alloy machining experiment of the robotic abrasive belt grinding is designed. The mapping relationship between the grinding force and the grinding depth is obtained, and the grinding force ratio in the downgrinding and upgrinding mode is discussed. The experimental and theoretical comparisons results show that with the increase of the grinding depress depth, both the grinding depth and the grinding force show an irregular increasing trend, and the increasing trend of the grinding force (increases by about 344.44%–445.45%) is obviously greater than that of the grinding depth (increases by about 52.94%). When the grinding depress depth is large (greater than 3 mm), the feed direction force and the normal force appear obvious secondary pressure peaks at the beginning and end of grinding, which has not been seen in previous studies. In addition, regardless of whether it is downgrinding or upgrinding, the grinding force ratio decreases with the increase of the depress depth, and the grinding force ratio of downgrinding (average 0.668) is smaller than that of upgrinding (average 0.724). This study provides a reference for robotic abrasive belt grinding, and the surface quality of Inconel 718 alloy of robotic abrasive belt grinding can be further improved through the optimization of force and depth.
TL;DR: Wang et al. as discussed by the authors proposed a general digital twin model update framework at first and further explored the update methods for multi-dimensional models for machine tools, which is essential for realizing the digitalization and servitization of machine tools.
Abstract: Digital twin, as an effective means to realize the fusion between physical and virtual spaces, has attracted more and more attention in the past few years. Based on ultra-fidelity models, more accurate service, e.g. real-time monitoring and failure prediction, can be reached. Against the background, some scholars studied the related theories and methods on modeling to depict various features of physical objects. Some scholars studied how to use Internet of Things to realize the connections and interactions, thereby keeping the consistency between the virtual and physical spaces. During this process, a new question arises that how to update the models once digital twin models are inconsistent with the practical situations. To solve the problem, this paper proposed a general digital twin model update framework at first. Then, the update methods for multi-dimension models are further explored. The cutting tool is the core component of machine tools which are the key equipment in industry. The precise cutting tool models are essential for realizing the digitalization and servitization of machine tools. Therefore, this paper takes a cutting tool as the application object to discuss how to conduct physics model update based on the proposed framework and methods. Through model update, a more accurate and updated tool wear model could be obtained, which contributes to the prognostics and health management for machine tools.
TL;DR: In this article , a new dynamic decoupling method employing dual force sensors (DFSs) is proposed to address the coupling dynamics between the macro and the mini, which can improve the contact force response rate and tracking accuracy.
Abstract: A robotic polishing system includes a force-controlled end-effector (mini manipulator) and a position-controlled industrial robot (macro manipulator). This combination mode has a fast response and a large workspace. However, the force-controlled axis component of the macro motions and the geometric of the workpiece surfaces will affect the contact force response rate and tracking accuracy due to the coupling dynamics between the macro and mini, limiting system performance. A new dynamic decoupling method employing dual force sensors (DFSs) is proposed to address these problems. One of the force sensors installed between the endpoint of the macro and the fixed platform of the mini realizes the dynamic decoupling of the macro and mini. The other one is added at the endpoint of the mini to obtain the interaction force in contact with the environment and feed it back to the control loop. When the disturbances produced by the macro trajectories and the uncertainties coming from the workpiece are introduced into the system, the proposed method can improve force response rate and tracking accuracy without knowing the dynamic models and parameters of the macro and the geometric of the workpiece surface. Several experiments are carried out under various conditions. Experimental results indicate that the contact force response rate and tracking error of DFSs are better than those of the conventional force-controlled and impedance matching methods, proving the proposed method’s effectiveness. In addition, the last comparison experiment verifies that the DFSs method applies to different kinds of end-effectors with various dynamics. • This paper proposes a new dynamic decoupling method based on force sensors. • It can obtain the coupling force to realize dynamic decoupling regardless of models. • It can reduce the disturbance effects of the macro and the workpiece on the force. • The proposed method can improve contact force response rate and tracking accuracy.
TL;DR: In this article , a learning framework that enables robotic arms to replicate new skills from human demonstration is proposed, which makes use of online human motion data acquired using wearable devices as an interactive interface for providing the anticipated motion to the robot in an efficient and user-friendly way.
Abstract: In this article, a learning framework that enables robotic arms to replicate new skills from human demonstration is proposed. The learning framework makes use of online human motion data acquired using wearable devices as an interactive interface for providing the anticipated motion to the robot in an efficient and user-friendly way. This approach offers human tutors the ability to control all joints of the robotic manipulator in real-time and able to achieve complex manipulation. The robotic manipulator is controlled remotely with our low-cost wearable devices for easy calibration and continuous motion mapping. We believe that our approach might lead to improving the human-robot skill learning, adaptability, and sensitivity of the proposed human-robot interaction for flexible task execution and thereby giving room for skill transfer and repeatability without complex coding skills. • This article presents a system that enables remote teleoperation based on wearable sensors for 6-Degree of freedom robotic manipulation control appication. • A skill learning framework is proposed based on dynamic movement primitives model from human manipulability to robot with the capability of replicating human demonstration.
TL;DR: In this paper , a novel deep learning architecture combines the output of a convolutional neural network (that takes melt pool images as inputs) with scalar variables (process and trajectory data).
Abstract: Laser Metal Deposition (LMD) is an additive manufacturing technology that attracts great interest from the industry, thanks to its potential to realize parts with complex geometries in one piece, and to repair damaged ones, while maintaining good mechanical properties. Nevertheless, the complexity of this process has limited its widespread adoption, since different part geometries, strategies and boundary conditions can yield very different results in terms of external shapes and inner flaws. Moreover, monitoring part quality during the process execution is very challenging, as direct measurements of both structural and geometrical properties are mostly impracticable. This work proposes an on-line monitoring and prediction approach for LMD that exploits coaxial melt pool images, together with process input data, to estimate the size of a track deposited by LMD. In particular, a novel deep learning architecture combines the output of a convolutional neural network (that takes melt pool images as inputs) with scalar variables (process and trajectory data). Various network architectures are evaluated, suggesting to use at least three convolutional layers. Furthermore, results imply a certain degree of invariance to the number and size of dense layers. The effectiveness of the proposed method is demonstrated basing on experiments performed on single tracks deposited by LMD using powders of Inconel 718, a relevant material for the aerospace and automotive sectors. • LMD process requires on-line control for achieving predictable results. • Matrix and scalar variables are mixed in the proposed deep network structure. • Geometry prediction by AI is tested on single-track experiments. • Image information extracted by the CNN improves on-line track size prediction.
TL;DR: In this article , a hybrid flow shop scheduling problem considering multi-skilled workers and fatigue factors is studied, where an agent-based simulation system is established to cope with the uncertainties in the worker fatigue model.
Abstract: In the past few decades, more and more studies have begun to consider the impact of human factors on manufacturing systems. This paper studies a hybrid flow shop scheduling problem considering multi-skilled workers and fatigue factors. An agent-based simulation system is established to cope with the uncertainties in the worker fatigue model. Furthermore, this paper proposes a novel simulation-based optimization (SBO) framework, which combines genetic algorithm (GA) and reinforcement learning (RL) to address the hybrid flow shop scheduling problem. Numerical experiments are conducted on several instances with different production configurations. In particular, a pharmaceutical production facility is modeled as a hybrid flow shop to demonstrate the feasibility and effectiveness of the proposed SBO method.
TL;DR: A survey of reinforcement learning in contact-rich manipulation tasks can be found in this paper , where the authors examine the state-of-the-art and the commonalities among the studies.
Abstract: Research and application of reinforcement learning in robotics for contact-rich manipulation tasks have exploded in recent years. Its ability to cope with unstructured environments and accomplish hard-to-engineer behaviors has led reinforcement learning agents to be increasingly applied in real-life scenarios. However, there is still a long way ahead for reinforcement learning to become a core element in industrial applications. This paper examines the landscape of reinforcement learning and reviews advances in its application in contact-rich tasks from 2017 to the present. The analysis investigates the main research for the most commonly selected tasks for testing reinforcement learning algorithms in both rigid and deformable object manipulation. Additionally, the trends around reinforcement learning associated with serial manipulators are explored as well as the various technological challenges that this machine learning control technique currently presents. Lastly, based on the state-of-the-art and the commonalities among the studies, a framework relating the main concepts of reinforcement learning in contact-rich manipulation tasks is proposed. The final goal of this review is to support the robotics community in future development of systems commanded by reinforcement learning, discuss the main challenges of this technology and suggest future research directions in the domain.
TL;DR: In this paper , a cloud-edge collaboration-based CPMT architecture is proposed, which makes full use of the computing resources of existing devices in the industrial sites, offloads digital twin (DT) modeling and data processing from the cloud to the edge, and provides microservice interfaces for users at the edge.
Abstract: • Proposes an architecture for cloud edge collaboration-based CPMT. • Proposes a task offloading technique to improve responsiveness. • Proposes an offloading algorithm to provide load balancing while maintaining system real-time and throughput rate requirements. • An application case is given for PHM of cutting tool. The Cyber-Physical Machine Tool (CPMT) is a promising solution for the next generation of machine tool digitalization and servitization due to its excellent interconnection, intelligence, adaptability, and autonomy. The rapid development of next-generation information technologies, such as the Internet of Things (IoT) and artificial intelligence (AI), provided richer services for CPMT but also led to problems of idle on-site computing resources, and excessive pressure on the cloud, slow service response and poor privacy. To solve the above problems, this paper proposes a cloud-edge collaboration-based CPMT architecture, which makes full use of the computing resources of existing devices in the industrial sites, offloads digital twin (DT) modeling and data processing from the cloud to the edge, and provides microservice interfaces for users at the edge. Given the limited computing resources available in the field and the demand for latency-sensitive applications, task offloading methods aimed at response speed and load balancing are proposed, respectively. Finally, a case of machine tool Prognostics and Health Management (PHM) service is presented, in which the proposed method is used to perform tool wear monitoring, prediction, and health management.
TL;DR: In this article , a vision-based Robotic Laser Cladding Repair Cell (RLCRC) is proposed for the repair of worn fixed bends, which has two features: (a) an intelligent inspection system that uses a deep learning model to automatically detect the damaged region in an image; and (b) employing computer visionbased calibration and 3D scanning techniques to precisely identify the geometries of damaged area.
Abstract: Repair technologies have been considered as sustainable approaches due to their capability to restore value in a damaged component and bring it to like-new condition. However, in contrast to a manufacturing process benefiting from an automated environment, the automation level for repair and remanufacturing processes remains low. With the aim of moving the repair industry towards autonomy, this study proposes a novel repair framework. The developed methodology presents a vision-based Robotic Laser Cladding Repair Cell (RLCRC) that has two features: (a) an intelligent inspection system that uses a deep learning model to automatically detect the damaged region in an image; (b) employing computer vision-based calibration and 3D scanning techniques to precisely identify the geometries of damaged area. The repair of fixed bends is selected as the case study. The results obtained validate the efficacy of the proposed framework, enabling automatic damage detection and damaged volume extraction for worn fixed bends. Following the suggested framework, a time reduction of more than 63% is reported.
TL;DR: In this paper , the authors proposed a hybrid learning-based digital twin for manufacturing process, which integrated AI techniques and machine tool expertise using aggregated data along the product development process, and tested it in an external industrial project exemplarily for realtime workpiece quality monitoring.
Abstract: Digital twin (DT) and artificial intelligence (AI) technologies are powerful enablers for Industry 4.0 toward sustainable resilient manufacturing. Digital twins of machine tools and machining processes combine advanced digital techniques and production domain knowledge, facilitate the enhancement of agility, traceability, and resilience of production systems, and help machine tool builders achieve a paradigm shift from one-time products provision to on-going service delivery. However, the adaptability and accuracy of digital twins at the shopfloor level are restricted by heterogeneous data sources, modeling precision as well as uncertainties from dynamical industrial environments. This article proposes a novel modeling framework to address these inadequacies by in-depth integrating AI techniques and machine tool expertise using aggregated data along the product development process. A data processing procedure is constructed to contextualize metadata sources from the design, planning, manufacturing, and quality stages and link them into a digital thread. On this consistent data basis, a modeling pipeline is presented to incorporate production and machine tool prior knowledge into AI development pipeline, while considering the multi-fidelity nature of data sources in dynamic industrial circumstances. In terms of implementation, we first introduce our existing work for building digital twins of machine tool and manufacturing process. Within this infrastructure, we developed a hybrid learning-based digital twin for manufacturing process following proposed modeling framework and tested it in an external industrial project exemplarily for real-time workpiece quality monitoring. The result indicates that the proposed hybrid learning-based digital twin enables learning uncertainties of the interaction of machine tools and machining processes in real industrial environments, thus allows estimating and enhancing the modeling reliability, depending on the data quality and accessibility. Prospectively, it also contributes to the reparametrization of model parameters and to the adaptive process control.
TL;DR: In this paper , a hybrid genetic algorithm based on variable neighborhood search (GAVNS) solution method is proposed, given the NP-hard nature of the problem, and three different decoding methods are specially designed according to the formation of optimization objective TEC.
Abstract: Remanufacturing system scheduling is an essential and effective approach to realize the digitization and greening of the remanufacturing industry. However, previous researches on the remanufacturing system scheduling problem mainly consider a single or two production stages and economic objectives. In this paper, by integrating the three core production stages, i.e., disassembly, reprocessing and reassembly together, we study the energy-aware remanufacturing system scheduling problem in which the well-accepted Turn Off and On strategy is also considered. First, a mathematical model aiming at minimizing the total energy consumption (TEC) of the remanufacturing system is established. Then, a hybrid genetic algorithm based on variable neighborhood search (GAVNS) solution method is proposed, given the NP-hard nature of the problem. In GAVNS, each chromosome is encoded by a job sequence and three different decoding methods are specially designed according to the formation of optimization objective TEC. To enhance the algorithm's local search capability, the variable neighborhood search technique is introduced. The feasibility and effectiveness of GAVNS in addressing the energy-aware remanufacturing system scheduling problem is verified through simulation experiments on a set of designed test instances. Experimental results also demonstrate that: (1) the Turn Off and On strategy can effectively reduce TEC of the remanufacturing system, which can reach an energy saving rate of 6.68%; (2) the performance of those decoding methods varies with respect to the problem size; (3) the decoding method based on minimizing the energy consumption of the remanufacturing system (namely DM3) has the best performance among the three decoding methods in most cases; (4) GAVNS is more effective than its four peers, i.e., a variant GAVNS_R, iterated greedy algorithm (IG), extended artificial bee colony algorithm (EABC), discrete invasive weed optimization algorithm (DIWO) in seeking the optimal schedule.
TL;DR: In this article , a digital twin-driven virtual commissioning method is proposed to simulate the machining processes in a virtual environment and perform virtual-commissioning to obtain better commissioning results.
Abstract: The commissioning of Computerized Numerical Control Machine Tools (CNCMTs) is particularly important and the commissioning quality directly affects its product processing. However, traditional commissioning methods are not suitable for complex and changeable machining conditions during operation, and the derived commissioning results have limited effectiveness. Therefore, this paper proposes a digital twin-driven virtual commissioning method to simulate the machining processes in a virtual environment and perform virtual commissioning to obtain better commissioning results. Firstly, a digital twin model is constructed using a multi-domain unified modeling language combined with a virtual-real mapping strategy to describe the response characteristics of CNCMTs. Secondly, a complex machining scene is simulated based on the twin model, and a virtual commissioning strategy and platform are constructed in this environment. Finally, the effectiveness of the proposed method is verified by taking the spindle system of CNCMTs as an example. The experimental results show a 13% short in response time and a 54% reduction in total systematic error along with a decrease in commissioning time.
TL;DR: In this article , the authors proposed an automatic joint motion planning method for a nine-axis industrial robot to achieve the shortest processing time, where offline programming is designed to generate paths for the complex surface of the hub, and the Greedy Best First Search (GBFS) and Sine cosine algorithm (SCA) are combined to find the optimal joint motion efficiently.
Abstract: Automatic joint motion planning is very important in robotic wheel hub polishing systems. Higher flexibility is achieved based on the joint configuration with multiple solutions, which means that the robot has kinematic redundancy for machining tasks. Redundant joints can be used to optimize the motion of the robot, but less research has been done on multi-dimensional redundant optimization. In this paper, a 6-axis robot with a 3-axis actuator is designed for wheel hub polishing. We propose an automatic joint motion planning method for a nine-axis industrial robot to achieve the shortest processing time. Firstly, offline programming is designed to generate paths for the complex surface of the hub. In order to reduce the machining path points on the surface of the hub, a improved Douglas-Peucker (DP) algorithm is proposed, which can take into account the change of the path point posture. Secondly, the Greedy Best First Search (GBFS) and Sine cosine algorithm (SCA) are combined to find the optimal joint motion efficiently. Moreover, we use nested SCA for comparison to test whether the combined algorithm can avoid local optima. Finally, the performance and computational efficiency of the method are validated in both simulation and real environments based on the hub surface.
TL;DR: Wang et al. as mentioned in this paper proposed a knowledge augmented broad learning system with a knowledge module and broad selective sampling module, which provides a multichannel selective sampling network to decouple the mixed-type defects.
Abstract: Defect detection is a critical measurement process for intelligent manufacturing systems to provide insights for product quality improvement. For complex products such as integrated circuit wafers, several types of defects are usually coupled in a piece of wafer to form a mixed-type defect, which poses a challenge to current defect detection methods. This paper proposed a knowledge augmented broad learning system with a knowledge module and broad selective sampling module, which provides a multichannel selective sampling network to decouple the mixed-type defects. In this model, each channel is equipped with a pre-trained deformable convolution model to extract the feature of a fixed single-type defect. The knowledge module is designed to activate the candidate network channel by pre-detection of wafer maps. The experiment results indicated that the proposed model outperforms conventional models and other deep learning models, which demonstrated that the knowledge augmented broad selective sampling mechanism is effective for mixed-type defect detection.
TL;DR: Wang et al. as mentioned in this paper proposed a prediction and compensation method of robot tracking error based on temporal convolutional network (TCN), where the pose-dependent effect of load on joint tracking error is considered.
Abstract: Industrial robots are widely used because of their high flexibility and low cost compared with CNC machine tools, but the low tracking accuracy limits their application in the field of high-precision manufacturing. To improve the tracking accuracy and solve the complex modeling problems, a prediction and compensation method of robot tracking error is proposed based on temporal convolutional network (TCN), where the pose-dependent effect of load on joint tracking error is considered. The terminal load is decomposed to joint load by using Jacobian matrix and then used as the pose-dependent information of the data-based model. A prediction model based on TCN is used to predict the tracking error of joints. Finally, a pre-compensation method is adopted to improve the joint tracking accuracy based on the predicted errors. Experimental results show that the model presents good prediction and compensation accuracy. The mean absolute tracking errors are increased by more than 80% in the test path. This method can effectively compensate the tracking errors of the robot joints and therefore greatly improve the tracking accuracy of the tool center point and tool orientation in the Cartesian coordinate system.
TL;DR: In this article , a bio-inspired LIDA cognitive-based Digital Twin architecture is proposed to achieve unmanned maintenance of machine tools through a self-constructed, self-evaluated, and self-optimized manner.
Abstract: Affected by COVID-19, the maintenance process of machine tools is significantly hindered, while unmanned maintenance becomes an emerging trend in such background. So far, three challenges, namely, the dependence on maintenance experts, the dynamic maintenance environments, and unsynchronized interactions between physical and information sides, exist as the main obstacles in its widespread applications. In order to fill this gap, a bio-inspired LIDA cognitive-based Digital Twin architecture is proposed, so as to achieve unmanned maintenance of machine tools through a self-constructed, self-evaluated, and self-optimized manner. A three phases process in the architecture, including the physical phase, virtual phase, and service phase, is further introduced to support the cognitive cycle for unmanned maintenance of machine tools. An illustrative example is depicted in the unmanned fault diagnosis on the rolling bearing of a drilling platform, which validates the feasibility and advantages of the proposed architecture. As an explorative study, it is wished that this work provides useful insights for unmanned maintenance of machine tools in a dynamic production environment.