Showing papers in "Robotics and Computer-integrated Manufacturing in 2022"
TL;DR: A novel taxonomy of levels of interaction between humans and robots along the lines of SAEs guidelines for autonomous vehicles is introduced in response to a need for standard definitions and evolving nature of the field.
Abstract: Increased global competition has placed a premium on customer satisfaction, and there is a greater demand for manufacturers to be flexible with their products and services. This challenge is usually addressed with the introduction of human operators for precise tasks that require dexterity, flexibility and cognitive decision making. On the other hand, robots, through automation, are very effective in carrying out repetitive, non-ergonomic tasks. Owing to the complementary nature of robots’ and humans’ capabilities, there is an increased interest towards a shared workspace for humans and robots to work together collaboratively, forming the motivation behind the field of human-robot collaboration (HRC). Research in HRC in industry is concerned with the safety of the humans and robots, extent, and modes of collaboration among them, and the level of autonomy and adaptability of robots that can be trained for different tasks. This paper introduces a novel taxonomy of levels of interaction between humans and robots along the lines of SAEs guidelines for autonomous vehicles in response to a need for standard definitions and evolving nature of the field. Research into modes of communication for HRC driven by machine learning are reviewed followed by broad definitions of the types of machine learning. The authors also present a comprehensive review of the machine learning (ML) methodologies and industrial applications of the same in the context of adaptable collaborative robots.
82 citations
TL;DR: A taxonomy of levels of interaction between humans and robots along the lines of SAEs guidelines for autonomous vehicles is presented in this article , in response to a need for standard definitions and evolving nature of the field of human-robot collaboration.
Abstract: Increased global competition has placed a premium on customer satisfaction, and there is a greater demand for manufacturers to be flexible with their products and services. This challenge is usually addressed with the introduction of human operators for precise tasks that require dexterity, flexibility and cognitive decision making. On the other hand, robots, through automation, are very effective in carrying out repetitive, non-ergonomic tasks. Owing to the complementary nature of robots’ and humans’ capabilities, there is an increased interest towards a shared workspace for humans and robots to work together collaboratively, forming the motivation behind the field of human-robot collaboration (HRC). Research in HRC in industry is concerned with the safety of the humans and robots, extent, and modes of collaboration among them, and the level of autonomy and adaptability of robots that can be trained for different tasks. This paper introduces a novel taxonomy of levels of interaction between humans and robots along the lines of SAEs guidelines for autonomous vehicles in response to a need for standard definitions and evolving nature of the field. Research into modes of communication for HRC driven by machine learning are reviewed followed by broad definitions of the types of machine learning. The authors also present a comprehensive review of the machine learning (ML) methodologies and industrial applications of the same in the context of adaptable collaborative robots.
54 citations
TL;DR: In this article , the authors proposed a novel integrated mixed reality (MR) system for safety-aware human-robot collaboration using deep learning and digital twin generation, which can accurately measure the minimum safe distance in real-time and provide MR-based task assistance to the human operator.
Abstract: For human-robot collaboration (HRC), one of the most practical methods to ensure human safety with a vision-based system is establishing a minimum safe distance. This study proposes a novel integrated mixed reality (MR) system for safety-aware HRC using deep learning and digital twin generation. The proposed approach can accurately measure the minimum safe distance in real-time and provide MR-based task assistance to the human operator. The approach integrates MR with safety-related monitoring by tracking the shared workplace and providing user-centric visualization through smart MR glasses for safe and effective HRC. Two RGB-D sensors are used to reconstruct and track the working environment. One sensor scans one area of the physical environment through 3D point cloud data. The other also scans another area of the environment and tracks the user's 3D skeletal information. In addition, the two partially scanned environments are registered together by applying a fast global registration method to two sets of the 3D point cloud. Furthermore, deep learning-based instance segmentation is applied to the target object's 3D point cloud to increase the registration between the real robot and its virtual robot, the digital twin of the real robot. While only 3D point cloud data are widely used in previous studies, this study proposes a simple yet effective 3D offset-based safety distance calculation method based on the robot's digital twin and the human skeleton. The 3D offset-based method allows for real-time applicability without sacrificing the accuracy of safety distance calculation for HRI. In addition, two comparative evaluations were conducted to confirm the originality and advantage of the proposed MR-based HRC.
53 citations
TL;DR: In this paper, the authors proposed a novel integrated mixed reality (MR) system for safety-aware human-robot collaboration using deep learning and digital twin generation, which can accurately measure the minimum safe distance in real-time and provide MR-based task assistance to the human operator.
Abstract: For human-robot collaboration (HRC), one of the most practical methods to ensure human safety with a vision-based system is establishing a minimum safe distance. This study proposes a novel integrated mixed reality (MR) system for safety-aware HRC using deep learning and digital twin generation. The proposed approach can accurately measure the minimum safe distance in real-time and provide MR-based task assistance to the human operator. The approach integrates MR with safety-related monitoring by tracking the shared workplace and providing user-centric visualization through smart MR glasses for safe and effective HRC. Two RGB-D sensors are used to reconstruct and track the working environment. One sensor scans one area of the physical environment through 3D point cloud data. The other also scans another area of the environment and tracks the user's 3D skeletal information. In addition, the two partially scanned environments are registered together by applying a fast global registration method to two sets of the 3D point cloud. Furthermore, deep learning-based instance segmentation is applied to the target object's 3D point cloud to increase the registration between the real robot and its virtual robot, the digital twin of the real robot. While only 3D point cloud data are widely used in previous studies, this study proposes a simple yet effective 3D offset-based safety distance calculation method based on the robot's digital twin and the human skeleton. The 3D offset-based method allows for real-time applicability without sacrificing the accuracy of safety distance calculation for HRI. In addition, two comparative evaluations were conducted to confirm the originality and advantage of the proposed MR-based HRC.
53 citations
TL;DR: In this article , a systematic review of computer vision-based holistic scene understanding in human-robot collaboration scenarios is presented, which mainly takes into account the cognition of object, human, and environment along with visual reasoning to gather and compile visual information into semantic knowledge for subsequent robot decision-making and proactive collaboration.
Abstract: Recently human–robot collaboration (HRC) has emerged as a promising paradigm for mass personalization in manufacturing owing to the potential to fully exploit the strength of human flexibility and robot precision. To achieve better collaboration, robots should be capable of holistically perceiving and parsing the information of a working scene in real-time to plan proactively and act accordingly. Although excessive attentions have been paid to human cognition in existing works of HRC, there is a lack of a holistic consideration of other crucial elements of a working scene, especially when taking a further step towards Proactive HRC. Aiming to fill the gap, this paper provides a systematic review of computer vision-based holistic scene understanding in HRC scenarios, which mainly takes into account the cognition of object, human, and environment along with visual reasoning to gather and compile visual information into semantic knowledge for subsequent robot decision-making and proactive collaboration. Finally, challenges and potential research directions that can be largely facilitated by enhanced holistic perception techniques are also discussed.
53 citations
TL;DR: This review aims to support the robotics community in the future development of HRCD systems, discuss identified literature gaps, and suggest future research directions in this area.
Abstract: Nowadays, numerous companies and industries introduce recycling processes in their production, aiming to increase the sustainable use of the planet’s natural resources. Nevertheless, these processes remain inefficient due to the high degree of complexity and variation in the products. In order to remedy this, industry stakeholders adopt the circular economy business model and introduce take-back programmes and remanufacturing processes for their End of Life products in their own supply chains. Take-back programmes enable the re-sourcing of sub-assemblies and components of previously manufactured products while remanufacturing processes encourage non-destructive disassembly. Due to the uncertain conditions of the re-sourced products, fully automated cells cannot cope with the demanding disassembly processes. Therefore, there is a need to establish hybrid disassembly robot cells where humans and robots work closely together in a process known as human–robot collaborative disassembly (HRCD). This paper examines the landscape of HRCD and reviews the progress in the field during the period 2009–2020. The analysis investigates principles and elements of human–robot collaboration in industrial environments such as safety standards and collaborative operation modes, HRI communication interfaces, and the design characteristics of a disassembly process. Additionally, the various technical challenges of HRCD are explored, and a review of existing systems supporting HRCD is presented. This review aims to support the robotics community in the future development of HRCD systems, discuss identified literature gaps, and suggest future research directions in this area.
52 citations
TL;DR: This research suggests that purposely developed service-oriented data schemas that capture the essential information for high-level cloud manufacturing decision-making via PnP IIoT technologies are a good solution for connecting field-level manufacturing equipment to a cloud manufacturing platform.
Abstract: Cloud manufacturing represents a service-oriented manufacturing paradigm that allows ubiquitous and on-demand access to various customisable manufacturing services in the cloud. While a vast amount of research in cloud manufacturing has focused on high-level decision-making tasks, such as service composition and scheduling, the link between field-level manufacturing data and the cloud manufacturing platform has not been well established. Efficient data acquisition, communication, storage, query, and analysis of field-level manufacturing equipment remain a significant challenge that hinders the development of cloud manufacturing systems. Therefore, this paper investigates the implementation of the emerging Industrial Internet of Things (IIoT) technologies in a cloud manufacturing system to address this challenge. We propose a service-oriented plug-and-play (PnP) IIoT gateway solution based on a generic system architecture of IIoT-supported cloud manufacturing system. Service-oriented data schemas for manufacturing equipment are developed to capture just-enough information about field-level manufacturing equipment and allow efficient data storage and query in a cloud time-series database (TSDB). We tested the feasibility and advantages of the proposed approach via the practical implementation of the IIoT gateways on a 3D printer and a machine tool. Our research suggests that purposely developed service-oriented data schemas that capture the essential information for high-level cloud manufacturing decision-making via PnP IIoT technologies are a good solution for connecting field-level manufacturing equipment to a cloud manufacturing platform.
47 citations
TL;DR: In this article , a multi-robot collaborative manufacturing system with human-in-the-loop control by leveraging the cutting-edge augmented reality (AR) and digital twin (DT) techniques is proposed.
Abstract: The teleoperation and coordination of multiple industrial robots play an important role in today’s industrial internet-based collaborative manufacturing systems. The user-friendly teleoperation approach allows operators from different manufacturing domains to reduce redundant learning costs and intuitively control the robot in advance. Nevertheless, only a few preliminary works have been introduced very recently, let alone its effective implementation in the manufacturing scenarios. To address the gap, this research proposes a novel multi-robot collaborative manufacturing system with human-in-the-loop control by leveraging the cutting-edge augmented reality (AR) and digital twin (DT) techniques. In the proposed system, the DTs of industrial robots are firstly mapped to physical robots and visualize them in the AR glasses. Meanwhile, a multi-robot communication mechanism is designed and implemented, to synchronize the state of robots in the twin. Moreover, a reinforcement learning algorithm is integrated into the robot motion planning to replace the conventional kinematics-based robot movement with corresponding target positions. Finally, three interactive AR-assisted DT modes, including real-time motion control, planned motion control, and robot monitoring mode are generated, which can be readily switched by human operators. Two experimental studies are conducted on (1) a single robot with a commonly used peg-in-hole experiment, and (2) the motion planning of multi-robot collaborative tasks, respectively. From the experimental results, it can be found that the proposed system can well handle the multi-robot teleoperation tasks with high efficiency and owns great potentials to be adopted in other complicated manufacturing scenarios in the near future.
45 citations
TL;DR: A human-robot collaborative reinforcement learning algorithm is proposed to optimize the task sequence allocation scheme in assembly processes and the result shows that the proposed method had great potential in dynamic division of human- robot collaborative tasks.
Abstract: The assembly process of high precision products involves a variety of delicate operations that are time-consuming and energy-intensive. Neither the human operators nor the robots can complete the tasks independently and efficiently. The human-robot collaboration to be applied in complex assembly operation would help reduce human workload and improve efficiency. However, human behavior can be unpredictable in assembly activities so that it is difficult for the robots to understand intentions of the human operations. Thus, the collaboration of humans and robots is challenging in industrial applications. In this regard, a human-robot collaborative reinforcement learning algorithm is proposed to optimize the task sequence allocation scheme in assembly processes. Finally, the effectiveness of the method is verified through experimental analysis of the virtual assembly of an alternator. The result shows that the proposed method had great potential in dynamic division of human-robot collaborative tasks.
45 citations
TL;DR: In this article , a generic service-oriented plug-and-play (PnP) IIoT gateway solution based on a generic system architecture for cloud manufacturing is proposed. But, the work in this paper is focused on high-level decision-making tasks and the link between field-level manufacturing data and the cloud manufacturing platform is not well established.
Abstract: • Proposed a generic system architecture of IIoT-supported cloud manufacturing as a strategic guideline for integrating IIoT technologies in a cloud manufacturing environment. • Proposed a service-oriented PnP IIoT gateway for efficient data acquisition, communication, query, analysis, and visualisation of manufacturing equipment. • Developed two generic service-oriented data schemas for machine tools and 3D printers that allow efficient data management and query for decision-making tasks in a cloud manufacturing platform. • Developed two practical PnP IIoT gateways for a 3D printer and a CNC machine tool that validate the feasibility of the proposed approach. Cloud manufacturing represents a service-oriented manufacturing paradigm that allows ubiquitous and on-demand access to various customisable manufacturing services in the cloud. While a vast amount of research in cloud manufacturing has focused on high-level decision-making tasks, such as service composition and scheduling, the link between field-level manufacturing data and the cloud manufacturing platform has not been well established. Efficient data acquisition, communication, storage, query, and analysis of field-level manufacturing equipment remain a significant challenge that hinders the development of cloud manufacturing systems. Therefore, this paper investigates the implementation of the emerging Industrial Internet of Things (IIoT) technologies in a cloud manufacturing system to address this challenge. We propose a service-oriented plug-and-play (PnP) IIoT gateway solution based on a generic system architecture of IIoT-supported cloud manufacturing system. Service-oriented data schemas for manufacturing equipment are developed to capture just-enough information about field-level manufacturing equipment and allow efficient data storage and query in a cloud time-series database (TSDB). We tested the feasibility and advantages of the proposed approach via the practical implementation of the IIoT gateways on a 3D printer and a machine tool. Our research suggests that purposely developed service-oriented data schemas that capture the essential information for high-level cloud manufacturing decision-making via PnP IIoT technologies are a good solution for connecting field-level manufacturing equipment to a cloud manufacturing platform.
40 citations
TL;DR: In this paper , a human-robot collaborative reinforcement learning algorithm is proposed to optimize the task sequence allocation scheme in assembly processes, and the result shows that the proposed method had great potential in dynamic division of human-robots collaborative tasks.
Abstract: The assembly process of high precision products involves a variety of delicate operations that are time-consuming and energy-intensive. Neither the human operators nor the robots can complete the tasks independently and efficiently. The human-robot collaboration to be applied in complex assembly operation would help reduce human workload and improve efficiency. However, human behavior can be unpredictable in assembly activities so that it is difficult for the robots to understand intentions of the human operations. Thus, the collaboration of humans and robots is challenging in industrial applications. In this regard, a human-robot collaborative reinforcement learning algorithm is proposed to optimize the task sequence allocation scheme in assembly processes. Finally, the effectiveness of the method is verified through experimental analysis of the virtual assembly of an alternator. The result shows that the proposed method had great potential in dynamic division of human-robot collaborative tasks.
TL;DR: A hybrid approach framework driven by digital twin technology (DT), to predict performance degradation of transmission system using the complementary advantages offered by the fusion of these methods to bridge the link between data-driven prediction and model-based prediction.
Abstract: Precision performance prediction of transmission system is considered as a key technology to modern equipment health management. Given the importance of maintaining a transmission system's precision, this paper presents a hybrid approach framework driven by digital twin technology (DT), to predict performance degradation. Firstly, a DT model based on meta-action theory is established, and real-time monitoring and digital simulation, driven by DT data, is realized in order to analyze the precision of the transmission units in machine tools. Secondly, the wear of gear in transmission unit is studied through Achard wear theory, which considered the comprehensive influence of gear load and speed on surface wear of the gear pair tooth, based on the model driving method. The performance degradation of the transmission unit is obtained by using the RBF neural network algorithm based on the data-driven method to extrapolate the wear data to the field-measurable precision index value. In addition, the hybrid predictive approach of the performance degradation model through the particle filter algorithm is built, and the real-time data is used to update the current state estimation to improve the prediction accuracy. By combining the mechanism of the physical degradation processes with the real-time and historical data and turning them into a cooperative architecture, this prediction method uses the complementary advantages offered by the fusion of these methods to bridge the link between data-driven prediction and model-based prediction. Finally, the method has been successfully applied to the precision prediction of the transmission unit in CNCMT turntable, and it is compared with the single prediction method to verify the effectiveness and feasibility.
TL;DR: In this article , a comprehensive disassembly sequence planning (DSP) algorithm in the human-robot collaboration (HRC) setting with consideration of several important factors including limited resources and human workers' safety is presented.
Abstract: This paper presents a comprehensive disassembly sequence planning (DSP) algorithm in the human–robot collaboration (HRC) setting with consideration of several important factors including limited resources and human workers’ safety. The proposed DSP algorithm is capable of planning and distributing disassembly tasks among the human operator, the robot, and HRC, aiming to minimize the total disassembly time without violating resources and safety constraints. Regarding the resource constraints, we consider one human operator and one robot, and a limited quantity of disassembly tools. Regarding the safety constraints, we consider avoiding potential human injuries from to-be-disassembled components and possible collisions between the human operator and the robot due to the short distance between disassembly tasks. In addition, the transitions for tool changing, the moving between disassembly modules, and the precedence constraint of components to be disassembled are also considered and formulated as constraints in the problem formulation. Both numerical and experimental studies on the disassembly of a used hard disk drive (HDD) have been conducted to validate the proposed algorithm. • Disassembly sequence planning considering human-robot collaboration (HRC) • Consideration of constraints including limited resources and human worker’s safety • Distributing and planning disassembly tasks among human, robot, and HRC • Experimental validation on disassembly of used hard disk drives (video available)
TL;DR: In this article, a Pareto-based collaborative multi-objective optimization algorithm (CMOA) is proposed to solve the distributed permutation flow shop problem with limited buffers (DPFSP-LB).
Abstract: Energy-efficient scheduling of distributed production systems has become a common practice among large companies with the advancement of economic globalization and green manufacturing. Nevertheless, energy-efficient scheduling of distributed permutation flow-shop problem with limited buffers (DPFSP-LB) does not receive adequate attention in the relevant literature. This paper is therefore the first attempt to study this DPFSP-LB with objectives of minimizing makespan and total energy consumption ( T E C ). To solve this energy-efficient DPFSP-LB, a Pareto-based collaborative multi-objective optimization algorithm (CMOA) is proposed. In the proposed CMOA, first, the speed scaling strategy based on problem property is designed to reduce T E C . Second, a collaborative initialization strategy is presented to generate a high-quality initial population. Third, three properties of DPFSP-LB are utilized to develop a collaborative search operator and a knowledge-based local search operator. Finally, we verify the effectiveness of each improvement component of CMOA and compare it against other well-known multi-objective optimization algorithms on instances. Experiment results demonstrate the effectiveness of CMOA in solving this energy-efficient DPFSP-LB. Especially, the CMOA is able to obtain excellent results on all problems regarding the comprehensive metric, and is also competitive to its rivals regarding the convergence metric.
TL;DR: In this article , a Pareto-based collaborative multi-objective optimization algorithm (CMOA) is proposed to solve the problem of distributed permutation flow shop problem with limited buffers.
Abstract: Energy-efficient scheduling of distributed production systems has become a common practice among large companies with the advancement of economic globalization and green manufacturing. Nevertheless, energy-efficient scheduling of distributed permutation flow-shop problem with limited buffers (DPFSP-LB) does not receive adequate attention in the relevant literature. This paper is therefore the first attempt to study this DPFSP-LB with objectives of minimizing makespan and total energy consumption ( T E C ). To solve this energy-efficient DPFSP-LB, a Pareto-based collaborative multi-objective optimization algorithm (CMOA) is proposed. In the proposed CMOA, first, the speed scaling strategy based on problem property is designed to reduce T E C . Second, a collaborative initialization strategy is presented to generate a high-quality initial population. Third, three properties of DPFSP-LB are utilized to develop a collaborative search operator and a knowledge-based local search operator. Finally, we verify the effectiveness of each improvement component of CMOA and compare it against other well-known multi-objective optimization algorithms on instances. Experiment results demonstrate the effectiveness of CMOA in solving this energy-efficient DPFSP-LB. Especially, the CMOA is able to obtain excellent results on all problems regarding the comprehensive metric, and is also competitive to its rivals regarding the convergence metric. • A green criterion is considered in the studied problem. • A new constraint of the limited buffers is introduced into this problem. • A multi-objective optimization algorithm is presented to solve this problem. • An effective energy saving strategy is proposed. • An initialization strategy and local search strategy are proposed.
TL;DR: An automatic construction framework for the process knowledge base in the field of machining based on knowledge graph (KG) is introduced and a hybrid algorithm based on an improved edit distance and attribute weighting is built to overcome the redundancy in the knowledge fusion stage.
Abstract: The process knowledge base is the key module in intelligent process design, it determines the intelligence degree of the design system and affects the quality of product design. However, traditional process knowledge base construction is non-automated, time consuming and requires much manual work, which is not sufficient to meet the demands of the modern manufacturing mode. Moreover, the knowledge base often adopts a single knowledge representation, and this may lead to ambiguity in the meaning of some knowledge, which will affect the quality of the process knowledge base. To overcome the above problems, an automatic construction framework for the process knowledge base in the field of machining based on knowledge graph (KG) is introduced. First, the knowledge is classified and annotated based on the function-behavior-states (FBS) design method. Second, a knowledge extraction framework based on BERT-BiLSTM-CRF is established to perform the automatic knowledge extraction of process text. Third, a knowledge representation method based on fuzzy comprehensive evaluation is established, forming three types of knowledge representation with the KG as the main, production rules and two-dimensional data linked list as a supplement. In addition, to overcome the redundancy in the knowledge fusion stage, a hybrid algorithm based on an improved edit distance and attribute weighting is built. Finally, a prototype system is developed, and quality analysis is carried out. Compared with the F values of BiLSTM-CRF and CNN-BiLSTM-CRF, that of the proposed extraction method in the machining domain is increased by 7.35% and 3.87%, respectively.
TL;DR: A novel CNN with the function of spectrum calculation and fault diagnosis is designed, in which the spectrum calculation network and the fault diagnosis network are connected in series and a new error cost function model is designed to guide the network parameters optimization in the direction of feature classification, which is conductive to improve the diagnosis accuracy.
Abstract: The combination of nonlinear spectrum and convolutional neural network (CNN) is efficient for fault diagnosis of nonlinear system. However, in traditional method, the nonlinear spectrum calculation was accomplished by identification algorithm outside the CNN, which reduced the diagnosis efficiency. To solve this problem, a novel CNN with the function of spectrum calculation and fault diagnosis is designed, in which the spectrum calculation network and the fault diagnosis network are connected in series. By extracting the optimized parameters of network, the nonlinear spectrum based on generalized frequency response function (GFRF) is obtained in the former network. Then, the GFRF spectrum is automatically put into the latter network for feature extraction and diagnosis. Hence, after determining the structure of the CNN, only by system input and output, the fault diagnosis can be realized, which avoids the complex process in traditional method. What's more, a new error cost function model is designed to guide the network parameters optimization in the direction of feature classification, which is conductive to improve the diagnosis accuracy. The proposed network model is applied to the heavy-duty industrial robot system, and the best performance is demonstrated by several experiments.
TL;DR: The experimental results indicate that this method can significantly improve the efficiency, accuracy, and flexibility of the robotic welding system and is demonstrated by integrating it into a subassembly welding robotic system.
Abstract: Offline programming is an intuitive and automatic programming generation technique that does not use real robotic systems, thus greatly decreasing the downtime required for system programming, and resulting in enormous savings in terms of labor costs. Currently, offline programming can be generally categorized into computer-aided-design-based (CAD-based) and vision-based approaches; these two types of offline programming approaches have been widely applied in robotic welding systems. However, owing to the highly complex and diverse workpieces needed in the shipbuilding industry, neither of the aforementioned offline programming approaches can fully support the automatic generation of welding programs. In this paper, a hybrid offline programming method systematically combining CAD-based, vision-based, and vision & CAD interactive activities is proposed to overcome the limitations of current automatic program generation methods for robotic welding systems. In the vision-based activities, the positions of the workpieces are obtained by using geometrical features gathered from the workpieces’ images, whereas in the CAD-based activities, the welding tasks are assigned to different mobile components of the welding torch; then, their welding paths are planned according to the workpieces’ CAD models. The vision & CAD interactive activities enable the mapping between the point cloud of a workpiece and its CAD model, so that the deviations caused by assembly errors can be detected and path compensation data can be determined. The effectiveness of the proposed hybrid offline programming method is demonstrated by integrating it into a subassembly welding robotic system. The experimental results indicate that this method can significantly improve the efficiency, accuracy, and flexibility of the robotic welding system.
TL;DR: The DSS was developed using a simulation-optimization approach by incorporating an artificial neural network and a genetic algorithm for problem representation and optimizing decision support solutions to help SMEs compile and exploit data, and supporting their decisions under business ambiguities.
Abstract: Elevated business uncertainties and competition over recent years have caused changes to the data-driven supply chain management of sourcing and inventories across industries. However, only large-sized enterprises have the resources to harness data for aiding their decision-making and planning. By contrast, small- and medium-sized enterprises (SMEs) commonly have limited resources and knowledge, which affects their ability to collect and utilize data. Thus, it is a challenge for them to implement advanced decision support tools to mitigate the effects of market uncertainties. This paper proposes a decision support system (DSS) for sourcing and inventory management, with the aims of helping SMEs compile and exploit data, and supporting their decisions under business ambiguities. The DSS was developed using a simulation-optimization approach by incorporating an artificial neural network and a genetic algorithm for problem representation and optimizing decision support solutions. The exploitation of observational and empirical data reduces the burden of data compilation obtained from unorganized data sources across SME operations. Further, uncertainty factors such as raw material demand, price, and supply lead time were considered. When implemented in a medium-sized food industry company, the DSS can provide decision support solutions that integrate the selection of recommended suppliers and optimal order quantities. It can also help decision-makers to shape their inventory management policies under uncertain raw material demands, while also considering service levels, sales promotions, lead times, and material availability from multiple suppliers. Consequently, implementation of the DSS helped to reduce the total purchased raw material costs by an average of 51.62% and reduced the order interval and on-hand inventory costs by an average of 54.24%.
TL;DR: In this article , a multi-agent manufacturing system based on deep reinforcement learning (DRL) is presented, which integrates the self-organization mechanism and self-learning strategy.
Abstract: Personalized orders bring challenges to the production paradigm, and there is an urgent need for the dynamic responsiveness and self-adjustment ability of the workshop. Traditional dispatching rules and heuristic algorithms solve the production planning and control problems by making schedules. However, the previous methods cannot work well in a changeable workshop environment when encountering a large number of stochastic disturbances of orders and resources. Recently, the potential of artificial intelligence (AI) algorithms in solving the dynamic scheduling problem has attracted researchers' attention. Therefore, this paper presents a multi-agent manufacturing system based on deep reinforcement learning (DRL), which integrates the self-organization mechanism and self-learning strategy. Firstly, the manufacturing equipment in the workshop is constructed as an equipment agent with the support of edge computing node, and an improved contract network protocol (CNP) is applied to guide the cooperation and competition among multiple agents, so as to complete personalized orders efficiently. Secondly, a multi-layer perceptron is employed to establish the decision-making module called AI scheduler inside the equipment agent. According to the perceived workshop state information, AI scheduler intelligently generates an optimal production strategy to perform task allocation. Then, based on the collected sample trajectories of scheduling process, AI scheduler is periodically trained and updated through the proximal policy optimization (PPO) algorithm to improve its decision-making performance. Finally, in the multi-agent manufacturing system testbed, dynamic events such as stochastic job insertions and unpredictable machine failures are considered in the verification experiments. The experimental results show that the proposed method is capable of obtaining the scheduling solutions that meet various performance metrics, as well as dealing with resource or task disturbances efficiently and autonomously.
TL;DR: In this article , the authors proposed the concept design of a novel end-effector based on constant-force mechanism for robotic polishing, which is used to polish rusty steel, and the recorded contact force signal during polishing demonstrates the effectiveness of the constant force mechanism in counteracting the force overshoot.
Abstract: Polishing is an important final machining process in manufacturing. For robotic polishing, active compliance control is the most frequently used approach to control the contact force between the end-effector and workpiece. However, it usually exhibits the problem of force overshoot at the start of contact, with poor force accuracy normally larger than 1 N. This paper proposes the concept design of a novel end-effector based on constant-force mechanism for robotic polishing. This design is the first constant-force mechanism based robotic end-effector for use in polishing experiment. An industrial robot is used to position the end-effector and the end-effector regulates the contact force passively. The constant-force motion range acts as a buffer to counteract the excessive displacement caused by inertia. As a result, there is no force overshoot, producing the consistency for the workpiece’s surface quality. Moreover, the property of the constant force ensures more accurate contact force without using a complex controller. Design, modeling, and simulation study have been performed to demonstrate the proposed idea. A prototype is fabricated for experimental testing. The end-effector is adopted to polish rusty steel, and the recorded contact force signal during polishing demonstrates the effectiveness of the constant-force mechanism in counteracting the force overshoot and improving the force accuracy. Results indicate that the polished surface is extremely uniform with steady contact force regulated by the constant-force mechanism.
TL;DR: In this article, the authors proposed the concept design of a novel end-effector based on constant-force mechanism for robotic polishing, which is used to polish rusty steel, and the recorded contact force signal during polishing demonstrates the effectiveness of the constant force mechanism in counteracting the force overshoot.
Abstract: Polishing is an important final machining process in manufacturing. For robotic polishing, active compliance control is the most frequently used approach to control the contact force between the end-effector and workpiece. However, it usually exhibits the problem of force overshoot at the start of contact, with poor force accuracy normally larger than 1 N. This paper proposes the concept design of a novel end-effector based on constant-force mechanism for robotic polishing. This design is the first constant-force mechanism based robotic end-effector for use in polishing experiment. An industrial robot is used to position the end-effector and the end-effector regulates the contact force passively. The constant-force motion range acts as a buffer to counteract the excessive displacement caused by inertia. As a result, there is no force overshoot, producing the consistency for the workpiece’s surface quality. Moreover, the property of the constant force ensures more accurate contact force without using a complex controller. Design, modeling, and simulation study have been performed to demonstrate the proposed idea. A prototype is fabricated for experimental testing. The end-effector is adopted to polish rusty steel, and the recorded contact force signal during polishing demonstrates the effectiveness of the constant-force mechanism in counteracting the force overshoot and improving the force accuracy. Results indicate that the polished surface is extremely uniform with steady contact force regulated by the constant-force mechanism.
TL;DR: Wang et al. as mentioned in this paper proposed an intelligent predictive maintenance approach for machine tools via multiple services cooperating within a single framework, which is supported by the combination of Convolutional Neural Network and Long Short-Term Memory (CNN-LSTM).
Abstract: • We solved the fault prediction problem with the combination of CNN and LSTM . • We solved the maintenance decision problem with the deep reinforcement learning. • The enhanced visual guidance of the maintenance process was realized by AR , which could integrate the invisible maintenance information into the machine tools. • The proposed predictive maintenance approach outperforms others in terms of accuracy and effectiveness. In the Industry 4.0 era, the number and complexity of machine tools are both increased, which is prone to cause malfunctions and downtime in the manufacturing process. Predictive Maintenance (PdM), as a pivotal part of Prognostics and Health Management (PHM), plays a vital role in enhancing the reliability of machine tools in the Internet of Things (IoT)-enabled manufacturing. In order to realize a highly reliable maintenance plan integrated with the fault prediction, the maintenance decision-making, and the Augmented Reality (AR)-enabled auxiliary maintenance, an intelligent predictive maintenance approach for machine tools is proposed in this paper via multiple services cooperating within a single framework. The fault prediction service is supported by the combination of Convolutional Neural Network and Long Short-Term Memory (CNN-LSTM). Specified features from massive production data acquired by IoT can be comprehensively extracted using CNN, and their nonlinear relationship can be fitted by LSTM. Based on the fault prediction result, deep reinforcement learning is adopted to achieve the production control and schedule maintenance personnel if a fault code appears. On top of this, the guidance information from the maintenance experience database can be integrated into the faulty machine tools in the form of visibility through AR, which can guide the maintenance personnel to complete maintenance tasks more efficiently. Moreover, the remote expert service is also integrated in the AR-supported auxiliary maintenance, which is activated to solve unexpected faults that are not stored in the maintenance experience database. Comparative experiments are conducted in the IoT-enabled manufacturing workshop with real-world case studies, and the results demonstrate that the proposed predictive maintenance approach is both effective and practical.
TL;DR: An object detection solution based on deep learning on 3D point clouds for a collaborative mobile robot manipulator to automate SME production and validate the object detection, localization algorithms, and tools developed and employed in production cases using the mobile robot manipulated.
Abstract: Increasing research attention has been attracted to automatic production processes in small and medium-sized enterprises (SMEs) using collaborative robotic systems. In this work, we develop an object detection solution based on deep learning on 3D point clouds for a collaborative mobile robot manipulator to automate SME production. In this solution, a 3D point cloud technology is adopted to measure the shape and depth information of targeted objects in SME production, for instance, name tags production and plug-in charging. Deep learning is then employed to deal with the uncertainty in 3D detection, such as inconsistent light conditions and the irregular distribution and structural ambiguity of point clouds. A 2D camera is employed to calibrate the relative positions of the mobile manipulator to workstations. The mobile robot manipulator is equipped with cameras, an in-house developed adaptive gripper, and a learning-based computer vision system developed in this work. The principle and procedures of the proposed 3D object detection and 2D calibration are presented in detail. The automatic name tags production and plug-in charging experiments are conducted to validate the object detection, localization algorithms, and tools developed and employed in production cases using the mobile robot manipulator.
TL;DR: An exact mixed-integer programming model is established to accurately obtain the minimum disassembly objectives: cycle time, energy consumption, and improved hazardous index and the superiority of HDA is proved by comparing the optimization results of a large-scale case with three other classic algorithms.
Abstract: To address the problem of considerable waste electromechanical product generation, a partial disassembly line balancing problem with multi-robot workstations that can synchronously disassemble multiple products (MPR-PDLBP) is investigated to improve the product capacity and efficiency of existing disassembly lines. First, an exact mixed-integer programming model is established to accurately obtain the minimum disassembly objectives: cycle time, energy consumption, and improved hazardous index. Second, compared with the conventional disassembly line balancing problem (DLBP), the solution space and optimization difficulty of MPR-PDLBP increase significantly. Thus, a multi-objective hybrid driving algorithm (HDA) based on a three-layer encoding method with a heuristic rule is proposed to effectively address MPR-PDLBP, and a driving strategy is proposed to improve the exploitation ability and convergence speed of HDA. Finally, validity of the proposed model and algorithm are verified by comparing the calculation results of GUROBI and HDA for two small-scale cases. The superiority of HDA is proved by comparing the optimization results of a large-scale case with three other classic algorithms.
TL;DR: In this article , the authors propose a rapid construction method of equipment model (RCMEM) for a discrete manufacturing DT workshop system, which can meet the requirements of complex manufacturing business scenarios, improve the efficiency and quality of the DT model construction at the equipment level, and form an efficient equipment model construction method.
Abstract: A large-scale manufacturing workshop includes a wide variety and a large amount of equipment. When equipment is mapped as a digital twin (DT) model, the DT model has complex and implicit or non-hierarchical relationships, which makes the development of a DT workshop system (DTWS) for a manufacturing plant challenging. To address this problem, this paper proposes a rapid construction method of equipment model (RCMEM) for a discrete manufacturing DT workshop system, which can meet the requirements of complex manufacturing business scenarios, improve the efficiency and quality of the DT model construction at the equipment level, and form an efficient equipment model construction method. First, a DT object (DTO), which represents a basic unit of the DT model, is defined. The DTO is used to derive the DT model at the equipment level. The DTO collects and wraps generalization-capable functions from equipment into components, and geometric, functional, and communicative descriptions are combined into a single model. The DT model can increase the scalability, reusability, and modeling quality by reusing the DTO capabilities. Further, a general construction process of the DTO model of multi-axis computer numerical control (CNC) machines is presented. This process standardizes the common capabilities of the equipment models and minimizes the construction time of machines’ DT models. Furthermore, a rapid mounting and communication configuration mechanism (RMCCM) is proposed. This mechanism enables fast DTO network communication and improves interoperability in a multi-source data environment. This paper also proposes a DTWorks Software development kit, which can help developers to build digital factory standalone applications. The RCMEM is implemented into the DTWorks. Finally, a real DTWS case is used to examine the differences in equipment model building before and after the application of the RCMEM. The results demonstrate the good effectiveness of the RCMEM in complex business settings.
TL;DR: In this paper, a knowledge-based system for predictive maintenance in Industry 4.0 (KSPMI) is developed based on a novel hybrid approach that leverages both statistical and symbolic AI technologies.
Abstract: In the context of Industry 4.0, smart factories use advanced sensing and data analytic technologies to understand and monitor the manufacturing processes. To enhance production efficiency and reliability, statistical Artificial Intelligence (AI) technologies such as machine learning and data mining are used to detect and predict potential anomalies within manufacturing processes. However, due to the heterogeneous nature of industrial data, sometimes the knowledge extracted from industrial data is presented in a complex structure. This brings the semantic gap issue which stands for the lack of interoperability among different manufacturing systems. Furthermore, as the Cyber-Physical Systems (CPS) are becoming more knowledge-intensive, uniform knowledge representation of physical resources and real-time reasoning capabilities for analytic tasks are needed to automate the decision-making processes for these systems. These requirements highlight the potential of using symbolic AI for predictive maintenance. To automate and facilitate predictive analytics in Industry 4.0, in this paper, we present a novel Knowledge-based System for Predictive Maintenance in Industry 4.0 (KSPMI). KSPMI is developed based on a novel hybrid approach that leverages both statistical and symbolic AI technologies. The hybrid approach involves using statistical AI technologies such as machine learning and chronicle mining (a special type of sequential pattern mining approach) to extract machine degradation models from industrial data. On the other hand, symbolic AI technologies, especially domain ontologies and logic rules, will use the extracted chronicle patterns to query and reason on system input data with rich domain and contextual knowledge. This hybrid approach uses Semantic Web Rule Language (SWRL) rules generated from chronicle patterns together with domain ontologies to perform ontology reasoning, which enables the automatic detection of machinery anomalies and the prediction of future events’ occurrence. KSPMI is evaluated and tested on both real-world and synthetic data sets.
TL;DR: In this article , an automatic construction framework for the process knowledge base in the field of machining based on knowledge graph (KG) is introduced, and a knowledge extraction framework based on BERT-BiLSTM-CRF is established to perform the automatic knowledge extraction of process text.
Abstract: • A framework for automatically constructing a knowledge base is developed. • The extraction effect of this framework is better than other frameworks. • A evaluation algorithm is proposed to judge the optimal expression of knowledge. • Semantic and attribute weighting factors among knowledge entities are considered. The process knowledge base is the key module in intelligent process design, it determines the intelligence degree of the design system and affects the quality of product design. However, traditional process knowledge base construction is non-automated, time consuming and requires much manual work, which is not sufficient to meet the demands of the modern manufacturing mode. Moreover, the knowledge base often adopts a single knowledge representation, and this may lead to ambiguity in the meaning of some knowledge, which will affect the quality of the process knowledge base. To overcome the above problems, an automatic construction framework for the process knowledge base in the field of machining based on knowledge graph (KG) is introduced. First, the knowledge is classified and annotated based on the function-behavior-states (FBS) design method. Second, a knowledge extraction framework based on BERT-BiLSTM-CRF is established to perform the automatic knowledge extraction of process text. Third, a knowledge representation method based on fuzzy comprehensive evaluation is established, forming three types of knowledge representation with the KG as the main, production rules and two-dimensional data linked list as a supplement. In addition, to overcome the redundancy in the knowledge fusion stage, a hybrid algorithm based on an improved edit distance and attribute weighting is built. Finally, a prototype system is developed, and quality analysis is carried out. Compared with the F values of BiLSTM-CRF and CNN-BiLSTM-CRF, that of the proposed extraction method in the machining domain is increased by 7.35% and 3.87%, respectively.
TL;DR: In this article , a knowledge-based system for predictive maintenance in Industry 4.0 (KSPMI) is developed based on a novel hybrid approach that leverages both statistical and symbolic AI technologies.
Abstract: In the context of Industry 4.0, smart factories use advanced sensing and data analytic technologies to understand and monitor the manufacturing processes. To enhance production efficiency and reliability, statistical Artificial Intelligence (AI) technologies such as machine learning and data mining are used to detect and predict potential anomalies within manufacturing processes. However, due to the heterogeneous nature of industrial data, sometimes the knowledge extracted from industrial data is presented in a complex structure. This brings the semantic gap issue which stands for the lack of interoperability among different manufacturing systems. Furthermore, as the Cyber-Physical Systems (CPS) are becoming more knowledge-intensive, uniform knowledge representation of physical resources and real-time reasoning capabilities for analytic tasks are needed to automate the decision-making processes for these systems. These requirements highlight the potential of using symbolic AI for predictive maintenance. To automate and facilitate predictive analytics in Industry 4.0, in this paper, we present a novel Knowledge-based System for Predictive Maintenance in Industry 4.0 (KSPMI). KSPMI is developed based on a novel hybrid approach that leverages both statistical and symbolic AI technologies. The hybrid approach involves using statistical AI technologies such as machine learning and chronicle mining (a special type of sequential pattern mining approach) to extract machine degradation models from industrial data. On the other hand, symbolic AI technologies, especially domain ontologies and logic rules, will use the extracted chronicle patterns to query and reason on system input data with rich domain and contextual knowledge. This hybrid approach uses Semantic Web Rule Language (SWRL) rules generated from chronicle patterns together with domain ontologies to perform ontology reasoning, which enables the automatic detection of machinery anomalies and the prediction of future events’ occurrence. KSPMI is evaluated and tested on both real-world and synthetic data sets.