Continuum robots (CRs) hold great potential for many medical and industrial applications where compliant interaction within the potentially confined environment is required. However, the navigation of CRs poses several challenges due to their limited actuation channels and the hyper-flexibility of their structure. Environmental uncertainty and characteristic hysteresis in such procedures add to the complexity of their operation. Therefore, the quality of trajectory tracking for continuum robots plays an essential role in the success of the application procedures. While there are a few different actuation configurations available for CRs, the focus of this paper will be placed on tendon-driven manipulators. In this research, a new fuzzy reinforcement learning (FRL) approach is introduced. The proposed FRL-based control parameters are tuned by the Taguchi method and evolutionary genetic algorithm (GA) to provide faster convergence to the Nash Equilibrium. The approach is verified through a comprehensive set of simulations using a Cosserat rod model. The results show a steady and accurate trajectory tracking capability for a CR.

A Fuzzy Reinforcement Learning Approach for Continuum Robot Control

Soft continuum robots have been accepted as a promising category of biomedical robots, accredited to the robots' inherent compliance that makes them safely interact with their surroundings. In its application of minimally invasive surgery, such a continuum concept shares the same view of robotization for conventional endoscopy/laparoscopy. Different from rigid-link robots with accurate analytical kinematics/dynamics, soft robots encounter modeling uncertainties due to intrinsic and extrinsic factors, which would deteriorate the model-based control performances. However, the trade-off between flexibility and controllability of soft manipulators may not be readily optimized but would be demanded for specific kinds of modeling approaches. To this end, data-driven modeling strategies making use of machine learning algorithms would be an encouraging way out for the control of soft continuum robots. In this article, we attempt to overview the current state of kinematic/dynamic model-free control schemes for continuum manipulators, particularly by learning-based means, and discuss their similarities and differences. Perspectives and trends in the development of new control methods are also investigated through the review of existing limitations and challenges.

/pdf/a-survey-for-machine-learning-based-control-of-continuum-1t7tzty22t.pdf

A Survey for Machine Learning-Based Control of Continuum Robots.

Since the emergence of soft robotics around two decades ago, research interest in the field has escalated at a pace. It is fuelled by the industry’s appreciation of the wide range of soft materials available that can be used to create highly dexterous robots with adaptability characteristics far beyond that which can be achieved with rigid component devices. The ability, inherent in soft robots, to compliantly adapt to the environment, has significantly sparked interest from the surgical robotics community. This article provides an in-depth overview of recent progress and outlines the remaining challenges in the development of soft robotics for minimally invasive surgery.

/pdf/soft-robot-assisted-minimally-invasive-surgery-and-2t8wz91s.pdf

Soft Robot-Assisted Minimally Invasive Surgery and Interventions: Advances and Outlook

In percutaneous intervention for treatment of coronary plaques, guidewire navigation is a primary procedure for stent delivery. Steering a flexible guidewire within coronary arteries requires considerable training, and the non-linearity between the control operation and the movement of the guidewire makes precise manipulation difficult. Here, we introduce a deep reinforcement learning(RL) framework for autonomous guidewire navigation in a robot-assisted coronary intervention. Using Rainbow, a segment-wise learning approach is applied to determine how best to accelerate training using human demonstrations with deep Q-learning from demonstrations (DQfD), transfer learning, and weight initialization. `State' for RL is customized as a focus window near the guidewire tip, and subgoals are placed to mitigate a sparse reward problem. The RL agent improves performance, eventually enabling the guidewire to reach all valid targets in `stable' phase. Our framework opens anew direction in the automation of robot-assisted intervention, providing guidance on RL in physical spaces involving mechanical fatigue.

Deep reinforcement learning for guidewire navigation in coronary artery phantom.

This paper presentsa safe and effective keyhole neurosurgery intra-operative planning framework for flexible neurosurgical robots. The framework is intended to support neurosurgeons during the intra-operative procedure to react to a dynamic environment.The proposed system integrates inverse reinforcement learning path planning algorithm combined with 1) a pre-operative path planning framework for fast and intuitive user interaction, 2) a realistic, time-bounded simulator based on Position-based Dynamics (PBD) simulation that mocks brain deformations due to catheter insertion and 3) a simulated robotic system.Simulation results performed on a human brain dataset show that the inverse reinforcement learning intra-operative planning method can guide a steerable needle with bounded curvature to a predefined target pose with an average targeting error of 1.34 ± 0.52 (25 th = 1.02, 75 th = 1.36) mm in position and 3.16 ± 1.06 (25 th = 2, 75 th = 4.94) degrees in orientation under a deformable simulated environment, with a re-planning time of 0.02 sec and a success rate of 100%.With this work, we demonstrate that the presented intra-operative steerable needle path planner is able to avoid anatomical obstacles while optimising surgical criteria.The results demonstrate that the proposed method is fast and can securely steer flexible needles with high accuracy and robustness. 

Inverse Reinforcement Learning Intra-Operative Path Planning for Steerable Needle

To reduce the radiation exposure of personnel during an interventional procedure for arrhythmia, a robot has been developed and implemented herein for use in interventional procedures. Studies on the control of an electrophysiology catheter by robots are being conducted. However, controlling a catheter using a robot has limited precision owing to external forces subjected on the catheter due to blood flow and pulse inside a heart. This study implements a reinforcement learning method for automated control of a catheter by a robot. Using the reinforcement learning method, this study aims to show that such a robot can learn to manipulate a catheter to reach a target in a simulated environment and subsequently control a catheter in an actual environment. Randomization noise is used during the simulation to reduce the differences between the simulation and actual learning environments. Each environment is implemented with different movement values depending on insertion angles and steps of the catheter model. When the results from the simulated learning model are implemented in the actual environment, the success rate of catheter reaching the designated target is 73 %. In addition, the noise-implemented model shows that the success rate can be increased up to 87 %. Through these experiments, the study verifies that a simulated learning model can be implemented in a robot system to control an actual catheter as well as that the success rate of the model can be increased using randomization noise.

Automatic control of cardiac ablation catheter with deep reinforcement learning method

This paper introduces a newly developed robotic gait training system for lower-limb rehabilitation of stroke patients. The system (Cyborg-Trainer L; Cyborg-Lab Co., Korea) provides a stair-climbing mode in addition to the conventional level-walking mode by leveraging a unique exoskeleton structure with separately operable foot plates. Unlike conventional end-effector type gait training robots, the subject’s feet are not constrained by foot plates, but are free to emulate the ground or a set of stairs. The ground reaction force is measured by force sensors in the foot plates and utilized to compensate for the vertical movement of pelvis. The exoskeleton structures are connected at hip, knee, and ankle joints, and these can support a patient’s weight to ensure a normal gait pattern. The system has four control modes with different levels of assistive or resistance force. To show the feasibility of the developed training mode, a series of experiments measuring muscle activity were conducted during 1) level-walking with the robot, 2) level-walking on a treadmill without a robot, 3) stair-climbing with the robot, and 4) actual stairclimbing without a robot. The muscle activation from the rectus femoris, biceps femoris, tibialis anterior, and gastrocnemius medialis of the dominant leg of five healthy adults were measured and analyzed. Results showed that all muscles had a rhythmic muscle activation pattern. Even though muscle activation patterns were different between gaits using the robotic gait system and those not using it, reduced amplitudes and phasic muscle activations were observed during the training in the robotic system. The developed system is a new type of robotic gait training system that could induce phasic lower limb muscle activation patterns, and its clinical efficacy will be validated in clinical trials after regulatory approval.

A Robotic Gait Training System with Stair-climbing Mode Based on a Unique Exoskeleton Structure with Active Foot Plates

Change in Risk of Dropout Due to Bleeding during Bloodletting-Cupping Therapy

이 연구는 육아휴직자 중 남성 비율이 급증한 2010년대 중반 이후 육아휴직을 사용한 남성들을 심층면접하여 남성의 육아휴직이 한국의 젠더관계 변화에 갖는 함의를 검토한다. 남성의 육아휴직을 둘러싼 제도적·사회적 변화 속에서 실제로 휴직을 한 남성들은 어떤 사람들이며 어떤 경험을 하고 있는지를 일과 돌봄의 맥락에서 읽어내고, 남성의 육아휴직이 한국사회 돌봄의 분배 체제를 탈젠더화하는 방향으로 변화시키고 ‘돌보는 남성성’을 등장시킬 계기가 될지 따져본다.BR 연구 결과, 연구참여자들은 맞벌이를 하며 아내의 소득과 일자리 안정성이 높은 사람들로 육아휴직 전부터 가사노동을 분담했으며, 승진을 위해 가족생활을 희생하지 않고자 하는 사람들이었다. 이들의 육아휴직 결정은 조부모 돌봄이 가능하지 않고 아내의 육아휴직이 소진된 다음에 이루어졌으며, 조직에 존재하는 ‘이상적 노동자상’이 휴직의 전 과정에서 걸림돌이 되었다. 이들은 휴직 기간 중 육아노동에 전방위적으로 참여했지만, 가사노동 참여는 그렇지 않았다. 이들이 육아휴직 경험에 부여하는 의미는 아버지-자녀 관계와 핵가족적 틀을 넘어서지 못했고, 남성성의 재구성을 향해 나아가는 모습은 드러나지 않았다. 이런 연구 결과는 남성 육아휴직자의 증가를 곧 돌봄의 탈젠더화 그 자체로 이해하는 것은 무리가 있으며, 노동시장의 성평등화를 위한 복합적인 개입과 ‘이상적 노동자상’의 변화가 필요함을 시사한다. 

Does fathers’ use of parental leave promote degenderization of care? : Focusing on fathers’ experience of parental leave since the mid-2010s

이 연구는 자신을 페미니스트로 정체화하는 젊은 여성들의 드럭스토어 이용 경험을 정치적 소비주의(political consumerism) 관점에서 분석하고, 이 과정에서 이들이 경험하는 갈등과 주체적 행위전략을 살펴본다. 드럭스토어는 K-뷰티 산업의 집약처이자 일상적 욕망의 코드를 담은 소매점으로서 도시 여성의 라이프스타일을 구성해 왔으나, 미용 관습을 전면 거부하는 ‘탈코르셋’ 운동이 대중적 페미니즘의 한 흐름으로 자리 잡으면서 이 공간의 유혹성은 페미니스트 정체성과 긴장관계에 놓이게 되었다. 연구는 탈코르셋 이후에도 많은 여성들이 드럭스토어를 방문한다는 점에 주목하여, 참여관찰 등 예비조사와 심층면접을 통해 페미니스트 여성들의 드럭스토어 이용 경험과 주관적 의미부여에 대한 질적 연구를 수행하였다.BR 연구 결과, 여성들은 드럭스토어를 이용하면서도 자신이 생각하는 페미니스트 가치에 부합하게 행동하고자 다양한 신체적ㆍ심리적 전략을 사용하고 있었다. 광고와 화려한 상품진열대 앞에서 여전히 구매 욕구를 느끼는 자신을 발견하지만 이러한 관습화된 욕망이 충동적인 미용 상품 소비로 이어지지 않도록 시선 및 동선과 사고방식을 바꾸며, 이는 미용 상품에 대한 보이콧(boycott) 및 친환경ㆍ비건 제품에 대한 바이콧(buycott)이라는 정치적 소비주의 실천으로 이어졌다. 여성들은 일상 속에서 할 수 있는 페미니스트 실천으로서 정치적 소비를 의미화하며 자신만의 페미니스트 정체성을 구성해간다. 이를 바탕으로 연구는 한국 여성의 ‘탈코르셋’이 일회적이고 단절적인 사건이나 상태가 아닌, 다양하고 지속적인 페미니스트 되기의 과정으로 이해되어야 함을 제시한다. 

Eunkyung Bae

Papers

Automatic control of cardiac ablation catheter with deep reinforcement learning method

A Robotic Gait Training System with Stair-climbing Mode Based on a Unique Exoskeleton Structure with Active Foot Plates

Change in Risk of Dropout Due to Bleeding during Bloodletting-Cupping Therapy

Does fathers’ use of parental leave promote degenderization of care? : Focusing on fathers’ experience of parental leave since the mid-2010s

Feminist Women’s Drugstore Usage after Tal-Corset: Political Consumerism as a Path to Becoming Feminist