S Phaniteja

Bio: S Phaniteja is an academic researcher from International Institute of Information Technology, Hyderabad. The author has contributed to research in topics: Reinforcement learning & Humanoid robot. The author has an hindex of 4, co-authored 7 publications receiving 55 citations. Previous affiliations of S Phaniteja include International Institute of Information Technology.

A deep reinforcement learning approach for dynamically stable inverse kinematics of humanoid robots

Abstract: Real time calculation of inverse kinematics (IK) with dynamically stable configuration is of high necessity in humanoid robots as they are highly susceptible to lose balance. This paper proposes a methodology to generate joint-space trajectories of stable configurations for solving inverse kinematics using Deep Reinforcement Learning (RL). Our approach is based on the idea of exploring the entire configuration space of the robot and learning the best possible solutions using Deep Deterministic Policy Gradient (DDPG). The proposed strategy was evaluated on the highly articulated upper body of a humanoid model with 27 degree of freedom (DoF). The trained model was able to solve inverse kinematics for the end effectors with 90% accuracy while maintaining the balance in double support phase.

A Deep Reinforcement Learning Approach for Dynamically Stable Inverse Kinematics of Humanoid Robots

Abstract: Real time calculation of inverse kinematics (IK) with dynamically stable configuration is of high necessity in humanoid robots as they are highly susceptible to lose balance This paper proposes a methodology to generate joint-space trajectories of stable configurations for solving inverse kinematics using Deep Reinforcement Learning (RL) Our approach is based on the idea of exploring the entire configuration space of the robot and learning the best possible solutions using Deep Deterministic Policy Gradient (DDPG) The proposed strategy was evaluated on the highly articulated upper body of a humanoid model with 27 degree of freedom (DoF) The trained model was able to solve inverse kinematics for the end effectors with 90% accuracy while maintaining the balance in double support phase

Parameter Sharing Reinforcement Learning Architecture for Multi Agent Driving

Abstract: Multi-agent learning provides a potential solution for frameworks to learn and simulate traffic behaviors. This paper proposes a novel architecture to learn multiple driving behaviors in a traffic scenario. The proposed architecture can learn multiple behaviors independently as well as simultaneously. We take advantage of the homogeneity of agents and learn in a parameter sharing paradigm. To further speed up the training process asynchronous updates are employed into the architecture. While learning different behaviors simultaneously, the given framework was also able to learn cooperation between the agents, without any explicit communication. We applied this framework to learn two important behaviors in driving: 1) Lane-Keeping and 2) Over-Taking. Results indicate faster convergence and learning of a more generic behavior, that is scalable to any number of agents. When compared the results with existing approaches, our results indicate equal and even better performance in some cases.

Parameter Sharing Reinforcement Learning Architecture for Multi Agent Driving Behaviors.

Abstract: Multi-agent learning provides a potential framework for learning and simulating traffic behaviors. This paper proposes a novel architecture to learn multiple driving behaviors in a traffic scenario. The proposed architecture can learn multiple behaviors independently as well as simultaneously. We take advantage of the homogeneity of agents and learn in a parameter sharing paradigm. To further speed up the training process asynchronous updates are employed into the architecture. While learning different behaviors simultaneously, the given framework was also able to learn cooperation between the agents, without any explicit communication. We applied this framework to learn two important behaviors in driving: 1) Lane-Keeping and 2) Over-Taking. Results indicate faster convergence and learning of a more generic behavior, that is scalable to any number of agents. When compared the results with existing approaches, our results indicate equal and even better performance in some cases.

Motion Planning Framework for Autonomous Vehicles: A Time Scaled Collision Cone Interleaved Model Predictive Control Approach

Abstract: Planning frameworks for autonomous vehicles must be robust and computationally efficient for real time realization. At the same time, they should accommodate the unpredictable behavior of the other participants and produce safe trajectories. In this paper, we present a computationally efficient hierarchical planning framework for autonomous vehicles that can generate safe trajectories in complex driving scenarios, which are commonly encountered in urban traffic settings. The first level of the proposed framework constructs a Model Predictive Control(MPC)routine using an efficient difference of convex programmingapproach, that generates smooth and collision-free trajectories. The constraints on curvature and road boundaries are seamlessly integrated into this optimization routine. The second layer is mainly responsible to handle the unpredictable behaviors that are typically exhibited by the other participants of traffic. It is built along the lines of time scaled collision cone(TSCC)which optimize for the velocities along the trajectory to handle such disturbances. We additionally show that our framework maintains optimal balance between temporal and path deviations while executing safe trajectories. To demonstrate the efficacy of the presented framework we validated it in extensive simulations in different driving scenarios like over taking, lane merging and jaywalking among many dynamic and static obstacles.

Distributed Deep Learning-based Offloading for Mobile Edge Computing Networks

Abstract: This paper studies mobile edge computing (MEC) networks where multiple wireless devices (WDs) choose to offload their computation tasks to an edge server. To conserve energy and maintain quality of service for WDs, the optimization of joint offloading decision and bandwidth allocation is formulated as a mixed integer programming problem. However, the problem is computationally limited by the curse of dimensionality, which cannot be solved by general optimization tools in an effective and efficient way, especially for large-scale WDs. In this paper, we propose a distributed deep learning-based offloading (DDLO) algorithm for MEC networks, where multiple parallel DNNs are used to generate offloading decisions. We adopt a shared replay memory to store newly generated offloading decisions which are further to train and improve all DNNs. Extensive numerical results show that the proposed DDLO algorithm can generate near-optimal offloading decisions in less than one second.

Multi-Server Multi-User Multi-Task Computation Offloading for Mobile Edge Computing Networks

Abstract: This paper studies mobile edge computing (MEC) networks where multiple wireless devices (WDs) offload their computation tasks to multiple edge servers and one cloud server. Considering different real-time computation tasks at different WDs, every task is decided to be processed locally at its WD or to be offloaded to and processed at one of the edge servers or the cloud server. In this paper, we investigate low-complexity computation offloading policies to guarantee quality of service of the MEC network and to minimize WDs’ energy consumption. Specifically, both a linear programing relaxation-based (LR-based) algorithm and a distributed deep learning-based offloading (DDLO) algorithm are independently studied for MEC networks. We further propose a heterogeneous DDLO to achieve better convergence performance than DDLO. Extensive numerical results show that the DDLO algorithms guarantee better performance than the LR-based algorithm. Furthermore, the DDLO algorithm generates an offloading decision in less than 1 millisecond, which is several orders faster than the LR-based algorithm.

Survey of Deep Reinforcement Learning for Motion Planning of Autonomous Vehicles

Abstract: Academic research in the field of autonomous vehicles has reached high popularity in recent years related to several topics as sensor technologies, V2X communications, safety, security, decision making, control, and even legal and standardization rules. Besides classic control design approaches, Artificial Intelligence and Machine Learning methods are present in almost all of these fields. Another part of research focuses on different layers of Motion Planning, such as strategic decisions, trajectory planning, and control. A wide range of techniques in Machine Learning itself have been developed, and this article describes one of these fields, Deep Reinforcement Learning (DRL). The paper provides insight into the hierarchical motion planning problem and describes the basics of DRL. The main elements of designing such a system are the modeling of the environment, the modeling abstractions, the description of the state and the perception models, the appropriate rewarding, and the realization of the underlying neural network. The paper describes vehicle models, simulation possibilities and computational requirements. Strategic decisions on different layers and the observation models, e.g., continuous and discrete state representations, grid-based, and camera-based solutions are presented. The paper surveys the state-of-art solutions systematized by the different tasks and levels of autonomous driving, such as car-following, lane-keeping, trajectory following, merging, or driving in dense traffic. Finally, open questions and future challenges are discussed.

Optimization-Based Inverse Model of Soft Robots With Contact Handling

Abstract: This letter presents a physically based algorithm to interactively simulate and control the motion of soft robots interacting with their environment. We use the finite-element method to simulate the nonlinear deformation of the soft structure, its actuators, and surroundings and propose a control method relying on a quadratic optimization to find the inverse of the model. The novelty of this work is that the deformations due to contacts, including self-collisions, are taken into account in the optimization process. We propose a dedicated and efficient solver to handle the linear complementarity constraints introduced by the contacts. Thus, the method allows interactive transfer of the motion of soft robots from their task space to their actuator space while interacting with their surrounding. The method is generic and tested on several numerical examples and on a real cable-driven soft robot.