Deeper neural networks are more difficult to train. We present a residual learning framework to ease the training of networks that are substantially deeper than those used previously. We explicitly reformulate the layers as learning residual functions with reference to the layer inputs, instead of learning unreferenced functions. We provide comprehensive empirical evidence showing that these residual networks are easier to optimize, and can gain accuracy from considerably increased depth. On the ImageNet dataset we evaluate residual nets with a depth of up to 152 layers—8× deeper than VGG nets [40] but still having lower complexity. An ensemble of these residual nets achieves 3.57% error on the ImageNet test set. This result won the 1st place on the ILSVRC 2015 classification task. We also present analysis on CIFAR-10 with 100 and 1000 layers. The depth of representations is of central importance for many visual recognition tasks. Solely due to our extremely deep representations, we obtain a 28% relative improvement on the COCO object detection dataset. Deep residual nets are foundations of our submissions to ILSVRC & COCO 2015 competitions1, where we also won the 1st places on the tasks of ImageNet detection, ImageNet localization, COCO detection, and COCO segmentation.

/pdf/deep-residual-learning-for-image-recognition-b75jg61qrq.pdf

Deep Residual Learning for Image Recognition

In this work we investigate the effect of the convolutional network depth on its accuracy in the large-scale image recognition setting. Our main contribution is a thorough evaluation of networks of increasing depth using an architecture with very small (3x3) convolution filters, which shows that a significant improvement on the prior-art configurations can be achieved by pushing the depth to 16-19 weight layers. These findings were the basis of our ImageNet Challenge 2014 submission, where our team secured the first and the second places in the localisation and classification tracks respectively. We also show that our representations generalise well to other datasets, where they achieve state-of-the-art results. We have made our two best-performing ConvNet models publicly available to facilitate further research on the use of deep visual representations in computer vision.

Very Deep Convolutional Networks for Large-Scale Image Recognition

Deeper neural networks are more difficult to train. We present a residual learning framework to ease the training of networks that are substantially deeper than those used previously. We explicitly reformulate the layers as learning residual functions with reference to the layer inputs, instead of learning unreferenced functions. We provide comprehensive empirical evidence showing that these residual networks are easier to optimize, and can gain accuracy from considerably increased depth. On the ImageNet dataset we evaluate residual nets with a depth of up to 152 layers---8x deeper than VGG nets but still having lower complexity. An ensemble of these residual nets achieves 3.57% error on the ImageNet test set. This result won the 1st place on the ILSVRC 2015 classification task. We also present analysis on CIFAR-10 with 100 and 1000 layers. 
The depth of representations is of central importance for many visual recognition tasks. Solely due to our extremely deep representations, we obtain a 28% relative improvement on the COCO object detection dataset. Deep residual nets are foundations of our submissions to ILSVRC & COCO 2015 competitions, where we also won the 1st places on the tasks of ImageNet detection, ImageNet localization, COCO detection, and COCO segmentation.

Deep learning is a form of machine learning that enables computers to learn from experience and understand the world in terms of a hierarchy of concepts. Because the computer gathers knowledge from experience, there is no need for a human computer operator to formally specify all the knowledge that the computer needs. The hierarchy of concepts allows the computer to learn complicated concepts by building them out of simpler ones; a graph of these hierarchies would be many layers deep. This book introduces a broad range of topics in deep learning. The text offers mathematical and conceptual background, covering relevant concepts in linear algebra, probability theory and information theory, numerical computation, and machine learning. It describes deep learning techniques used by practitioners in industry, including deep feedforward networks, regularization, optimization algorithms, convolutional networks, sequence modeling, and practical methodology; and it surveys such applications as natural language processing, speech recognition, computer vision, online recommendation systems, bioinformatics, and videogames. Finally, the book offers research perspectives, covering such theoretical topics as linear factor models, autoencoders, representation learning, structured probabilistic models, Monte Carlo methods, the partition function, approximate inference, and deep generative models. Deep Learning can be used by undergraduate or graduate students planning careers in either industry or research, and by software engineers who want to begin using deep learning in their products or platforms. A website offers supplementary material for both readers and instructors.

Deep Learning

BACKGROUND The human hand has extraordinary dexterity with more than 20 degrees of freedom (DOF) actuated by lightweight and efficient biological actuators (i.e., muscles). The average weight of human hand is only 400g [1]. Over the last few decades, research and commercialization effort has been dedicated to the development of novel robotic hands for humanoid or prosthetic application towards dexterous and biomimetic design [2]. However, due to the limitations of existing electric motors in terms of torque density and energy efficiency, the design of humanoid hands has to compromise between dexterity and weight. For example, commercial prosthetic terminal devices i-Limb [3] and Bebionic [4] prioritize the lightweight need (450g) and use 5-DOF motors to under-actuated 11 joints, which is only able to realize a few basic grasp postures. On the other hand, some humanoid robot hand devices like DLR-HIT I & II hands [5] prioritize the dexterity need (15 DOF) but weigh more than four times than their biological counterpart (2200g and 1500g, respectively). The contribution of this work is the development of a custom high-torque density electric actuator and its application

/pdf/a-modular-approach-for-lightweight-humanoid-hand-design-42b5ozf8fs.pdf

A Modular Approach for Lightweight Humanoid Hand Design Using High Torque Density Electric Actuators

Rapid start-up of PN/A process and efficient enrichment of functional bacteria: a novel aerobic-biofilm/anaerobic-granular nitrogen removal system (OANRS).

Walking disorders are common in post-stroke. Body weight support (BWS) systems have been proposed and proven to enhance gait training systems for recovering in individuals with hemiplegia. However, the fixed weight support and walking speed increase the risk of falling and decrease the active participation of the subjects. This paper proposes a strategy to enhance the efficiency of BWS treadmill training. It consists in regulating the height of the BWS system to track the height of the subject’s center of mass (CoM), whereby the CoM is estimated through a long-short term memory (LSTM) network and a locomotion recognition system. The LSTM network takes the walking speed, body-height to leg-length ratio, hip and knee joint angles of the hemiplegic subjects’ non-paretic side from the locomotion recognition system as input signals and outputs the CoM height to a BWS treadmill training robot. Besides, the hip and knee joints’ ranges of motion are increased by 34.54% and 25.64% under the CoM height regulation compared to the constant weight support, respectively. With the CoM height regulation strategy, the stance phase duration of the paretic side is significantly increased by 14.6% of the gait cycle, and the symmetry of the gait is also promoted. The CoM height kinematics by adjustment strategy is in good agreement with the mean values of the 14 non-disabled subjects, which demonstrated that the adjustment strategy improves the stability of CoM height during the training.

A Center of Mass Estimation and Control Strategy for Body-Weight-Support Treadmill Training

We propose DeepMetaHandles, a 3D conditional generative model based on mesh deformation. Given a collection of 3D meshes of a category and their deformation handles (control points), our method learns a set of meta-handles for each shape, which are represented as combinations of the given handles. The disentangled meta-handles factorize all the plausible deformations of the shape, while each of them corresponds to an intuitive deformation. A new deformation can then be generated by sampling the coefficients of the meta-handles in a specific range. We employ biharmonic coordinates as the deformation function, which can smoothly propagate the control points' translations to the entire mesh. To avoid learning zero deformation as meta-handles, we incorporate a target-fitting module which deforms the input mesh to match a random target. To enhance deformations' plausibility, we employ a soft-rasterizer-based discriminator that projects the meshes to a 2D space. Our experiments demonstrate the superiority of the generated deformations as well as the interpretability and consistency of the learned meta-handles.

DeepMetaHandles: Learning Deformation Meta-Handles of 3D Meshes with Biharmonic Coordinates

Supramolecular polymers (SPs) formed by self-assembly of peptide-based molecular units assume a variety of interesting one-dimensional (1D) morphologies. While the morphological complexity and phase behavior of self-assembling peptide conjugates bear some resemblance to those of low-molecular-weight and macromolecular surfactants, Y-junctions, or three-way connected constructs, a topological defect observed in traditional surfactants has not been identified, likely due to the intolerance of defective packing by the strong, associative interactions afforded by the peptide segments. Here we report our discovery of branched SPs with Y-junctions and occasionally enlarged spherical end-caps formed by micellization of a ferrocene-based peptide amphiphile in water. Our results suggest that the incorporation of two ferrocenes into the amphiphile design is key to ensure the formation of branched SPs. We hypothesize that the complex interplay of internal interactions limits the effective propagation of hydrogen bonding within the assemblies and, consequently, creates fragmented β-sheets that are more tolerant for supramolecular branching. Given the redox sensitivity of the ferrocene units, sequential addition of reductants and oxidants to the solution led the assemblies to reversibly transform between branched SPs and spherical aggregates.

Hao Su

Papers

A Modular Approach for Lightweight Humanoid Hand Design Using High Torque Density Electric Actuators

Rapid start-up of PN/A process and efficient enrichment of functional bacteria: a novel aerobic-biofilm/anaerobic-granular nitrogen removal system (OANRS).

A Center of Mass Estimation and Control Strategy for Body-Weight-Support Treadmill Training

DeepMetaHandles: Learning Deformation Meta-Handles of 3D Meshes with Biharmonic Coordinates

Discovery of Y-Shaped Supramolecular Polymers in a Self-Assembling Peptide Amphiphile System.