https://www2.econ.iastate.edu/tesfatsi/DeepLearningInNeuralNetworksOverview.JSchmidhuber2015.pdf

Deep learning in neural networks

/pdf/semi-supervised-learning-literature-survey-4ccxy3rmmj.pdf

Semi-Supervised Learning Literature Survey

IEEE Transactions on Pattern Analysis and Machine Intelligence

DNA sequencing with chain terminating inhibitors

One of the most exciting tools that have entered the material science toolbox in recent years is machine learning. This collection of statistical methods has already proved to be capable of considerably speeding up both fundamental and applied research. At present, we are witnessing an explosion of works that develop and apply machine learning to solid-state systems. We provide a comprehensive overview and analysis of the most recent research in this topic. As a starting point, we introduce machine learning principles, algorithms, descriptors, and databases in materials science. We continue with the description of different machine learning approaches for the discovery of stable materials and the prediction of their crystal structure. Then we discuss research in numerous quantitative structure–property relationships and various approaches for the replacement of first-principle methods by machine learning. We review how active learning and surrogate-based optimization can be applied to improve the rational design process and related examples of applications. Two major questions are always the interpretability of and the physical understanding gained from machine learning models. We consider therefore the different facets of interpretability and their importance in materials science. Finally, we propose solutions and future research paths for various challenges in computational materials science.

/pdf/recent-advances-and-applications-of-machine-learning-in-55gs85xsws.pdf

Recent advances and applications of machine learning in solid-state materials science

This chapter presents an introduction to recurrent neural networks for readers familiar with artificial neural networks in general, and multi-layer perceptrons trained with gradient descent algorithms (back-propagation) in particular. A recurrent neural network (RNN) is an artificial neural network with internal loops. These internal loops induce recursive dynamics in the networks and thus introduce delayed activation dependencies across the processing elements (PEs) in the network.

Recurrent Neural Networks

This chapter contains sections titled: Introduction Exponential Error Decay Dilemma: Avoiding Aradient Decay Prevents Long-Term Latching Remedies Conclusion 
]]>

Gradient Flow in Recurrent Nets: The Difficulty of Learning LongTerm Dependencies

From the Book:
The idea for this book emerged as the two of us were organizing the 1996 Neural Information Processing Systems (NIPS) workshop on recurrent networks. Many of the telephone calls between us turned to students and research. We both bemoaned the fact that we were handing large stacks of journal papers to our potential graduate students, in some cases, scaring them off to other advisors. Since we both were dealing with terminal masters students whom we would be lucky if they stuck around for a whole year, we needed introductory material that was self-contained, accessible, and devoid of distractionsa book that a first-year graduate student, or upper-division undergrad, could read and become productive in a short period of time. Unfortunately, there were no such books available at that time. Hertz, Krogh, and Palmers textbook serves well in getting students up to speed on back propagation, Hopfield networks, and Boltzmann machines. However, it did little justice to our research area, dynam-ical recurrent networks. Sometime before the workshop, we decided to fill this void. 
From the conception of this project, we both agreed that this book was not going to be yet another edited collection of papers. We wanted a coherent presentation of the field. One difficulty for students has been the arbitrary nomenclature and mathematical notation used to describe recurrent networks. Though not insurmountable, it does act as a speed bump, reducing the valuable time that they could be spending elsewhere. 
Like any other young field, we have several researchers with competing viewpoints. One problem with edited volumes is that the author of a chapter will have a biased view of the world. We decided to reduce this bias by assigning multiple authors to chapters. The goal of the assignments was to ensure a balanced presentation of the material, one that would be coming directly from the experts in that field. We also encouraged the authors to tie their chapters with other chapters in the book. Though difficult at times, it transforms the text from a collection of papers into a coherent book. 
The book is organized into five sections. The first section presents the range of dynamical recurrent network (DRN) architectures that will be used in the book. With these architectures in hand, we turn to examine their capabilities as computational devices. The third section presents several training algorithms for solving the network loading problem. Next, we look back and temper our enthusiasm by acknowledging the limitations of DRNs. The final section deals with applications of recurrent networks. 
The material in this book assumes a basic understanding of neural networks. The reader, we assume, has read an introductory textbook covering multilayered perceptrons and back-propagation. As such, this book would be ideal for a second course in neural networks.

Field Guide to Dynamical Recurrent Networks

New directions in fuzzy automata

The ability of a researcher to re-identify (re-ID) an individual animal upon re-encounter is fundamental for addressing a broad range of questions in the study of ecosystem function, community and population dynamics, and behavioural ecology. In this review, we describe a brief history of camera traps for re-ID, present a collection of computer vision feature engineering methodologies previously used for animal re-ID, provide an introduction to the underlying mechanisms of deep learning relevant to animal re-ID, highlight the success of deep learning methods for human re-ID, describe the few ecological studies currently utilizing deep learning for camera trap analyses, and our predictions for near future methodologies based on the rapid development of deep learning methods. By utilizing novel deep learning methods for object detection and similarity comparisons, ecologists can extract animals from an image/video data and train deep learning classifiers to re-ID animal individuals beyond the capabilities of a human observer. This methodology will allow ecologists with camera/video trap data to re-identify individuals that exit and re-enter the camera frame. Our expectation is that this is just the beginning of a major trend that could stand to revolutionize the analysis of camera trap data and, ultimately, our approach to animal ecology.

https://besjournals.onlinelibrary.wiley.com/doi/pdf/10.1111/2041-210X.13133

Stefan C. Kremer

Papers

Recurrent Neural Networks

Gradient Flow in Recurrent Nets: The Difficulty of Learning LongTerm Dependencies

Field Guide to Dynamical Recurrent Networks

New directions in fuzzy automata

Past, Present, and Future Approaches Using Computer Vision for Animal Re-Identification from Camera Trap Data