Having accurate, detailed, and up-to-date information about the location and behavior of animals in the wild would improve our ability to study and conserve ecosystems. We investigate the ability to automatically, accurately, and inexpensively collect such data, which could help catalyze the transformation of many fields of ecology, wildlife biology, zoology, conservation biology, and animal behavior into “big data” sciences. Motion-sensor “camera traps” enable collecting wildlife pictures inexpensively, unobtrusively, and frequently. However, extracting information from these pictures remains an expensive, time-consuming, manual task. We demonstrate that such information can be automatically extracted by deep learning, a cutting-edge type of artificial intelligence. We train deep convolutional neural networks to identify, count, and describe the behaviors of 48 species in the 3.2 million-image Snapshot Serengeti dataset. Our deep neural networks automatically identify animals with >93.8% accuracy, and we expect that number to improve rapidly in years to come. More importantly, if our system classifies only images it is confident about, our system can automate animal identification for 99.3% of the data while still performing at the same 96.6% accuracy as that of crowdsourced teams of human volunteers, saving >8.4 y (i.e., >17,000 h at 40 h/wk) of human labeling effort on this 3.2 million-image dataset. Those efficiency gains highlight the importance of using deep neural networks to automate data extraction from camera-trap images, reducing a roadblock for this widely used technology. Our results suggest that deep learning could enable the inexpensive, unobtrusive, high-volume, and even real-time collection of a wealth of information about vast numbers of animals in the wild.

https://www.pnas.org/content/pnas/115/25/E5716.full.pdf

Automatically identifying, counting, and describing wild animals in camera-trap images with deep learning.

Having accurate, detailed, and up-to-date information about the location and behavior of animals in the wild would revolutionize our ability to study and conserve ecosystems. We investigate the ability to automatically, accurately, and inexpensively collect such data, which could transform many fields of biology, ecology, and zoology into "big data" sciences. Motion sensor "camera traps" enable collecting wildlife pictures inexpensively, unobtrusively, and frequently. However, extracting information from these pictures remains an expensive, time-consuming, manual task. We demonstrate that such information can be automatically extracted by deep learning, a cutting-edge type of artificial intelligence. We train deep convolutional neural networks to identify, count, and describe the behaviors of 48 species in the 3.2-million-image Snapshot Serengeti dataset. Our deep neural networks automatically identify animals with over 93.8% accuracy, and we expect that number to improve rapidly in years to come. More importantly, if our system classifies only images it is confident about, our system can automate animal identification for 99.3% of the data while still performing at the same 96.6% accuracy as that of crowdsourced teams of human volunteers, saving more than 8.4 years (at 40 hours per week) of human labeling effort (i.e. over 17,000 hours) on this 3.2-million-image dataset. Those efficiency gains immediately highlight the importance of using deep neural networks to automate data extraction from camera-trap images. Our results suggest that this technology could enable the inexpensive, unobtrusive, high-volume, and even real-time collection of a wealth of information about vast numbers of animals in the wild.

Automatically identifying, counting, and describing wild animals in camera-trap images with deep learning

This book serves as an introduction to the flourishing field of super-resolution imaging. It is a compiled volume, with different authors for each of its 14 chapters. While not having a strong outline or textbook format, the chapters group into several sections.

Super-Resolution Imaging

Abstract With the advances in scientific foundations and technological implementations, optical metrology has become versatile problem-solving backbones in manufacturing, fundamental research, and engineering applications, such as quality control, nondestructive testing, experimental mechanics, and biomedicine. In recent years, deep learning, a subfield of machine learning, is emerging as a powerful tool to address problems by learning from data, largely driven by the availability of massive datasets, enhanced computational power, fast data storage, and novel training algorithms for the deep neural network. It is currently promoting increased interests and gaining extensive attention for its utilization in the field of optical metrology. Unlike the traditional “physics-based” approach, deep-learning-enabled optical metrology is a kind of “data-driven” approach, which has already provided numerous alternative solutions to many challenging problems in this field with better performances. In this review, we present an overview of the current status and the latest progress of deep-learning technologies in the field of optical metrology. We first briefly introduce both traditional image-processing algorithms in optical metrology and the basic concepts of deep learning, followed by a comprehensive review of its applications in various optical metrology tasks, such as fringe denoising, phase retrieval, phase unwrapping, subset correlation, and error compensation. The open challenges faced by the current deep-learning approach in optical metrology are then discussed. Finally, the directions for future research are outlined. 

/pdf/deep-learning-in-optical-metrology-a-review-2ggtgtrp.pdf

Deep learning in optical metrology: a review

Orbital angular momentum (OAM) beams allow for increased channel capacity in free-space optical communication. Conventionally, these OAM beams are multiplexed together at a transmitter and then propagated through the atmosphere to a receiver where, due to their orthogonality properties, they are demultiplexed. We propose a technique to demultiplex these OAM-carrying beams by capturing an image of the unique multiplexing intensity pattern and training a convolutional neural network (CNN) as a classifier. This CNN-based demultiplexing method allows for simplicity of operation as alignment is unnecessary, orthogonality constraints are loosened, and costly optical hardware is not required. We test our CNN-based technique against a traditional demultiplexing method, conjugate mode sorting, with various OAM mode sets and levels of simulated atmospheric turbulence in a laboratory setting. Furthermore, we examine our CNN-based technique with respect to added sensor noise, number of photon detections, number of pixels, unknown levels of turbulence, and training set size. Results show that the CNN-based demultiplexing method is able to demultiplex combinatorially multiplexed OAM modes from a fixed set with >99% accuracy for high levels of turbulence-well exceeding the conjugate mode demultiplexing method. We also show that this new method is robust to added sensor noise, number of photon detections, number of pixels, unknown levels of turbulence, and training set size.

Machine learning approach to OAM beam demultiplexing via convolutional neural networks.

As a means of increasing the channel capacity in free-space optical communication systems, two types of orbital angular momentum carrying beams, Bessel–Gauss and Laguerre–Gauss, are studied. In a series of numerical simulations, we show that Bessel–Gauss beams, pseudo-nondiffracting beams, outperform Laguerre–Gauss beams of various orders in channel efficiency and bit error rates.

Laguerre-Gauss and Bessel-Gauss beams propagation through turbulence: analysis of channel efficiency.

Free space optical communications utilizing orbital angular momentum beams have recently emerged as a new technique for communications with potential for increased channel capacity. Turbulence due to changes in the index of refraction emanating from temperature, humidity, and air flow patterns, however, add nonlinear effects to the received patterns, thus making the demultiplexing task more difficult. Deep learning techniques have been previously been applied to solve the demultiplexing problem as an image classification task. Here we make use of a newly developed theory suggesting a link between image turbulence and photon transport through the continuity equation to describe a method that utilizes a "shallow" learning method instead. The decoding technique is tested and compared against previous approaches using deep convolutional neural networks. Results show that the new method can obtain similar classification accuracies (bit error ratio) at a small fraction (1/90) of the computational cost, thus enabling higher bit rates.

De-multiplexing vortex modes in optical communications using transport-based pattern recognition

Multibeam large-angle beam steering is demonstrated in the visible spectral region by imprinting computer-generated holographic Fresnel zone plates on a liquid crystal spatial light modulator (SLM) configured as the first element of a telescope. The position and intensity of each beam are controlled independently. The laser beam is steered over a ±37° field of regard, with the power in the beam at 37° being greater than 50% of the on-axis power. The power delivered on axis for a single beam was 48% of the power incident on the SLM. The beam profile remained Gaussian over the full steering range, and the on-axis beam divergence is 2.1 mrad.

Efficient multibeam large-angle nonmechanical laser beam steering from computer-generated holograms rendered on a liquid crystal spatial light modulator.

The Shack–Hartmann wavefront sensor (SH-WFS) is known to produce incorrect measurements of the wavefront gradient in the presence of non-uniform illumination. Moreover, the most common least-squares phase reconstructors cannot accurately reconstruct the wavefront in the presence of branch points. We therefore developed the intensity/slopes network (ISNet), a deep convolutional-neural-network-based reconstructor that uses both the wavefront gradient information and the intensity of the SH-WFS’s subapertures to provide better wavefront reconstruction. We trained the network on simulated data with multiple levels of turbulence and compared the performance of our reconstructor to several other reconstruction techniques. ISNet produced the lowest wavefront error of the reconstructors we evaluated and operated at a speed suitable for real-time applications, enabling the use of the SH-WFS in stronger turbulence than was previously possible.

Abbie T. Watnik

Papers

Machine learning approach to OAM beam demultiplexing via convolutional neural networks.

Laguerre-Gauss and Bessel-Gauss beams propagation through turbulence: analysis of channel efficiency.

De-multiplexing vortex modes in optical communications using transport-based pattern recognition

Efficient multibeam large-angle nonmechanical laser beam steering from computer-generated holograms rendered on a liquid crystal spatial light modulator.

Intensity-enhanced deep network wavefront reconstruction in Shack-Hartmann sensors.