Machine Learning is the study of methods for programming computers to learn. Computers are applied to a wide range of tasks, and for most of these it is relatively easy for programmers to design and implement the necessary software. However, there are many tasks for which this is difficult or impossible. These can be divided into four general categories. First, there are problems for which there exist no human experts. For example, in modern automated manufacturing facilities, there is a need to predict machine failures before they occur by analyzing sensor readings. Because the machines are new, there are no human experts who can be interviewed by a programmer to provide the knowledge necessary to build a computer system. A machine learning system can study recorded data and subsequent machine failures and learn prediction rules. Second, there are problems where human experts exist, but where they are unable to explain their expertise. This is the case in many perceptual tasks, such as speech recognition, hand-writing recognition, and natural language understanding. Virtually all humans exhibit expert-level abilities on these tasks, but none of them can describe the detailed steps that they follow as they perform them. Fortunately, humans can provide machines with examples of the inputs and correct outputs for these tasks, so machine learning algorithms can learn to map the inputs to the outputs. Third, there are problems where phenomena are changing rapidly. In finance, for example, people would like to predict the future behavior of the stock market, of consumer purchases, or of exchange rates. These behaviors change frequently, so that even if a programmer could construct a good predictive computer program, it would need to be rewritten frequently. A learning program can relieve the programmer of this burden by constantly modifying and tuning a set of learned prediction rules. Fourth, there are applications that need to be customized for each computer user separately. Consider, for example, a program to filter unwanted electronic mail messages. Different users will need different filters. It is unreasonable to expect each user to program his or her own rules, and it is infeasible to provide every user with a software engineer to keep the rules up-to-date. A machine learning system can learn which mail messages the user rejects and maintain the filtering rules automatically. Machine learning addresses many of the same research questions as the fields of statistics, data mining, and psychology, but with differences of emphasis. Statistics focuses on understanding the phenomena that have generated the data, often with the goal of testing different hypotheses about those phenomena. Data mining seeks to find patterns in the data that are understandable by people. Psychological studies of human learning aspire to understand the mechanisms underlying the various learning behaviors exhibited by people (concept learning, skill acquisition, strategy change, etc.).

Machine learning

I have developed "tennis elbow" from lugging this book around the past four weeks, but it is worth the pain, the effort, and the aspirin. It is also worth the (relatively speaking) bargain price. Including appendixes, this book contains 894 pages of text. The entire panorama of the neural sciences is surveyed and examined, and it is comprehensive in its scope, from genomes to social behaviors. The editors explicitly state that the book is designed as "an introductory text for students of biology, behavior, and medicine," but it is hard to imagine any audience, interested in any fragment of neuroscience at any level of sophistication, that would not enjoy this book. The editors have done a masterful job of weaving together the biologic, the behavioral, and the clinical sciences into a single tapestry in which everyone from the molecular biologist to the practicing psychiatrist can find and appreciate his or

Principles of Neural Science

Time Series Classification (TSC) is an important and challenging problem in data mining. With the increase of time series data availability, hundreds of TSC algorithms have been proposed. Among these methods, only a few have considered Deep Neural Networks (DNNs) to perform this task. This is surprising as deep learning has seen very successful applications in the last years. DNNs have indeed revolutionized the field of computer vision especially with the advent of novel deeper architectures such as Residual and Convolutional Neural Networks. Apart from images, sequential data such as text and audio can also be processed with DNNs to reach state-of-the-art performance for document classification and speech recognition. In this article, we study the current state-of-the-art performance of deep learning algorithms for TSC by presenting an empirical study of the most recent DNN architectures for TSC. We give an overview of the most successful deep learning applications in various time series domains under a unified taxonomy of DNNs for TSC. We also provide an open source deep learning framework to the TSC community where we implemented each of the compared approaches and evaluated them on a univariate TSC benchmark (the UCR/UEA archive) and 12 multivariate time series datasets. By training 8730 deep learning models on 97 time series datasets, we propose the most exhaustive study of DNNs for TSC to date.

https://link.springer.com/content/pdf/10.1007/s10618-019-00619-1.pdf

Deep learning for time series classification: a review

In recent years, IP protection of FPGA hardware designs has become a requirement for many IP vendors. In [34], Simpson and Schaumont proposed a fundamentally different approach to IP protection on FPGAs based on the use of Physical Unclonable Functions (PUFs). Their work only assumes the existence of a PUF on the FPGAs without actually proposing a PUF construction. In this paper, we propose new protocols for the IP protection problem on FPGAs and provide the first construction of a PUF intrinsic to current FPGAs based on SRAM memory randomness present on current FPGAs. We analyze SRAM-based PUF statistical properties and investigate the trade offs that can be made when implementing a fuzzy extractor.

/pdf/fpga-intrinsic-pufs-and-their-use-for-ip-protection-4zcbpcd9em.pdf

FPGA Intrinsic PUFs and Their Use for IP Protection

Intermittently powered applications create a need for low-cost security and privacy in potentially hostile environments, supported by primitives including identification and random number generation. Our measurements show that power-up of SRAM produces a physical fingerprint. We propose a system of fingerprint extraction and random numbers in SRAM (FERNS) that harvests static identity and randomness from existing volatile CMOS memory without requiring any dedicated circuitry. The identity results from manufacture-time physically random device threshold voltage mismatch, and the random numbers result from runtime physically random noise. We use experimental data from high-performance SRAM chips and the embedded SRAM of the WISP UHF RFID tag to validate the principles behind FERNS. For the SRAM chip, we demonstrate that 8-byte fingerprints can uniquely identify circuits among a population of 5,120 instances and extrapolate that 24-byte fingerprints would uniquely identify all instances ever produced. Using a smaller population, we demonstrate similar identifying ability from the embedded SRAM. In addition to identification, we show that SRAM fingerprints capture noise, enabling true random number generation. We demonstrate that a 512-byte SRAM fingerprint contains sufficient entropy to generate 128-bit true random numbers and that the generated numbers pass the NIST tests for runs, approximate entropy, and block frequency.

Power-Up SRAM State as an Identifying Fingerprint and Source of True Random Numbers

Rapid development in miniature implantable electronics are expediting advances in neuroscience by allowing observation and control of neural activities. The first stage of an implantable biosignal recording system, a low-noise biopotential amplifier (BPA), is critical to the overall power and noise performance of the system. In order to integrate a large number of front-end amplifiers in multichannel implantable systems, the power consumption of each amplifier must be minimized. This paper introduces a closed-loop complementary-input amplifier, which has a bandwidth of 0.05 Hz to 10.5 kHz, an input-referred noise of 2.2 μ Vrms, and a power dissipation of 12 μW. As a point of comparison, a standard telescopic-cascode closed-loop amplifier with a 0.4 Hz to 8.5 kHz bandwidth, input-referred noise of 3.2 μ Vrms, and power dissipation of 12.5 μW is presented. Also for comparison, we show results from an open-loop complementary-input amplifier that exhibits an input-referred noise of 3.6 μ Vrms while consuming 800 nW of power. The two closed-loop amplifiers are fabricated in a 0.13 μ m CMOS process. The open-loop amplifier is fabricated in a 0.5 μm SOI-BiCMOS process. All three amplifiers operate with a 1 V supply.

Design of Ultra-Low Power Biopotential Amplifiers for Biosignal Acquisition Applications

A 128b 6.3pJ/b, 96%-stable chip-ID generation circuit using process variation is designed in a 0.13mum CMOS technology. The circuit consumes 162nW from a 1V supply at low readout frequencies and 6.34muW at 1 Mb/s. Two layout techniques are designed and fabricated to provide a performance comparison of power consumption and ID reliability

A 1.6pJ/bit 96% Stable Chip-ID Generating Circuit using Process Variations

A 128-bit, 1.6 pJ/bit, 96% stable chip ID generation circuit utilizing process variations is designed in a 0.13 mum CMOS process. The circuit consumes 162 nW from a 1 V supply at low readout frequencies and 1.6 muW at 1 Mb/s. Cross-coupled logic gates were employed to simultaneously generate, amplify, and digitize the random circuit offset to create a stable unique digital chip ID code. A thorough statistical analysis is presented in order to explore the ID circuit reliability and stability. Two ID generators with different layout techniques were designed and fabricated to provide a performance comparison of power consumption, ID stability, and ID statistical robustness.

A Digital 1.6 pJ/bit Chip Identification Circuit Using Process Variations

Direct processing of raw high-dimensional data such as images and video by machine learning systems is impractical both due to prohibitive power consumption and the “curse of dimensionality,” which makes learning tasks exponentially more difficult as dimension increases. Deep machine learning (DML) mimics the hierarchical presentation of information in the human brain to achieve robust automated feature extraction, reducing the dimension of such data. However, the computational complexity of DML systems limits large-scale implementations in standard digital computers. Custom analog or mixed-mode signal processors have been reported to yield much higher energy efficiency than DSP [1-4], presenting the means of overcoming these limitations. However, the use of volatile digital memory in [1-3] precludes their use in intermittently-powered devices, and the required interfacing and internal A/D/A conversions add power and area overhead. Nonvolatile storage is employed in [4], but the lack of learning capability requires task-specific programming before operation, and precludes online adaptation.

30.10 A 1TOPS/W analog deep machine-learning engine with floating-gate storage in 0.13μm CMOS

In this paper we present a pre-amplifier designed for neural recording applications. Extremely low power dissipation is achieved by operating in an open-loop configuration, restricting the circuit to a single current branch, and reusing current to improve noise performance. Our amplifier exhibits 3.5 microVrms of input-referred noise and has a digitally-controlled gain between 36dB and 44dB. The amplifier is AC-coupled, with a pass-band from 0.3 Hz to 4.7kHz. The circuit is implemented in a 0.5 microm SOI Bi-CMOS process and consumes 805nA from a 1.0V supply, corresponding to a noise efficiency factor (NEF) of 1.8, which is the lowest reported NEF to date.

Jeremy Holleman

Papers

Design of Ultra-Low Power Biopotential Amplifiers for Biosignal Acquisition Applications

A 1.6pJ/bit 96% Stable Chip-ID Generating Circuit using Process Variations

A Digital 1.6 pJ/bit Chip Identification Circuit Using Process Variations

30.10 A 1TOPS/W analog deep machine-learning engine with floating-gate storage in 0.13μm CMOS

A Sub-Microwatt Low-Noise Amplifier for Neural Recording