Machine Learning is the study of methods for programming computers to learn. Computers are applied to a wide range of tasks, and for most of these it is relatively easy for programmers to design and implement the necessary software. However, there are many tasks for which this is difficult or impossible. These can be divided into four general categories. First, there are problems for which there exist no human experts. For example, in modern automated manufacturing facilities, there is a need to predict machine failures before they occur by analyzing sensor readings. Because the machines are new, there are no human experts who can be interviewed by a programmer to provide the knowledge necessary to build a computer system. A machine learning system can study recorded data and subsequent machine failures and learn prediction rules. Second, there are problems where human experts exist, but where they are unable to explain their expertise. This is the case in many perceptual tasks, such as speech recognition, hand-writing recognition, and natural language understanding. Virtually all humans exhibit expert-level abilities on these tasks, but none of them can describe the detailed steps that they follow as they perform them. Fortunately, humans can provide machines with examples of the inputs and correct outputs for these tasks, so machine learning algorithms can learn to map the inputs to the outputs. Third, there are problems where phenomena are changing rapidly. In finance, for example, people would like to predict the future behavior of the stock market, of consumer purchases, or of exchange rates. These behaviors change frequently, so that even if a programmer could construct a good predictive computer program, it would need to be rewritten frequently. A learning program can relieve the programmer of this burden by constantly modifying and tuning a set of learned prediction rules. Fourth, there are applications that need to be customized for each computer user separately. Consider, for example, a program to filter unwanted electronic mail messages. Different users will need different filters. It is unreasonable to expect each user to program his or her own rules, and it is infeasible to provide every user with a software engineer to keep the rules up-to-date. A machine learning system can learn which mail messages the user rejects and maintain the filtering rules automatically. Machine learning addresses many of the same research questions as the fields of statistics, data mining, and psychology, but with differences of emphasis. Statistics focuses on understanding the phenomena that have generated the data, often with the goal of testing different hypotheses about those phenomena. Data mining seeks to find patterns in the data that are understandable by people. Psychological studies of human learning aspire to understand the mechanisms underlying the various learning behaviors exhibited by people (concept learning, skill acquisition, strategy change, etc.).

Machine learning

Learning goal-directed behavior in environments with sparse feedback is a major challenge for reinforcement learning algorithms. The primary difficulty arises due to insufficient exploration, resulting in an agent being unable to learn robust value functions. Intrinsically motivated agents can explore new behavior for its own sake rather than to directly solve problems. Such intrinsic behaviors could eventually help the agent solve tasks posed by the environment. We present hierarchical-DQN (h-DQN), a framework to integrate hierarchical value functions, operating at different temporal scales, with intrinsically motivated deep reinforcement learning. A top-level value function learns a policy over intrinsic goals, and a lower-level function learns a policy over atomic actions to satisfy the given goals. h-DQN allows for flexible goal specifications, such as functions over entities and relations. This provides an efficient space for exploration in complicated environments. We demonstrate the strength of our approach on two problems with very sparse, delayed feedback: (1) a complex discrete stochastic decision process, and (2) the classic ATARI game `Montezuma's Revenge'.

Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivation

We are witnessing a growing interest in Networks on Chips (NoC) that is related to the evolution of integrated circuit technology and to the growing requirements in performance and portability of electronic systems. Current integrated circuits contain several processing cores, and even relatively simple systems, such as cellular telephones, behave as multiprocessors. Moreover, many electronic systems consist of heterogeneous components and they require efficient on-chip communication. In the last few years, multiprocessing platforms have been developed to address high performance computation, such as image rendering. Examples are Sony’s emotion engine [OKA] and IBM’s cell chip [PHAM] where on-chip communication efficiency is key to the overall system performance.

Networks on chip

VLSI Test Principles and Architectures: Design for Testability (Systems on Silicon)

Test access mechanisms (TAMs) and test wrappers are integral parts of a system-on-chip (SoC) test architecture. Prior research has concentrated on only one aspect of the TAM/wrapper design problem at a time, i.e., either optimizing the TAMs for a set of pre-designed wrappers, or optimizing the wrapper for a given TAM width. In this paper, we address a more general problem, that of carrying out TAM design and wrapper optimization in conjunction. We present an efficient algorithm to construct wrappers that reduce the testing time for cores. Our wrapper design algorithm improves on earlier approaches by also reducing the TAM width required to achieve these lower testing times. We present new mathematical models for TAM optimization that use the core testing time values calculated by our wrapper design algorithm. We further present a new enumerative method for TAM optimization that reduces execution time significantly when the number of TAMs being designed is small. Experimental results are presented for an academic SoC as well as an industrial SoC.

/pdf/test-wrapper-and-test-access-mechanism-co-optimization-for-26982t0yh5.pdf

Test wrapper and test access mechanism co-optimization for system-on-chip

A wrapper is a thin shell around the core, that provides the switching between functional, and core-internal and core-external test modes. Together with a test access mechanism (TAM), the core test wrapper forms the test access infrastructure to embedded reusable cores. Various company-internal as well as industry-wide standardized but scalable wrappers have been proposed. This paper deals with the design of such core test wrappers. It gives a general architecture for wrappers, and describes how a wrapper can be built up from a library of wrapper cells which are selected on basis of the terminal types of the core. We show that the ordering and partitioning of wrapper cells and core-internal scan chains over TAM chains determine the test time of the core. An heuristic approach for the NP-hard problem of partitioning the TAM chain items for minimal test time is presented and its usage is illustrated by means of an example. Finally we sketch how wrapper generation and verification can be automated.

Wrapper design for embedded core test

This paper deals with the design of test architectures for modular SOC testing. These architectures consist of wrappers and TAMs (test access mechanisms). For a given SOC, with specified parameters of modules and their tests, we design architectures which minimize the required ATE vector memory depth and test application time. In this paper, we formulate the problems of test architecture design both for modules with fixed- and flexible-length scan chains. Subsequently, we derive a formulation of an architecture-independent test time lower bound for SOCs and list the lower bound values for the 'ITC'02 SOC test benchmarks'. We present a novel architecture-independent heuristic algorithm that effectively optimizes the test architecture for a given SOC. The algorithm efficiently determines the number of TAMs and their widths, the assignment of modules to TAMs, and the wrapper design per module. We show how this algorithm can be used for optimizing both test bus and testrail architectures with serial and parallel test schedules. Experimental results for the 'ITC'02 SOC test benchmarks' show that, compared to previously published algorithms, we obtain comparable or better test times at negligible compute time.

Effective and efficient test architecture design for SOCs

For large, complex ICs, engineers need efficient techniques for debugging first silicon. The system presented here consists of an on-chip debug infrastructure and supporting debugger software,which interacts with the infrastructure to make the chip's features accessible through a serial interface.

Design for debug: catching design errors in digital chips

In this paper, we present a core-based scan architecture for silicon debug, which is currently being standardized within Philips. The reasons behind the core-based debug architecture, together with implementation details, are described. The choices that were made during its development are explained using the experiences gained from two large Philips system chips that each utilize core-based design and test, and scan-based silicon debug. The results of an area-cost evaluation of the presented architecture for these two large system chips are also presented.

Core-based scan architecture for silicon debug

This article deals with the design of on-chip architectures for testing large system chips (SOCs) for manufacturing defects in a modular fashion. These architectures consist of wrappers and test access mechanisms (TAMs). For an SOC with specified parameters of modules and their tests, we design an architecture that minimizes the required tester vector memory depth and test application time. In this article, we formulate the test architecture design problems for both modules with fixed- and flexible-length scan chains, assuming the relevant module parameters and a maximal SOC TAM width are given. Subsequently, we derive a formulation for an architecture-independent lower bound for the SOC test time. We analyze three types of TAM under-utilization that make the theoretical lower bound unachievable in most practical architecture instances. We present a novel architecture-independent heuristic algorithm that effectively optimizes the test architecture for a given SOC. The algorithm efficiently determines the number of TAMs and their widths, the assignment of modules to TAMs, and the wrapper design per module. We show how this algorithm can be used for optimizing both test bus and TestRail architectures with either serial or parallel test schedules. Experimental results for the ITC'02 SOC Test Benchmarks show that, compared to manual best-effort engineering approaches, we can save up to 75p in test times, while compared to previously published algorithms, we obtain comparable or better test times at negligible compute time.

Sandeep Kumar Goel

Papers

Wrapper design for embedded core test

Effective and efficient test architecture design for SOCs

Design for debug: catching design errors in digital chips

Core-based scan architecture for silicon debug

SOC test architecture design for efficient utilization of test bandwidth