Multilayer neural networks trained with the back-propagation algorithm constitute the best example of a successful gradient based learning technique. Given an appropriate network architecture, gradient-based learning algorithms can be used to synthesize a complex decision surface that can classify high-dimensional patterns, such as handwritten characters, with minimal preprocessing. This paper reviews various methods applied to handwritten character recognition and compares them on a standard handwritten digit recognition task. Convolutional neural networks, which are specifically designed to deal with the variability of 2D shapes, are shown to outperform all other techniques. Real-life document recognition systems are composed of multiple modules including field extraction, segmentation recognition, and language modeling. A new learning paradigm, called graph transformer networks (GTN), allows such multimodule systems to be trained globally using gradient-based methods so as to minimize an overall performance measure. Two systems for online handwriting recognition are described. Experiments demonstrate the advantage of global training, and the flexibility of graph transformer networks. A graph transformer network for reading a bank cheque is also described. It uses convolutional neural network character recognizers combined with global training techniques to provide record accuracy on business and personal cheques. It is deployed commercially and reads several million cheques per day.

Gradient-based learning applied to document recognition

A new heuristic approach for minimizing possibly nonlinear and non-differentiable continuous space functions is presented. By means of an extensive testbed it is demonstrated that the new method converges faster and with more certainty than many other acclaimed global optimization methods. The new method requires few control variables, is robust, easy to use, and lends itself very well to parallel computation.

/pdf/differential-evolution-a-simple-and-efficient-heuristic-for-4vrb5o5ldc.pdf

Differential Evolution – A Simple and Efficient Heuristic for Global Optimization over Continuous Spaces

Deposits of clastic carbonate-dominated (calciclastic) sedimentary slope systems in the rock record have been identified mostly as linearly-consistent carbonate apron deposits, even though most ancient clastic carbonate slope deposits fit the submarine fan systems better. Calciclastic submarine fans are consequently rarely described and are poorly understood. Subsequently, very little is known especially in mud-dominated calciclastic submarine fan systems. Presented in this study are a sedimentological core and petrographic characterisation of samples from eleven boreholes from the Lower Carboniferous of Bowland Basin (Northwest England) that reveals a >250 m thick calciturbidite complex deposited in a calciclastic submarine fan setting. Seven facies are recognised from core and thin section characterisation and are grouped into three carbonate turbidite sequences. They include: 1) Calciturbidites, comprising mostly of highto low-density, wavy-laminated bioclast-rich facies; 2) low-density densite mudstones which are characterised by planar laminated and unlaminated muddominated facies; and 3) Calcidebrites which are muddy or hyper-concentrated debrisflow deposits occurring as poorly-sorted, chaotic, mud-supported floatstones. These

Phd by thesis

In this paper we present a new data structure for representing Boolean functions and an associated set of manipulation algorithms. Functions are represented by directed, acyclic graphs in a manner similar to the representations introduced by Lee [1] and Akers [2], but with further restrictions on the ordering of decision variables in the graph. Although a function requires, in the worst case, a graph of size exponential in the number of arguments, many of the functions encountered in typical applications have a more reasonable representation. Our algorithms have time complexity proportional to the sizes of the graphs being operated on, and hence are quite efficient as long as the graphs do not grow too large. We present experimental results from applying these algorithms to problems in logic design verification that demonstrate the practicality of our approach.

/pdf/graph-based-algorithms-for-boolean-function-manipulation-2u3osq5jc0.pdf

Graph-Based Algorithms for Boolean Function Manipulation

Our growing dependence on increasingly complex computer and software systems necessitates the development of formalisms, techniques, and tools for assessing functional properties of these systems. One such technique that has emerged in the last twenty years is model checking, which systematically (and automatically) checks whether a model of a given system satisfies a desired property such as deadlock freedom, invariants, and request-response properties. This automated technique for verification and debugging has developed into a mature and widely used approach with many applications. Principles of Model Checking offers a comprehensive introduction to model checking that is not only a text suitable for classroom use but also a valuable reference for researchers and practitioners in the field. The book begins with the basic principles for modeling concurrent and communicating systems, introduces different classes of properties (including safety and liveness), presents the notion of fairness, and provides automata-based algorithms for these properties. It introduces the temporal logics LTL and CTL, compares them, and covers algorithms for verifying these logics, discussing real-time systems as well as systems subject to random phenomena. Separate chapters treat such efficiency-improving techniques as abstraction and symbolic manipulation. The book includes an extensive set of examples (most of which run through several chapters) and a complete set of basic results accompanied by detailed proofs. Each chapter concludes with a summary, bibliographic notes, and an extensive list of exercises of both practical and theoretical nature.

/pdf/principles-of-model-checking-249iimickz.pdf

Principles of Model Checking

Certain formal verification tasks require reasoning about Boolean combinations of non-linear arithmetic constraints over the real numbers. In this paper, we present a new technique for satisfiability solving of Boolean combinations of non-linear constraints that are convex. Our approach applies fundamental results from the theory of convex programming to realize a satisfiability modulo theory (SMT) solver. Our solver, CalCS, uses a lazy combination of SAT and a theory solver. A key step in our algorithm is the use of complementary slackness and duality theory to generate succinct infeasibility proofs that support conflict-driven learning. Moreover, whenever non-convex constraints are produced from Boolean reasoning, we provide a procedure that generates conservative approximations of the original set of constraints by using geometric properties of convex sets and supporting hyperplanes. We validate CalCS on several benchmarks including formulas generated from bounded model checking of hybrid automata and static analysis of floating-point software.

CalCS: SMT solving for non-linear convex constraints

A platform is an abstraction layer that hides the details of several possible implementation refinements of the underlying layers. It is a library of elements characterized by models that represent their functionalities and offer an estimation of (physical) quantities that are of importance for the designer. The library contains interconnects and rules that define what are the legal composition of the elements. A legal composition of elements and interconnects is called a platform instance. Platform-based design is a meet-in-the-middle process, where successive refinements of specifications meet with abstractions of potential implementations that are captured in the models of the elements of the platform. It is this characteristic that makes platform-based design a novel design method. We argue for the importance of structuring precisely the platform layers and we discuss how to define formally the transitions from one platform to the next. In particular, we emphasize the interplay of top-down constraint propagation and bottomup performance estimation while illustrating the notion of articulation point in the design process. In this context, we study the key role played by the API platform together with the micro-architecture platform in embedded system design. Also, we report on three applications of platform-based design: at the system-level, we discuss network platforms for communication protocol design and fault-tolerant platforms for the design of safety-critical applications; at the implementation level, we present analog platforms for mixed-signal integrated circuit design.

/pdf/platform-based-design-for-embedded-systems-4xrd9gkadk.pdf

Platform-Based Design for Embedded Systems.

The theory of latency insensitive design is presented as the foundation of a new correct by construction methodology to design very large digital systems by assembling blocks of Intellectual Properties. Latency insensitive designs are synchronous distributed systems and are realized by assembling functional modules exchanging data on communication channels according to an appropriate protocol. The goal of the protocol is to guarantee that latency insensitive designs composed of functionally correct modules, behave correctly independently of the wire delays. A latency-insensitive protocol is presented that makes use of relay stations buffering signals propagating along long wires. To guarantee correct behavior of the overall system, modules must satisfy weak conditions. The weakness of the conditions makes our method widely applicable.

/pdf/latency-insensitive-protocols-4j9afwvqpg.pdf

Latency Insensitive Protocols

This dissertation addresses three separate, but related problems concerning the formal analysis of synchronous circuits and their associated finite state machines. The first problem is the logical analysis of synchronous circuits containing combinational cycles. The presence of such cycles can cause unstable behavior at the outputs of a circuit, but this is not necessarily always the case. This work determines when cycles are harmless, and when they are not. In particular, three classes of circuits are defined that tradeoff time to decide the class, with the permissiveness of the class. For each class, the complexity of the corresponding decision problem is proven and a procedure to decide the class is given. In addition, if a circuit is determined to be within a given class, then a new circuit can be generated with the same input/output behavior, but without combinational cycles. This is an important utility, as many CAD tools do not accept circuits with combinational cycles.
The second problem that is addressed is the CTL model checking of interacting FSMs. A state equivalence is presented that is defined with respect to a given CTL formula. Since it does not attempt to preserve all CTL formulas, like bisimulation does, we can expect to compute coarser equivalences. This equivalence is used to manage the size of the transition relations encountered when model checking a system of interacting FSMs. Specifically, the equivalence is used to reduce the size of each component FSM, so that their product will be smaller. We show how to apply the method, whether an explicit representation is used for the FSMs, or BDDs are used. Also, we show that in some cases this approach can detect if a formula passes or fails, without composing all the component machines. The method is exact and completely automatic, and handles full CTL.
These two problems are PSPACE-hard (in the number of flip-flops) to decide; approximate methods may be useful to find a solution in affordable CPU time. To demonstrate the use of approximate methods in logical analysis, we address the the state reachability problem in FSMs, which is the problem of determining if one set of states can reach another. State reachability has broad applications in formal verification, synthesis, and testing of synchronous circuits. This work attacks this problem by making a series of under- and over-approximations to the state transition graph, using the over-approximations to guide the search in the under-approximations for a potential path from one state set to the other. Central to this method is an algorithm to approximate a Boolean function by another function having a smaller BDD.

Formal analysis of synchronous circuits

Circuit simulation programs have proven to be most important computer-aided design tools for the analysis of the electrical performance of integrated circuits. One of the most common analyses performed by circuit simulators and the most expensive in terms of computer time is nonlinear time-domain transient analysis. Conventional circuit simulators were designed initially for the cost-effective analysis of circuits containing a few hundred transistors or less. Because of the need to verify the performance of larger circuits, many users have successfully simulated circuits containing thousands of transistors despite the cost.Recently, a new class of algorithms has been applied to the electrical IC simulation problem. New simulators using these methods provide accurate waveform information with up to two orders of magnitude speed improvement for large circuits. These programs use relaxation methods for the solution of the set of ordinary differential equations, which describe the circuit under analysis, rather than the direct sparse-matrix methods on which standard circuit simulators are based.In this paper, the techniques used in relaxation-based electrical simulation are presented in a rigorous and unified framework, and the numerical properties of the various methods are explored. Both the advantages and the limitations of these techniques for the analysis of large IC's are described.

Alberto Sangiovanni-Vincentelli

Papers

CalCS: SMT solving for non-linear convex constraints

Platform-Based Design for Embedded Systems.

Latency Insensitive Protocols

Formal analysis of synchronous circuits

Relaxation-Based Electrical Simulation