Text-to-image generation has traditionally focused on finding better modeling assumptions for training on a fixed dataset. These assumptions might involve complex architectures, auxiliary losses, or side information such as object part labels or segmentation masks supplied during training. We describe a simple approach for this task based on a transformer that autoregressively models the text and image tokens as a single stream of data. With sufficient data and scale, our approach is competitive with previous domain-specific models when evaluated in a zero-shot fashion.

/pdf/zero-shot-text-to-image-generation-3ra9fg0mfm.pdf

Zero-Shot Text-to-Image Generation

Table of Contents CHAPTER 1 INTRODUCTION 1.1 WHAT IS AN OPERATING SYSTEM? 1.2 HISTORY OF OPERATING SYSTEMS 1.3 OPERATING SYSTEM CONCEPTS 1.4 SYSTEM CALLS 1.5 OPERATING SYSTEM STRUCTURE 1.6 OUTLINE OF THE REST OF THIS BOOK 1.7 SUMMARY CHAPTER 2 PROCESSES 2.1 INTRODUCTION TO PROCESSES 2.2 INTERPROCESS COMMUNICATION 2.3 CLASSICAL IPC PROBLEMS 2.4 SCHEDULING 2.5 OVERVIEW OF PROCESSES IN MINIX 3 2.6 IMPLEMENTATION OF PROCESSES IN MINIX 3 2.7 THE SYSTEM TASK IN MINIX 3 2.8 THE CLOCK TASK IN MINIX 3 2.9 SUMMARY CHAPTER 3 INPUT/OUTPUT 3.1 PRINCIPLES OF I/O HARDWARE 3.2 PRINCIPLES OF I/O SOFTWARE 3.3 DEADLOCKS 3.4 OVERVIEW OF I/O IN MINIX 3 3.5 BLOCK DEVICES IN MINIX 3 3.6 RAM DISKS 3.7 DISKS 3.8 TERMINALS 3.9 SUMMARY CHAPTER 4 MEMORY MANAGEMENT 4.1 BASIC MEMORY MANAGEMENT 4.2 SWAPPING 4.3 VIRTUAL MEMORY 4.4 PAGE REPLACEMENT ALGORITHMS 4.5 DESIGN ISSUES FOR PAGING SYSTEMS 4.6 SEGMENTATION 4.7 OVERVIEW OF THE MINIX 3 PROCESS MANAGER 4.8 IMPLEMENTATION OF THE MINIX 3 PROCESS MANAGER 4.9 SUMMARY CHAPTER 5 FILE SYSTEMS 5.1 FILES 5.2 DIRECTORIES 5.3 FILE SYSTEM IMPLEMENTATION 5.4 SECURITY 5.5 PROTECTION MECHANISMS 5.6 OVERVIEW OF THE MINIX 3 FILE SYSTEM 5.7 IMPLEMENTATION OF THE MINIX 3 FILE SYSTEM 5.8 SUMMARY CHAPTER 6 READING LIST AND BIBLIOGRAPHY 6.1 SUGGESTIONS FOR FURTHER READING 6.2 ALPHABETICAL BIBLIOGRAPHY APPENDIX A - INSTALLING MINIX 3 APPENDIX B - MINIX 3 SOURCE CODE LISTING APPENDIX C - INDEX TO FILES INDEX

Operating Systems: Design and Implementation

Domain-specific hardware is becoming a promising topic in the backdrop of improvement slow down for general-purpose processors due to the foreseeable end of Moore’s Law. Machine learning, especially deep neural networks (DNNs), has become the most dazzling domain witnessing successful applications in a wide spectrum of artificial intelligence (AI) tasks. The incomparable accuracy of DNNs is achieved by paying the cost of hungry memory consumption and high computational complexity, which greatly impedes their deployment in embedded systems. Therefore, the DNN compression concept was naturally proposed and widely used for memory saving and compute acceleration. In the past few years, a tremendous number of compression techniques have sprung up to pursue a satisfactory tradeoff between processing efficiency and application accuracy. Recently, this wave has spread to the design of neural network accelerators for gaining extremely high performance. However, the amount of related works is incredibly huge and the reported approaches are quite divergent. This research chaos motivates us to provide a comprehensive survey on the recent advances toward the goal of efficient compression and execution of DNNs without significantly compromising accuracy, involving both the high-level algorithms and their applications in hardware design. In this article, we review the mainstream compression approaches such as compact model, tensor decomposition, data quantization, and network sparsification. We explain their compression principles, evaluation metrics, sensitivity analysis, and joint-way use. Then, we answer the question of how to leverage these methods in the design of neural network accelerators and present the state-of-the-art hardware architectures. In the end, we discuss several existing issues such as fair comparison, testing workloads, automatic compression, influence on security, and framework/hardware-level support, and give promising topics in this field and the possible challenges as well. This article attempts to enable readers to quickly build up a big picture of neural network compression and acceleration, clearly evaluate various methods, and confidently get started in the right way.

Model Compression and Hardware Acceleration for Neural Networks: A Comprehensive Survey

Interactive AI-powered services require low-latency evaluation of deep neural network (DNN) models—aka ""real-time AI"". The growing demand for computationally expensive, state-of-the-art DNNs, coupled with diminishing performance gains of general-purpose architectures, has fueled an explosion of specialized Neural Processing Units (NPUs). NPUs for interactive services should satisfy two requirements: (1) execution of DNN models with low latency, high throughput, and high efficiency, and (2) flexibility to accommodate evolving state-of-the-art models (e.g., RNNs, CNNs, MLPs) without costly silicon updates. This paper describes the NPU architecture for Project Brainwave, a production-scale system for real-time AI. The Brainwave NPU achieves more than an order of magnitude improvement in latency and throughput over state-of-the-art GPUs on large RNNs at a batch size of 1. The NPU attains this performance using a single-threaded SIMD ISA paired with a distributed microarchitecture capable of dispatching over 7M operations from a single instruction. The spatially distributed microarchitecture, scaled up to 96,000 multiply-accumulate units, is supported by hierarchical instruction decoders and schedulers coupled with thousands of independently addressable high-bandwidth on-chip memories, and can transparently exploit many levels of fine-grain SIMD parallelism. When targeting an FPGA, microarchitectural parameters such as native datapaths and numerical precision can be "synthesis specialized" to models at compile time, enabling atypically high FPGA performance competitive with hardened NPUs. When running on an Intel Stratix 10 280 FPGA, the Brainwave NPU achieves performance ranging from ten to over thirty-five teraflops, with no batching, on large, memory-intensive RNNs.

A configurable cloud-scale DNN processor for real-time AI

Many important applications must run continuously and without interruption, yet must be changed to fix bugs or upgrade functionality. No prior general-purpose methodology for dynamic updating achieves a practical balance between flexibility, robustness, low overhead, and ease of use.We present a new approach for C-like languages that provides type-safe dynamic updating of native code in an extremely flexible manner (code, data, and types may be updated, at programmer-determined times) and permits the use of automated tools to aid the programmer in the updating process. Our system is based on dynamic patches that both contain the updated code and the code needed to transition from the old version to the new. A novel aspect of our patches is that they consist of verifiable native code (e.g. Proof-Carrying Code [17] or Typed Assembly Language [16]), which is native code accompanied by annotations that allow on-line verification of the code's safety. We discuss how patches are generated mostly automatically, how they are applied using dynamic-linking technology, and how code is compiled to make it updateable.To concretely illustrate our system, we have implemented a dynamically-updateable web server, FlashEd. We discuss our experience building and maintaining FlashEd. Performance experiments show that for FlashEd, the overhead due to updating is typically less than 1%.

/pdf/dynamic-software-updating-1e7lw6g411.pdf

Dynamic software updating

Deep neural networks are commonly developed and trained in 32-bit floating point format. Significant gains in performance and energy efficiency could be realized by training and inference in numerical formats optimized for deep learning. Despite advances in limited precision inference in recent years, training of neural networks in low bit-width remains a challenging problem. Here we present the Flexpoint data format, aiming at a complete replacement of 32-bit floating point format training and inference, designed to support modern deep network topologies without modifications. Flexpoint tensors have a shared exponent that is dynamically adjusted to minimize overflows and maximize available dynamic range. We validate Flexpoint by training AlexNet, a deep residual network and a generative adversarial network, using a simulator implemented with the \emph{neon} deep learning framework. We demonstrate that 16-bit Flexpoint closely matches 32-bit floating point in training all three models, without any need for tuning of model hyperparameters. Our results suggest Flexpoint as a promising numerical format for future hardware for training and inference.

/pdf/flexpoint-an-adaptive-numerical-format-for-efficient-542kdhnyz2.pdf

Flexpoint: An Adaptive Numerical Format for Efficient Training of Deep Neural Networks

As partial evaluation gets more mature, it is now possible to use this program transformation technique to tackle realistic languages and real-size application programs. However, this evolution raises a number of critical issues that need to be addressed before the approach becomes truly practical.

/pdf/a-uniform-approach-for-compile-time-and-run-time-4pgdbxwm2w.pdf

A Uniform Approach for Compile-Time and Run-Time Specialization

Permission to make digital/hard copy of part or all of this work for personal or classroom use is granted without fee provided that the copies are not made or distributed for profit or commercial advantage, the copyright notice, the title of the publication, and its date appear, and notice is given that copying is by permission of the ACM, Inc. To copy otherwise, to republish, to post on servers, or to redistribute to lists, requires prior specific permission and/or a fee.

Tempo: specializing systems applications and beyond

Specializing programs with respect to run time values has been shown to drastically improve code performance on realistic programs ranging from operating systems to graphics. Recently, various approaches to specializing code at run time have been proposed. However, these approaches still suffer from shortcomings that limit their applicability: they are manual, too expensive, or require programs to be written in a dedicated language. We solve these problems by introducing new techniques to implement run time specialization. The key to our approach is the use of code templates. Templates are automatically generated from ordinary programs and are optimized before run time, allowing high quality code to be quickly generated at run time. Experimental results obtained on scientific and graphics code indicate that our approach is highly effective. Little run time overhead is introduced, since code generation primarily consists of copying instructions. Run time specialized programs run up to 10 times faster, and are nearly as fast as fully optimized programs (80% on average). The combination of low run time overhead and high code quality enables specialization to be amortized in as few as 3 runs. Although this approach is highly effective, its implementation is relatively simple since it exploits existing partial evaluation and compiler technologies.

Automatic, template-based run-time specialization: implementation and experimental study

A certifying compiler takes a source language program and produces object code, as well as a “certificate” that can be used to verify that the object code satisfies desirable properties, such as type safety and memory safety. Certifying compilation helps to increase both compiler robustness and program safety. Compiler robustness is improved since some compiler errors can be caught by checking the object code against the certificate immediately after compilation. Program safety is improved because the object code and certificate alone are sufficient to establish safety: even if the object code and certificate are produced on an unknown machine by an unknown compiler and sent over an untrusted network, safe execution is guaranteed as long as the code and certificate pass the verifier.

Existing work in certifying compilation has addressed statically generated code. In this paper, we extend this to code generated at run time. Our goal is to combine certifying compilation with run-time code generation to produce programs that are both fast and verifiably safe. To achieve this goal, we present two new languages with explicit run-time code generation constructs: Cyclone, a type safe dialect of C, and TAL/T, a type safe assembly language. We have designed and implemented a system that translates a safe C program into Cyclone, which is then compiled to TAL/T, and finally assembled into executable object code. This paper focuses on our overall approach and the front end of our systems details about TAL/T will appear in a subsequent paper.

Luke Hornof

Papers

Flexpoint: An Adaptive Numerical Format for Efficient Training of Deep Neural Networks

A Uniform Approach for Compile-Time and Run-Time Specialization

Tempo: specializing systems applications and beyond

Automatic, template-based run-time specialization: implementation and experimental study

Certifying Compilation and Run-Time Code Generation