From the Publisher: 
The accessible presentation of this book gives both a general view of the entire computer vision enterprise and also offers sufficient detail to be able to build useful applications. Users learn techniques that have proven to be useful by first-hand experience and a wide range of mathematical methods. A CD-ROM with every copy of the text contains source code for programming practice, color images, and illustrative movies. Comprehensive and up-to-date, this book includes essential topics that either reflect practical significance or are of theoretical importance. Topics are discussed in substantial and increasing depth. Application surveys describe numerous important application areas such as image based rendering and digital libraries. Many important algorithms broken down and illustrated in pseudo code. Appropriate for use by engineers as a comprehensive reference to the computer vision enterprise.

Computer vision : a modern approach = 计算机视觉 : 一种现代的方法

We propose a localized peer-to-peer (P2P) electricity trading model for locally buying and selling electricity among plug-in hybrid electric vehicles (PHEVs) in smart grids Unlike traditional schemes, which transport electricity over long distances and through complex electricity transportation meshes, our proposed model achieves demand response by providing incentives to discharging PHEVs to balance local electricity demand out of their own self-interests However, since transaction security and privacy protection issues present serious challenges, we explore a promising consortium blockchain technology to improve transaction security without reliance on a trusted third party A localized P 2P E lectricity T rading system with CO nsortium blockchai N (PETCON) method is proposed to illustrate detailed operations of localized P2P electricity trading Moreover, the electricity pricing and the amount of traded electricity among PHEVs are solved by an iterative double auction mechanism to maximize social welfare in this electricity trading Security analysis shows that our proposed PETCON improves transaction security and privacy protection Numerical results based on a real map of Texas indicate that the double auction mechanism can achieve social welfare maximization while protecting privacy of the PHEVs

Enabling Localized Peer-to-Peer Electricity Trading Among Plug-in Hybrid Electric Vehicles Using Consortium Blockchains

Building models that can be rapidly adapted to novel tasks using only a handful of annotated examples is an open challenge for multimodal machine learning research. We introduce Flamingo, a family of Visual Language Models (VLM) with this ability. We propose key architectural innovations to: (i) bridge powerful pretrained vision-only and language-only models, (ii) handle sequences of arbitrarily interleaved visual and textual data, and (iii) seamlessly ingest images or videos as inputs. Thanks to their flexibility, Flamingo models can be trained on large-scale multimodal web corpora containing arbitrarily interleaved text and images, which is key to endow them with in-context few-shot learning capabilities. We perform a thorough evaluation of our models, exploring and measuring their ability to rapidly adapt to a variety of image and video tasks. These include open-ended tasks such as visual question-answering, where the model is prompted with a question which it has to answer; captioning tasks, which evaluate the ability to describe a scene or an event; and close-ended tasks such as multiple-choice visual question-answering. For tasks lying anywhere on this spectrum, a single Flamingo model can achieve a new state of the art with few-shot learning, simply by prompting the model with task-specific examples. On numerous benchmarks, Flamingo outperforms models fine-tuned on thousands of times more task-specific data.

Flamingo: a Visual Language Model for Few-Shot Learning

The paper surveys evaluation methods of natural language generation (NLG) systems that have been developed in the last few years We group NLG evaluation methods into three categories: (1) human-centric evaluation metrics, (2) automatic metrics that require no training, and (3) machine-learned metrics For each category, we discuss the progress that has been made and the challenges still being faced, with a focus on the evaluation of recently proposed NLG tasks and neural NLG models We then present two examples for task-specific NLG evaluations for automatic text summarization and long text generation, and conclude the paper by proposing future research directions

Evaluation of Text Generation: A Survey

Transformers have proved effective in many NLP tasks. However, their training requires non-trivial efforts regarding carefully designing cutting-edge optimizers and learning rate schedulers (e.g., conventional SGD fails to train Transformers effectively). Our objective here is to understand __what complicates Transformer training__ from both empirical and theoretical perspectives. Our analysis reveals that unbalanced gradients are not the root cause of the instability of training. Instead, we identify an amplification effect that influences training substantially—for each layer in a multi-layer Transformer model, heavy dependency on its residual branch makes training unstable, since it amplifies small parameter perturbations (e.g., parameter updates) and results in significant disturbances in the model output. Yet we observe that a light dependency limits the model potential and leads to inferior trained models. Inspired by our analysis, we propose Admin (Adaptive model initialization) to stabilize the early stage’s training and unleash its full potential in the late stage. Extensive experiments show that Admin is more stable, converges faster, and leads to better performance

/pdf/understanding-the-difficulty-of-training-transformers-21t3jai92y.pdf

Understanding the Difficulty of Training Transformers

Deep networks often suffer from vanishing or exploding gradients due to inefficient signal propagation, leading to long training times or convergence difficulties. Various architecture designs, sophisticated residual-style networks, and initialization schemes have been shown to improve deep signal propagation. Recently, Pennington et al. used free probability theory to show that dynamical isometry plays an integral role in efficient deep learning. We show that the simplest architecture change of gating each residual connection using a single zero-initialized parameter satisfies initial dynamical isometry and outperforms more complex approaches. Although much simpler than its predecessors, this gate enables training thousands of fully connected layers with fast convergence and better test performance for ResNets trained on CIFAR-10. We apply this technique to language modeling and find that we can easily train 120-layer Transformers. When applied to 12 layer Transformers, it converges 56% faster on enwiki8.

ReZero is All You Need: Fast Convergence at Large Depth.

We propose a novel approach using representation learning for tackling the problem of extracting structured information from form-like document images. We propose an extraction system that uses knowledge of the types of the target fields to generate extraction candidates and a neural network architecture that learns a dense representation of each candidate based on neighboring words in the document. These learned representations are not only useful in solving the extraction task for unseen document templates from two different domains but are also interpretable, as we show using loss cases.

/pdf/representation-learning-for-information-extraction-from-form-3kq07jddvw.pdf

Representation Learning for Information Extraction from Form-like Documents

This paper proposes a double auction mechanism for energy trade between buying and selling agents. The framework is general enough, requiring neither the agents' preferences nor the energy pricing to be fixed values across the spatially distributed agents. A microgrid controller implements a distributed algorithm to maximize individual participating agents' utilities as well as the social welfare. This is accomplished by the controller in an iterative manner, such that the need to obtain private information pertaining to individual agents' preferences is obviated. Simulation results with a set of seven buyers and an equal number of sellers indicate that the proposed iterative double auction can establish social welfare maximization, requiring only a reasonable amount of computational overhead.

An efficient iterative double auction for energy trading in microgrids

Existing approaches to recipe generation are unable to create recipes for users with culinary preferences but incomplete knowledge of ingredients in specific dishes. We propose a new task of personalized recipe generation to help these users: expanding a name and incomplete ingredient details into complete natural-text instructions aligned with the user’s historical preferences. We attend on technique- and recipe-level representations of a user’s previously consumed recipes, fusing these ‘user-aware’ representations in an attention fusion layer to control recipe text generation. Experiments on a new dataset of 180K recipes and 700K interactions show our model’s ability to generate plausible and personalized recipes compared to non-personalized baselines.

/pdf/generating-personalized-recipes-from-historical-user-35wvgojcgp.pdf

Generating Personalized Recipes from Historical User Preferences.

Stories generated with neural language models have shown promise in grammatical and stylistic consistency. However, the generated stories are still lacking in common sense reasoning, e.g., they often contain sentences deprived of world knowledge. We propose a simple multi-task learning scheme to achieve quantitatively better common sense reasoning in language models by leveraging auxiliary training signals from datasets designed to provide common sense grounding. When combined with our two-stage fine-tuning pipeline, our method achieves improved common sense reasoning and state-of-the-art perplexity on the WritingPrompts (Fan et al., 2018) story generation dataset.

/pdf/improving-neural-story-generation-by-targeted-common-sense-ec84ut7q7x.pdf

Bodhisattwa Prasad Majumder

Papers

ReZero is All You Need: Fast Convergence at Large Depth.

Representation Learning for Information Extraction from Form-like Documents

An efficient iterative double auction for energy trading in microgrids

Generating Personalized Recipes from Historical User Preferences.

Improving Neural Story Generation by Targeted Common Sense Grounding.