We investigate conditional adversarial networks as a general-purpose solution to image-to-image translation problems. These networks not only learn the mapping from input image to output image, but also learn a loss function to train this mapping. This makes it possible to apply the same generic approach to problems that traditionally would require very different loss formulations. We demonstrate that this approach is effective at synthesizing photos from label maps, reconstructing objects from edge maps, and colorizing images, among other tasks. Moreover, since the release of the pix2pix software associated with this paper, hundreds of twitter users have posted their own artistic experiments using our system. As a community, we no longer hand-engineer our mapping functions, and this work suggests we can achieve reasonable results without handengineering our loss functions either.

/pdf/image-to-image-translation-with-conditional-adversarial-5058t24et2.pdf

Image-to-Image Translation with Conditional Adversarial Networks

Image-to-image translation is a class of vision and graphics problems where the goal is to learn the mapping between an input image and an output image using a training set of aligned image pairs. However, for many tasks, paired training data will not be available. We present an approach for learning to translate an image from a source domain X to a target domain Y in the absence of paired examples. Our goal is to learn a mapping G : X → Y such that the distribution of images from G(X) is indistinguishable from the distribution Y using an adversarial loss. Because this mapping is highly under-constrained, we couple it with an inverse mapping F : Y → X and introduce a cycle consistency loss to push F(G(X)) ≈ X (and vice versa). Qualitative results are presented on several tasks where paired training data does not exist, including collection style transfer, object transfiguration, season transfer, photo enhancement, etc. Quantitative comparisons against several prior methods demonstrate the superiority of our approach.

Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks

We investigate conditional adversarial networks as a general-purpose solution to image-to-image translation problems. These networks not only learn the mapping from input image to output image, but also learn a loss function to train this mapping. This makes it possible to apply the same generic approach to problems that traditionally would require very different loss formulations. We demonstrate that this approach is effective at synthesizing photos from label maps, reconstructing objects from edge maps, and colorizing images, among other tasks. Indeed, since the release of the pix2pix software associated with this paper, a large number of internet users (many of them artists) have posted their own experiments with our system, further demonstrating its wide applicability and ease of adoption without the need for parameter tweaking. As a community, we no longer hand-engineer our mapping functions, and this work suggests we can achieve reasonable results without hand-engineering our loss functions either.

Presents a review of 200 references in content-based image retrieval. The paper starts with discussing the working conditions of content-based retrieval: patterns of use, types of pictures, the role of semantics, and the sensory gap. Subsequent sections discuss computational steps for image retrieval systems. Step one of the review is image processing for retrieval sorted by color, texture, and local geometry. Features for retrieval are discussed next, sorted by: accumulative and global features, salient points, object and shape features, signs, and structural combinations thereof. Similarity of pictures and objects in pictures is reviewed for each of the feature types, in close connection to the types and means of feedback the user of the systems is capable of giving by interaction. We briefly discuss aspects of system engineering: databases, system architecture, and evaluation. In the concluding section, we present our view on: the driving force of the field, the heritage from computer vision, the influence on computer vision, the role of similarity and of interaction, the need for databases, the problem of evaluation, and the role of the semantic gap.

/pdf/content-based-image-retrieval-at-the-end-of-the-early-years-q45iy9ejai.pdf

Content-based image retrieval at the end of the early years

Rendering the semantic content of an image in different styles is a difficult image processing task. Arguably, a major limiting factor for previous approaches has been the lack of image representations that explicitly represent semantic information and, thus, allow to separate image content from style. Here we use image representations derived from Convolutional Neural Networks optimised for object recognition, which make high level image information explicit. We introduce A Neural Algorithm of Artistic Style that can separate and recombine the image content and style of natural images. The algorithm allows us to produce new images of high perceptual quality that combine the content of an arbitrary photograph with the appearance of numerous wellknown artworks. Our results provide new insights into the deep image representations learned by Convolutional Neural Networks and demonstrate their potential for high level image synthesis and manipulation.

/pdf/image-style-transfer-using-convolutional-neural-networks-xq8m0q3iz5.pdf

Image Style Transfer Using Convolutional Neural Networks

A method described herein includes an act of receiving a query from a user, wherein the query is configured to search over a plurality of documents belonging to a particular domain. The method also includes an act of providing data to the user for display on a display screen of a computing apparatus, wherein the data is provided based at least in part upon a statistical analysis undertaken with respect to structured data pertaining to the particular domain, wherein the structured data is based at least in part upon data included in the plurality of documents.

Performing query expansion based upon statistical analysis of structured data

FIELD: information technology. ^ SUBSTANCE: architecture may employ high level language templates which widely determine model limits for adapting information content to multiple sizes and dimensions with wide variety of information content in a wide variety of formats. Also, a description of high quality documents in high level language may be transformed into low level limits for use in a system for creating adaptive models of documents based on a mesh, which leads to a sharp drop in the number of templates required for that system, while simultaneously increasing functionalities of templates and simplifying creation and maintenance of the templates. ^ EFFECT: design of an architecture used to crate a high quality document, which is a document having the outward appearance, whereby the document displays information content of a given type with given size of the displayed document. ^ 17 cl, 28 dwg

Various types of formatting with harmonic layout for dynamically aggregated documents

Described is a technology by which a user pastes selected data into a command line of a program, including when the selected data is non-textual. Upon detecting the paste (or drop) action, a variable name is automatically generated and inserted at the current point in a command line, where it acts as a proxy for the pasted data itself. A data structure comprising the selected data or transformed data corresponding to that selected data is maintained in program storage, e.g., RAM allocated to the program. In one aspect, a handler may be used to transform the data from one format into another that may be used by a particular program. For example, text may be reformatted into an array on which the program operates. The handler may be selected from a plurality of possible handlers, including customized handlers.

Pasting Various Data into a Programming Environment

A system and process for a server to provide a page of a document generated by an adaptive document layout application in a fixed-layout document format to a client computer over a client-server based network is presented. The document page is provided in response to a request for the page from the client, which may also specify a desired window size and/or desired formatting. In general, the server, referred to hereinafter as the Adaptive Document Layout (ADL) server, receives the request from a client computer and generates the requested document page using the aforementioned adaptive document layout application. The generated page is then converted into a prescribed fixed-layout document format. Finally, the ADL server transmits the page to the client computer via the client-server based network.

Charles E. Jacobs

Papers

Performing query expansion based upon statistical analysis of structured data

Various types of formatting with harmonic layout for dynamically aggregated documents

Pasting Various Data into a Programming Environment

Adaptive system and process for client/server based document layout

Erkennung von Zeichen mit niedriger Auflösung für mit einer Kamera erfassten Dokumente