Background: PLINK 1 is a widely used open-source C/C++ toolset for genome-wide association studies (GWAS) and research in population genetics. However, the steady accumulation of data from imputation and whole-genome sequencing studies has exposed a strong need for faster and scalable implementations of key functions, such as logistic regression, linkage disequilibrium estimation, and genomic distance evaluation. In addition, GWAS and population-genetic data now frequently contain genotype likelihoods, phase information, and/or multiallelic variants, none of which can be represented by PLINK 1’s primary data format. Findings: To address these issues, we are developing a second-generation codebase for PLINK. The first major release from this codebase, PLINK 1.9, introduces extensive use of bit-level parallelism, O √ n -time/constant-space Hardy-Weinberg equilibrium and Fisher’s exact tests, and many other algorithmic improvements. In combination, these changes accelerate most operations by 1-4 orders of magnitude, and allow the program to handle datasets too large to fit in RAM. We have also developed an extension to the data format which adds low-overhead support for genotype likelihoods, phase, multiallelic variants, and reference vs. alternate alleles, which is the basis of our planned second release (PLINK 2.0). Conclusions: The second-generation versions of PLINK will offer dramatic improvements in performance and compatibility. For the first time, users without access to high-end computing resources can perform several essential analyses of the feature-rich and very large genetic datasets coming into use.

/pdf/second-generation-plink-rising-to-the-challenge-of-larger-2i1coy4h8p.pdf

Second-generation PLINK: rising to the challenge of larger and richer datasets

This paper presents and characterizes the Princeton Application Repository for Shared-Memory Computers (PARSEC), a benchmark suite for studies of Chip-Multiprocessors (CMPs). Previous available benchmarks for multiprocessors have focused on high-performance computing applications and used a limited number of synchronization methods. PARSEC includes emerging applications in recognition, mining and synthesis (RMS) as well as systems applications which mimic large-scale multithreaded commercial programs. Our characterization shows that the benchmark suite covers a wide spectrum of working sets, locality, data sharing, synchronization and off-chip traffic. The benchmark suite has been made available to the public.

/pdf/the-parsec-benchmark-suite-characterization-and-jiajft2rem.pdf

The PARSEC benchmark suite: characterization and architectural implications

PLINK 1 is a widely used open-source C/C++ toolset for genome-wide association studies (GWAS) and research in population genetics. However, the steady accumulation of data from imputation and whole-genome sequencing studies has exposed a strong need for even faster and more scalable implementations of key functions. In addition, GWAS and population-genetic data now frequently contain probabilistic calls, phase information, and/or multiallelic variants, none of which can be represented by PLINK 1's primary data format. 
To address these issues, we are developing a second-generation codebase for PLINK. The first major release from this codebase, PLINK 1.9, introduces extensive use of bit-level parallelism, O(sqrt(n))-time/constant-space Hardy-Weinberg equilibrium and Fisher's exact tests, and many other algorithmic improvements. In combination, these changes accelerate most operations by 1-4 orders of magnitude, and allow the program to handle datasets too large to fit in RAM. This will be followed by PLINK 2.0, which will introduce (a) a new data format capable of efficiently representing probabilities, phase, and multiallelic variants, and (b) extensions of many functions to account for the new types of information. 
The second-generation versions of PLINK will offer dramatic improvements in performance and compatibility. For the first time, users without access to high-end computing resources can perform several essential analyses of the feature-rich and very large genetic datasets coming into use.

We explore the effect of dimensionality on the nearest neighbor problem. We show that under a broad set of conditions (much broader than independent and identically distributed dimensions), as dimensionality increases, the distance to the nearest data point approaches the distance to the farthest data point. To provide a practical perspective, we present empirical results on both real and synthetic data sets that demonstrate that this effect can occur for as few as 10-15 dimensions. These results should not be interpreted to mean that high-dimensional indexing is never meaningful; we illustrate this point by identifying some high-dimensional workloads for which this effect does not occur. However, our results do emphasize that the methodology used almost universally in the database literature to evaluate high-dimensional indexing techniques is flawed, and should be modified. In particular, most such techniques proposed in the literature are not evaluated versus simple linear scan, and are evaluated over workloads for which nearest neighbor is not meaningful. Often, even the reported experiments, when analyzed carefully, show that linear scan would outperform the techniques being proposed on the workloads studied in high (10-15) dimensionality!.

When is nearest neighbor meaningful

Similarity search finds application in specialized database systems handling complex data such as images or videos, which are typically represented by high-dimensional features and require specific indexing structures. This paper tackles the problem of better utilizing GPUs for this task. While GPUs excel at data-parallel tasks, prior approaches are bottlenecked by algorithms that expose less parallelism, such as k-min selection, or make poor use of the memory hierarchy. 
We propose a design for k-selection that operates at up to 55% of theoretical peak performance, enabling a nearest neighbor implementation that is 8.5x faster than prior GPU state of the art. We apply it in different similarity search scenarios, by proposing optimized design for brute-force, approximate and compressed-domain search based on product quantization. In all these setups, we outperform the state of the art by large margins. Our implementation enables the construction of a high accuracy k-NN graph on 95 million images from the Yfcc100M dataset in 35 minutes, and of a graph connecting 1 billion vectors in less than 12 hours on 4 Maxwell Titan X GPUs. We have open-sourced our approach for the sake of comparison and reproducibility.

Billion-scale similarity search with GPUs

Techniques for an omni-channel approach for displaying simulated digital apparel content are presented herein. A machine can detect an available amount of a computing resource on a client device. A determination that the client device is to render only a three-dimensional body model, among a set of models that includes the three-dimensional body model and a three-dimensional garment model, and that a server is to render the three-dimensional garment model, may occur based on the detected available amount of the computing resource on the client device. The machine can provide the client device with the three-dimensional garment model draped on the three-dimensional body model. The machine can cause the server to render at least a portion of the three-dimensional garment model in accordance with the determination. The machine can cause the client device to render at least a portion of the three-dimensional body model in accordance with the determination.

Omni-channel simulated digital apparel content display

We investigate the performance and scalability of the randomized CX low-rank matrix factorization and demonstrate its applicability through the analysis of a 1TB mass spectrometry imaging (MSI) dataset, using Apache Spark on an Amazon EC2 cluster, a Cray XC40 system, and an experimental Cray cluster. We implemented this factorization both as a parallelized C implementation with hand-tuned optimizations and in Scala using the Apache Spark high-level cluster computing framework. We obtained consistent performance across the three platforms: using Spark we were able to process the 1TB size dataset in under 30 minutes with 960 cores on all systems, with the fastest times obtained on the experimental Cray cluster. In comparison, the C implementation processed the 1TB size dataset 21X faster on the Amazon EC2 system, due to careful cache optimizations, bandwidth-friendly access of matrices and vector computation using SIMD units. We report these results and their implications on the hardware and software issues arising in supporting data-centric workloads in parallel and distributed environments.

/pdf/a-multi-platform-evaluation-of-the-randomized-cx-low-rank-1itxzknqcd.pdf

A Multi-Platform Evaluation of the Randomized CX Low-Rank Matrix Factorization in Spark

Methods and systems to determine view cell occlusion, including to project objects of a 3-dimensional graphics environment to a 2-dimensional image plane with respect to the view point, to reduce sizes of corresponding object images, to generate an occluder map from the reduced-size object images, to compare at least a portion of the object images to the occluder map, and to identify an object as occluded with respect to the view cell when pixel depth values of the object image are greater than corresponding pixel depth values of the occluder map. Methods and systems to reduce an object image size include methods and systems to nullify pixel depth values within a radius of an edge pixel, and to determine the radius as a distance from the edge pixel to a second pixel so that a line between the view point and the second pixel is parallel with one or more of a line and a plane that is tangential to a sphere enclosing the view cell and a point on the object that corresponds to the edge pixel.

Methods and systems to determine conservative view cell occlusion

A method of computing a collision-free velocity ( 117, 217 ) for an agent ( 110 ) in a crowd simulation environment ( 100 ) comprises identifying a quadratic optimization problem that corresponds to the collision-free velocity, and finding an exact solution for the quadratic optimization problem by using a geometric approach.

Optimization-Based exact formulation and solution of crowd simulation in virtual worlds

We present ISOSLIDER, a system for interactive exploration of isosurfaces of a scalar field Our algorithm focuses on fast update of isosurfaces for interactive display as a user makes small changes to the isovalue of the desired surface We exploit the coherence of this update Larger changes are supported as well The update to the isosurface is made at a correct level of detail so that not too many operations need be performed nor too many triangles need be rendered ISOSLIDER does not need to retain the entire volume in the main memory and stores most data out of core The central idea of the ISOSLIDER algorithm is to determine salient isovalues where surface topology changes and pre-encode these changes so as to facilitate fast updates to the triangulation

/pdf/isoslider-a-system-for-interactive-exploration-of-3qo18vpw48.pdf

Jatin Chhugani

Papers

Omni-channel simulated digital apparel content display

A Multi-Platform Evaluation of the Randomized CX Low-Rank Matrix Factorization in Spark

Methods and systems to determine conservative view cell occlusion

Optimization-Based exact formulation and solution of crowd simulation in virtual worlds

ISOSLIDER: a system for interactive exploration of isosurfaces