MapReduce is a programming model and an associated implementation for processing and generating large data sets. Users specify a map function that processes a key/value pair to generate a set of intermediate key/value pairs, and a reduce function that merges all intermediate values associated with the same intermediate key. Many real world tasks are expressible in this model, as shown in the paper.

Programs written in this functional style are automatically parallelized and executed on a large cluster of commodity machines. The run-time system takes care of the details of partitioning the input data, scheduling the program's execution across a set of machines, handling machine failures, and managing the required inter-machine communication. This allows programmers without any experience with parallel and distributed systems to easily utilize the resources of a large distributed system.

Our implementation of MapReduce runs on a large cluster of commodity machines and is highly scalable: a typical MapReduce computation processes many terabytes of data on thousands of machines. Programmers find the system easy to use: hundreds of MapReduce programs have been implemented and upwards of one thousand MapReduce jobs are executed on Google's clusters every day.

/pdf/mapreduce-simplified-data-processing-on-large-clusters-16emdgg5h5.pdf

MapReduce: simplified data processing on large clusters

MapReduce is a programming model and an associated implementation for processing and generating large datasets that is amenable to a broad variety of real-world tasks. Users specify the computation in terms of a map and a reduce function, and the underlying runtime system automatically parallelizes the computation across large-scale clusters of machines, handles machine failures, and schedules inter-machine communication to make efficient use of the network and disks. Programmers find the system easy to use: more than ten thousand distinct MapReduce programs have been implemented internally at Google over the past four years, and an average of one hundred thousand MapReduce jobs are executed on Google's clusters every day, processing a total of more than twenty petabytes of data per day.

/pdf/mapreduce-simplified-data-processing-on-large-clusters-1zth35tzla.pdf

Many problems of recent interest in statistics and machine learning can be posed in the framework of convex optimization. Due to the explosion in size and complexity of modern datasets, it is increasingly important to be able to solve problems with a very large number of features or training examples. As a result, both the decentralized collection or storage of these datasets as well as accompanying distributed solution methods are either necessary or at least highly desirable. In this review, we argue that the alternating direction method of multipliers is well suited to distributed convex optimization, and in particular to large-scale problems arising in statistics, machine learning, and related areas. The method was developed in the 1970s, with roots in the 1950s, and is equivalent or closely related to many other algorithms, such as dual decomposition, the method of multipliers, Douglas–Rachford splitting, Spingarn's method of partial inverses, Dykstra's alternating projections, Bregman iterative algorithms for l1 problems, proximal methods, and others. After briefly surveying the theory and history of the algorithm, we discuss applications to a wide variety of statistical and machine learning problems of recent interest, including the lasso, sparse logistic regression, basis pursuit, covariance selection, support vector machines, and many others. We also discuss general distributed optimization, extensions to the nonconvex setting, and efficient implementation, including some details on distributed MPI and Hadoop MapReduce implementations.

/pdf/distributed-optimization-and-statistical-learning-via-the-8s9gtddw0d.pdf

Distributed Optimization and Statistical Learning Via the Alternating Direction Method of Multipliers

A general non-parametric technique is proposed for the analysis of a complex multimodal feature space and to delineate arbitrarily shaped clusters in it. The basic computational module of the technique is an old pattern recognition procedure: the mean shift. For discrete data, we prove the convergence of a recursive mean shift procedure to the nearest stationary point of the underlying density function and, thus, its utility in detecting the modes of the density. The relation of the mean shift procedure to the Nadaraya-Watson estimator from kernel regression and the robust M-estimators; of location is also established. Algorithms for two low-level vision tasks discontinuity-preserving smoothing and image segmentation - are described as applications. In these algorithms, the only user-set parameter is the resolution of the analysis, and either gray-level or color images are accepted as input. Extensive experimental results illustrate their excellent performance.

Mean shift: a robust approach toward feature space analysis

Today, visual recognition systems are still rarely employed in robotics applications. Perhaps one of the main reasons for this is the lack of demanding benchmarks that mimic such scenarios. In this paper, we take advantage of our autonomous driving platform to develop novel challenging benchmarks for the tasks of stereo, optical flow, visual odometry/SLAM and 3D object detection. Our recording platform is equipped with four high resolution video cameras, a Velodyne laser scanner and a state-of-the-art localization system. Our benchmarks comprise 389 stereo and optical flow image pairs, stereo visual odometry sequences of 39.2 km length, and more than 200k 3D object annotations captured in cluttered scenarios (up to 15 cars and 30 pedestrians are visible per image). Results from state-of-the-art algorithms reveal that methods ranking high on established datasets such as Middlebury perform below average when being moved outside the laboratory to the real world. Our goal is to reduce this bias by providing challenging benchmarks with novel difficulties to the computer vision community. Our benchmarks are available online at: www.cvlibs.net/datasets/kitti

/pdf/are-we-ready-for-autonomous-driving-the-kitti-vision-zygefho352.pdf

Are we ready for autonomous driving? The KITTI vision benchmark suite

The aim of this special issue of IJCV is to bring you a snapshot of, and reference to, the state of the art in detecting depth or disparity from stereo, multi-baseline, multi-view and novel image sensors. The primary concern here is with correspondence methods especially as they relate to speed, accuracy and density of results. Techniques for camera calibration and stereo rectification are not a major theme here since classical techniques already work well (source code for which is in the OpenCV library referenced below). Above, is what this special IJCV issue is about. Why we want depth is two fold: Depth allows cameras and machines to move in and understand the 3D world in which they are embedded; Depth is also one more cue that helps solve tough problems in computer vision such as object segmentation and tracking. Indeed, depth may be the cue that delivers reliable vision systems in the near term. This issue begins on page 7 with an article by Scharstein and Szeliski covering an extensive taxonomy and evaluation of dense stereo correspondence algorithms including some of the new techniques appearing in this issue. One may use this as a launching off point into the rest of the issue. Topics covered are:

Guest Editorial: Stereo and Multi-Baseline Vision

본원에서는, 물리적 소매 위치에 있는 아이템 또는 제품(예컨대, 타겟 제품)을 구매하려고 하는 사용자에게 대체물들 구매에 관한 정보를 제공하는 AR(augmented reality) 시스템이 개시된다 일부 변형들에서, 제품 및/또는 대체 제품을 구매하는 오퍼들이 AR 시스템을 통해 상인 및/또는 경쟁자들에 의해 제공된다 ONS(offer negotiation server)는 다양한 EP(external party)들이 제공한 오퍼 데이터를 어그리게이팅하고, 사용자가 타겟 제품의 구매를 고려하고 있고 있을 때 사용자에게 이 오퍼들을 디스플레이한다 일부 변형들에서, AR 시스템은 소매 위치에서 아이템들의 구매 프로세스를 가능하게 하도록 구성될 수 있다

구매를 위한 증강 현실 디바이스들, 시스템들 및 방법들

In one embodiment, the present invention includes a method of successively splitting an analog function into high and low ranges and calculating a binary mask for these ranges to obtain a plurality of data regions at a plurality of split levels, and training binary classifiers on the plurality of data regions of at least one of the split levels. In such manner, binary classifiers may be used to classify an analog function. Other embodiments are described and claimed.

Classifying an analog voltage in a control system using binary classification of time segments determined by voltage level

Mean Shift today, is widely used for mode detection and clustering. The technique though, is challenged in practice due to assumptions of isotropicity and homoscedasticity. We present an adaptive Mean Shift methodology that allows for full anisotropic clustering, through unsupervised local bandwidth selection. The bandwidth matrices evolve naturally, adapting locally through agglomeration, and in turn guiding further agglomeration. The online methodology is practical and effecive for low-dimensional feature spaces, preserving better detail and clustering salience. Additionally, conventional Mean Shift either critically depends on a per instance choice of bandwidth, or relies on offline methods which are inflexible and/or again data instance specific. The presented approach, due to its adaptive design, also alleviates this issue - with a default form performing generally well. The methodology though, allows for effective tuning of results.

/pdf/anisotropic-agglomerative-adaptive-mean-shift-4nh5blpfy1.pdf

Anisotropic Agglomerative Adaptive Mean-Shift

In a visual tracking task, the object may exhibit rich dynamic behavior in complex environments that can corrupt target observations via background clutter and occlusion. Such dynamics and background induce nonlinear, nonGaussian and multimodal observation densities. These densities are difficult to model with traditional methods such as Kalman filter models (KFMs) due to their Gaussian assumptions. Dynamic Bayesian networks (DBNs) provide a more general framework in which to solve these problems. DBNs generalize KFMs by allowing arbitrary probability distributions, not just (unimodal) linear-Gaussian. Under the DBN umbrella, a broad class of learning and inference algorithms for time-series models can be used in visual tracking. Furthermore, DBNs provide a natural way to combine multiple vision cues. In this chapter, we describe some DBN models for tracking in nonlinear, nonGaussian and multimodal situations, and present a prediction method to assist feature extraction part by making a hypothesis for the new observations. IGI PUBLISHING This paper appears in the publication, Bayesian Network Technologies: Applications and Graphical Models edited by A. Mittal and A. Kassim © 2007, IGI Global 701 E. Chocolate Avenue, Suite 200, Hershey PA 17033-1240, USA Tel: 717/533-8845; Fax 717/533-8661; URL-http://www.igi-pub.com ITB14285

Gary Bradski

Papers

Guest Editorial: Stereo and Multi-Baseline Vision

구매를 위한 증강 현실 디바이스들, 시스템들 및 방법들

Classifying an analog voltage in a control system using binary classification of time segments determined by voltage level

Anisotropic Agglomerative Adaptive Mean-Shift

DBN Models for Visual Tracking and Prediction