MapReduce is a programming model and an associated implementation for processing and generating large data sets. Users specify a map function that processes a key/value pair to generate a set of intermediate key/value pairs, and a reduce function that merges all intermediate values associated with the same intermediate key. Many real world tasks are expressible in this model, as shown in the paper.

Programs written in this functional style are automatically parallelized and executed on a large cluster of commodity machines. The run-time system takes care of the details of partitioning the input data, scheduling the program's execution across a set of machines, handling machine failures, and managing the required inter-machine communication. This allows programmers without any experience with parallel and distributed systems to easily utilize the resources of a large distributed system.

Our implementation of MapReduce runs on a large cluster of commodity machines and is highly scalable: a typical MapReduce computation processes many terabytes of data on thousands of machines. Programmers find the system easy to use: hundreds of MapReduce programs have been implemented and upwards of one thousand MapReduce jobs are executed on Google's clusters every day.

/pdf/mapreduce-simplified-data-processing-on-large-clusters-16emdgg5h5.pdf

MapReduce: simplified data processing on large clusters

MapReduce is a programming model and an associated implementation for processing and generating large datasets that is amenable to a broad variety of real-world tasks. Users specify the computation in terms of a map and a reduce function, and the underlying runtime system automatically parallelizes the computation across large-scale clusters of machines, handles machine failures, and schedules inter-machine communication to make efficient use of the network and disks. Programmers find the system easy to use: more than ten thousand distinct MapReduce programs have been implemented internally at Google over the past four years, and an average of one hundred thousand MapReduce jobs are executed on Google's clusters every day, processing a total of more than twenty petabytes of data per day.

/pdf/mapreduce-simplified-data-processing-on-large-clusters-1zth35tzla.pdf

Software vulnerabilities have had a devastating effect on the Internet. Worms such as CodeRed and Slammer can compromise hundreds of thousands of hosts within hours or even minutes, and cause millions of dollars of damage [26, 43]. To successfully combat these fast automatic Internet attacks, we need fast automatic attack detection and filtering mechanisms. In this paper we propose dynamic taint analysis for automatic detection of overwrite attacks, which include most types of exploits. This approach does not need source code or special compilation for the monitored program, and hence works on commodity software. To demonstrate this idea, we have implemented TaintCheck, a mechanism that can perform dynamic taint analysis by performing binary rewriting at run time. We show that TaintCheck reliably detects most types of exploits. We found that TaintCheck produced no false positives for any of the many different programs that we tested. Further, we describe how TaintCheck could improve automatic signature generation in

/pdf/dynamic-taint-analysis-for-automatic-detection-analysis-and-2bodbjb0mi.pdf

Dynamic Taint Analysis for Automatic Detection, Analysis, and Signature Generation of Exploits on Commodity Software

We present a simple architectural mechanism called dynamic information flow tracking that can significantly improve the security of computing systems with negligible performance overhead. Dynamic information flow tracking protects programs against malicious software attacks by identifying spurious information flows from untrusted I/O and restricting the usage of the spurious information.Every security attack to take control of a program needs to transfer the program's control to malevolent code. In our approach, the operating system identifies a set of input channels as spurious, and the processor tracks all information flows from those inputs. A broad range of attacks are effectively defeated by checking the use of the spurious values as instructions and pointers.Our protection is transparent to users or application programmers; the executables can be used without any modification. Also, our scheme only incurs, on average, a memory overhead of 1.4% and a performance overhead of 1.1%.

/pdf/secure-program-execution-via-dynamic-information-flow-2qbo0v1zn5.pdf

Secure program execution via dynamic information flow tracking

We describe a new, general approach for safeguarding systems against any type of code-injection attack. We apply Kerckhoff's principle, by creating process-specific randomized instruction sets (e.g., machine instructions) of the system executing potentially vulnerable software. An attacker who does not know the key to the randomization algorithm will inject code that is invalid for that randomized processor, causing a runtime exception. To determine the difficulty of integrating support for the proposed mechanism in the operating system, we modified the Linux kernel, the GNU binutils tools, and the bochs-x86 emulator. Although the performance penalty is significant, our prototype demonstrates the feasibility of the approach, and should be directly usable on a suitable-modified processor (e.g., the Transmeta Crusoe).Our approach is equally applicable against code-injecting attacks in scripting and interpreted languages, e.g., web-based SQL injection. We demonstrate this by modifying the Perl interpreter to permit randomized script execution. The performance penalty in this case is minimal. Where our proposed approach is feasible (i.e., in an emulated environment, in the presence of programmable or specialized hardware, or in interpreted languages), it can serve as a low-overhead protection mechanism, and can easily complement other mechanisms.

/pdf/countering-code-injection-attacks-with-instruction-set-1m4firgxpp.pdf

Countering code-injection attacks with instruction-set randomization

The exploitation of buffer overflow vulnerabilities in process stacks constitutes a significant portion of security attacks. We present two new methods to detect and handle such attacks. In contrast to previous work, the new methods work with any existing pre-compiled executable and can be used transparently per-process as well as on a system-wide basis. The first method intercepts all calls to library functions known to be vulnerable. A substitute version of the corresponding function implements the original functionality, but in a manner that ensures that any buffer overflows are contained within the current stack frame. The second method uses binary modification of the process memory to force verification of critical elements of stacks before use. We have implemented both methods on Linux as dynamically loadable libraries and shown that both libraries detect several known attacks. The performance overhead of these libraries range from negligible to 15%.

Transparent run-time defense against stack smashing attacks

Charlotte: metacomputing on the Web

The importance of adapting networks of workstations for use as parallel processing platforms is well established. However current solutions do not always address important issues that exist in real networks. External factors like the sharing of resources, unpredictable behavior of the network and failures, are present in multiuser networks and must be addressed. CALYPSO is a prototype software system for writing and executing parallel programs on non-dedicated platforms, based on COTS networked workstations operating systems, and compilers. Among notable properties of the system are: (1) simple programming paradigm incorporating shared memory constructs and separating the programming and the execution parallelism, (2) transparent utilization of unreliable shared resources by providing dynamic load balancing and fault tolerance, and (3) effective performance for large classes of coarse-grained computations. We present the system and report our initial experiments and performance results in settings that closely resemble the dynamic behavior of a "real" network. Under varying work-load conditions, resource availability and process failures, the efficiency of the test program we present ranged from 84% to 94% bench-marked against a sequential program.

CALYPSO: a novel software system for fault-tolerant parallel processing on distributed platforms

A data image management system (DIMS) that includes a local data image manager (LDIM), a remote data image manager (RDIM), and a remote persistent storage device (RPSD). The LDIM communicates with the RDIM through a direct communication link or through a communication network. The RDIM can store data on and retrieve data from the RPSD. In an environment where an LDIM has been installed in a computer having a “local” persistent storage device (LPSD), the DIMS allows for the storing of the LPSD's data image on the RPSD, with the LPSD serving as a persistent, consistent cache of the data image. The data image stored on the RPSD is referred to as the “master data image” and the data image cached on the LPSD is referred to as the “local data image” or “cached data image.” The LDIM functions to intercept read/write requests that are intended to be received by the LPSD. The read/write requests specify an address of the LPSD. Upon intercepting a read request, the LDIM is programmed to determine whether the portion of the cached data image that is stored at the specified address is up-to-date. If it is up-to-date, the LDIM retrieves the requested data from the LPSD and passes the data back to the component or device from which it received the request. If it is not up-to-date, the LDIM transmits the read request to the RDIM. Upon receiving the read request, the RDIM locates and reads the requested data from the master data image stored on the RPSD and then transmits the data back to the LDIM.

Data image management via emulation of non-volatile storage device

The exploitation of buffer overﬂow vulnerabilities in proc ess stacks constitutes a signiﬁcant portion of securityattacks. We present a new method to detect and handle such attacks. In contrast to previous methods, this newmethod works with any existing pre-compiled executable andcan be used transparently, even on a system-wide basis.The method intercepts all calls to library functions that are known to be vulnerable. A substitute version of thecorresponding function implements the original functionality, but in a manner that ensures that any buffer overﬂowsare contained within the current stack frame. This method has been implemented on Linux as a dynamically loadablelibrarycalled libsafe. Libsafe has been shown to detect several known attacks and can potentially prevent yet unknownattacks. Experiments indicate that the performance overhead of libsafe is negligible. 1 Introduction As the Internet has grown, the opportunities for attempts to access remote systems improperly have increased. Severalsecurity attacks, such as the 1988 Internet Worm [7, 18, 19], have even become entrenched in Internet history. Someattacks, such as the Internet Worm, merely annoy or occupy system resources. However, other attacks are moreinsidious because they seize root privileges and modify, corrupt, or steal data.0510152025301988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999YearNumber of alertsbuffer overflow total

Arash Baratloo

Papers

Transparent run-time defense against stack smashing attacks

Charlotte: metacomputing on the Web

CALYPSO: a novel software system for fault-tolerant parallel processing on distributed platforms

Data image management via emulation of non-volatile storage device

Libsafe: Protecting Critical Elements of Stacks