Accelerating Dynamic Data Race Detection Using Static Thread Interference Analysis
12 Mar 2016-pp 30-39
TL;DR: A new framework is presented that eliminates redundant race check and boosts the dynamic race detection by performing static optimizations on top of a series of thread interference analysis phases.
Abstract: Precise dynamic race detectors report an error if and only if more than one thread concurrently exhibits conflict on a memory access. They insert instrumentations at compile-time to perform runtime checks on all memory accesses to ensure that all races are captured and no spurious warnings are generated. However, a dynamic race check for a particular memory access statement is guaranteed to be redundant if the statement can be statically identified as thread interference-free. Despite significant recent advances in dynamic detection techniques, the redundant check remains a critical factor that leads to prohibitive overhead of dynamic race detection for multithreaded programs.In this paper, we present a new framework that eliminates redundant race check and boosts the dynamic race detection by performing static optimizations on top of a series of thread interference analysis phases. Our framework is implemented on top of LLVM 3.5.0 and evaluated with an industry dynamic race detector TSAN which is available as a part of LLVM tool chain. 11 benchmarks from SPLASH2 are used to evaluate the effectiveness of our approach in accelerating TSAN by eliminating redundant interference-free checks. The experimental result demonstrates our new approach achieves from 1.4x to 4.0x (2.4x on average) speedup over original TSAN under 4 threads setting, and achieves from 1.3x to 4.6x (2.6x on average) speedup under 16 threads setting.
Citations
More filters
17 Mar 2016
TL;DR: SVF, which is fully implemented in LLVM, allows value-flow construction and pointer analysis to be performed in an iterative manner, thereby providing increasingly improved precision for both.
Abstract: This paper presents SVF, a tool that enables scalable and precise interprocedural Static Value-Flow analysis for C programs by leveraging recent advances in sparse analysis. SVF, which is fully implemented in LLVM, allows value-flow construction and pointer analysis to be performed in an iterative manner, thereby providing increasingly improved precision for both. SVF accepts points- to information generated by any pointer analysis (e.g., Andersen’s analysis) and constructs an interprocedural memory SSA form, in which the def-use chains of both top-level and address-taken variables are captured. Such value-flows can be subsequently exploited to support various forms of program analysis or enable more precise pointer analysis (e.g., flow-sensitive analysis) to be performed sparsely. By dividing a pointer analysis into three loosely coupled components: Graph, Rules and Solver, SVF provides an extensible interface for users to write their own solutions easily. SVF is publicly available at http://unsw-corg.github.io/SVF.
241 citations
Posted Content•
TL;DR: Experiments show that MUZZ outperforms AFL in both multithreading-relevant seed generation and concurrency-vulnerability detection, and by replaying the target programs against the generated seeds, MUZZ also reveals more Concurrency-bugs than AFL.
Abstract: Grey-box fuzz testing has revealed thousands of vulnerabilities in real-world software owing to its lightweight instrumentation, fast coverage feedback, and dynamic adjusting strategies. However, directly applying grey-box fuzzing to input-dependent multithreaded programs can be extremely inefficient. In practice, multithreading-relevant bugs are usually buried in sophisticated program flows. Meanwhile, the existing grey-box fuzzing techniques do not stress thread-interleavings which affect execution states in multithreaded programs. Therefore, mainstream grey-box fuzzers cannot effectively test problematic segments in multithreaded programs despite they might obtain high code coverage statistics.
To this end, we propose MUZZ, a new grey-box fuzzing technique that hunts for bugs in multithreaded programs. MUZZ owns three novel thread-aware instrumentations, namely coverage-oriented instrumentation, thread-context instrumentation, and schedule-intervention instrumentation. During fuzzing, these instrumentations engender runtime feedback to stress execution states caused by thread interleavings. By leveraging the feedback in the dynamic seed selection and execution strategies, MUZZ preserves more valuable seeds that expose bugs in a multithreading context.
We evaluate MUZZ on 12 real-world software programs. Experiments show that MUZZ outperforms AFL in both multithreading-relevant seed generation and concurrency-vulnerability detection. Further, by replaying the target programs against the generated seeds, MUZZ also reveals more concurrency-bugs (e.g., data-races, thread-leaks) than AFL. In total, MUZZ detected 8 new concurrency-vulnerabilities and 19 new concurrency-bugs. At the time of writing, 4 CVE IDs have been assigned to the reported issues.
27 citations
Cites background or methods from "Accelerating Dynamic Data Race Dete..."
...During A :instrumentation (§4), for a multithreaded program Po, MUZZ firstly computes thread-aware interprocedural control flow graph (ICFG) and the code segments that are likely to interleave with others during execution [11,45], namely suspicious interleaving scope, in §4....
[...]
..., may-happen-in-parallel [11, 45]) or dynamic concurrency-bug detection algorithms (e....
[...]
...It is worth noting that line 15, although protected by m, may still happen-in-parallel [11, 45] with lines 10 and 11....
[...]
Proceedings Article•
01 Jan 2020
TL;DR: In this paper, the authors propose MUZZ, a new grey-box fuzzing technique that hunts for bugs in multithreaded programs by leveraging the feedback in the dynamic seed selection and execution strategies.
Abstract: Grey-box fuzz testing has revealed thousands of vulnerabilities in real-world software owing to its lightweight instrumentation, fast coverage feedback, and dynamic adjusting strategies. However, directly applying grey-box fuzzing to input-dependent multithreaded programs can be extremely inefficient. In practice, multithreading-relevant bugs are usually buried in sophisticated program flows. Meanwhile, the existing grey-box fuzzing techniques do not stress thread-interleavings which affect execution states in multithreaded programs. Therefore, mainstream grey-box fuzzers cannot effectively test problematic segments in multithreaded programs despite they might obtain high code coverage statistics. To this end, we propose MUZZ, a new grey-box fuzzing technique that hunts for bugs in multithreaded programs. MUZZ owns three novel thread-aware instrumentations, namely coverage-oriented instrumentation, thread-context instrumentation, and schedule-intervention instrumentation. During fuzzing, these instrumentations engender runtime feedback to stress execution states caused by thread interleavings. By leveraging the feedback in the dynamic seed selection and execution strategies, MUZZ preserves more valuable seeds that expose bugs in a multithreading context. We evaluate MUZZ on 12 real-world software programs. Experiments show that MUZZ outperforms AFL in both multithreading-relevant seed generation and concurrency-vulnerability detection. Further, by replaying the target programs against the generated seeds, MUZZ also reveals more concurrency-bugs (e.g., data-races, thread-leaks) than AFL. In total, MUZZ detected 8 new concurrency-vulnerabilities and 19 new concurrency-bugs. At the time of writing, 4 CVE IDs have been assigned to the reported issues.
9 citations
TL;DR: A tool, which provides an adjustable balance between precise and slow software model checkers and fast and imprecise static analyzers, and an abstraction over the precise thread interaction and analysis for each thread in a separate way, but together with a specific environment, which models effects of other threads.
Abstract: Most of the software model checker tools do not scale well on complicated software. Our goal was to develop a tool, which provides an adjustable balance between precise and slow software model checkers and fast and imprecise static analyzers. The key idea of the approach is an abstraction over the precise thread interaction and analysis for each thread in a separate way, but together with a specific environment, which models effects of other threads. The environment contains a description of potential actions over the shared data and synchronization primitives, and conditions for its application. Adjusting the precision of the environment, one can achieve a required balance between speed and precision of the complete analysis. A formal description of the suggested approach was performed within a Configurable Program Analysis theory. It allows formulating assumptions and proving the soundness of the approach under the assumptions. For efficient data race detection we use a specific memory model, which allows to distinguish memory domains into the disjoint set of regions, which correspond to a data types. An implementation of the suggested approach into the CPAchecker framework allows reusing an existed approaches with minimal changes. Implementation of additional techniques according to the extended theory allows to increase the precision of the analysis. Results of the evaluation allow confirming scalability and practical usability of the approach.
7 citations
References
More filters
TL;DR: A new tool, called Eraser, is described, for dynamically detecting data races in lock-based multithreaded programs, which uses binary rewriting techniques to monitor every shared-monory reference and verify that consistent locking behavior is observed.
Abstract: Multithreaded programming is difficult and error prone. It is easy to make a mistake in synchronization that produces a data race, yet it can be extremely hard to locate this mistake during debugging. This article describes a new tool, called Eraser, for dynamically detecting data races in lock-based multithreaded programs. Eraser uses binary rewriting techniques to monitor every shared-monory reference and verify that consistent locking behavior is observed. We present several case studies, including undergraduate coursework and a multithreaded Web search engine, that demonstrate the effectiveness of this approach.
1,553 citations
"Accelerating Dynamic Data Race Dete..." refers background or methods in this paper
...Lockset-based algorithms attempt to detect inconsistent use of locks by different threads [37]....
[...]
...Most typical dynamic race detectors are based on two techniques: Lockset computation [37], Happens-Before (HB) ordering [9, 15], and a hybrid of these two [52, 50, 13, 32]....
[...]
01 Oct 1997
TL;DR: Eraser as mentioned in this paper uses binary rewriting techniques to monitor every shared memory reference and verify that consistent locking behavior is observed in lock-based multi-threaded programs, which can be used to detect data races.
Abstract: Multi-threaded programming is difficult and error prone. It is easy to make a mistake in synchronization that produces a data race, yet it can be extremely hard to locate this mistake during debugging. This paper describes a new tool, called Eraser, for dynamically detecting data races in lock-based multi-threaded programs. Eraser uses binary rewriting techniques to monitor every shared memory reference and verify that consistent locking behavior is observed. We present several case studies, including undergraduate coursework and a multi-threaded Web search engine, that demonstrate the effectiveness of this approach.
1,424 citations
19 Oct 2003
TL;DR: RacerX is a static tool that uses flow-sensitive, interprocedural analysis to detect both race conditions and deadlocks and uses novel strategies to infer checking information such as which locks protect which operations, which code contexts are multithreaded, and which shared accesses are dangerous.
Abstract: This paper describes RacerX, a static tool that uses flow-sensitive, interprocedural analysis to detect both race conditions and deadlocks. It is explicitly designed to find errors in large, complex multithreaded systems. It aggressively infers checking information such as which locks protect which operations, which code contexts are multithreaded, and which shared accesses are dangerous. It tracks a set of code features which it uses to sort errors both from most to least severe. It uses novel techniques to counter the impact of analysis mistakes. The tool is fast, requiring between 2-14 minutes to analyze a 1.8 million line system. We have applied it to Linux, FreeBSD, and a large commercial code base, finding serious errors in all of them. RacerX is a static tool that uses flow-sensitive, interprocedural analysis to detect both race conditions and deadlocks. It uses novel strategies to infer checking information such as which locks protect which operations, which code contexts are multithreaded, and which shared accesses are dangerous. We applied it to FreeBSD, Linux and a large commercial code base and found serious errors in all of them.
788 citations
"Accelerating Dynamic Data Race Dete..." refers background in this paper
...The problems caused by data races have motivated much work on detecting races via static [1, 5, 33, 49, 26, 14, 16] or dynamic analysis [13, 28, 48, 52]....
[...]
15 Jun 2009
TL;DR: This paper exploits the insight that the full generality of vector clocks is unnecessary in most cases to replace heavyweight vector clocks with an adaptive lightweight representation that, for almost all operations of the target program, requires only constant space and supports constant-time operations.
Abstract: \begin{}Multithreaded programs are notoriously prone to race conditions. Prior work on dynamic race detectors includes fast but imprecise race detectors that report false alarms, as well as slow but precise race detectors that never report false alarms. The latter typically use expensive vector clock operations that require time linear in the number of program threads.This paper exploits the insight that the full generality of vector clocks is unnecessary in most cases. That is, we can replace heavyweight vector clocks with an adaptive lightweight representation that, for almost all operations of the target program, requires only constant space and supports constant-time operations. This representation change significantly improves time and space performance, with no loss in precision.Experimental results on Java benchmarks including the Eclipse development environment show that our FastTrack race detector is an order of magnitude faster than a traditional vector-clock race detector, and roughly twice as fast as the high-performance DJIT+ algorithm. FastTrack is even comparable in speed to Eraser on our Java benchmarks, while never reporting false alarms.
657 citations
"Accelerating Dynamic Data Race Dete..." refers methods in this paper
...The state-of-practice HB-based detector TSAN adopts improved FastTrack [15] to achieve the better performance....
[...]
...piler instrumentation module and a run-time library to perform dynamic detection algorithm similar to FastTrack[15]....
[...]
...Most typical dynamic race detectors are based on two techniques: Lockset computation [37], Happens-Before (HB) ordering [9, 15], and a hybrid of these two [52, 50, 13, 32]....
[...]
...It consists of a compiler instrumentation module and a run-time library to perform dynamic detection algorithm similar to FastTrack[15]....
[...]
...IFRit is faster than FastTrack, but may miss some races....
[...]
Book•
01 Jan 2008TL;DR: A novel technique for static race detection in Java programs, comprised of a series of stages that employ a combination of static analyses to successively reduce the pairs of memory accesses potentially involved in a race.
Abstract: We present a novel technique for static race detection in Java programs, comprised of a series of stages that employ a combination of static analyses to successively reduce the pairs of memory accesses potentially involved in a race. We have implemented our technique and applied it to a suite of multi-threaded Java programs. Our experiments show that it is precise, scalable, and useful, reporting tens to hundreds of serious and previously unknown concurrency bugs in large, widely-used programs with few false alarms.
526 citations
"Accelerating Dynamic Data Race Dete..." refers background in this paper
...A number of studies have appeared, introducing a variety of advanced techniques for discovering interleaving information in a program [27, 26]....
[...]
...The problems caused by data races have motivated much work on detecting races via static [1, 5, 33, 49, 26, 14, 16] or dynamic analysis [13, 28, 48, 52]....
[...]