Showing papers on "Heap (data structure) published in 2018"

Revery: From Proof-of-Concept to Exploitable

Abstract: Automatic exploit generation is an open challenge. Existing solutions usually explore in depth the crashing paths, i.e., paths taken by proof-of-concept (POC) inputs triggering vulnerabilities, and generate exploits when exploitable states are found along the paths. However, exploitable states do not always exist in crashing paths. Moreover, existing solutions heavily rely on symbolic execution and are not scalable in path exploration and exploit generation. In addition, few solutions could exploit heap-based vulnerabilities. In this paper, we propose a new solution revery to search for exploitable states in paths diverging from crashing paths, and generate control-flow hijacking exploits for heap-based vulnerabilities. It adopts three novel techniques:(1) a digraph to characterize a vulnerability's memory layout and its contributor instructions;(2) a fuzz solution to explore diverging paths, which have similar memory layouts as the crashing paths, in order to search more exploitable states and generate corresponding diverging inputs;(3) a stitch solution to stitch crashing paths and diverging paths together, and synthesize EXP inputs able to trigger both vulnerabilities and exploitable states. We have developed a prototype of revery based on the binary analysis engine angr, and evaluated it on a set of 19 real world CTF (capture the flag) challenges. Experiment results showed that it could generate exploits for 9 (47%) of them, and generate EXP inputs able to trigger exploitable states for another 5 (26%) of them.

Finding root causes of floating point error

Abstract: Floating-point arithmetic plays a central role in science, engineering, and finance by enabling developers to approximate real arithmetic. To address numerical issues in large floating-point applications, developers must identify root causes, which is difficult because floating-point errors are generally non-local, non-compositional, and non-uniform. This paper presents Herbgrind, a tool to help developers identify and address root causes in numerical code written in low-level languages like C/C++ and Fortran. Herbgrind dynamically tracks dependencies between operations and program outputs to avoid false positives and abstracts erroneous computations to simplified program fragments whose improvement can reduce output error. We perform several case studies applying Herbgrind to large, expert-crafted numerical programs and show that it scales to applications spanning hundreds of thousands of lines, correctly handling the low-level details of modern floating point hardware and mathematical libraries and tracking error across function boundaries and through the heap.

Espresso: Brewing Java For More Non-Volatility with Non-volatile Memory

Abstract: Fast, byte-addressable non-volatile memory (NVM) embraces both near-DRAM latency and disk-like persistence, which has generated considerable interests to revolutionize system software stack and programming models. However, it is less understood how NVM can be combined with managed runtime like Java virtual machine (JVM) to ease persistence management. This paper proposes Espresso, a holistic extension to Java and its runtime, to enable Java programmers to exploit NVM for persistence management with high performance. Espresso first provides a general persistent heap design called Persistent Java Heap (PJH) to manage persistent data as normal Java objects. The heap is then strengthened with a recoverable mechanism to provide crash consistency for heap metadata. Espresso further provides a new abstraction called Persistent Java Object (PJO) to provide an easy-to-use but safe persistence programming model for programmers to persist application data. Evaluation confirms that Espresso significantly outperforms state-of-art NVM support for Java (i.e., JPA and PCJ) while being compatible to data structures in existing Java programs.

Skyway: Connecting Managed Heaps in Distributed Big Data Systems

Abstract: Managed languages such as Java and Scala are prevalently used in development of large-scale distributed systems. Under the managed runtime, when performing data transfer across machines, a task frequently conducted in a Big Data system, the system needs to serialize a sea of objects into a byte sequence before sending them over the network. The remote node receiving the bytes then deserializes them back into objects. This process is both performance-inefficient and labor-intensive: (1) object serialization/deserialization makes heavy use of reflection, an expensive runtime operation and/or (2) serialization/deserialization functions need to be hand-written and are error-prone. This paper presents Skyway, a JVM-based technique that can directly connect managed heaps of different (local or remote) JVM processes. Under Skyway, objects in the source heap can be directly written into a remote heap without changing their formats. Skyway provides performance benefits to any JVM-based system by completely eliminating the need (1) of invoking serialization/deserialization functions, thus saving CPU time, and (2) of requiring developers to hand-write serialization functions.

Automatic Heap Layout Manipulation for Exploitation

Abstract: Heap layout manipulation is integral to exploiting heap-based memory corruption vulnerabilities. In this paper we present the first automatic approach to the problem, based on pseudo-random black-box search. Our approach searches for the inputs required to place the source of a heap-based buffer overflow or underflow next to heap-allocated objects that an exploit developer, or automatic exploit generation system, wishes to read or corrupt. We present a framework for benchmarking heap layout manipulation algorithms, and use it to evaluate our approach on several real-world allocators, showing that pseudo-random black box search can be highly effective. We then present SHRIKE, a novel system that can perform automatic heap layout manipulation on the PHP interpreter and can be used in the construction of control-flow hijacking exploits. Starting from PHP's regression tests, SHRIKE discovers fragments of PHP code that interact with the interpreter's heap in useful ways, such as making allocations and deallocations of particular sizes, or allocating objects containing sensitive data, such as pointers. SHRIKE then uses our search algorithm to piece together these fragments into programs, searching for one that achieves a desired heap layout. SHRIKE allows an exploit developer to focus on the higher level concepts in an exploit, and to defer the resolution of heap layout constraints to SHRIKE. We demonstrate this by using SHRIKE in the construction of a control-flow hijacking exploit for the PHP interpreter.

High-Performance Sparse Matrix-Matrix Products on Intel KNL and Multicore Architectures

Abstract: Sparse matrix-matrix multiplication (SpGEMM) is a computational primitive that is widely used in areas ranging from traditional numerical applications to recent big data analysis and machine learning. Although many SpGEMM algorithms have been proposed, hardware specific optimizations for multi- and many-core processors are lacking and a detailed analysis of their performance under various use cases and matrices is not available. We firstly identify and mitigate multiple bottlenecks with memory management and thread scheduling on Intel Xeon Phi (Knights Landing or KNL). Specifically targeting multi- and many-core processors, we develop a hash-table-based algorithm and optimize a heap-based shared-memory SpGEMM algorithm. We examine their performance together with other publicly available codes. Different from the literature, our evaluation also includes use cases that are representative of real graph algorithms, such as multi-source breadth-first search or triangle counting. Our hash-table and heap-based algorithms are showing significant speedups from libraries in the majority of the cases while different algorithms dominate the other scenarios with different matrix size, sparsity, compression factor and operation type. We wrap up in-depth evaluation results and make a recipe to give the best SpGEMM algorithm for target scenario. A critical finding is that hash-table-based SpGEMM gets a significant performance boost if the nonzeros are not required to be sorted within each row of the output matrix.

CUP: Comprehensive User-Space Protection for C/C++

Abstract: Memory corruption vulnerabilities in C/C++ applications enable attackers to execute code, change data, and leak information. Current memory sanitizers do not provide comprehensive coverage of a program»s data. In particular, existing tools focus primarily on heap allocations with limited support for stack allocations and globals. Orthogonally, existing tools focus on the main executable with limited support for system libraries. Existing tools also suffer from both false positives and false negatives. We present Comprehensive User-Space Protection for C/C++, \sysname, an LLVM sanitizer that provides complete spatial and probabilistic temporal memory safety for C/C++ programs on 64-bit architectures (with a prototype implementation for x86\_64). \sysname uses a hybrid metadata scheme that supports all program data including globals, heap, or stack and maintains Application Binary Interface (ABI) compatibility. Existing approaches have false positives and 8%-25% false negatives on the NIST Juliet test suite. In contrast, \sysname has no false negatives or false positives. \sysname instruments all user-space code, including libc and other system libraries, removing these libraries from the trusted computing base. Supporting all of user space allows \sysname to treat a missed check as a failed check, leading to no false negatives for \sysname. The overhead introduced by \sysname is half that of the state-of-the-art full memory protection on benchmarks where both mechanisms run, and imposes 1.58x overhead when compared to baseline on all benchmarks. Consequently, \sysname is intended as a sanitizer for use by system developers, and to protect truly critical systems.

HeapHopper: Bringing Bounded Model Checking to Heap Implementation Security

Abstract: Heap metadata attacks have become one of the primary ways in which attackers exploit memory corruption vulnerabilities. While heap implementation developers have introduced mitigations to prevent and detect corruption, it is still possible for attackers to work around them. In part, this is because these mitigations are created and evaluated without a principled foundation, resulting, in many cases, in complex, inefficient, and ineffective attempts at heap metadata defenses. In this paper, we present HEAPHOPPER, an automated approach, based on model checking and symbolic execution, to analyze the exploitability of heap implementations in the presence of memory corruption. Using HEAPHOPPER, we were able to perform a systematic analysis of different, widely used heap implementations, finding surprising weaknesses in them. Our results show, for instance, how a newly introduced caching mechanism in ptmalloc (the heap allocator implementation used by most of the Linux distributions) significantly weakens its security. Moreover, HEAPHOPPER guided us in implementing and evaluating improvements to the security of ptmalloc, replacing an ineffective recent attempt at the mitigation of a specific form of heap metadata corruption with an effective defense.

Practical memory safety with REST

Abstract: In this paper, we propose Random Embedded Secret Tokens (REST), a simple hardware primitive to provide content-based checks, and show how it can be used to mitigate common types of spatial and temporal memory errors at very low cost. REST is simply a very large random value that is embedded into programs. To provide memory safety, REST is used to bookend data structures during allocation. If the hardware accesses a REST value during execution, due to programming errors or adversarial actions, it reports a privileged memory safety exception. Implementing REST requires 1 bit of metadata per L1 data cache line and a comparator to check for REST tokens during a cache fill. The software infrastructure to provide memory safety with REST reuses a production-quality memory error detection tool, AddressSanitizer, by changing less than 1.5K lines of code. REST based memory safety offers several advantages com-pared to extant methods: (1) it does not require significant redesign of hardware or software, (2) the overhead of heap and stack safety is 2% compared to 40% for AddressSanitizer, (3) the security of the memory safety implementation is improved compared AddressSanitizer, and (4) REST based memory safety can mitigate heap safety errors in legacy binaries without recom-pilation or source code. These advantages provide a significant step towards continuous runtime memory safety monitoring and mitigation for legacy and new binaries.

Object-oriented recovery for non-volatile memory

Abstract: New non-volatile memory (NVM) technologies enable direct, durable storage of data in an application's heap. Durable, randomly accessible memory facilitates the construction of applications that do not lose data at system shutdown or power failure. Existing NVM programming frameworks provide mechanisms to consistently capture a running application's state. They do not, however, fully support object-oriented languages or ensure that the persistent heap is consistent with the environment when the application is restarted. In this paper, we propose a new NVM language extension and runtime system that supports object-oriented NVM programming and avoids the pitfalls of prior approaches. At the heart of our technique is object reconstruction, which transparently restores and reconstructs a persistent object's state during program restart. It is implemented in NVMReconstruction, a Clang/LLVM extension and runtime library that provides: (i) transient fields in persistent objects, (ii) support for virtual functions and function pointers, (iii) direct representation of persistent pointers as virtual addresses, and (iv) type-specific reconstruction of a persistent object during program restart. In addition, NVMReconstruction supports updating an application's code, even if this causes objects to expand, by providing object migration. NVMReconstruction also can compact the persistent heap to reduce fragmentation. In experiments, we demonstrate the versatility and usability of object reconstruction and its low runtime performance cost.

Cache-Oblivious MPI All-to-All Communications Based on Morton Order

Abstract: Many-core systems with a rapidly increasing number of cores pose a significant challenge to parallel applications to use their complex memory hierarchies efficiently. Many such applications rely on collective communications in performance-critical phases, which become a bottleneck if they are not optimized. We address this issue by proposing cache-oblivious algorithms for MPI_Alltoall, MPI_Allgather, and the MPI neighborhood collectives to exploit the data locality. To implement the cache-oblivious algorithms, we allocate the send and receive buffers on a shared heap and use Morton order to guide the memory copies. Our analysis shows that our algorithm for MPI_Alltoall is asymptotically optimal. We show an extension to our algorithms to minimize the communication distance on NUMA systems while maintaining optimality within each socket. We further demonstrate how the cache-oblivious algorithms can be applied to multi-node machines. Experiments are conducted on different many-core architectures. For MPI_Alltoall, our implementation achieves on average 1.40X speedup over the naive implementation based on shared heap for small and medium block sizes (less than 16 KB) on a Xeon Phi KNC, achieves on average 3.03X speedup over MVAPICH2 on a Xeon E7-8890, and achieves on average 2.23X speedup over MVAPICH2 on a 256-node Xeon E5-2680 cluster for block sizes less than 1 KB.

Structuring the Synthesis of Heap-Manipulating Programs - Extended Version

Abstract: This paper describes a deductive approach to synthesizing imperative programs with pointers from declarative specifications expressed in Separation Logic. Our synthesis algorithm takes as input a pair of assertions---a pre- and a postcondition---which describe two states of the symbolic heap, and derives a program that transforms one state into the other, guided by the shape of the heap. The program synthesis algorithm rests on the novel framework of Synthetic Separation Logic (SSL), which generalises the classical notion of heap entailment $\mathcal{P} \vdash \mathcal{Q}$ to incorporate a possibility of transforming a heap satisfying an assertion $\mathcal{P}$ into a heap satisfying an assertion $\mathcal{Q}$. A synthesized program represents a proof term for a transforming entailment statement $\mathcal{P} \leadsto \mathcal{Q}$, and the synthesis procedure corresponds to a proof search. The derived programs are, thus, correct by construction, in the sense that they satisfy the ascribed pre/postconditions, and are accompanied by complete proof derivations, which can be checked independently. We have implemented a proof search engine for SSL in a form the program synthesizer called SuSLik. For efficiency, the engine exploits properties of SSL rules, such as invertibility and commutativity of rule applications on separate heaps, to prune the space of derivations it has to consider. We explain and showcase the use of SSL on characteristic examples, describe the design of SuSLik, and report on our experience of using it to synthesize a series of benchmark programs manipulating heap-based linked data structures.

Influence of man-made edaphotopes of the spoil heap on biota

Abstract: The aim of the work is to investigate the influence of man-made edaphotopes of colliery spoil heap on biota by bioindication method. The physicochemical properties of edaphotopes were investigated: humidity, substrate temperature, humus content at different intervals of their depth and on different sections of the waste heap. The storage technologies of mineral waste in the dumps disregard its further use and transforming that occurs over time. It was revealed that on the northern slope exposures the greatest amount of moisture was concentrated and temperature regimes were acceptable for the activity of the soil biota, which was proved by the largest amount of biomass. New dependences of humus content in edaphotopes of the heap in different intervals of their depth from biomass of humus-forming organisms were established. Slight successional processes on the surface of the heap were observed, where young flora began to develop. This positive phenomenon will contribute to the humus of the genetic horizons and the development of tree and shrub vegetation. Favourable conditions for the development of neo-terrain and the formation of a new landscape in mining regions were confirmed.

L* Algorithm—A Linear Computational Complexity Graph Searching Algorithm for Path Planning

Abstract: The state-of-the-art graph searching algorithm applied to the optimal global path planning problem for mobile robots is the A* algorithm with the heap structured open list. In this paper, we present a novel algorithm, called the L* algorithm, which can be applied to global path planning and is faster than the A* algorithm. The structure of the open list with the use of bidirectional sublists (buckets) ensures the linear computational complexity of the L* algorithm because the nodes in the current bucket can be processed in any sequence and it is not necessary to sort the bucket. Our approach can maintain the optimality and linear computational complexity with the use of the cost expressed by floating-point numbers. The paper presents the requirements of the L* algorithm use and the proof of the admissibility of this algorithm. The experiments confirmed that the L* algorithm is faster than the A* algorithm in various path planning scenarios. We also introduced a method of estimating the execution time of the A* and the L* algorithm. The method was compared with the experimental results.

Geovisualisation of Relief in a Virtual Reality System on the Basis of Low-Level Aerial Imagery

Abstract: The aim of the following paper was to present the geomatic process of transforming low-level aerial imagery obtained with unmanned aerial vehicles (UAV) into a digital terrain model (DTM) and implementing the model into a virtual reality system (VR). The object of the study was a natural aggretage heap of an irregular shape and denivelations up to 11 m. Based on the obtained photos, three point clouds (varying in the level of detail) were generated for the 20,000-m2-area. For further analyses, the researchers selected the point cloud with the best ratio of accuracy to output file size. This choice was made based on seven control points of the heap surveyed in the field and the corresponding points in the generated 3D model. The obtained several-centimetre differences between the control points in the field and the ones from the model might testify to the usefulness of the described algorithm for creating large-scale DTMs for engineering purposes. Finally, the chosen model was implemented into the VR system, which enables the most lifelike exploration of 3D terrain plasticity in real time, thanks to the first person view mode (FPV). In this mode, the user observes an object with the aid of a Head- mounted display (HMD), experiencing the geovisualisation from the inside, and virtually analysing the terrain as a direct animator of the observations.

Heap Leaching: Modelling and Forecasting Using CFD Technology

Abstract: Heap leach operations typically employ some form of modelling and forecasting tools to predict cash flow margins and project viability. However, these vary from simple spreadsheets to phenomenological models, with more complex models not commonly employed as they require the greatest amount of time and effort. Yet, accurate production modelling and forecasting are essential for managing production and potentially critical for successful operation of a complex heap, time and effort spent in setting up modelling tools initially may increase profitability in the long term. A brief overview of various modelling approaches is presented, but this paper focuses on the capabilities of a computational fluid dynamics (CFD) model. Advances in computational capability allow for complex CFD models, coupled with leach kinetic models, to be applied to complex ore bodies. In this paper a comprehensive hydrodynamic CFD model is described and applied to chalcopyrite dissolution under heap operating conditions. The model is parameterized against experimental data and validated against a range of experimental leach tests under different thermal conditions. A three-dimensional ‘virtual’ heap, under fluctuating meteorological conditions, is simulated. Continuous and intermittent irrigation is investigated, showing copper recovery per unit volume of applied leach solution to be slightly increased for pulse irrigation.

HEAP: An Efficient and Fault-Tolerant Authentication and Key Exchange Protocol for Hadoop-Assisted Big Data Platform

Abstract: Hadoop framework has been evolved to manage big data in cloud. Hadoop distributed file system and MapReduce, the vital components of this framework, provide scalable and fault-tolerant big data storage and processing services at a lower cost. However, Hadoop does not provide any robust authentication mechanism for principals’ authentication. In fact, the existing state-of-the-art authentication protocols are vulnerable to various security threats, such as man-in-the-middle, replay, password guessing, stolen-verifier, privileged-insider, identity compromization, impersonation, denial-of-service, online/off-line dictionary, chosen plaintext, workstation compromization, and server-side compromisation attacks. Beside these threats, the state-of-the-art mechanisms lack to address the server-side data integrity and confidentiality issues. In addition to this, most of the existing authentication protocols follow a single-server-based user authentication strategy, which, in fact, originates single point of failure and single point of vulnerability issues. To address these limitations, in this paper, we propose a fault-tolerant authentication protocol suitable for the Hadoop framework, which is called the efficient authentication protocol for Hadoop (HEAP). HEAP alleviates the major issues of the existing state-of-the-art authentication mechanisms, namely operating-system-based authentication, password-based approach, and delegated token-based schemes, respectively, which are presently deployed in Hadoop. HEAP follows two-server-based authentication mechanism. HEAP authenticates the principal based on digital signature generation and verification strategy utilizing both advanced encryption standard and elliptic curve cryptography. The security analysis using both the formal security using the broadly accepted real-or-random (ROR) model and the informal (non-mathematical) security shows that HEAP protects several well-known attacks. In addition, the formal security verification using the widely used automated validation of Internet security protocols and applications ensures that HEAP is resilient against replay and man-in-the-middle attacks. Finally, the performance study contemplates that the overheads incurred in HEAP is reasonable and is also comparable to that of other existing state-of-the-art authentication protocols. High security along with comparable overheads makes HEAP to be robust and practical for a secure access to the big data storage and processing services.

Reconceiving the grain heap: Margins and movements on the market floor

Abstract: This article returns to what was once an ethnographic staple in the sociology of India: the post-harvest grain heap. Having long occupied centre stage in analyses of a moneyless, redistributive transactional order widely known as the jajmani system, it has also been the subject of influential critique, where it has been argued that the misconceived heap sustained a powerful anthropological fiction. Moving beyond these positions, which seem to have left the heap grounded in the past, the grain heap in this work is reconceptualised as a critical entry point and analytic for the study of contemporary commodity markets. Based on long-term ethnographic fieldwork in an agricultural market (mandi) in Madhya Pradesh, it finds that it is along the seams or internal margins of the market, at routine sites of physical transfer and exchange, assembly and dispersal, integration and disruption, that heaps of agricultural produce materialise. An analysis of critical aspects of the heap—its position, composition, measure...

High-performance sparse matrix-matrix products on Intel KNL and multicore architectures

Abstract: Sparse matrix-matrix multiplication (SpGEMM) is a computational primitive that is widely used in areas ranging from traditional numerical applications to recent big data analysis and machine learning. Although many SpGEMM algorithms have been proposed, hardware specific optimizations for multi- and many-core processors are lacking and a detailed analysis of their performance under various use cases and matrices is not available. We firstly identify and mitigate multiple bottlenecks with memory management and thread scheduling on Intel Xeon Phi (Knights Landing or KNL). Specifically targeting multi- and many-core processors, we develop a hash-table-based algorithm and optimize a heap-based shared-memory SpGEMM algorithm. We examine their performance together with other publicly available codes. Different from the literature, our evaluation also includes use cases that are representative of real graph algorithms, such as multi-source breadth-first search or triangle counting. Our hash-table and heap-based algorithms are showing significant speedups from libraries in the majority of the cases while different algorithms dominate the other scenarios with different matrix size, sparsity, compression factor and operation type. We wrap up in-depth evaluation results and make a recipe to give the best SpGEMM algorithm for target scenario. A critical finding is that hash-table-based SpGEMM gets a significant performance boost if the nonzeros are not required to be sorted within each row of the output matrix.

The benefits and costs of writing a POSIX kernel in a high-level language

Abstract: This paper presents an evaluation of the use of a high-level language (HLL) with garbage collection to implement a monolithic POSIX-style kernel. The goal is to explore if it is reasonable to use an HLL instead of C for such kernels, by examining performance costs, implementation challenges, and programmability and safety benefits.The paper contributes Biscuit, a kernel written in Go that implements enough of POSIX (virtual memory, mmap, TCP/IP sockets, a logging file system, poll, etc.) to execute significant applications. Biscuit makes liberal use of Go's HLL features (closures, channels, maps, interfaces, garbage collected heap allocation), which subjectively made programming easier. The most challenging puzzle was handling the possibility of running out of kernel heap memory; Biscuit benefited from the analyzability of Go source to address this challenge.On a set of kernel-intensive benchmarks (including NGINX and Redis) the fraction of kernel CPU time Biscuit spends on HLL features (primarily garbage collection and thread stack expansion checks) ranges up to 13%. The longest single GC-related pause suffered by NGINX was 115 microseconds; the longest observed sum of GC delays to a complete NGINX client request was 600 microseconds. In experiments comparing nearly identical system call, page fault, and context switch code paths written in Go and C, the Go version was 5% to 15% slower.

Shooting from the heap: ultra-scalable static analysis with heap snapshots

Abstract: Traditional whole-program static analysis (e.g., a points-to analysis that models the heap) encounters scalability problems for realistic applications. We propose a ``featherweight'' analysis that combines a dynamic snapshot of the heap with otherwise full static analysis of program behavior. The analysis is extremely scalable, offering speedups of well over 3x, with complexity empirically evaluated to grow linearly relative to the number of reachable methods. The analysis is also an excellent tradeoff of precision and recall (relative to different dynamic executions): while it can never fully capture all program behaviors (i.e., it cannot match the near-perfect recall of a full static analysis) it often approaches it closely while achieving much higher (3.5x) precision.

Hydrometallurgical Processing of Low-Grade Sulfide Ore and Mine Waste in the Arctic Regions: Perspectives and Challenges

Abstract: The authors describe the opportunities of low-grade sulfide ores and mine waste processing with heap and bacterial leaching methods. By the example of gold and silver ores, we analyzed specific issues and processing technologies for heap leaching intensification in severe climatic conditions. The paper presents perspectives for heap leaching of sulfide and mixed ores from the Udokan (Russia) and Talvivaara (Finland) deposits, as well as technogenic waste dumps, namely, the Allarechensky Deposit Dumps (Russia). The paper also shows the laboratory results of non-ferrous metals leaching from low-grade copper-nickel ores of the Monchepluton area, and from tailings of JSC Kola Mining and Metallurgical Company.

Poster: Testing Heap-Based Programs with Java StarFinder

Abstract: We present Java StarFinder (JSF), a tool for automated test case generation and error detection for Java programs having inputs in the form of complex heap-manipulating data structures. The core of JSF is a symbolic execution engine that uses separation logic with existential quantifiers and inductively-defined predicates to precisely represent the (unbounded) symbolic heap. The feasibility of a heap con figuration is checked by a satisfiability solver for separation logic. At the end of each feasible path, a concrete model of the symbolic heap (returned by the solver) is used to generate a test case, e.g., a linked list or an AVL tree, that exercises that path. We show the effectiveness of JSF by applying it on non-trivial heap-manipulating programs and evaluated it against JBSE, a state-of-the-art symbolic execution engine for heap-based programs. Experimental results show that our tool significantly reduces the number of invalid test inputs and improves the test coverage.

Smooth heaps and a dual view of self-adjusting data structures

Abstract: We present a new connection between self-adjusting binary search trees (BSTs) and heaps, two fundamental, extensively studied, and practically relevant families of data structures (Allen,Munro, 1978; Sleator, Tarjan, 1983; Fredman, Sedgewick, Sleator, Tarjan, 1986; Wilber, 1989; Fredman, 1999; Iacono, Ozkan, 2014). Roughly speaking, we map an arbitrary heap algorithm within a broad and natural model, to a corresponding BST algorithm with the same cost on a dual sequence of operations (i.e. the same sequence with the roles of time and key-space switched). This is the first general transformation between the two families of data structures. There is a rich theory of dynamic optimality for BSTs (i.e. the theory of competitiveness between BST algorithms). The lack of an analogous theory for heaps has been noted in the literature (e.g. Pettie; 2005, 2008). Through our connection, we transfer all instance-specific lower bounds known for BSTs to a general model of heaps, initiating a theory of dynamic optimality for heaps. On the algorithmic side, we obtain a new, simple and efficient heap algorithm, which we call the smooth heap. We show the smooth heap to be the heap-counterpart of Greedy, the BST algorithm with the strongest proven and conjectured properties from the literature, conjectured to be instance-optimal (Lucas, 1988; Munro, 2000; Demaine et al., 2009). Assuming the optimality of Greedy, the smooth heap is also optimal within our model of heap algorithms. Intriguingly, the smooth heap, although derived from a non-practical BST algorithm, is simple and easy to implement (e.g. it stores no auxiliary data besides the keys and tree pointers). It can be seen as a variation on the popular pairing heap data structure, extending it with a “power-of-two-choices” type of heuristic. For the smooth heap we obtain instance-specific upper bounds, with applications in adaptive sorting, and we see it as a promising candidate for the long-standing question of a simpler alternative to Fibonacci heaps.

Modular Termination Verification of Single-Threaded and Multithreaded Programs

Abstract: We propose an approach for the modular specification and verification of total correctness properties of object-oriented programs. The core of our approach is a specification style that prescribes a way to assign a level expression to each method such that each callee’s level is below the caller’s, even in the presence of dynamic binding. The specification style yields specifications that properly hide implementation details. The main idea is to use multisets of method names as levels, and to associate with each object levels that abstractly reflect the way the object is built from other objects. A method’s level is then defined in terms of the method’s own name and the levels associated with the objects passed as arguments.We first present the specification style in the context of programs that do not modify object fields. We then combine it with separation logic and abstract predicate families to obtain an approach for programs with heap mutation. In a third step, we address concurrency, by incorporating an existing approach for verifying deadlock freedom of channels and locks. Our main contribution here is to achieve information hiding by using the proposed termination levels for lock ordering as well. Also, we introduce call permissions to enable elegant verification of termination of programs where threads cause work in other threads, such as in thread pools or fine-grained concurrent algorithms involving compare-and-swap loops.We explain how our approach can be used also to verify the liveness of nonterminating programs.

User-defined Classification and Multi-level Grouping of Objects in Memory Monitoring

Abstract: Software becomes more and more complex. Performance degradations and anomalies can often only be understood by using monitoring approaches, e.g., for tracing the allocations and lifetimes of objects on the heap. However, this leads to huge amounts of data that have to be classified, grouped and visualized in order to be useful for developers. In this paper, we present a flexible offline memory analysis approach that allows classifying heap objects based on arbitrary criteria. A small set of predefined classification criteria such as the type and the allocation site of an object can further be extended by additional user-defined criteria. In contrast to state-of-the-art tools, which group objects based on a single criterion, our approach allows the combination of multiple criteria using multi-level grouping. The resulting classification trees allow a flexible in-depth analysis of the data and a natural hierarchical visualization of the results.