Nick Sumner's Research

Dual Execution For On The Fly Fine Grained Execution Comparison

wsumner@sfu.ca (Nick Sumner) — Sat, 14 Mar 2015 00:00:00 -0700

Execution comparison has many applications in debugging, malware analysis, software feature identification, and intrusion detection. Existing comparison techniques have various limitations. Some can only compare at the system event level and require executions to take the same input. Some require storing instruction traces that are very space-consuming and have difficulty dealing with nondeterminism. In this paper, we propose a novel dual execution technique that allows on-the-fly comparison at the instruction level. Only differences between the executions are recorded. It allows executions to proceed in a coupled mode such that they share the same input sequence with the same timing, reducing nondeterminism. It also allows them to proceed in a decoupled mode such that the user can interact with each one differently. Decoupled executions can be recoupled to share the same future inputs and facilitate further comparison. We have implemented a prototype and applied it to identifying functional components for reuse, comparative debugging with new GDB primitives, and understanding real world regression failures. Our results show that dual execution is a critical enabling technique for execution comparison.

Reuse Oriented Reverse Engineering Of Functional Components From X86 Binaries

wsumner@sfu.ca (Nick Sumner) — Sat, 31 May 2014 00:00:00 -0700

Locating, extracting, and reusing the implementation of a feature within an existing binary program is challenging. This paper proposes a novel algorithm to identify modular functions corresponding to such features and to provide usable interfaces for the extracted functions. We provide a way to represent a desired feature with two executions that both execute the feature but with different inputs. Instead of reverse engineering the interface of a function, we wrap the existing interface and provide a simpler and more intuitive interface for the function through concretization and redirection. Experiments show that our technique can be applied to extract varied features from several real world applications including a malicious application.

Identifying Execution Points For Dynamic Analyses

wsumner@sfu.ca (Nick Sumner) — Tue, 12 Nov 2013 00:00:00 -0800

Dynamic analyses rely on the ability to identify points within or across executions. In spite of this being a core task for dynamic analyses, new solutions are frequently developed without an awareness of existing solutions, their strengths, their weaknesses, or their caveats. This paper surveys the existing approaches for identifying execution points and examines their analytical and empirical properties that researchers and developers should be aware of when using them within an analysis. In addition, based on limitations in precision, correctness, and efficiency for techniques that identify corresponding execution points across multiple executions, we designed and implemented a new technique, Precise Execution Point IDs. This technique avoids correctness and precision issues in prior solutions, enabling analyses that use our approach to also produce more correct results. Empirical comparison with the surveyed techniques shows that our approach has 25% overhead on average, several times less than existing solutions.

DRIP: A Framework for Purifying Trojaned Kernel Drivers

wsumner@sfu.ca (Nick Sumner) — Mon, 24 Jun 2013 00:00:00 -0700

Kernel drivers are usually provided in the form of loadable kernel extensions, which can be loaded/unloaded dynamically at runtime and execute with the same privilege as the core operating system kernel. The unrestricted security access from the drivers to the kernel is nevertheless a double-edged sword that makes them become susceptible targets of driver trojan attacks. Given a benign driver, it is now easy to implant malicious logic with existing hacking tools at hand. Once implanted, such malicious logic will be difficult to detect. In this paper we propose DRIP, a framework for detecting and eliminating malicious logic embedded in a kernel driver through iteratively eliminating unnecessary kernel API invocations from the driver. When provided with the binary of a trojaned driver, DRIP generates a purified driver with benign functionalities preserved and malicious ones eliminated. Our evaluation shows that DRIP successfully eliminates malicious effects of trojaned drivers in the system, with the purified drivers maintaining or even improving their performance over the trojaned driver.

Lucid Debugging

wsumner@sfu.ca (Nick Sumner) — Sat, 18 May 2013 00:00:00 -0700

Lucid Debugging provides a framework for automatically summarizing and explaining observed software failures. Exploiting fine-grained execution comparison allows for concise and precise explanations for how a fault in a program caused an observed failure.

Comparative Causality: Explaining The Differences Between Executions

wsumner@sfu.ca (Nick Sumner) — Sat, 18 May 2013 00:00:00 -0700

We propose a novel fine-grained causal inference technique. Given two executions and some observed differences between them, the technique reasons about the causes of such differences. The technique does so by state replacement, i.e. replacing part of the program state at an earlier point to observe whether the target differences can be induced. It makes a number of key advances: it features a novel execution model that avoids undesirable entangling of the replaced state and the original state; it properly handles differences of omission by symmetrically analyzing both executions; it also leverages a recently developed slicing technique to limit the scope of causality testing while ensuring that no relevant state causes can be missed. The application of the technique on automated debugging shows that it substantially improves the precision and efficiency of causal inference compared to state of the art techniques.

Modeling Software Execution Environment

wsumner@sfu.ca (Nick Sumner) — Thu, 18 Oct 2012 00:00:00 -0700

Software execution environment, interfaced with software through library functions and system calls, constitutes an important aspect of the software's semantics. Software analysis ought to take the execution environment into consideration. However, due to lack of source code and the inherent implementation complexity of these functions, it is quite difficult to co-analyze software and its environment. In this paper, we propose to extend program synthesis techniques to construct models for system and library functions. The technique samples the behavior of the original implementation of a function. The samples are used as the specification to synthesize the model, which is a C program. The generated model is iteratively refined. We have developed a prototype that can successfully construct models for a pool of system and library calls from their real world complex binary implementations. Moreover, our experiments have shown that the constructed models can improve dynamic test generation and failure tolerance.

Libepid

wsumner@sfu.ca (Nick Sumner) — Wed, 10 Oct 2012 00:00:00 -0700

Libepid provides an LLVM library for identifying execution points at runtime. Different abstractions like calling contexts and execution indices allow for a balance between efficiency and precision.

Libepid on Google Code

Precise Calling Context Encoding

wsumner@sfu.ca (Nick Sumner) — Sat, 01 Sep 2012 00:00:00 -0700

Calling contexts are very important for a wide range of applications such as profiling, debugging, and event logging. Most applications perform expensive stack walking to recover contexts. The resulting contexts are often explicitly represented as a sequence of call sites and hence bulky. We propose a technique to encode the current calling context of any point during an execution. In particular, an acyclic call path is encoded into one number through only integer additions. Recursive call paths are divided into acyclic subsequences and encoded independently. We leverage stack depth in a safe way to optimize encoding: if a calling context can be safely and uniquely identified by its stack depth, we do not perform encoding. We propose an algorithm to seamlessly fuse encoding and stack depth based identification. The algorithm is safe because different contexts are guaranteed to have different IDs. It also ensures contexts can be faithfully decoded. Our experiments show that our technique incurs negligible overhead (1.89% on average). For most medium-sized programs, it can encode all contexts with just one number. For large programs, we are able to encode most calling contexts to a few numbers.

Comparing Executions To Explain Software Failures

wsumner@sfu.ca (Nick Sumner) — Thu, 02 Feb 2012 00:00:00 -0800

Marathon: Detecting Atomic Set Serializability Violations With Conflict Graphs

wsumner@sfu.ca (Nick Sumner) — Wed, 28 Sep 2011 00:00:00 -0700

Recent research has proposed several analyses to mitigate the fact that finding concurrency bugs in multi-threaded software is notoriously hard. This work proposes a new analysis based on a correctness criterion called "atomic-set serializability", which incorporates both race conditions and traditional atomicity/serializability. We present a novel analysis based on conflict cycle detection that is guaranteed to find all violations in the intercepted execution trace. A set of heuristics automatically determines all annotations required for atomic-set serializability. We implemented the analysis and evaluated it on a suite consisting of real programs and benchmarks. The evaluation demonstrates the usefulness of our heuristics by finding a number of known (as well as new) violations with competitive overhead and a very low false positive rate.

Selecting Peers For Execution Comparison

wsumner@sfu.ca (Nick Sumner) — Thu, 21 Jul 2011 00:00:00 -0700

Execution comparison is becoming more common as a means of debugging faulty programs or simply explaining program behavior. Often, such as when debugging, the goal is to understand particular aspects of a single execution, and it is not immediately clear against what we should compare this execution. Prior work has led to approaches for acquiring a second execution, or peer, with which to compare the first. The earliest of these searched test suites for suitable candidates. More recent work focuses on synthesizing a new execution, either by generating new input for the program or by directly mutating the execution itself. In spite of these proposals, it is unclear what advantages these different techniques for finding peers might have over each other. In this paper, we implement five different existing techniques and examine their impact on 20 real bugs. These bugs represent the full set of reported bugs for three programs during one year. We present a metric to evaluate the quality of the peers. It is based on the similarity of the peers to the executions of the patched programs. We also discuss in detail the different scenarios where these techniques hold advantages.

Unified Debugging Of Distributed Systems With Recon

wsumner@sfu.ca (Nick Sumner) — Tue, 28 Jun 2011 00:00:00 -0700

To scale to today's complex distributed software systems, debugging and replaying techniques mostly focus on single facets of software, e.g., local concurrency, distributed messaging, or data representation. This forces developers to tediously combine different technologies such as instruction-level dynamic tracing, event log analysis, or global state reconstruction to gradually explain non-trivial defects. This paper proposes Recon, a debugging system that provides iterative and interactive homogeneous debugging services. As related systems, Recon promotes SQL-like queries for debugging distributed systems. Unlike other approaches, however, Recon allows for all system artifacts including nodes, communication channels, events, or instructions to be uniformly described by relations. Also, an application in Recon originally runs with a lightweight logger that only collects replay logs for individual nodes. Developers debug a complete program by replaying the execution with fine-grained instrumentation that is capable of exposing instruction-level information. We illustrate the effectiveness of Recon on programs as diverse as BerkeleyDB, i3/Chord, RandTree, and Pastry. Our evaluation includes executions in local clusters as well as in Amazon EC2 and exhibits an unreported bug in RandTree.

Toward Generating Reducible Replay Logs

wsumner@sfu.ca (Nick Sumner) — Tue, 07 Jun 2011 00:00:00 -0700

Logging and replay is important to reproducing software failures and recovering from failures. Replaying a long execution is time consuming, especially when replay is further integrated with runtime techniques that require expensive instrumentation, such as dependence detection. In this paper, we propose a technique to reduce a replay log while retaining its ability to reproduce a failure. While traditional logging records only system calls and signals, our technique leverages the compiler to selectively collect additional information on the fly. Upon a failure, the log can be reduced by analyzing itself. The collection is highly optimized. The runtime overhead is trivial (2.6% average) and the size of additional log is comparable to the original log. Substantial reduction can be cost effectively achieved through a search based algorithm. The reduced log is guaranteed to reproduce the failure.

Coalescing Executions For Fast Uncertainty Analysis

wsumner@sfu.ca (Nick Sumner) — Fri, 27 May 2011 00:00:00 -0700

Uncertain data processing is critical in a wide range of applications such as scientific computation handling data with inevitable errors and financial decision making relying on human provided parameters. While increasingly studied in the area of databases, uncertain data processing is often carried out by software, and thus software based solutions are attractive. In particular, Monte Carlo (MC) methods execute software with many samples from the uncertain inputs and observe the statistical behavior of the output. In this paper, we propose a technique to improve the cost-effectiveness of MC methods. Assuming only part of the input is uncertain, the certain part of the input always leads to the same execution across multiple sample runs. We remove such redundancy by coalescing multiple sample runs in a single run. In the coalesced run, the program operates on a vector of values if uncertainty is present and a single value otherwise. We handle cases where control flow and pointers are uncertain. Our results show that we can speed up the execution time of 30 sample runs by an average factor of 2.3 without precision lost or by up to 3.4 with negligible precision lost.

Memory Indexing: Canonicalizing Addresses Across Executions

wsumner@sfu.ca (Nick Sumner) — Wed, 10 Nov 2010 00:00:00 -0800

Fine-grained program execution comparison examines different executions generated by different program versions, different inputs, or by perturbations. It has a wide range of applications in debugging, regression testing, program comprehension, and security. Meaningful comparison demands that executions are aligned before they are compared, otherwise the resulting differences do not reflect semantic differences. Prior work has focused on aligning executions along the control flow dimension. In this paper, we observe that the memory dimension is also critical and propose a novel solution to align memory locations across different executions. We introduce a canonical representation for memory locations and pointer values called memory indexing. Aligned memory locations across runs share the same index. We formally define the semantics of memory indexing and present a cost-effective design. We also show that memory indexing overcomes an important challenge in automated debugging by enabling robust state replacement across runs.

Analyzing Concurrency Bugs Using Dual Slicing

wsumner@sfu.ca (Nick Sumner) — Fri, 16 Jul 2010 00:00:00 -0700

Recently, there has been much interest in developing analyzes to detect concurrency bugs that arise because of data races, atomicity violations, execution omission, etc. However, determining whether reported bugs are in fact real, and understanding how these bugs lead to incorrect behavior, remains a labor-intensive process. This paper proposes a novel dynamic analysis that automatically produces the causal path of a concurrent failure leading from the root cause to the failure. Given two schedules, one inducing the failure and the other not, our technique collects traces of the two executions, and compares them to identify salient differences. The causal relation between the differences is disclosed by leveraging a novel slicing algorithm called dual slicing that slices both executions alternatively and iteratively, producing a slice containing trace differences from both runs. Our experiments show that dual slices tend to be very small, often an order of magnitude or more smaller than the corresponding dynamic slices; more importantly, they enable precise analysis of real concurrency bugs for large programs, with reasonable overhead.

Precise Calling Context Encoding

wsumner@sfu.ca (Nick Sumner) — Sat, 08 May 2010 00:00:00 -0700

Algorithms For Automatically Computing The Causal Paths Of Failures

wsumner@sfu.ca (Nick Sumner) — Sun, 29 Mar 2009 00:00:00 -0700

We have proposed an automated debugging technique that explains a failure by computing its causal path leading from the root cause to the failure. Given a failing execution, the technique first searches for a dynamic patch. Fine-grained execution comparison between the failing run and the patched run is performed to isolate the causal path. The comparison is enabled by precisely aligning the two executions. We herein propose and study two algorithms aiming at efficiency. We also evaluate the effectiveness and cost of our technique on a set of real bugs, including requirement bugs in which no a single or small set of statements can be blamed as the root cause. In such cases, understanding a failure is more important.

Efficient Program Execution Indexing

wsumner@sfu.ca (Nick Sumner) — Fri, 06 Jun 2008 00:00:00 -0700

Update: If you have used execution indexing in the past, you will likely wish to look at the PEPID paper that resolves some of the efficiency and accuracy issues present in this original work.

Execution indexing uniquely identifies a point in an execution. Desirable execution indices reveal correlations between points in an execution and establish correspondence between points across multiple executions. Therefore, execution indexing is essential for a wide variety of dynamic program analyses, for example, it can be used to organize program profiles; it can precisely identify the point in a re-execution that corresponds to a given point in an original execution and thus facilitate debugging or dynamic instrumentation. In this paper, we formally define the concept of execution index and propose an indexing scheme based on execution structure and program state. We present a highly optimized online implementation of the technique. We also perform a client study, which targets producing a failure inducing schedule for a data race by verifying the two alternative happens-before orderings of a racing pair. Indexing is used to precisely locate corresponding points across multiple executions in the presence of non-determinism so that no heavyweight tracing/replay system is needed.