Compositional Shape Analysis by Means of Bi-Abduction 1: The Best Abductive Solution for Points-To Symbolic Heaps Is a Computable Disjunction of Symbolic HeapsResearch Paper
Motivation
Automatic verification of programs that manipulate pointers needs procedure specifications: preconditions saying which part of the heap a procedure touches and postconditions saying what it leaves behind. Writing them by hand does not scale to millions of lines of code. Calcagno, Distefano, O'Hearn and Yang (J. ACM 58(6), 2011) infer them compositionally with bi-abduction: when symbolic execution reaches a call whose precondition is not implied by the current state, it asks for the missing memory that would make it hold. This abduction question in separation logic,
is the engine behind the Abductor tool and, later, Facebook Infer.
The question has many solutions ( is always one), so its quality matters. The paper defines an order on solutions that prefers less memory and then logical strength, shows that a best solution always exists as a semantic object, and then asks whether it can be written down and computed in the language that the analysis actually manipulates: symbolic heaps. For the fragment whose only spatial predicate is points-to, it answers yes. That answer, Theorem 3.25, is the goal of this mission.
Setting
Values are natural numbers; is nil and the nonzero naturals are the locations. A heap is a finite partial map from locations to values; a stack assigns a value to every variable; a state is a pair . Write when have disjoint domains and is their union. A predicate is an arbitrary set of states. On predicates:
- iff with and ;
- iff for every disjoint from with , we have ;
- holds in the empty heap, everywhere, and , , , , are the Boolean and quantifier operations; is inclusion.
Symbolic heaps of the Points-to Instantiation. Expressions are variables or natural-number constants. A pure formula is a conjunction of and . A spatial formula is a -conjunction of points-to facts , possibly with ; here iff and is the one-cell heap . A symbolic heap is , and a disjunction . In the abduction question (5), , the left-hand side is quantifier-free.
Orders. iff (spatially smaller), and
keeps the states of no proper subheap of which (with the same stack) is in . says that some separate heap satisfies , and the compatible preorder is . Subtraction is .
Formalization targets
Goal: Theorem 3.25
There is a single algorithm such that for every quantifier-free and symbolic heap of the Points-to Instantiation, the disjunction satisfies
Both halves are part of the goal: the best solution is expressible as a disjunction of symbolic heaps, and it is computable.
Milestones, in the order the paper's argument uses them
- Lemma 3.16. If is a minimal solution w.r.t. , then is the minimal solution w.r.t. .
- Figure 2 derives solutions (§3.4.2). If is derivable, then .
- Lemma 3.18. For all some with is derivable, and the proof search terminates.
- Lemma 3.19. If is derivable and , then .
- Lemma 3.20. and are semantically equivalent.
- Lemma 3.23. Six algebraic properties of subtraction, including with .
- Quantifier removal for pure formulae (§3.4.4). is equivalent to a computable pure formula without .
- Subtraction with quantifiers (§3.4.4). is a computable disjunction of symbolic heaps.
Significance
The result. Semantically the best solution is (Theorem 3.13 of the paper), a formula that nests and negation around , which no automatic separation-logic prover handles and which says nothing directly about the shape of the missing memory. Theorem 3.25 shows that in the points-to fragment this object collapses to a finite disjunction of symbolic heaps, the format in which an analysis draws "pictures of memory", and that it can be computed. It gives an optimal abduction procedure against which the paper's heuristic procedure can be measured.
Formalizing it. The theorem is proved in the paper, but several steps are given as sketches ("the proof of each of these properties is straightforward", "a straightforward exercise"), Lemma 3.18 has only a proof sketch, and the quantifier-free subtraction display on p. 33 is incomplete as printed (it inspects only the points-to facts and not the pure part of the subtracted heap). A machine-checked development would supply the missing arguments, fix the subtraction step, and give a verified reference implementation of optimal abduction for points-to symbolic heaps. No part of it is known to have been formalized.
Difficulty
The obvious route is to compute directly, but quantifies over all heaps, and symbolic heaps have no syntax for it. The paper splits the problem into three phases (a -minimal solution from a proof system, the incompatible solutions, and ), and each phase carries its own difficulty. The proof system's minimality (Lemma 3.19) must hold against arbitrary semantic predicates, not only symbolic heaps, and it fails as soon as may denote more than one heap for a given stack (the paper notes that adding list segments breaks it). Computing needs subtraction of symbolic heaps with quantifiers, which in turn needs quantifier elimination for Boolean combinations of equalities over an infinite domain, and the paper's quantifier-free subtraction step has to be repaired. Showing that the final disjunction is -least, not merely a solution, needs all three phases.
Formalization scope
All statements live in the namespace BiAbduction.Systematic. Values are ℕ with nil 0, locations are the nonzero naturals, variables are ℕ (program and logical variables form one sort), and heaps are maps ℕ → Option ℕ with 0 unallocated and finite domain. Finiteness of heaps is essential, since relies on well-foundedness of the subheap order. Predicates are arbitrary Set (Stack × Heap). The syntax is built from ℕ, Bool, Sum, Prod and List by abbreviations, so "computed by an algorithm" is Mathlib's Computable₂ with the standard Primcodable encodings. The left-hand side has no true conjunct, as in the paper's proofs; the right-hand side may have one. Figure 2 is an inductive relation in which psto may match any right-hand points-to fact. The soundness and -minimality statements assume the paper's bound-variable convention (no variable bound in occurs in ); the goal does not need it.
The goal cannot be met by a degenerate reading. The competitors range over all predicates, not only over disjunctions, and computability refers to fixed standard encodings. The points-to semantics excludes nil addresses, and heaps are finite.
A complete development needs the separation-logic algebra of predicates (associativity and commutativity of , the adjunction between and ), properties of under finite heaps, the soundness and minimality of the proof system, quantifier elimination for equality logic, and a computability argument for list-manipulating procedures. The semantic layer and the quantifier-elimination result are reusable beyond this mission. Contributions to any milestone, and to the supporting lemmas of §3.3 (Lemmas 3.10–3.11, Theorem 3.13), are welcome.
Theorem numbers and page references are those of the accepted manuscript of the J. ACM article (72 pages), not of the typeset journal version.
Selected references
- C. Calcagno, D. Distefano, P. W. O'Hearn, H. Yang, Compositional Shape Analysis by means of Bi-Abduction, Journal of the ACM 58(6), Article 26, 2011. https://doi.org/10.1145/2049697.2049700
- C. Calcagno, D. Distefano, P. W. O'Hearn, H. Yang, Compositional Shape Analysis by means of Bi-Abduction, POPL 2009. https://doi.org/10.1145/1480881.1480917
- J. Berdine, C. Calcagno, P. W. O'Hearn, Symbolic Execution with Separation Logic, APLAS 2005. https://doi.org/10.1007/11575467_5
- J. C. Reynolds, Separation Logic: A Logic for Shared Mutable Data Structures, LICS 2002. https://doi.org/10.1109/LICS.2002.1029817
- D. Pym, P. W. O'Hearn, H. Yang, Possible Worlds and Resources: The Semantics of BI, Theoretical Computer Science 315(1), 2004. https://doi.org/10.1016/j.tcs.2003.11.020