Efﬁcient Intuitionistic Theorem Proving with the Polarized Inverse Method

. The inverse method is a generic proof search procedure applicable to non-classical logics satisfying cut elimination and the subformula property. In this paper we describe a general architecture and several high-level optimizations that enable its efﬁcient implementation. Some of these rely on logic-speciﬁc properties, such as polarization and focusing, which have been shown to hold in a wide range of non-classical logics. Others, such as rule subsumption and recursive backward subsumption apply in general. We empirically evaluate our techniques on ﬁrst-order intuitionistic logic with our implementation Imogen and demonstrate a substantial improvement over all other existing intuitionistic theorem provers on problems from the ILTP problem library.


Introduction
The inverse method [11,6] uses forward saturation, generalizing resolution to nonclassical logics satisfying the subformula property and cut elimination.Focusing [1,10] reduces the search space in a sequent calculus by restricting the application of inference rules based on the polarities [9] of the connectives and atomic formulas.In this paper we describe a framework for reasoning in such logics, and exhibit a concrete implementation of a theorem prover for intuitionistic predicate logic, called Imogen, that is by some measure the most effective first order intuitionistic theorem prover: on the ILTP library of intuitionistic challenge problems [17], a collection similar to the well known TPTP [18] library, Imogen solves over 100 more problems than its closest competitor ileanCoP [13]. 1his work continues a line of research on building efficient theorem provers for nonclassical logics using the inverse method, following Tammet [19] (for intuitionistic and linear logic), Linprover [4,3] (for intuitionistic linear logic), and the early propositional version of Imogen [12].We briefly highlight the principal contributions of this paper.On the logical side, explicit polarization of a given input formula determines basic characteristics of the search space, refining ideas from Chaudhuri et al. [5].Our architecture provides a clean interface between the specification of basic logical inference (the front end) on one side and saturating proof search (the back end) on the other.This separation allows for generic logic-independent optimizations.Our back end maintains dynamic collections of sequents and inference rules, both of which are subject to redundancy elimination and heuristic selection.Novel redundancy elimination techniques include inference rule subsumption and recursive backward subsumption.We conclude with an extensive analysis of the effects of individual optimizations.

Backward Polarized Sequent Calculus
The inverse method upon which our methods are based is a forward saturation strategy.However, we limit the search space of the inverse method by searching only for focused proofs [1] that could be found in the backwards direction.To make the relation clear, in this section we give the rules for the (ground) backward polarized sequent calculus.This ground calculus will then be lifted to a free variable calculus and proof search will proceed in the forward direction, from the initial sequents to the goal, following the inverse method recipe [6].One novelty of our approach is the use of explicit polarization in formulas that syntactically mark the polarity of the atoms and connectives.We first describe polarized formulas, and then show the backward sequent calculus.

Polarized Formulas
A connective is positive if its left rule in the sequent calculus is invertible and negative if its right rule is invertible.As shown below, our proof search fundamentally depends on the polarity of connectives.In intuitionistic logic, the status of conjunction and truth is ambiguous in the sense that they are both positive and negative, while their status in linear logic is uniquely determined.We therefore syntactically distinguish positive and negative formulas with so-called shift operators [9] explicitly coercing between them.Even though we use the notation of linear logic, the behavior of the connectives is not linear.
In the following, the meta-variable P ranges over atomic formulas which have the form p(t 1 , . . ., t n ) for predicates p.Note also that both in formulas and sequents, the signs are not actual syntax but mnemonic guides to the reader.

Positive formulas A
The translation A − of an (unpolarized) formula F in intuitionistic logic is nondeterministic, subject only to the constraints that the erasure defined below coincides with the original formula: |A − | = F , and all predicates are assigned a consistent polarity.

Erasure of polarized formulas
For example, the formula ((p ∨ r) ∧ (q ⊃ r)) ⊃ (p ⊃ q) ⊃ r can be interpreted as any of the following polarized formulas (among others): Shift operators have highest binding precedence in our presentation of the examples.As we will see from the inference rules given below, the choice of translation determines the search behavior on the resulting polarized formula.Different choices can lead to search spaces with radically different structure [5,12].

Backward Polarized Sequent Calculus
The backward calculus is a refinement of Gentzen's LJ that eliminates don't-care nondeterministic choices, and manages don't-know nondeterminism by chaining such inferences in sequence.Andreoli was the first to define this focusing strategy and prove it complete [1], and similar proofs for other logics soon followed [8,10,20].Therefore, polarization can be applied to optimize search in a wide variety of logics.The polarized calculus is defined via four mutually recursive judgments.In the judgments, we separate the antecedents into positive and negative zones.We write Γ for an unordered collection of negative formulas or positive atoms.Dually, C stands for a positive formula or a negative atom.
The first two judgments concern formulas with invertible rules on the right and left.Together, the two judgments form the inversion phase of focusing.In the rules RA-∀ and LA-∃, a is a new parameter. 2The context ∆ + is consists entirely of positive formulas and is ordered so that inference rules can only be applied to the rightmost formula, eliminating don't-care nondeterminism.
The next two judgments are concerned with non-invertible rules.These two judgments make up the focusing phase.

Γ
A + (Right Focusing) Backward search for a proof of A − would start with inversion from •; • =⇒ A − ; • and then alternate between focusing and inversion phases.Call a focusing phase followed by an inversion phase a block.The boundary between blocks is of particular importance.The sequents at the boundary have the form Γ ; • =⇒ •; C, which we call stable sequents.There are two rules that control the phase changes and make the choices at block boundaries.
An example of a backward derivation highlighting the block structure, is shown in Figure 2. a, b, and c are negative atoms.The elided sections are deterministic application of the above rules.Note that the nondeterminism occurs only at block boundaries and during focusing choices.

Fig. 2. Backward proof, with blocks
Theorem 1 [10] If there exists a intuitionistic derivation of A, then for any polarization A − of A, there exists a focused derivation of •; • =⇒ A − ; •.

Synthetic Connectives and Derived Rules
We have already observed that backward proofs have the property that the proof is broken into blocks, with stable sequents at the boundary.The only rules applicable to stable sequents are the rules that select a formula on which to focus.It is the formulas occurring in stable sequents that form the primary objects of our further inquiry.It helps to think of such formulas, abstracted over their free variables, as synthetic connectives [2].Define the synthetic connectives of a formula A as all subformulas of A that could appear in stable sequents in a focused backward proof.In a change of perspective, we can consider each block of a proof as the application of a left or right rule for a synthetic connective.The rules operating on synthetic connectives are derived from the rules for its constituent formulas.We can thus consider a backward proof as a proof using only these synthetic (derived) rules.Each derived rule then corresponds to a block of the original proof.
Since we need only consider stable sequents and synthetic connectives, we can simplify notation, and ignore the (empty) positive left and negative right zones in the derived rules.Write Γ ; • =⇒ •; C as Γ =⇒ C. As a further simplification, we can give formulas a predicate label and abstract over its free variables.This labeling technique is described in detail in [6].For the remainder, we assume this labeling has been carried out.Define an atomic formula as either a label or a predicate applied to a (possibly empty) list of terms.After labeling, our sequents consist entirely of atomic formulas.
Example 1 In Figure 2, frame boxes surround the three blocks of the proof.The synthetic connectives are a, ↓a b and ↓(↓a b) c.There is a single derived rule for each synthetic connective (though this is not the case in general).We implicitly carry the principal formula of a left rule to all of its premises.
These rules correspond to the blocks shown in Figure 2. Corresponding labeled rules for Then the blocks of the proof from Figure 2 can be compressed to the succinct 3 The Polarized Inverse Method In the previous section we developed the notion of a focused backwards proof; in this section we turn it into a forward proof search strategy.

Forward Rules
Recall the following rules from the backward (unfocused) intuitionistic sequent calculus: Interpreting these rules for forward search shows some difficulties.In the forward direction we want to guess neither the context Γ , nor the formula A in the ⊥-L rule.We therefore allow the succedent to be empty, and ignore Γ in such initial sequents.The analogous forward sequents have the form We maintain a connection with the backward calculus by saying that a forward sequent stands for all of its weakening and substitution instances.This new form of sequent requires a more complicated notion of matching (or applying) inference rules.We now define the operations necessary for proof search in the forward polarized sequent calculus lifted [6] to free variables.We assume the reader is familiar with the usual notions of substitutions and most general unifiers.The variables of a sequent are implicitly universally quantified outside the sequent.This means every time a sequent is used, we have to rename its variables to be fresh.We will apply such renamings tacitly.
Definition 2 (Inference rules) An inference rule with name id, has premises H 1 , . . ., H n and conclusion Q, all of which are sequents.Π is a set of parameters (the fixed parameters) that are introduced during the inversion phase.Inference rules are schematic in their variables, which can range over formulas or terms.

Matching
Given some known sequents, we wish to derive new sequents using the inference rules.
The process of inferring new sequents from known sequents using inference rules is called matching (or rule application).In the backward direction of intuitionistic logic, matching is a simple matter, consisting of decomposing the principle connective and copying the context.The presence of fixed parameters, possibly empty succedents, and the union of contexts significantly complicate the operation in the forward direction.We therefore define matching in stages.(Recall [3] throughout that if a parameter a is in the domain of a substitution θ, then aθ is also a parameter.)First, a sequent can only match a premise of a rule if their respective succedents are unifiable.When matching a premise of an inference rule, we need to match antecedents as well.

Definition 3 (Succedent
Because a sequent stands for all of it weakening and substitution instances, not all specified antecedents in the premise of a rule actually need to be present in the sequent. There are therefore many ways the antecedents of a sequent can match the antecedents in a premise of a rule, depending on the antecedents that are held (that is, not matched).This consideration leads to the following: Definition 5 (Premise Matching) Sequent Γ −→ C matches premise Γ −→ C with θ if C matches C with θ 1 and Γ matches Γ with θ 2 and θ = mgu(θ 1 , θ 2 ).
The first four properties assure our equivalent of the eigenvariable condition: every parameter that is generated by the synthetic rule from a (derived) use of LA-∃ or RA-∀ is new.The final property assures that the rule is consistent.If the succedent is unspecified, then it either remains unspecified in the new sequent, or is specified to exactly one sequent.If all of the above hold, then the result of the match is the sequent The definition of matching assures that the forward application simulates a backward rule application.Since we always combine unused premises in the same way, in the rest of the paper we omit the contexts Γ in forward inference rules.
Example 2 If the synthetic connective is L 1 = ↓(((∃y.↓p(y)) ∀x.(p(x) & q(x))) on the right, then the backward and forward synthetic rules are

Proof Search
Before we can turn our observations into a method for proof search, we need two more crucial definitions.First, the inverse method cannot in general prove a given sequent exactly, but sometimes only a stronger form of it.This is captured by the subsumption relation.

Definition 7 (Subsumption
there exists a substitution θ such that |Γ 1 θ| = |Γ 1 | (i.e., θ does not contract Γ 1 ) and Suppose Q ≤ Q and we are trying to prove Q .Since weakening is an admissible rule in the backward calculus, given a backward proof D of Q, we could modify D by weakening, yielding a proof of Q .
The second definition comes from the following observation.It is not the case that (p(X, Y ), p(Y, X) −→ g) ≤ (p(Z, Z) −→ g), even though (p(Z, Z) −→ g) is identical to (p(X, Y ), p(Y, X) −→ g) under substitution {X → Z, Y → Z}. (Remember that we maintain the premises as a set.)Since a sequent stands for its substitution instances, we should be able to infer the latter sequent.This consideration motivates the definition of contraction: Now we have the basic operations necessary to define forward search using the polarized inverse method.We begin with a negative polarized input formula A − .We first decompose the problem into stable sequents by applying the backward rules, inverting the sequent •; • =⇒ A − ; •.The leaves of the backward inversion are stable sequents.Each stable sequent is solved independently.(This is why the bottom portion of Figure 2 is not contained in a block.)For each stable sequent, we determine the sequent's synthetic formulas, and generate the corresponding derived rules.We begin with a sequent database containing the initial sequents, those synthetic rules with no premises.We repeatedly match the synthetic rules to known sequents in the forward direction.The resulting matches, along with all of their contraction instances, are added to the database.We continue in this way until we either generate a sequent that subsumes the goal, or until the database is saturated, that is, any further inference would only add sequents subsumed by something already in the database.Due to the undecidability of the problem, if the goal is not provable, it is possible that the database will never saturate.
Theorem 2 (Completeness) If there exists a (ground) backward focused derivation of a polarized formula A, then such a derivation can be constructed using the polarized inverse method.Proof Analogous to the corresponding proof in [3].

An Implementation Framework
We turn now to our implementation, called Imogen.The implementation is designed as two distinct modules, referred to respectively as the front end and the back end.The front end deals with the specifics of a particular logic and focusing strategy.It takes a formula as input and returns the initial stable sequents, and for each sequent a complete set of synthetic inference rules and initial sequents.The back end maintains a database of known sequents, and applies the rules to the database using a fair strategy, generating new sequents.It stops when it finds a sequent that subsumes the goal, or when the database is saturated.
This design makes it possible to use the same back end for different logics.While Imogen now only supports two front ends, intuitionistic first order logic and an optimized front end for the propositional fragment, it would be straightforward to extend to other logics.

The Front End: Rule Generation and Matching
The front end has two distinct tasks.The first is to generate the initial rules and sequents given an input formula and a focusing strategy.This is achieved by, for each synthetic connective, evaluating the inference rules of Section 2 in the backward direction.Each block of a potential backward proof becomes an inference rule.
The second is to define the main functions outlined in the last section: subsumption, contraction, and rule matching.Subsumption can be an expensive operation, but is straightforward to implement.Contraction can be a problematic operation because if a sequent has antecedents with the same label or predicate symbol, there can be an exponential number of contraction instances.It is not uncommon for Imogen to generate tens of thousands of contraction instances of a single sequent.
To implement the function match of Definition 6, we use the technique of partial rule application.Instead of having a fixed rule set and matching all the hypotheses simultaneously, we have an expanding rule set, and match one premise at a time.The match of a rule with n premises yields a new residual rule with n − 1 premises.
yields the new inference rule Similar to contraction, if both a rule and sequent have multiple instances of the same label or predicate, matching can produce an inordinate number of new rules or sequents.

The Back End: Rule Application and Subsumption
The back end takes the initial sequents and rules from the front end, along with the definitions of matching, subsumption and contraction.Then it uses a modified form of the Otter loop to search for proofs.
The Otter Loop.The "Otter loop" is a general strategy for automated reasoning using forward inference.In our version, there are two databases of sequents, called kept and active, and two databases of rules (because of our partial rule matching strategy) also so named.The active sequents (AS), consist of all the sequents that have already been matched to rules in active rules (AR).Symmetrically, the active rules are the rules that have been matched to the active sequents.The other databases, the kept rules (KR) and sequents (KS), have not yet been considered for matching.A step of the loop proceeds as follows, as shown in the figure below.Imogen chooses either a kept sequent or a kept rule according to some fair strategy.Suppose it chooses a sequent.Then we are in the situation in diagram (a).The sequent is added to the active sequents, and then is matched to all active rules (b).This matching will generate new sequents (when the matched rule has a single premise), and new rules (when the matched rule has multiple premises).The new rules and sequents are added to the respective kept databases (c).A symmetric process occurs when choosing a kept rule.

Subsumption
Redundancy elimination is an important part of an efficient implementation of the polarized inverse method.Imogen performs subsumption in a variety of ways.The first is forward subsumption: new sequents generated during the matching process that are subsumed by existing sequents are never added to the kept database.Another form of subsumption occurs when a new sequent subsumes an existing active or kept sequent.
There are two forms of backward subsumption in Imogen.The first, simply called backward subsumption is where we delete the subsumed sequent from the database.In recursive backward subsumption we delete not only the subsumed sequent, but all of that sequent's descendents except those justifying the subsuming sequent.The idea being that Imogen, with the new, stronger sequent, will eventually recreate equal or stronger forms of the rules and sequents that were deleted.A final version of subsumption is called rule subsumption.Rule subsumption occurs when a new sequent subsumes the conclusion of an inference rule.In this case, whatever the content of the premises, the resulting conclusion would be forward subsumed, as matching only instantiates variables and adds to the antecedents.Thus, such a rule can be safely deleted.
Theorem 3 If there exists a derivation of A, then there exists a derivation that respects forward, backward, and rule subsumption.
Proof For forward and backward subsumption, the proof is analogous to the one given by Degtyarev and Voronkov [6].Since each sequent that could be generated by a subsumed rule would itself be subsumed, their argument extends easily to our framework.
For recursive backward subsumption, while the soundness is clear, it is not as easy to see that our strategy is still complete.
Theorem 4 Recursive backward subsumption is nondeterministically complete, that is, if the database saturates without subsuming the goal, then the goal can not be derived.
Proof For every recursively deleted sequent we either retain a stronger sequent, or we retain the possibility to recreate a stronger sequent.For the database to be saturated, we must have subsumed or recreated all the deleted sequents.
This is enough to obtain the correctness of our prover: by soundness (and, in addition, through an independently verifiable natural deduction proof object) we have a proof when the goal is subsumed.If the databased is saturated without subsuming the goal, there cannot be a proof.We conjecture that recursive backward subsumption is also complete in the stronger sense that if there is a proof we could in principle always find it (since rule and sequent selection are fair), but we do not at present have a rigorous proof.
Besides fairness, the proofs of completeness under the various forms of redundancy elimination rely mainly on the following of the (derived) rules used in the forward direction: If the premise of a rule is subsumed, either the conclusion is already subsumed or we can reapply the rule and obtain a new conclusion which subsumes the old one.

Other Features
The back end implements a few other notable features.Since in a backward proof, the antecedents of the goal sequent will occur in every sequent in the proof after the initial stabilization phase.We can globalize these antecedents [5], which reduces the space required to store sequents and avoids unnecessary operations on them.Imogen implements a variety of term indexing algorithms [7], including path and substitution tree indexing to quickly retrieve elements of the databases.Experimental results show that in our case path indexing is more efficient than substitution tree indexing.The back end also maintains a descendent graph of the rules and sequents.This graph is used by the front end to reconstruct a natural deduction proof term of a proved formula.This gives external evidence that the proof found by Imogen is indeed a proof.

Performance Evaluation
We now give some performance statistics and internal comparisons of the effects of different optimizations.All of the Imogen statistics from this paper are from a 2.4 Ghz Intel Macintosh, Darwin 9.6.0,with 2Gb of memory.Imogen is written in Standard ML, and is compiled with MLton.
ILTP.We evaluated Imogen on ILTP, the Intuitionistic Logic Theorem Proving library [17].The statistics from the ILTP website [16] are shown below.Currently the library gives detailed results for 6 intuitionistic theorem provers on 2550 problems, with a time limit of 10 minutes.The other provers from ILTP use various optimizations of backward search.The non-Imogen statistics are run on a Xeon 3.4 GHz Linux, Mandrake 10.2.The amount of memory is not given on the website.
Besides solving the most total problems, Imogen does much better than other provers at disproving non-theorems.This is a similar result to Imogen's intuitionistic propositional theorem prover described in McLaughlin and Pfenning [12].

Proved Refuted
Subsumption.The following table shows the performance of Imogen with different settings for subsumption.Because of a lack of computer resources, we only ran the internal experiments on the a subset of the ILTP library (about 800 problems), and only for 10 seconds each.The first three columns are for backward subsumption settings.The last column does no rule subsumption.

Solved
Polarization.One benefit of the polarized inverse method is that it is simple to simulate different focusing strategies using appropriate placement of double shifts.For instance, if we wish to measure the effect of the inversion phase without the focusing phase, or vice versa, we can strategically insert double shifts ↓↑ or ↑↓ at the locations where focusing (or inversion) would take place.The double shifts will break the current phase and generate a block boundary.The following table gives the performance of some of these strategies, again using 10 second timeouts.Single Step simulates the unfocused inverse method.Weak Focusing makes all focusing phases complete, but breaks the inversion phases into single steps.Weak Inversion makes the inversion phase complete, but breaks the focusing phase into single steps.Fair Weak Focusing is like weak focusing but allows the initial stabilization phase to run unchecked.In all of these experiments, we assigned negative polarity to all atoms.Positive Atoms makes all atoms positive, but otherwise does no unnecessary shifting.

Conclusion
In this paper we presented a basis for forward reasoning using the polarized inverse method, and demonstrated its practical effectiveness in the case of intuitionistic logic.In related work, Pientka et.al. [15] describe an experimental implementation of a focused inverse method for LF.Chaudhuri [3] describes a focused inverse method prover for linear logic.Earlier work is by Tammet [19] who describes an implementation of a forward intuitionistic theorem prover.We did not compare Imogen to his system because it is not part of ILTP.According to the ILTP website [16], the existing implementation is unsound.
Our work is by no means complete.While the current implementation is flexible with polarities, we have not investigated more than rather trivial heuristics for selecting the polarity of atoms, conjunctions, and inserting shifts.It is known, for instance [5], that using positive atoms simulates backward chaining in the inverse method.In our experiments however, we find that Imogen performs poorly on some problems that backchaining solves quickly.Using positive atoms, we can solve at least 50 more ILTP problems in less than a minute, but knowing when to choose positive atoms is still an open question.
One new optimization would be to consider a subordination relation on propositions [14].This would prune the search space by deleting sequents of the form Γ, p −→ q if no proof of q could depend on a proof of p as determined by the subordination relation.We are currently extending Imogen to other logics, including first order logic with constraints, and LF.

Definition 1 (
Forward Sequents) A forward sequent has the form Γ −→ C where Γ is a set of atomic formulas and C is either the empty set or a set containing a single atomic formula.It is written Γ −→ A in the case that C = {A}, or Γ −→ • in case C = ∅.The set Γ consists of the antecedents, and C is called the succedent.Define the functions succ(Γ −→ C) = C and ants(Γ −→ C) = Γ .