Derivative grammars: a symbolic approach to parsing with derivatives

We present a novel approach to context-free grammar parsing that is based on generating a sequence of grammars called emph{derivative grammars} from a given context-free grammar and input string. The generation of the derivative grammars is des cribed by a few simple inference rules. We present an $O(n^2)$ space and $O(n^3)$ time recognition a lgorithm, which can be extended to generate parse trees in $O(n^3)$ time and $O(n^2log{n})$ space. Derivative grammars can be viewed as a emph{symbolic} approach to implementing the notion of emph {derivative languages}, which was introduced by Brzozowski. Might and others have explored an emph{operational} approach to implementing derivative languages i n which the context-free grammar is encoded as a collection of recursive algebraic data types in a f unctional language like Haskell. Functional language implementation features like knot-tying and lazy evaluation are exploited to ensure that parsing is done correctly and efficiently in spite of complications like left-recursion. In contrast, our symbolic approach using inference rules can be imple mented easily in any programming language and we obtain better space bounds for parsing. Reifying derivative languages by encoding them symbolically as grammars also enables formal connecti ons to be made for the first time between the derivatives approach and classical parsing methods lik e the Earley and LL/LR parsers. In particular, we show that the sets of Earley items maintained by the Earley parser implicitly enco de derivative grammars and we give a procedure for producing derivative grammars from these sets. Co nversely, we show that our derivative grammar recognizer can be transformed into the Earley recogniz er by optimizing some of its bookkeeping. These results suggest that derivative grammars may provide a new foundation for context-free grammar recognition and parsing.