dk.brics.grammar.operations
Class GrammarTokenizer
java.lang.Object
dk.brics.grammar.operations.GrammarTokenizer
public class GrammarTokenizer
- extends Object
Grammar tokenizer.
Replaces every regexp terminal by a string terminal.
Note that this is sound (for ambiguity analysis) only if the terminals represent tokens
such that every string can be split in at most one way into tokens.
Methods inherited from class java.lang.Object |
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
GrammarTokenizer
public GrammarTokenizer()
- Constructs a new grammar tokenizer.
tokenize
public void tokenize(Grammar g)
- Tokenizes the given grammar.
- Parameters:
g
- grammar
Copyright © 2005-2008 Anders Møller.