Age | Commit message (Collapse) | Author | |
---|---|---|---|
2015-02-22 | Adapted old Tokenizer infrastructure to new Tokens.hpp | Andreas Stöckel | |
2015-02-22 | Implemented TokenizedData, a facility to store data with tokens where tokens ↵ | Andreas Stöckel | |
can be dynamically enabled and the whitespace mode specified at the moment the tokens are read | |||
2015-02-22 | Implemented SourceOffsetVector -- a class for storing the SourceOffset for ↵ | Andreas Stöckel | |
each character in a sequence in an fairly efficient manner | |||
2015-02-15 | Moved TokenTrieTest to new directory | Andreas Stöckel | |
2015-02-14 | Moved DynamicTokenizer and TokenTrie to parser/utils | Andreas Stöckel | |