summaryrefslogtreecommitdiff
path: root/src/core/utils/Tokenizer.cpp
AgeCommit message (Collapse)Author
2014-11-21According to my lords holy command I enacted his graces order of moving all ↵Benjamin Paassen
peasants and lordlings, big and small, young and old, to our common land and inner sanctuary to be safe from outer threats by those usurpers.
2014-11-19fixed a bug preventing the Tokenizer from finding the right token if a parse ↵Benjamin Paassen
was incomplete beforehand. Also cleared the buffers if a subclass returns false from doPrepare. Failing to clear the buffers lead to subsequent problems.
2014-11-14implemented tokenizer test and started implementing CodeTokenizer under ↵Benjamin Paassen
supervision of Maester Stoeckel.
2014-10-31finished first draft of tokenizer code.Benjamin Paassen
git-svn-id: file:///var/local/svn/basicwriter@90 daaaf23c-2e50-4459-9457-1e69db5a47bf
2014-10-31finished work on TokenTree and tested it.Benjamin Paassen
git-svn-id: file:///var/local/svn/basicwriter@88 daaaf23c-2e50-4459-9457-1e69db5a47bf
2014-10-31added tokenizer class code.Benjamin Paassen
git-svn-id: file:///var/local/svn/basicwriter@87 daaaf23c-2e50-4459-9457-1e69db5a47bf