summaryrefslogtreecommitdiff
path: root/src/core/utils/CodeTokenizer.hpp
AgeCommit message (Collapse)Author
2014-11-21According to my lords holy command I enacted his graces order of moving all ↵Benjamin Paassen
peasants and lordlings, big and small, young and old, to our common land and inner sanctuary to be safe from outer threats by those usurpers.
2014-11-19one slight formatting change.Benjamin Paassen
2014-11-191.) fixed the word tokenization mechanism in the CodeTokenizer which ↵Benjamin Paassen
returned rubbish previously. 2.) Allowed multiline tokens if the mode is LINEBREAK (obsiously). 3.) returned LINEBREAK tokens in normal mode. 4.) added a CodeTokenizer test.
2014-11-19added documentation for CodeTokenizer.Benjamin Paassen
2014-11-14implemented tokenizer test and started implementing CodeTokenizer under ↵Benjamin Paassen
supervision of Maester Stoeckel.