@Vogel612 Maybe ! The morning will be quieter that's for sure. But the weird things is since it's so unusual normally I have more troubled sleeping
@SimonForsberg My wife have a dentist appointment, we have to go to costco then do the groceries and latter she's getting a tatoo. After well be really free
But right bow im playing the switch in the car waiting
The TokenStream and the indices are not necessary for the problem. They are just there to be able to reproduce the matched text and in case you want to do some operations based on the context.
Let me explain what the TokenStream is. A typical way to parse code is to first turn it in a lexer into a stream of tokens, i.e. atomic items for parsing.
E.g. if you have the formula (111 + 13) * 22 would be turned into tokens LPAREN, 111, PLUS, 13, RPAREN, MULT, 22. That simplifies the parsing since unnecessary things like whitespace are already gone and things like number literals area already condensed into a single token.
BTW, we exclude the generated lexer and parser from git because they are just a derivative. So, you have to look after a build in order to see the generated lexer and parser.