Conversation started Mar 3, 2022 at 15:41.
Mar 3, 2022 15:41
In tokenizer.py:
You need two variables: current_float and float_contents.
Example: 12.34
The program will think 1 and 2 are numbers and will store them in tha variable number.
When the program sees the decimal point, because of this code the char is not a digit, so the program will try to add number as a token.
You need to add an if-statement before the latter to check if the char is a decimal point:
If it is then it has to add the contents of number and a decimal point to float_contents and set current_float to True.
Now 3 and 4 will be added to number again.
And when you check after this, you need to check if current_float is True and add all the contents of number to float_contents, and finally, you add float_contents to tokens
 
Conversation ended Mar 3, 2022 at 15:42.