Understanding the VBA grammar, tokens, lexer, parser and parse trees

IvenBach

Mar 8, 2017 04:01

VBALexer.g4 open

Mat's Mug

every commit requires a comment/description

IvenBach

and it looks all foreign to me

Mat's Mug

it's ANTLR's language

this is the lexer grammar

IvenBach

This may take me a while but I'll try.

Mat's Mug

the difference between a lexer and a parser it how they work with tokens

the lexer creates them out of fragments; the parser uses them in parser rules

the syntax is pretty simple:

IvenBach

Mar 8, 2017 04:03

what is a token?

Mat's Mug

we have a string that contains the code of a given module, we got that from the VBIDE API

if the best we can do is searches in that string, we're doomed ;-)

IvenBach

quora.com/What-is-the-difference-between-a-lexer-and-a-parser

Mat's Mug

so what we do instead is, we tokenize it

IvenBach

Is that a good explanation?

Comintern

:thumbsup:

IvenBach

Mar 8, 2017 04:06

Basically: 'Parsers work at the grammatical level, lexers work at the word level.'

Mat's Mug

yeah. the lexer grammar defines a set of lexer rules

ThunderFrame

IIUC, "End" is a keyword, but depending on context, it can be an "End" statement token, or an "End" Property token, or an "End" If token etc.

IvenBach

Ok.

Mat's Mug

RULENAME : {match pattern}

actually, scroll all the way to the bottom

the smallest parse unit in the grammar is a fragment

IvenBach

:boom: Oh man.... It just hit the fan and got real messy in here!

Mat's Mug

Mar 8, 2017 04:07

the lexer rules use these fragments

 fragment A:('a'|'A');

fragment JANUARY : J A N U A R Y;

there's a rule for everything you could possibly see in a VBA program

NEWLINE : '\r' '\n' | [\r\n\u2028\u2029];
SINGLEQUOTE : '\'';
UNDERSCORE : '_';
WS : [ \t];

well, one we can parse anyway

a WS lexer token is either a space or a tab

IvenBach

\r \n are the unicode characters?

WS = WhiteSpace

Mat's Mug

yeah

IvenBach

'\' denotes the escape character?

Mat's Mug

as in regex

@IvenBach exactly

IvenBach

so WS : [<space>\t];

Mat's Mug

Mar 8, 2017 04:12

as far as Rubberduck gives a duck, yeah

IvenBach

In laymans terms. WS is taken to mean either a <space> or <tab character>

Mat's Mug

yep

then you scroll up and you get the whole list of VB keywords and operators

IvenBach

NEWLINE is taken to mean <return> <newline> or either [<return>,<newline>,<u2028>,<u2029>]

Mat's Mug

yep

IvenBach

Think I'm understanding

Mat's Mug

Mar 8, 2017 04:14

lexer rules are always SCREAM_CASE

IvenBach

So anything thats a fragment can be to be something that can later be built upon?

Mat's Mug

fragments are always all_lowercase

exactly

as are lexer tokens themselves

EXIT_DO : E X I T (WS | LINE_CONTINUATION)+ D O;
EXIT_FOR : E X I T (WS | LINE_CONTINUATION)+ F O R;
EXIT_FUNCTION : E X I T (WS | LINE_CONTINUATION)+ F U N C T I O N;
EXIT_PROPERTY : E X I T (WS | LINE_CONTINUATION)+ P R O P E R T Y;
EXIT_SUB : E X I T (WS | LINE_CONTINUATION)+ S U B;

I never thought one could put a line continuation there

IvenBach

EXIT_SUB : E X I T (WS | LINE_CONTINUATION)+ S U B; the <+> symbol at the end denotes 1 or more?

Mat's Mug

exactly

* denotes any number

IvenBach

HAHAHA. Who would put a LINE_CONTINUATION character between Exit and Sub

Mat's Mug

Mar 8, 2017 04:16

IKR!

IvenBach

Now I understand you you @Comintern @ThunderFrame were talking about.

Mat's Mug

// Apparently END_ENUM and END_TYPE don't allow line continuations (in the VB editor)
END_ENUM : E N D WS+ E N U M;

IvenBach

goodreads.com/quotes/…

Comintern

Sub Foo()
    Exit _
 _
 _
 _
 _
 _
    Sub
    Debug.Print "Does not print."
End Sub

IvenBach

Or for that matter a single individual, myself included.

Mat's Mug

Mar 8, 2017 04:17

so far so good?

IvenBach

I think so.

Mat's Mug

that's all there is to the lexer really

bring up VBAParser.g4

IvenBach

For END_ENUM : <Stuff in here>

does everything have to be separated by a space? Always?

Mat's Mug

yes - E N D is the E fragment, the N fragment, the D fragment

Comintern

The 'E N D ...` are fragments.

They're specified that way so they can be case-insensitive.

Mat's Mug

Mar 8, 2017 04:19

^

Comintern

fragment E:('e'|'E');

IvenBach

and the fragments (e|E) (n|N) (d|D) constitute END

Mat's Mug

exactly

IvenBach

which itself is a fragment that 'could' be further built upon.

See you are explaining this fine.
It's the monkey you're teaching that has to take it slowly.

Mat's Mug

propertyGetStmt :
    (visibility whiteSpace)? (STATIC whiteSpace)? PROPERTY_GET whiteSpace functionName (whiteSpace? argList)? (whiteSpace asTypeClause)? endOfStatement
    block
    END_PROPERTY
;

that's a parser rule

parser rules are camelCase (aka javaCase)

IvenBach

Mar 8, 2017 04:21

It's also <thing> : <rules> syntax?

Mat's Mug

yep

IvenBach

Ok

Mat's Mug

this is how Rubberduck sees a module:

module :
    endOfStatement
    moduleAttributes
    moduleHeader?
    moduleAttributes
    moduleConfig?
    moduleAttributes
    moduleDeclarations
    moduleAttributes
    moduleBody
    moduleAttributes
    // A module can consist of WS as well as line continuations only.
    whiteSpace?
;

actually, this:

startRule : module EOF;

FreeMan

@Mat'sMug I was catching up until right about here. I've got 5484 outgoing commits showing in VS Team Explorer. It won't let me pull, the only thing I can do is push to Origin|Rubberduck-VBA|Upstream

Mat's Mug

@FreeMan you're F'd

Comintern

Mar 8, 2017 04:23

^

Did you clone main?

FreeMan

I just did the merge from upstream yesterday/Sunday to pick up all the .12 changes...

IvenBach

Give me a bit for the RD seeing a module part

Mat's Mug

sure

FreeMan

I made the couple of changes for 2803 in NonReturningFunctionInspection and wanted to keep those, so I did a commit on them, and that's what it's telling me now.

:(

Mat's Mug

github won't let you push to rubberduck-vba/Rubberduck/next

Comintern

Mar 8, 2017 04:25

@FreeMan - It looks like you forked master instead of next.

> This branch is 5517 commits behind rubberduck-vba:master.

Mat's Mug

huh, you fork a repo, not a branch?

FreeMan

crap. Yup...

just saw that...

Comintern

Go to your git.

Change the branch dropdown to next.

Mat's Mug

grabs popcorn

FreeMan

GitHub, WinGit or TeamExp

Comintern

Mar 8, 2017 04:26

github.com/daFreeMan/Rubberduck

FreeMan

create a PR for this?

Comintern

No.

IvenBach

@Mat'sMug This is going to take me a little longer to digest. Working on it.

Comintern

When you switch to next, it should show a commit or 2 ahead of next

FreeMan

This branch is 2 commits ahead of rubberduck-vba:next.

Comintern

Mar 8, 2017 04:27

Perfect.

FreeMan

yup, I see that.

Comintern

Delete your entire local directory

2

Then hit the green [Clone or download] button on github and copy the URL.

FreeMan

including the .git and other . directories?

Comintern

Yup. Including the directory itself.

FreeMan

closes VS first

Comintern

Mar 8, 2017 04:29

^ And any open VBE hosts.

FreeMan

^none at the moment.

deleted & URL in clipboard

Comintern

OK, open VS as admin.

IvenBach

<Token>? means 0 or 1 occurences?

Comintern

When it loads, go to team explorer.

FreeMan

then Add?

Comintern

Mar 8, 2017 04:31

Clone

Then paste the url.

FreeMan

that makes more sense.

Cloning

Mat's Mug

@IvenBach isn't Antlr awesome? You can use it to define a formal grammar for any language you want.. even make a new one just for your application

Comintern

Make absolutely sure you clone to the same directory path you had before.

FreeMan

Yup

and... done

Comintern

If you don't, it screws up your registry.

OK, now open the solution.

FreeMan

Mar 8, 2017 04:32

double click & open

IvenBach

@Mat'sMug I'm unable to determine if there is awesome in what I'm looking at. Give me a few more minutes to get back to you on that.

Comintern

Right-click on Rubberduck then properties.

Mat's Mug

you're looking at the parser grammar?

FreeMan

presume I don't need to install the 3rd-party Git command prompt tools, since I've already got wingit installed?

Comintern

The only thing that was blown away was your startup project.

IvenBach

Mar 8, 2017 04:33

@Mat'sMug Right now and traipsing through each one to get an idea.

Comintern

@FreeMan Those follow VS, not the solution.

IvenBach

I just can't find EOF anywhere. I know it stands for End Of File

FreeMan

this is the first time I've opened TE and it keeps prompting me to install them. I don't need another set of command line tools, I shouldn't think

Rubberduck, right-click, properties

Comintern

Nah, don't worry about it. If you have wingit installed, you can do that anytime.

OK, in properties, go to Debug.

FreeMan

start external program -> Excel.exe?

Mat's Mug

Mar 8, 2017 04:35

@IvenBach correct. ANTLR see that and knows it can return from the startRule

Comintern

@FreeMan Bingo.

Hit save after that, and then F6 to build.

You should be good to go.

IvenBach

I've got a general loose idea of what it's doing. Not as good as the lexer since that was much simpler but I think I'm ok to proceed. Bear with me if I don't get it all.

Mat's Mug

the startRule is the single parser rule we need to call to process an entire code file

startRule : module EOF;

FreeMan

huh... Mine says Ctrl-Shift-B to build

IvenBach

Ok

FreeMan

Mar 8, 2017 04:36

========== Build: 2 succeeded, 0 failed, 0 up-to-date, 1 skipped ==========

Mat's Mug

each parser rule, once processed by Antlr, becomes a ParserRuleContext class

and within a parser rule, every time another parser rule is referenced, it becomes a method of that class

FreeMan

thanks again, @Comintern.

Comintern

NP, I could do that in my sleep at this point.

Mat's Mug

so StartRuleContext.module() returns a ModuleContext

module :
    endOfStatement
    moduleAttributes
    moduleHeader?
    moduleAttributes
    moduleConfig?
    moduleAttributes
    moduleDeclarations
    moduleAttributes
    moduleBody
    moduleAttributes
    // A module can consist of WS as well as line continuations only.
    whiteSpace?
;

so the module parser rule becomes a ModuleContext object

that ModuleContext object has a method for each of these referenced rules

let's go do moduleBody

moduleBody :
    whiteSpace?
    (moduleBodyElement endOfStatement)*;

moduleBodyElement :
    functionStmt
    | propertyGetStmt
    | propertySetStmt
    | propertyLetStmt
    | subStmt
;

a module's body is made of whitespace and as many moduleBodyElement+endOfStatement as we can take

IvenBach

the ParserRuleContext used by Rubberduck.Common.CodeModuleExtensions.GetStmtContext which it instantiates?

Mat's Mug

Mar 8, 2017 04:44

yep

Comintern

@IvenBach I wouldn't worry about the code at this point.

IvenBach

@Comintern Don't hide the magic from me!

Mat's Mug

lol

we'll get there :)

IvenBach

@Mat'sMug Getting this understood right now.

Comintern

The parser generates the objects, and ANTLR generates the code. If you thought our code was sausage, you should peek in VBParser.cs sometime...

Mat's Mug

Mar 8, 2017 04:46

^

not too long, it's a HeadacheFactory

@Comintern he meant the extension method though

(there's a GetStmtContext method generated by Antlr in the VBAParser.cs class that Antlr creates from that grammar)

FreeMan

gonna stop while I'm ahead. Will take another crack at #2803 tomorrow.

IvenBach

@Mat'sMug I've somewhat understood it. Enough to take another step.

Comintern

Yep. The GetStmtContext method is using objects generated by the parser.

Mat's Mug

@IvenBach ok. Ctrl+F and search for functionStmt :

Comintern

@FreeMan That one might be a bit more in depth than I thought at first glance - it's something deep in the bowels of the resolver.

Mat's Mug

Mar 8, 2017 04:48

functionStmt :
    (visibility whiteSpace)? (STATIC whiteSpace)? FUNCTION whiteSpace? functionName (whiteSpace? argList)? (whiteSpace? asTypeClause)? endOfStatement
    block
    END_FUNCTION
;
functionName : identifier;

now we're deep enough in the tree that we're starting to see lexer tokens (they're ALL_CAPS)

the rest are other [lower-level] parser rules

IvenBach

(visibility whiteSpace)? means what?

Mat's Mug

Ctrl+F whiteSpace :

whiteSpace : (WS | LINE_CONTINUATION)+;

Comintern

If you see a non-lexer identifier, it's another parser rule.

Mat's Mug

^

Comintern

visibility : PRIVATE | PUBLIC | FRIEND | GLOBAL;

FreeMan

Mar 8, 2017 04:50

@Comintern I couldn't even find what I was looking at yesterday. Def time to crash. see ya in the morning.

IvenBach

visibility <-- Wouldn't it have been better to use accessModifier ?

Mat's Mug

all parser rules either refer to other parser rules, or to lexer tokens

Comintern

@FreeMan Give me a ping when you're working on it - we can team debug it if you want.

@IvenBach The terminology comes from the VBA language specification for the most part.

IvenBach

Ok

Mat's Mug

also we can't just go and rename parser rules on a whim - just renaming VisibilityContext would break the parse tree listeners handling these rules

but let's not get into listeners and tree walkers just yet

arg : (OPTIONAL whiteSpace)? ((BYVAL | BYREF) whiteSpace)? (PARAMARRAY whiteSpace)? unrestrictedIdentifier (whiteSpace? LPAREN whiteSpace? RPAREN)? (whiteSpace? asTypeClause)? (whiteSpace? argDefaultValue)?;

IvenBach

Mar 8, 2017 04:55

@Mat'sMug We gotta stay focused on making the sausage and eating it after.

Mat's Mug

lol

so parser rules are really just abstraction layers: the more parser rules, the more expressive the grammar becomes

IvenBach

OK

Mat's Mug

when you're done drilling through parser rules, down at the bottom of every branch of that parse tree, is lexer tokens

the fun part is that parser rules can be recursive

// 5.6 Expressions

the # stuff at the right isn't a comment; these identifiers become derived types

AddOpContext : ExpressionContext

IvenBach

Starting to not follow...

Mat's Mug

| expression whiteSpace? (PLUS | MINUS) whiteSpace? expression                                  # addOp

IvenBach

Mar 8, 2017 04:58

derived types? Kind of like Base class and derived class?

Mat's Mug

exactly

Comintern

^

IvenBach

so <addOP> = expression whiteSpace? (PLUS | MINUS) whiteSpace? expression

Mat's Mug

yeah

Comintern

Have you read about "is-a" in the context of OOP?

IvenBach

Mar 8, 2017 05:00

Does it include or exclude the "|" character?

Mat's Mug

addOp "is-a" expression

Comintern

addOp is a AddOpContext.

Mat's Mug

the pipe is an alternative - the expression rule has a couple of those

IvenBach

I've heard it and read it but haven't really "got it" yet

Comintern

| is the same as regex- it's a binary or.

Mat's Mug

Mar 8, 2017 05:01

literalExpression :
    numberLiteral
    | DATELITERAL
    | STRINGLITERAL
    | literalIdentifier typeHint?
;

^ that could be written as:

IvenBach

And a BinaryOR | is different from an ExclusiveOR XOR

Mat's Mug

literalExpression : numberLiteral  | DATELITERAL | STRINGLITERAL | literalIdentifier typeHint?;

Comintern

There actually are several written like that - they're just easier to read vertically.

IvenBach

@Comintern What I was going to ask.

Comintern

Especially if there are a ton of them.

Mat's Mug

Mar 8, 2017 05:02

more maintenance-friendly too, if you want to add a #subType

Comintern

^

IvenBach

Ok

Comintern

One thing to keep in mind about ANTLR (and C#) is that unlike VBA, new-lines are just like any other whitespace.

IvenBach

which is why in C# you use ; to denote the end of a line?

Mat's Mug

of an instruction

Comintern

Mar 8, 2017 05:04

Technically end of statement

^^

Mat's Mug

;-)

in Antlr it terminates a rule

IvenBach

Ok

Comintern

; in C style languages is the equivalent to a newline in VBA for all intents and purposes.

Mat's Mug

^

Comintern

Or :...

IvenBach

Mar 8, 2017 05:06

All this spoonfeeding is making eating this stuff a lot easier.

Mat's Mug

:)

so, we "build" the grammar - Antlr processes it and generates a number of C# source files for us

IvenBach

Which are those files?

Mat's Mug

one of them is VBALexer.cs; I think I only ever opened that one by accident

another is VBAParser.cs

one interesting thing to know is that all ParserRuleContext classes are nested types declared inside VBAParser

so the StartRuleContext looks like this:

IvenBach

_serializedATN sweet googalie moogalie... Glad you didn't have to type that.

Mat's Mug

	public partial class StartRuleContext : ParserRuleContext {
		public ITerminalNode Eof() { return GetToken(VBAParser.Eof, 0); }
		public ModuleContext module() {
			return GetRuleContext<ModuleContext>(0);
		}

IvenBach

Mar 8, 2017 05:09

@Mat'sMug This is the limit of my current knowledge.

Mat's Mug

startRule : module EOF;

Hosch250

@Mat'sMug Beautiful formatting, no?

Mat's Mug

not bad for generated code

Comintern

...if you like Java.

Mat's Mug

Antlr has a heavy Java background

Comintern

Mar 8, 2017 05:10

The code generation is done in Java.

Mat's Mug

^

Hosch250

Roslyn makes heavy use of generated code, and it is nicer :)

Mat's Mug

it's written by C#ers

Comintern

Roslyn isn't written in Java.

Hosch250

Oh, that's right--Java people like their braces all funny like that.

Comintern

Mar 8, 2017 05:11

If it was, it would break every 6 months or so.

IvenBach

@Mat'sMug <access modifier> <Something I Don't Know> class <ClassName> : <Inherits From> {<Don't understand this part, at all>}

Mat's Mug

every parser rule becomes a partial class nested inside the generated VBAParser class, and derived from ParserRuleContext (an Antlr runtime type)

IvenBach

I've no experience with Partial | Abstract class stuff

Mat's Mug

partial classes are awesome - Antlr generating partial classes means we can extend their definition in another code file...

Comintern

A Partial class is one that can be split into several files.

That means you can add additional functionality to it.

Mat's Mug

Mar 8, 2017 05:14

...and make them, say, implement interfaces we'd like them to implement

Comintern

An Abstract class is one that has to be inherited by a concrete class.

For example, if Animal is an abstract class, you might have Cat and Dog classes that inherit from it as a base class.

IvenBach

@Comintern public class Cat : Animal or public class Dog : Animal

Mat's Mug

yep

Comintern

You can create a new Dog or Cat object, but you can't create a new Animal - it doesn't make sense - it's just an abstraction of the commonalities between cats and dogs.

IvenBach

That I get the theory of. Still am not very comfortable with it in usage.

The idea of partial class still eludes me.

Mat's Mug

Mar 8, 2017 05:17

it's just a class that can be defined across multiple files

Comintern

All partial class means is that I can add whatever I want to it.

Mat's Mug

it was all the rage when C# 2.0 introduced them

Comintern

UserForms in VBA are kind of like partial classes.

IvenBach

so rather than public class Cat : Animal you'd have

Mat's Mug

forms in WinForms (.net) are partial classes

Comintern

Mar 8, 2017 05:17

^

You would have a partial Cat.

IvenBach

public class Cat : Feline, Mamalia , Animalia ?

Mat's Mug

that's the catch: there can be only one

IvenBach

where you're pulling parts from Feline, Mamalia, and Animalia to constitute a complete Cat?

Mat's Mug

otherwise it's multiple inheritance and compilers don't like that

Comintern

If the existing Cat implementation was missing a Meow() method, you could just add it in your own code and it would become part of the compiled class.

IvenBach

Mar 8, 2017 05:19

@Mat'sMug So pretty much Highlander style?

Mat's Mug

exactly

so abstract class Animalia, class Mamalia : Animalia, class Feline : Mamalia and class Cat : Feline

Comintern

I would love multiple inheritance in C#.

but...

Multiple inheritance

Multiple inheritance is a feature of some object-oriented computer programming languages in which an object or class can inherit characteristics and features from more than one parent object or parent class. It is distinct from single inheritance, where an object or class may only inherit from one particular object or class. Multiple inheritance has been a sensitive issue for many years, with opponents pointing to its increased complexity and ambiguity in situations such as the "diamond problem", where it may be ambiguous as to which parent class a particular feature is inherited from if more than...

IvenBach

the diamond problem

Comintern

^ bingo.

IvenBach

Again the idea of it I get. I just lack the experience to see it in use.

Mat's Mug

Mar 8, 2017 05:21

that's fine

Comintern

Partial classes sidestep that by allowing you to "tack-on" functionality without multiple inheritance.

It's not quite the same, but it solves some of the same problems.

IvenBach

Bear with the following psuedocode

Mat's Mug

	public partial class StartRuleContext : ParserRuleContext {
		public ITerminalNode Eof() { return GetToken(VBAParser.Eof, 0); }
		public ModuleContext module() {
			return GetRuleContext<ModuleContext>(0);
		}

bring up the object browser and lookup ParserRuleContext

you'll find all the members StartRuleContext inherits

IvenBach

public class Human, class Walk : SomeWalkingClass, class Talk : SomeTalkingClass, class Think : NopeNotDoingIt

Mat's Mug

in C# any class can inherit any other. unless it's sealed. sealed classes cannot be inherited

Comintern

Mar 8, 2017 05:23

...safely...without reflection...

Mat's Mug

shhh

IvenBach

@Mat'sMug Sealed = Gandalf vs Balrog?

Comintern

lol

Sealed just means you can't inherit from it.

Mat's Mug

public sealed class YouShallNotInheritMe

IvenBach

@Mat'sMug That's what I was thinking

Comintern

Mar 8, 2017 05:24

Usually it's because the sealed class wraps something that it doesn't want you to extend.

Mat's Mug

and then class Foobar : YouShallNotInheritMe won't compile

Hosch250

Until someone writes something that edits the emitted IL and can write that.

Or whatever.

Mat's Mug

so back to StartRuleContext, you can see it has a module() method that returns a ModuleContext

IvenBach

@Mat'sMug Antlr4.Runtime.ParserRuleContext : Antlr4.Runtime.RuleContext is what you were referring to?

Mat's Mug

yep

then take a peek at Antlr4.Runtime.IToken

there's a Text property

a Line property

and a Column property

IvenBach

Mar 8, 2017 05:28

Channel, Column, Inputstream, ... , Type

Mat's Mug

yeah

IvenBach

I can see that. Superficially I understand that they are members that are supplied to ParserRuleContext through inheritance

Mat's Mug

TokenIndex is the position of the token in the input stream (it all started with a string containing a module's code, right?)

every token knows its position in the token stream, and in the source string

Rubberduck uses token positions to locate everything

that's one crucial point

Comintern

When it doesn't, the wheels come off.

IvenBach

So the Token = composed of ParseThings = composed of LexerThings

Mat's Mug

Mar 8, 2017 05:31

a token comes from a lexer rule; lexer rules can be made of other lexer rules; then parser rules are made from lexer rules and other parser rules

Comintern

ParseThings are composed of LexerThings which implement Token interfaces.

IvenBach

SmallThing becomes part of LittleBiggerThing becomes part of MediumThing becomes part of LargeThing becomes part of RD. To find out where a SmallThing is locate you use the IToken.TokenIndex?

Mat's Mug

tokens' line and column properties tell us where they are in the module

...but only because we gave the parse the whole module

oh dear god

Comintern

?

IvenBach

He has to wash the walls down now. Nothing else is sticking to the walls since there's too much sausage up there already.

Comintern

Mar 8, 2017 05:34

Is the random eyeball staring back at you yet?

Mat's Mug

@Comintern if we know method DoSomething has been modified, and we use the VBIDE API to get its start line S, then we can offset/prepend that string (the DoSomething member) with S empty lines, parse that, and bingo we've only parsed a single procedure and we haven't broken the token positions!!!!

is that not possible? what am I missing? I must be missing something, it can't be that simple, can it?

Comintern

Yeah, that's possible.

The problem is tracking the offsets.

Do the positions update themselves if you use a stream re-writer?

Mat's Mug

yes

uh

yeah they have to

in the rewriter I mean

Comintern

We should be using that for code re-writing then. Always.

Mat's Mug

^

Comintern

Mar 8, 2017 05:37

That way we wouldn't have to reparse after refactoring.

IvenBach

Has your RubberDucking explanation to me made you find an epiphany?

Mat's Mug

and you can dump the rewriter into the code module in one call

@IvenBach yes

Comintern

Yep.

Which makes undo exactly like the indenter.

Mat's Mug

that's huge

IvenBach

My job here is done then Yep dusts off hands I do good work.

Mat's Mug

Mar 8, 2017 05:39

lol

not kidding, I'm going to have to get to bed soon

IvenBach

Sorry I just have a bit of a dry sense of humor.

Comintern

@Future, I think you could probably link tokens to text segments in AvalonEdit.

Mat's Mug

wipes tears

Comintern

Some user named "Future" just got a random ping there...

Mat's Mug

lol

moderator power-ping gets no match for @Future, you're good

IvenBach

Mar 8, 2017 05:41

@Mat'sMug So that pretty much concludes todays lesson?

Comintern

Take that a step further though. If an inspection is attached to a context, the quickfix would be to rewrite itself.

Mat's Mug

@IvenBach the next step is to look at how RD calls the Antlr-generated code and consumes the parse trees

@Comintern that's ...crazy

Comintern

And trust me, that is not an appetizing meal.

IvenBach

Ok.

Mat's Mug

eh, it's just rewriting, what, half of Rubberduck?

Comintern

Mar 8, 2017 05:43

Nah, maybe a third.

tops.

Mat's Mug

so about 80K lines of code then

IvenBach

@Comintern you can do that by tomorrow 7am?

Mat's Mug

#ThisLooksLikeAJobFor

Comintern

I'm on it.

Mat's Mug

lol

Comintern

Mar 8, 2017 05:44

I make no promises as to whether it will compile.

2

Mat's Mug

I'm going to sleep on it

Understanding the VBA grammar, tokens, lexer, parser and parse trees

VBA Rubberducking

Participants