VBA Rubberducking - 2018-01-19 (page 2 of 3)

« first day (1323 days earlier) ← previous day next day → last day (1857 days later) »

5:02 PM

^ reads that as bbl, figuring out a way to hijack a typelib GUID

heh. FWIW i'm fine w/ helpfile hijack as long it can handle legit path to helpfile which is in use by 0.000000000000000001% of VBA projects out there.

@this that's the thing: we're breaking any legit path to helpfile. the basic assumption being no project uses that property

yeah but the way I figure it, it would be even rarer to have 2 projects use same help file. Or barring, add something that is compatible (e.g. C:\path\to\foo\help.hlp#xxxx if VBE will ignore stuff after #xxxx). Or make it totally useless by providing better help system within RD.

I still dream of a better help system for internal APIs. Those are PITA to manage, IMO.

[rubberduck-vba/Rubberduck] Vogel612 deleted branch AV-Fix

[rubberduck-vba/Rubberduck] Vogel612 deleted branch codecov-behaviour-updates

[rubberduck-vba/Rubberduck] Vogel612 deleted branch codecov

5:28 PM

@this can't have both

and if the helpfile changes, RD gets massively confused

IOW VBProject.HelpFile no longer exists, as far as users are concerned - UR HELPFILE R BELONG TO US NOW

wait, why would changing a helpfile even matter? I thought it was to uniquely identify the project, no more

didn't we also use it to determine dirtiness?

so RD's all snooty and haughty? "Yer code is filthy... pig." :p

@Vogel612 nope. we assign it as soon as we get ahold of the IVBProject, and then use it in .Equals checks everywhere

@this because, caching

i.e. change the helpfile, then save & exit and restart RD, will be good.

but change the helpfile and continue, I expect problems

also you're right, we don't break the helpfile - we're using the path as the ID if it's there

and yes, it's a problem if two projects use the same helpfile

but since nobody uses that, it's a relatively safe thing to do

and it's the best we've got anyway

best we had anyway

5:45 PM

@Mat'sMug Sure they do.

Just copy/paste a file.

@Mat'sMug oh ok, then we're on same page - I wasn't really thinking of changing the helpfile midway but I can see how that'd be a problem

@Hosch250 yeah. but you wouldn't have both copies open at once

Sure you would :grin:

Not sure I follow. We are just using the string, not the file. File could very well be at /dev/null and it'll be AOK w/ RD

ayup

5:48 PM

@Hosch250 unless it's in the same RD instance, no problem there

Unless you are using source control.

wow VS, you're effing drunk again

user image

VS: "17 compile errors, we're missing a reference to Antlr 4.6.4"
ME: *~restarts VS*
VS: "6 compile errors, we're missing a reference to Antlr 4.6.4"

seriously though??

it's Very Stable

5:52 PM

Did you do a clean and rebuild?

And make sure you updated all the projects referencing ANTLR?

would be nice if VS could wipe its own ass

kinda got it fixed.

Why did we add Antlr.Runtime.net45 to RubberduckTests manually?

hmm, no idea

@Mat'sMug Name me a program that actually does that. ;) Heck, I'll go easy on you. Name me a program that even has an ass.

ASSembler IDE

</wiping>

5:56 PM

LOL. Last time I checked Assembly doesn't understand XML.

welp ...

apparently the endToken.EndColumn() of code with three lines with three spaces at the end is 8 ...

hmm. what's the code in question?

@Vogel612 How many columns does the line before it have?

0 ...

That was one of the issues that was easy to fix--it was looking at the wrong line.

Hmm. After?

5:58 PM

it's about SelectionExtensionTests

@Hosch250 0

just EOF

Hmm. What's the code?

Oh, just three new lines and three spaces?

yeap

Hmm. It's got it as one line?

I think it interprets <EOF> as text and not as token ...

which is a bit of a problem ...

i don't remember writing that test.

6:00 PM

So, 3 * \r + 3 * \n + 3 * {space} = 9 - 1 = 8.

yeah look like that to me.

but the EndColumn is supposed to consider the last line only

VS says it was Hosch... about a month ago ...

Wow?

or maybe you only changed it...

dunno.

But exactly what is the token?

6:01 PM

either way, if I add an empty line at the end, the end column is 5...

Oh, the NUnit stuff.

which makes even less sense.

Wow, lol.

@Hosch250 ahhh that makes sense..

so less sense is more sense, got it.

2

6:02 PM

mdoerner edited and @this wrote the test

3 and 4 months ago respectively

Hm. Ok. Still, I don't remember testing just 3 blank lines. :\

i guess i'm getting old.

regardless - i still ask - what is the token?

more precisely if you look at the token's text, does it agree?

with stuff like...

Public _
  Sub _
 Foo

the token should have the whole 3 lines in its Text property

and thus EndColumn should report 4

but i'm not sure about the 3 lines with just whitespaces.

the token says "'<EOF>'"

which would explain the column-number 5...

huh

that means Antlr4.6 breaks column positioning

which means we're F'd

LOL.

hmm. i can't believe they'd just put in metadata in Text property like that.

6:09 PM

I'm pretty sure what actually happened is that the Parser doesn't correctly access the token definitions in VBALexer

which then of course screws it over, because <EOF> isn't a single token, but 5 separate tokens.

that sounds more plausible. That kind of change would be a major breaking change. I don't think they'd even do that.

Huh? If it were, wouldn't the Text property have ', for first, < second, E for third, and so forth?

I'm using the experimental target code generation without Java that's available since 4.6.?

@this dunno... I'll investigate some more.

I'm pleasantly surprised that it only blew up 18 tests...

and I still have 2.3k passing tests, so there's that

anyway the EndColumn splits the lines without any regards to the line continuation ( we split on Environment.NewLine if there are any), but it very much rely on having the Text contain the whole ..... "phrase"?

[rubberduck-vba/Rubberduck] CHR-IS-B commented on issue #3714: VBA Project won't disappear after the workbook has been closed.

> I ran into this same problem after installing RD 2.1.2.2733-pre yesterday. I don't recall which RD version I was running before.

@Vogel612 How many erroring?

6:13 PM

18

and 25 skipped

I meant skipped.

I don't remember how many are skipped, but 25 is pretty close.

There might be a couple more in there that need tweaking, but not many.

ffs, test-explorer doesn't differentiate between ignored and inconclusive tests

@Vogel612 IKR

current changes: github.com/Vogel612/Rubberduck/commit/…

-42kLoC

@Vogel612 Nope. FWIW, R# doesn't either.

6:16 PM

this change probably means that we don't need to keep versioning the generated ANTLR files

then they all failed, Hosch. I think it's silly that they would pretend the two results are same thing.

@Vogel612 and it yet compiles?

yup

the files don't even show up in the solution explorer

or is it actually because we are referencing the files outside the build?

IIRC, they will stick generated files in the obj folder

which isn't tracked

user image

I do wonder if it'll stil build on AV - if it doesn't have the auto-generated files, it'll flip out

6:19 PM

especially relevant: github.com/Vogel612/Rubberduck/commit/…

actually I can open a PR to check exactly that

yeah would be good idea - looks like it's probably the package taking care of that for you

but still, I can't help but wonder. (Remember, I'm the guy who wears suspenders and belts)

[rubberduck-vba/Rubberduck] Vogel612 created pull request #3715: Migrate to ANTLR 4.6.4 to merge Vogel612/Rubberduck/ANTLR-Upgrade into rubberduck-vba/Rubberduck/next

> Delete all the manually generated codefiles and add build-actions for
all *.g4 files that can update all our stuff.

Note that this commit **WILL** break tests.
See #2498

^^ let's see what AV says

oh crap ...

[rubberduck-vba/Rubberduck] build for commit 8fc6039d on unknown branch: AppVeyor build failed

@Vogel612 and we'll know the build failed because of tests, not because it was missing stuff?

6:25 PM

BUILD FAILURE!

@this it failed because it was missing stuff ...

I'm not quite sure what exactly it was missing, but it's definitely missing some stuff

also apparently the *.g4 files are not encoded in UTF-8

@Vogel612 we normally copy the generated .cs files from \obj into that folder

which is a minor bit of a problem, but it can be pretty easily adjusted.

@Mat'sMug I know. I explicitly didn't.

ok

because currently the generator just runs on build

6:27 PM

hmm

and outputs where?

Let me investigate.

I think I should set copyToOutput to true on the ANTLR goal...

Hey @Mat'sMug do we have a special reason to save the grammars in cp-1252?

nope

okay. I'm moving the grammars to UTF-8 without BOM then

[rubberduck-vba/Rubberduck] Vogel612 synchronized pull request #3715: Migrate to ANTLR 4.6.4

@Mat'sMug to obj

no subfolders or anything.

6:41 PM

[rubberduck-vba/Rubberduck] build for commit 27601ed1 on unknown branch: AppVeyor build failed

BUILD FAILURE!

can't have a post-build event that copies the files, RD won't build without them in the first place

can't have pre-build event to do it, because grammars build, well, on build

so wtf is the use for a build action again?

hmm ...

it says it's missing members on VBAParser. ..

missing VBAParser too I suppose

which implies to me that it's not actually generating or building the class

or at least making an empty skeleton

cleaning and trying to repro the AV failure on my local

hrngh...

nothing of the sort. ..

20 warnings, 58 messages, no errors

6:44 PM

@Vogel612 you probably have the [old] generated files under \grammar

removed those

hmm

@Vogel612 because I don't believe that you actually deleted the file --- they are built... somewhere

they've been deleted in the commit

that's why it has 42kLoC removed

user image

the VBAParser may not be in the solution but it has to be somewhere in order to build. Maybe try to find where the VBAParser.cs is now

I bet you it's not referenced by the solution

6:48 PM

under obj/Debug

right - and AV builds using Release

lemme build in release to check...

make sure you clean the obj/Debug, too

@this gah!!

?

6:50 PM

oh wtf VS?

54 build errors by switching the configuration .

bingo

could it be because the build action is configuration-specific?

oh carp... maybe?

I manually edited the csproj file...

check the build action under the release. is it still the same as it was debug?

yep, it is

ok.

what about that custom build action for the antlr generation?

6:55 PM

that wasn't the build action?

@Vogel612 ...

IDK - i'm probably talking out of wrong orifice. I thought that the code generator required a manual tweak to MSBuild XML to ensure that it'll generate code.

ya, a bit.

38 mins ago, by Vogel612

especially relevant: https://github.com/Vogel612/Rubberduck/commit/8fc6039da509b490c1569d9039ef51aa50‌615d37#diff-79b7969d09a96ce303abf11b8684488c

not to MSBuild XML though...

to the csproj

then I don't know what else would cause build errors simply by changing the configuration

do we still have any build events?

none inside the Rubberduck.Parsing project AFAICT

aaand now it builds correctly...

what the

7:01 PM

hmm. i think this is significant

VBAParser.g4(22,24): warning AC0160: cannot find tokens file 'obj\Release\VBALexer.tokens' [C:\projects\rubberduck\Rubberduck.Parsing\Rubberduck.Parsing.csproj]

let's try that again

empty out the obj\Release

build. do you get build errors?

If so, build one more time.

checking right now :)

aaand none anymore..

now how do I define interdependencies ...

hmm - you're using 4.6.4, Sam's fork, right?

nope. official release

so, 4.7.2?

actually I just upgraded our dep ..

7:08 PM

@Vogel612 We are using Sam's.

kk... we're using Sam's..

Sam's to do the parsing, and the official runtime.

huh...

now - if it's Sam's 4.6.4 --

it still needs Java runtime

AV has that?

If it needs the runtime, we need to include the compiled files.

Because we can't make our users install the runtime too.

7:10 PM

AFAICT, it does

no no no

@this nope. there's been a change

not for end users

there's experimental support for a C# parser generator

so we're using release 4.5.0-alpha003?

4.6.4

7:12 PM

> Starting with release 4.5.0-alpha003, users are no longer required to install the Java Runtime in order to compile .NET applications using ANTLR 4. However, installing Java will dramatically improve the performance of the code generation process. It is highly recommended, especially on developer machines where background code generation is used for IntelliSense functionality.

sounds to me if you're using 4.6.4, AV needs JRT

still not correct

uh, tell that to Sam?

"no longer required to install the Java Runtime"

read the clause before that....

Starting with release 4.5.0-alpha003,

and 4.6.4 is definitely after that

7:15 PM

...... welp, I need to take math class again

i was thinking 4.4.6 even though I was typing 4.6.4. :(

sorry

no worries ;)

unfortunately it seems that the Antlr4 task does not support DependendentUpon

@this we never built the grammars on AV

@Mat'sMug I understood that we are going to.

with this branch Vogel's on

sure. But this also means it's likely AV environment does NOT include the Java runtime

@Vogel612 hacky but I think one way you can manage that is to dump them into their own project.

7:18 PM

no we can't since we need to specify the order that the Antlr tool invocations need to be run in.

@Mat'sMug hence why I was making a tuff with Vogel earlier. ;) Vogel came out right, though. With 4.6.4 (Sam's fork), we don't.

but I think I can hack a pre-build step into it that generates the lexer stuff

if you can, that'd be better than a separate project. But in case it doesn't, I think it suffices to just make Parsing project dependent on the new project without specifying anything else about other projects' dependency via the solution configuration.

[rubberduck-vba/Rubberduck] Vogel612 synchronized pull request #3715: Migrate to ANTLR 4.6.4

@this maybe that isn't even necessary

apparently the definition order of Tasks is relevant..

somewhat annoying

7:24 PM

[rubberduck-vba/Rubberduck] Vogel612 added label PR-Status: WIP to pull request #3715: Migrate to ANTLR 4.6.4

let's see.

would explain why first build fails but succeed on 2nd, too.

ayup

build passed, tests about to fail

that's very good. so we know we can run with that in AV.

nice!!

can't wait to see how much of performance boost we get from those expressions with lot of left recurisions...

7:34 PM

We are about to break 1k tests at work.

Which is up about 200 since I started.

[rubberduck-vba/Rubberduck] build for commit 22c706ea on unknown branch: AppVeyor build failed

BUILD FAILURE!

Which isn't particularly bad since we can't just write the tests, but we have to entirely restructure much of the code to make it testable. Moving things out of projects and all, so they can access the right interfaces.

@this as of now we're mostly getting erroring if-statements, actually

well... let's try running all those tests in Sll mode

@Vogel612 I think we need to move the expressions up.

hmm ..

7:44 PM

FWIW if our unit tests consistently pass with SLL mode, I think that's even better; the more we can avoid LL mode, the better.

it works in Sll AFAICT

github.com/rubberduck-vba/Rubberduck/blob/next/…

I think a few of those need to come up.

OK - IDK if test will tell us if it passed using SLL or if it had to fall down to LL

expression :
	// Literal Expression has to come before lExpression, otherwise it'll be classified as simple name expression instead.
	whiteSpace? LPAREN whiteSpace? expression whiteSpace? RPAREN									# parenthesizedExpr
	| NEW whiteSpace expression																		# newExpr
	| expression whiteSpace? POW whiteSpace? expression												# powOp
	| MINUS whiteSpace? expression																	# unaryMinusOp
	| expression whiteSpace? (MULT | DIV) whiteSpace? expression									# multOp
	| expression whiteSpace? INTDIV whiteSpace? expression											# intDivOp

Try that.

FWIW, here's what I had: github.com/Hosch250/Rubberduck/tree/antlrUpdate

The grammar tests all run in SLL exclusively.

7:54 PM

nope, they don't

Huh?

I mean those in TestGrammar.

okay, that I don't know.

we also have VBAParserTests. those run in LL by default

there's a few that fail unless run in Sll

The ones using the mocks use the usual approach.

Ok might be that as well.

@Hosch250 I think the important part here is that lExpression comes after the "primitive" expressions

The last time I looked at them, the parser tests all ran in SLL.

It has to come after the boolean ones.

7:57 PM

it didn't, apparently...

Hm, than Sam's generator still creates significantly different grammars than the official one.

With the official one, it would identify NOT at lExpression.

apparently it's closer to the official one now

because that's what broke one of the RemoveEmptyIfBlockQuickFix tests

Oh

okay, so we're at least getting closer.

still 25 skipped, but only 7 failing now

The entire problem with NOT is that, according to the spec, keywords are legal identifiers in VBA.

Not that the VBE would honor that.

8:01 PM

I'm really close to letting off a flurry of swearwords in here

Me too. VS is busy committing suicide.

I mean, I've not even been in it for several minutes.

I swear TFS makes VS less stable. It never crashes at home.

All this contacting the server. I mean, it takes well over a minute to rename a #@(*$(! file...

hmm ... apparently DefTypes are now ModuleAttributes...

user image

at least partially...

This is just thinking in long term -- do we need to have unit tests to test the shape of parse tree?

@this It's not so much that.

cos if the shape of hte parse tree has changed in future version, we want to know about it, no?

8:09 PM

It's more that we are testing grammar edge cases.

Oh, you mean should we have them? I don't know.

yeah, because otherwise we would have missed the DefType - (I'm assuming we don't test each grammar unit in VBA currently)

and our unit tests (for inspections/quickfixes/etc) shouldn't rely on us having had implemented something that make use of those grammar units

oh crap... why is singleLetter in letterSpec for defDirectives an unrestrictedIdentifier?

I mean... the VBAL says so, but DefDirectives only use the first character...

and currently a singleLetter does match B-C. ...

which is a minor bit of a problem .

@Vogel612 This is where I always get stuck.

you know what? I'm pretty sure that we can afford to not adhere to the VBA Language spec there.

because it makes no sense to have unrestricted identifiers there

i did just try stuff like DefInt AB or DefInt Public and both turn up red

8:25 PM

aight. ditching the VBA spec

could you try umlauts for me?

and underscore as well, please

hm

not sure if I'm understanding this right -

msdn.microsoft.com/en-us/library/ee199674.aspx

> IDENTIFIER = <any lex-identifier that is not a reserved-identifier>

yet our unrestrictedIdentifier includes reserved keywords?

hmm ... seems like it.

I'd guess we are enforcing that through precedence rules wherever necessary

but we do have IDENTIFIER

for DefType it might suffice to just use that directly

and thus skip all those alternatives

underscore => line continuation -- if used in range or in a comma separated list, red

accented letters => red (tested with é and ñ)

what about üöä?

all red, regardless if appearing singly, in a CSV or in a range

8:33 PM

good..

makes sense, since the documentation above explicitly enforce the range

basically A-Za-z

to the 26 letters, both lowercase and uppercase... yeah

which is a tad less than LETTER, but ...

single-letter = IDENTIFIER ; %x0041-005A / %x0061-007A

yep

i think they do additional checking to enforce that, I guess.

8:37 PM

if the VBE can't execute it, we won't parse it...

I'm clamping that down in the grammar now.

@this AFAIR, the IDENTIFIER whose definition you found in the spec is not used in the spec anywhere.

should we tell Spolsky?

:)

@M.Doerner github.com/rubberduck-vba/Rubberduck/blob/next/…

@M.Doerner it is. for letter-spec

He'd just shift the blame.

8:44 PM

then what is this?

I mean in the spec.

We just define things the other way around.

The spec defines identifier with the subset reserved identifier, which we call keyword.

Btw, our IDENTIFIER is a token.

^ as is anything in SCREAMCASE

Hmm.

We cannot distinguish by case as the VBA spec does because the case has a semantic meaning in antlr grammars.

but what about this? msdn.microsoft.com/en-us/library/ee199674.aspx

IDENTIFIER = <any lex-identifier that is not a reserved-identifier>

8:50 PM

@this Pocket protectors too?

@IvenBach pfft. Don't be silly. I don't settle for a mere pocket protectors. I encase them in a metal case. Of a sufficient strength to stop bullets. Just in case.

@this tl;dr: in Antlr grammars UPPERCASE rules define lexer tokens, and camelCase rules define parser syntax

@this for that we spell out all the reserved identifiers, and they take precedence over IDENTIFIER when tokenizing.

see VBALexer.g4

I see

the IDENTIFIER rule is pretty much the last lexer rule; only LINE_CONTINUATION comes after it - beyond that all we have are fragment rules

even that is a bit more than what is allowed as per the specs for DefType as Vogel earlier said.

8:57 PM

hmm

defDirective : defType whiteSpace letterSpec (whiteSpace? COMMA whiteSpace? letterSpec)*;
defType :
        DEFBOOL | DEFBYTE | DEFINT | DEFLNG | DEFLNGLNG | DEFLNGPTR | DEFCUR |
        DEFSNG | DEFDBL | DEFDATE |
        DEFSTR | DEFOBJ | DEFVAR
;

not sure how/why they'd come up as identifiers

it's the letterSpec

letterSpec : singleLetter | universalLetterRange | letterRange;
singleLetter : unrestrictedIdentifier;

oooh

hmm

should be an inline lexer token

yeah, I'm sure that's what Vogel is going to do to better align to the VBAL specs.

universalLetterRange : upperCaseA whiteSpace? MINUS whiteSpace? upperCaseZ;
upperCaseA : {_input.Lt(1).Text.Equals("A")}? unrestrictedIdentifier;
upperCaseZ : {_input.Lt(1).Text.Equals("Z")}? unrestrictedIdentifier;
letterRange : firstLetter whiteSpace? MINUS whiteSpace? lastLetter;
firstLetter : unrestrictedIdentifier;
lastLetter : unrestrictedIdentifier;

the whole block needs revisiting

« first day (1323 days earlier) ← previous day next day → last day (1857 days later) »

Transcript for

Jan '1819

VBA Rubberducking

This chat has moved to Discord: discord.gg/MYX9RECenJ

antlr com-interop parsing rubberduck vb6 vba whiskey-tango-foxtrot

join this room
about this room

00:00

06:00

12:00

18:00

all times are UTC

site design / logo © 2024 Stack Exchange Inc; legal

mobile