VBA Rubberducking

Duga

16:13

[rubberduck-vba/Rubberduck] Gener4tor opened issue #4652: Memory consumption

> The memory consumption of rubberduck is far to high for me.

Im my case - a 100 mb access-mdb with >100 Forms - MS-access uses ~110 MB RAM (open VBE but no rubberduck refresh)

After clicking "Refresh" in the code explorer, the Memory consumption grows and grows and grows and stops at ~ 680 MB.

This is far to much in my case because when I start working with my program, I get OutOfMemory-Errors soon.

Even worse: Features like e.g. intender or some renames tends to open a lot of fo

[rubberduck-vba/Rubberduck] Gener4tor edited issue #4652: Memory consumption

[rubberduck-vba/Rubberduck] retailcoder created comment on issue #4652: Memory consumption

> 32-bit host?

this

@Duga intender? ;-)

Mathieu Guindon

if you go "indent project", it's going to open every code pane it needs to indent

this

yeah.

Comintern

That's basically a dup of like 3 issues.

this

I wonder if that would be faster using exports/imports.

^^

Comintern

16:21

I think Gener4tor opened 2 of them.

this

yes he did

Duga

[rubberduck-vba/Rubberduck] Gener4tor created comment on issue #4652: Memory consumption

> Version 2.3.0.4221
OS: Microsoft Windows NT 10.0.16299.0, x64
Host Product: Microsoft Office 2016 x86
Host Version: 16.0.4711.1000
Host Executable: MSACCESS.EXE

Mathieu Guindon

yup, 32-bit

this

and I'm pretty sure we closed them before, linking to the other issue.

Comintern

@this Wouldn't that lose undo support?

this

16:22

but.... what is the difference?

Duga

[rubberduck-vba/Rubberduck] retailcoder edited comment on issue #4652: Memory consumption

> 32-bit host?

We could (should) probably close whatever module we open to perform any given task - but I don't think we can modify a module without opening it. Indenting at narrower scope than the entire current project should help reducing the number of modules that need to be opened.

this

you only have only 20 slots in the undo stack.

if you have more than that, you're doomed anyway.

Mathieu Guindon

10, given half of them are the rewriter clearing the module contents

Comintern

And that's per VBE instance, or per code pane?

Mathieu Guindon

per VBE instance

VBE doesn't keep a per-pane stack

this

16:24

IMO, Indent Project should be considered irreversible.

Mathieu Guindon

yeah

Comintern

I suppose we could keep our own stack, at the cost of some minor memory consumption...

^ self defeating.

Mathieu Guindon

and IMO a project with 100 forms in a 32-bit host should be considered a liability

this

640 KB should be enough for anyone.

Mathieu Guindon

lol

this

16:25

@MathieuGuindon uh, sorry but that has been working for several years.

I've seen Access with even 500 forms. O_O

Comintern

A project with 100 forms should be considered a liability with any bitness.

Mathieu Guindon

@this o_O

mansellan

@this meh, just patch the runtime

this

The point being, is that memory issue has been unnecessarily exacerbated with new versions of Office.

mansellan

vbforums.com/…

Mathieu Guindon

16:26

I'd probably consider making all the forms dynamic / runtime-generated around the time the 10th form is made

Comintern

IKR?

If you can't see them all in the explorer at the same time...

this

Oh definitely. There's a big code smell right there.

But if it's "working" and they can't use RD to clean it up....

Comintern

TooManyFingFormsInspection

this

That is a problem.

Comintern

How hard would it be to optionally cap RD's memory usage?

I also think we should explicitly set the GC pressure on startup to account for the host.

this

16:32

can we even?

I still think we have to first confirm what is really going on in that mysterious blob chunk of unmanaged memory.

Comintern

Right, but if the GC thinks it has the entire process space, it's really mistaken.

this

AIUI, the pressure is on a per-object basis

probably need to allocate some value for the SCW wrappers. But I don't know how we'd tell it about the host's consumption which can change dramatically.

e.g. load PowerPivot add-in in Excel => OMG!

Comintern

This reads like it's assembly-wide: docs.microsoft.com/en-us/dotnet/api/…

I.e., we call GC.AddMemoryPressure at startup, tell it that the host is using, say 1GB or so (configurable?) and let it do its thing.

this

hmm i see.

I see why I was thinking per-object since the doc was suggesting using a dispose pattern

Comintern

Which makes sense, but we just attach it to our application object.

The dispose pattern is to release the memory pressure with the resource. We shouldn't really care about that.

IvenBach

16:38

Operation Yank Wisdom Teeth in T-minus 2hour 30 mins. Wish me luck pond.

Comintern

'luck.

this

Iven, are you sure? You need all wisdom you have.

IvenBach

If you thought I was slow before, watch out.

this

(seriously, it'll be worth it)

<= minus 4 wisdom teeth

IvenBach

They aren't painful or impacted at all. Just excessively far back, according to dentist.

Mathieu Guindon

16:41

@IvenBach you'll do fine :)

bruglesco

@IvenBach g'luck. and maybe they will give you candy. err meds.

Mathieu Guindon

Hey, my car is broken. Well, it works. But sometimes it doesn't. I can't show you the car. Can you diagnose? Do you see the problem with your question? — urdearboy 2 mins ago

3

#golden

aaand gone

BigBen

dang it, I missed it

this

16:58

@Comintern just another idea - would it make sense to use GC notifications and adjust the memory pressure ?

that way we can tell it the actual working set amount of host's memory at that instant.

Mathieu Guindon

the 'References' CE node could probably mention the project it's for, in the bottom panel; currently just says 'References'

Comintern

@this Maybe - the problem is that would only cover the managed side of the equation.

We're guests in somebody else's memory space.

@MathieuGuindon That's in the CE PR coming this evening.

Hosch250

@IvenBach My dentist said the same, but I declined the operation.

this

@Comintern not sure I follow. We add/remove pressure based on the host's memory used via WMI (or whatever) in order to give GC a better picture of how much memory is in use by the process.

Comintern

Oh, now I follow.

So make it dynamic instead of a hard carve-out.

this

17:06

yeah

that would help accommodate for scenarios like Excel with PowerPivot

or other like that.

Comintern

Right. I wonder how responsive that would be to memory space changes. I've never monitored memory in my own process space before.

this

yeah, I was thinking of keeping a coarse granularity. 25 MB chunk, always rounding up for instance.

Comintern

25 MB chunks are probably even a bit fine.

this

#NeedExperimenting

Comintern

^

this

17:09

besides i'm not sure if adding memory pressure will necessarily prevent OOM.

It may only mean more eager GC, which may do nothing to alleviate of the black blob of unmanaged memory

Comintern

More eager GC can't hurt if it's getting close though.

And it would also allow for selectively disabling functionality based on memory pressure.

this

wow GH's filtering for the issue sucks.

Duga

[rubberduck-vba/Rubberduck] bclothier created comment on issue #4652: Memory consumption

> We acknowledge that this is a big issue and agree something needs to be done about it. However, it is not easy to fix and frankly, we need help with it. That's why we still have the issue #3347 open. Note that it has a number of linked issues. I will close this as a duplicate of that issue. Even so, we do welcome any assistance, especially on identifying what is really inside the large blob of unmanaged memory in order to resolve it.

[rubberduck-vba/Rubberduck] bclothier closed issue #4652: Memory consumption

[rubberduck-vba/Rubberduck] bclothier added label duplicate to issue #4652: Memory consumption

this

no point in having several issues about same thing, IMO.

Duga

[rubberduck-vba/Rubberduck] bclothier added label memory-management to issue #4652: Memory consumption

Comintern

17:22

^^

BZngr

Wondering if the following scenario is experienced by anyone else: 1. Add Class_Initialize to a class (no content in the body) module. 2. Save the module. 3. Click "Ready" button to re-parse. The Class_Initialize Sub is removed during parsing. Note: If I put some content within the Class_Initialize body, then it is retained.

this

I don't think that's RD's thing

that's VBE's thing

I see it all the time and not just for that Class_Initialize but with an empty event handler, too.

BZngr

If I save and close without reparsing...the Sub remains.

when I re-open

this

try compiling it.

and you'll see that effect.

nothing to do w/ the saving.

BZngr

yup - OK thanks.

this

17:27

@Comintern / @MathieuGuindon I'm thinking of opening a SO question for suggestions on how to analyze the unmanaged memory; I've already looked at few memory analyzer but they fell short in giving any meaningful data. thoughts?

Comintern

I'd support that.

It's quite possible that we're leaking a crap-load of memory via WPF.

this

yeah, i'll tag WPF just in case as well as COM.

oh hey, did we fix that leak in converter?

Comintern

I should run it under a profiler.

@this Which one was that?

this

wait aminute, WPF leaks?

and I thought it was managed

Comintern

One of the reasons I have a distaste for data binding.

this

17:33

@Comintern having trouble finding that... I thought Hosch250 found a memory leak with one of WPF converter for bitmap

e.g. not disposing the stream or something like that

Comintern

Throwing converters around, attachable behavior objects, all the other stuff that's required for non-vanilla control binding.

The big one is ending up with WPF objects that hold references to what should be short term view objects.

this

I'm sorry, can you explain to me once again exactly why is WPF awesome?

Comintern

Because it makes ridiculously hard things easy and easy things ridiculously hard?

Mathieu Guindon

lol

this

yeah I saw that remark somewhere else.

given that the easy things are the 90% of the time.... #NotSold

Comintern

17:38

TBH, once you start using attached behaviors and converters, you end up with WinForms code that's been chopped up and abstracted about 6 times.

this

blah, I can't find that chat snippet about the leaking converter (if it's even a converter)

Mathieu Guindon

WinForms renders off user32.dll, which has code from user.dll, which first shipped in Win 3.x

Comintern

I was fixing leaks as I ran across them.

Mathieu Guindon

WPF renders off DX

Comintern

Right, but the code looks the same.

this

17:39

yeah, I just wanted to be sure that was fixed but darned if I can remember where that was....

Mathieu Guindon

also, hierarchical bindings in WPF, vs pain-o-matic treeview control in WinForms

this

@MathieuGuindon it can render off unicorn farts for all I care. I just want it to stay out of my way and let me write logic without getting tangled up in all its intricate plumbing details.

3

</crankyoldman>

Mathieu Guindon

make me a unicorn-shaped button in WinForms

Comintern

TreeView has it's share of pain-o-matic in WPF too. Getting the AssociatedObject from its view in a behavior is ridiculous.

Mathieu Guindon

but then you get to make a control template for every node if you want, wheras in WinForms you get a label and an icon

Comintern

17:43

@MathieuGuindon Easy. Hook the repaint event, grab a GC, and draw anything you want.

Mathieu Guindon

IOW step out of the framework, do what you want ...with tech from 30 years ago

this

age has nothing to do with it. If it makes easy thing hard, it's doing it wrong.

Comintern

Subclassing a control and manually painting it isn't really different from writing an xaml control - it's just the level of abstraction and how you chose to have it rendered.

this

</okstoprantingalreadyoldman>

@Comintern GTK to know that we can use the dotMemory to diagnose. I know that Max and I ran that before but I wasn't really paying attention to the WPF back then.

Comintern

I'll play with it tonight. I'm especially curious about what's going on in the inspection results LV.

@MatthewHagemann Can you edit your question to include the WER report that you're looking for then? — Comintern 8 secs ago

WTF? I swear I'm going to throw my VTC onto that pile.

Mathieu Guindon

17:54

DVd

> You voted to close this question Dec 13 at 17:16

Comintern

It's going to go like this. OP is going to edit in the WER. I'm going to look at it, then edit my answer to be "yeah, from the report it looks like you killed it from your process".

Mathieu Guindon

$2 the crash is caused by a bug in the C# code

Comintern

If OP is lucky, I'll add a demo program in c# to do the same thing, and end with: Go look for something like this in your sh!tty code.

Mathieu Guindon

OP is tackling that bug from the wrong end

Comintern

^

Mathieu Guindon

17:59

attach a debugger, reproduce user steps, watch VS tell you about an unhandled exception, hit F5, watch the APPCRASH get written.

this

I assume that he does not even know how.

Mathieu Guindon

or rather, attach a debugger, reproduce user steps, watch VS tell you about an unhandled exception, go "oh crap, it's a NRE", hit stop, fix the bug, redeploy, done.

Comintern

Also, if you need to "figure out a way for my program to handle an Excel crash", just wrap your COM calls in a freak'n try-catch and watch for a host disconnect error.

this

(else, he wouldn't be asking the question... right?)

Comintern

Probably true. I'll work under the assumption that OP's application isn't a rocket science extension.

Mathieu Guindon

18:07

if it is, OP is in the wrong field

this

or a noob.

I can see how he'd have hacked together a template extension put in his code and "it works!!"

Mathieu Guindon

we still don't know whether the C# code is running in-process or not

I think it could be some WinForms app creating an invisible Excel instance and crashing it

Comintern

Probably using it as a back-end database.

this

if it's a C# front-end with Excel database.....

ain't enough Picards for it.

KySoto

the whole crew of several thousand in a group picture all face palming...

Comintern

18:17

Huh, this is interesting. the 64-bit version of that crash code doesn't take Excel down.

Unless the RD subclass is "protecting" it.

Duga

[rubberduck-vba/Rubberduck] comintern synchronized pull request #4650: Locked references removal fix

Comintern

18:45

@MatthewHagemann OK, silly question. I checked my event log and found the application error events triggered by this code: "Faulting application name: EXCEL.EXE, version: 15.0.5093.1000, time stamp: 0x5bed2d14 Faulting module name: VBE7.DLL, version: 7.1.10.68, time stamp: 0x58def2a6". Is it possible that you just don't have WER configured to log them? — Comintern 9 secs ago

Listening for the sound of the deafening facepalm...

Duga

[rubberduck-vba/Rubberduck] build for commit 16d9f9b2 on unknown branch: AppVeyor build failed

BUILD FAILURE!

Comintern

@Duga ...and that deafening facepalm is mine.

3

Mathieu Guindon

lol

KySoto

the timing was perfect...

Mathieu Guindon

@Duga rather interesting that it's apparently only shadowed declarations tests failing

Hosch250

18:55

So, now that I'm about to write my articles for my app, I need to decide how to structure it.

I think I'll structure it as a 10-week course, kind of like a masters-level course.

Mathieu Guindon

intro-body-conclusion works pretty nicely usually

oh

Hosch250

@MathieuGuindon I mean, structure the whole course.

Mathieu Guindon

lol

Hosch250

I think I'll do 1. setup (install VS, run console apps--I need VS installed for my fiddle anyway).

2. Basic console input/output.

3. Control flow: if/while/for/foreach/do while.

4. Debugging.

5. Unit tests.

6. Advanced IO and more complex control flow (recursion) tentatively.

this

anyone else getting malformed class when using predeclared class template?

Hosch250

18:58

And that should be enough.

Comintern

@MathieuGuindon They were adding project references without a filepath, which was throwing in the COMReferenceSynchronizer. The tests were set up in configuration that is impossible to do in a live environment, but adding a filename broke the tests. I think I'm missing a commit in there for creating the mock environment "correctly".

Hosch250

The only reason I put unit tests so low is because you kind of need to know basic programming before you can write a test.

Mathieu Guindon

@this works fine here

you edited your template? :)

@Hosch250 0. git basics

this

I worked it out - somehow it got LF ending instead of CRLF endings.

Hosch250

Hmm, an appendix?

this

19:03

which is unexpected. I hadn't edited that template.

IvenBach

@Hosch250 adult tooth that never came in is part of the reason I’m doing it. Plus I’m a glutton for punishment. Self evident by returning to this pond :p

Hosch250

@IvenBach Ah.

IvenBach

Might only be doing 2 today.

I’ll only be a half-wit.

Hosch250

Ouch.

@skiwi had this done recently.

Got good insurance?

IvenBach

We live in ‘Murica. Nope.

Mathieu Guindon

19:07

@IvenBach they normally do that in 2 appointments... unless you want to eat with a straw for 2 weeks

IvenBach

Even if you have “good” insurance they’ll still ream you in the end.

Hosch250

But... But... Socialist California?

:P

IvenBach

@MathieuGuindon if rather be done in one go. I got Christmas break to recover.

I’m fine eating avocado samiches for my recovery. Along with bananas.

Hosch250

Banana slices between avocado slices?

IvenBach

Couple that with the fact there’s opioids on every corner I’ll be A’ok

Mathieu Guindon

19:09

lol?

Hosch250

Strawberry smoothies for desert?

this

Weird. I don't remember going in for more than one extraction.

Mathieu Guindon

actually the last thing you'll want is to use a straw for anything whatsoever for a week or two

this

Of course, I was eating tomato soups for few days and out of it.

IvenBach

Suction has likelihood of removing the blood clot.

Duga

19:11

[rubberduck-vba/Rubberduck] imbachb starred us

4

Mathieu Guindon

exactly

@Duga 7 stars ahead!

IvenBach

@this attributed to drugs.

Mathieu Guindon

LOL

IvenBach

Drugs improve my humor factor By a non negative number greater than 0

this

oh yeah, i was half-consicious

IvenBach

19:13

The proof!

this

remember being full awake but feh. not really there.

IvenBach

I don’t mind it. Last time I had a tooth extracted they only numbed me up. Felt the roof being ripped out. Interesting noise and sensation in your skull.

this

did they plug you into the funny gas for that one? I think some are done with just shots, without needing the funny gas.

IvenBach

No gas.

For a bone graft I was gassed. Waking up dry heaving was one of my least —-memorable—- enjoyable experiences.

this

yeah

Comintern

19:30

> I've been looking at other scripts but can't figure out which lines of code I have to personalize myself.

"Personalize"?

Is that some weird euphemism for "code"?

Mathieu Guindon

think so. likely non-English speaker

skiwi

@Hosch250 Correct, and I'm still alive

IvenBach

19:54

3 out of 4 ain’t bad. Last one left, for now, ensures I’m not a complete moron.

Duga

20:11

[rubberduck-vba/Rubberduck] Vogel612 added label up-for-grabs to issue #4646: Request: Export/import project including folder structure

[rubberduck-vba/Rubberduck] Vogel612 added label good first issue to issue #4646: Request: Export/import project including folder structure

[rubberduck-vba/Rubberduck] Vogel612 added label feature-export to issue #4646: Request: Export/import project including folder structure

[rubberduck-vba/Rubberduck] Vogel612 added label difficulty-02-ducky to issue #4646: Request: Export/import project including folder structure

[rubberduck-vba/Rubberduck] Vogel612 added label enhancement to issue #4646: Request: Export/import project including folder structure

[rubberduck-vba/Rubberduck] Vogel612 milestoned issue #4193: COM Collector needs to handle indexed TLBs and referenced, but unregistered type libraries

[rubberduck-vba/Rubberduck] Vogel612 did something with some project card

[rubberduck-vba/Rubberduck] Vogel612 demilestoned issue #2531: Add/Remove dialog should behave like a dialog

[rubberduck-vba/Rubberduck] Vogel612 milestoned issue #2531: Add/Remove dialog should behave like a dialog

Mathieu Guindon

@IvenBach "it ain't bad", says the guy with the mouth frozen up to the ears ;-)

Duga

[rubberduck-vba/Rubberduck] Vogel612 is checking what fun stuff @Duga can say about project cards

[rubberduck-vba/Rubberduck] Vogel612 milestoned issue #1593: Code Explorer should show References to other VBA projects, just like Project Explorer

[rubberduck-vba/Rubberduck] Vogel612 is bored so why not move a project card

[rubberduck-vba/Rubberduck] Vogel612 demilestoned issue #1276: Add/Remove Project References Dialog

[rubberduck-vba/Rubberduck] Vogel612 milestoned issue #1276: Add/Remove Project References Dialog

[rubberduck-vba/Rubberduck] Vogel612 is checking what fun stuff @Duga can say about project cards

[rubberduck-vba/Rubberduck] Vogel612 created project card

[rubberduck-vba/Rubberduck] Vogel612 milestoned issue #4630: MissingAttributeInspection needs a quickfix

[rubberduck-vba/Rubberduck] Vogel612 milestoned issue #4626: @Description annotation

[rubberduck-vba/Rubberduck] Vogel612 is bored so why not move a project card

Mathieu Guindon

(at least, I hope you're solidly frozen for this)

M.Doerner

20:36

@this It is the inspection results view where I found the potential memory leak.

this

Yeah, but I could not find the code snippet in the chat history

I wanted to check if it was fixed in the next already

IvenBach

@MathieuGuindon not yet it ain’t bad. Ask me in 3-4 hours.

M.Doerner

I do not think anybody touched the inspection results view.

Anyway, I think we should load the five or so bitmaps into a resource dictionary once and then reference them instead of loading them via the converter from file each time.

this

it would be good to compare the before & after to see if it had a observable effect on the memory leak.

Mathieu Guindon

hmm, and bitmaps are disposables and involve unmanaged memory... could this be our "big blob of unknown memory"?

this

20:40

I don't know if you saw, but Comintern linked to an blog that shows how to debug WPF memory leak

@MathieuGuindon yes that would be a likely suspect at this point.

Mathieu Guindon

atomic-facepalm: armed & ready

this

Frankly, I think that if it turned out to be just simple as that, we're going to ^^^

Vogel612

ugh... it seems like at least one of my RAM bars is faulty...

can't use more than 6 of the installed 8 GB without something going haywire

this

and that's why you shouldn't buy Dodge. ;-)

Mathieu Guindon

LOL

Vogel612

20:42

~badumm, tss

Mathieu Guindon

they're in Germany. they have VW, BMW, Audi ....who needs a Dodge?

this

for dodgy operations?

Vogel612

apropos dodgy operations: do we have an open issue for the NRE in AssignmentNotUsedInspection?

I'm kinda out of it

this

this one was the only one I found

not sure if that's what you're looking for

Vogel612

aight.

thanks

Mathieu Guindon

20:49

@this that's a PR :)

Duga

[rubberduck-vba/Rubberduck] Vogel612 opened issue #4653: NRE in AssignmentNotUsedInspection

> Taken from the Logs presented in #4645, apparently there's a NRE in the AssignmentNotUsedInspection:

```
2018-12-17 15:02:39.0728;WARN-2.3.1.4327;Rubberduck.Inspections.Rubberduck.Inspections.Inspector;System.NullReferenceException: Object reference not set to an instance of an object.
at Rubberduck.Inspections.CodePathAnalysis.Walker.GenerateTree(IParseTree tree, Declaration declaration) in C:\projects\rubberduck\Rubberduck.CodeAnalysis\CodePathAnalysis\Walker.cs:line 72
at Rubber

[rubberduck-vba/Rubberduck] Vogel612 added label bug to issue #4653: NRE in AssignmentNotUsedInspection

[rubberduck-vba/Rubberduck] Vogel612 added label code-path-analysis to issue #4653: NRE in AssignmentNotUsedInspection

[rubberduck-vba/Rubberduck] Vogel612 created project card

[rubberduck-vba/Rubberduck] Vogel612 added label bug to issue #4645: False positive 'Parameter can be passed by value'

[rubberduck-vba/Rubberduck] Vogel612 added label feature-inspections to issue #4645: False positive 'Parameter can be passed by value'

[rubberduck-vba/Rubberduck] Vogel612 added label inspection-false-positive to issue #4645: False positive 'Parameter can be passed by value'

[rubberduck-vba/Rubberduck] Vogel612 did something with some project card

[rubberduck-vba/Rubberduck] daFreeMan opened issue #4654: "Find all references" gives an error for References

> Version 2.3.1.4330
OS: Microsoft Windows NT 10.0.15063.0, x64
Host Product: Microsoft Office 2016 x64
Host Version: 16.0.4711.1000
Host Executable: MSACCESS.EXE

Expand the `References` section of the _Code Explorer_
Select _any_ of the referenced libraries.
Click `Find All References...` at the bottom of the _CE_
Boom:
![2018-12-18 15_51_00-microsoft visual basic for applications - roireport - reportdefinitioncodebehin](https://user-images.githubusercontent.com/11889733/50182323-096

[rubberduck-vba/Rubberduck] Vogel612 added label bug to issue #4654: "Find all references" gives an error for References

[rubberduck-vba/Rubberduck] Vogel612 added label feature-reference-explorer to issue #4654: "Find all references" gives an error for References

[rubberduck-vba/Rubberduck] Vogel612 added label feature-code-explorer to issue #4654: "Find all references" gives an error for References

this

@MathieuGuindon yeah i know - that was the only one that mentioned NRE w/ that inspection.

Duga

[rubberduck-vba/Rubberduck] Vogel612 is checking what fun stuff @Duga can say about project cards

Comintern

Pretty sure I touched that stuff for the worksheet inspections. I can take a look at it.

Vogel612

it might be within the execution graph generation, but it manifests in the AssignmentNotUsedInspection ...

Comintern

20:58

I also found an issue in the unused case inspection last night that I need to open an issue for.

Transcript for