« first day (1629 days earlier)      last day (1551 days later) » 

12:01 AM
RELOAD!
[rubberduck-vba/Rubberduck] 4 opened issues. 1 closed issue. 6 issue comments.
[rubberduck203/VBEX] 1 issue comment.
 
Brother picked up a virus. That means I have a miserable 2-3 weeks ahead of me in the near future.
Just when I'm totally swamped at work, too...
 
#TIL all of the web links in VB6 (Main Menu->Help->Microsoft on the Web) are dead. No 404, just a blank page. Only an MS favicon to hint that it's actually resolved.
Wonder if anyone at MS even notices the occasional hit on them
 
12:20 AM
#TIL the importance of planning out a deployment. Can't change file path else it'll break those that currently use it.
 
[rubberduck-vba/Rubberduck] build for commit a0a59f5c on unknown branch: AppVeyor build succeeded
 
12:45 AM
@mansellan well, if you find yourself on a no-fly list, then yes.
 
uh, i guess I could have phrased that better!
 
Seriously I'm sure their response will be "what? We don't 404 it?"
 
^^
 
1:23 AM
Achievement unlock: make a XAML that crashes Excel
 
Wut?
 
 
11 hours later…
12:26 PM
 
12:41 PM
@IvenBach I made an answer out of that comment. I figure it's been more than a year, and I credited the original guy, so I'm covered.
@mansellan Need to be able to theme the RD windows to match the VBE...
@Hosch250 Nuke it from orbit - complete reformat and reinstall - it's the only way to be sure!
 
12:59 PM
@FreeMan or theme the VBE windows to match RD:-)
 
ummm... I've already got black on white. I'm not so thrilled with that theme. :(
I might be OK with a duck on black theme, though!
 
1:55 PM
Looking at the source, it feels like it will be impossible to improve memory and CPU performance :c The way Declarations are handled is impossible to rewrite without breaking lots of stuff
I wish the project was far less harder to understand. I've been doing C# for a few years and it's a nightmare to understand RD :c
 
@Elcan have you done multithreading or parallel task library in your C# work?
 
> Woohoo! Thank You All, I've finally got it!

**Code for x64 version** of Office (for x32 you have to replace `LongPtr` by `Long` and remove `PtrSafe`):
```
Option Explicit

Declare PtrSafe Function FindWindow Lib "user32" Alias "FindWindowA" ( _
ByVal lpClassName As String _
, ByVal lpWindowName As String _
) As LongPtr

Public Declare PtrSafe Function SendMessageA Lib "user32" ( _
ByVal hWnd As LongPtr _
> Woohoo! Thank You All, I've finally got it!

**Code for x64 version** of Office (for x32 you have to replace `LongPtr` by `Long` and remove `PtrSafe`):
```
Option Explicit

Declare PtrSafe Function FindWindow Lib "user32" Alias "FindWindowA" ( _
ByVal lpClassName As String _
, ByVal lpWindowName As String _
) As LongPtr

Public Declare PtrSafe Function SendMessageA Lib "user32" ( _
ByVal hWnd As LongPtr _
> Woohoo! Thank You All, I've finally got it!

**VBA code for 64 bit** Office (for x32 you have to replace `LongPtr` by `Long` and remove `PtrSafe`):
```
Option Explicit

Declare PtrSafe Function FindWindow Lib "user32" Alias "FindWindowA" ( _
ByVal lpClassName As String _
, ByVal lpWindowName As String _
) As LongPtr

Public Declare PtrSafe Function SendMessageA Lib "user32" ( _
ByVal hWnd As LongPtr _
> Nice work @ATolokolnikov !
 
2:18 PM
oh wow.... magic
 
wonders if we can squeeze that into a fix in 2.3...
 
it's not like that issue is tagged 04 for nothing
 
@mansellan the crucial part is just sendmessage
in theory, add a button? menu? that calls a method that does the sendmessage, done.
 
@Elcan It's mostly the parser/grammar that's hard.
The rest of the project is pretty straightforward, but those parts are quite complex.
 
TBH, I think to work in parser area of RD, you have to be quite strong in MT.
 
2:30 PM
Oh man, yes.
That's where I learned MT, and that's why I'm one of the best, if not the best, at it in my company.
 
That's why I asked about the MT and PTL. I know just enough C# to get in myself in trouble and I know that MT is very hard. Heck, I couldn't finish the suspension action without lot of help from Max.
 
The parser suspending?
 
Yes
 
Yeah, that's hard.
I remember doing that in my internship before you guys changed it up a bit.
 
@this yep shouldn't be hard. the perfect solution would be to make it act just like default (i.e. a 2-item context menu attached to the window header), but that's overkill for a quick fix
 
2:31 PM
It's insane how many pieces you have to juggle.
 
Took me a while to grasp the big picture that Max was trying to tell me regarding the locks.
 
Yeah, those were awesome.
And then the cancellation token.
That was hard to figure out.
Because when you cancel the token, then assign it to a new one--everything starts reading the new one!
 
@Elcan Nothing's impossible! Exceedingly difficult and not worth the effort maybe, but not impossible.
 
So, that's where that funky list thing comes in.
So the reference doesn't get overwritten.
 
@Hosch250 oh wow. I would never have expected that.
 
2:41 PM
Yeah, it's a reference type.
So do like this:
public void Foo()
{
    var token = new CancellationToken();
    Bar(token);
    token.Cancel();
    token = new CancellationToken();
}

public async void Bar(CancellationToken token){}
Bar's token isn't the first one--it was reference passed, and re-assigned to the new one.
private List<CancellationToken> _tokens;

public void Foo()
{
    _tokens = new List<CancellationToken>();
    _tokens.Add(new CancellationToken());

    Bar(_tokens.First());

    _tokens.First().Cancel();
    _tokens.Clear();
    _tokens.Add(new CancellationToken());
}

public async void Bar(CancellationToken token){}
 
Right makes sense. But if I went at it naively, I would have trusted too much that the token would tell me it's been cancelled.
 
This way, we don't overwrite the reference, we just remove a reference to the reference.
 
i mean, Microsoft made the class. They're smarter than me, right?!?
 
@this Same.
It took me a while to figure it out.
 
But that's very GTK- thanks for sharing. That means I have to actually think it through even when I'm using Microsoft's provided classes.
 
2:52 PM
@this I have already done lots of multithreading, but it rarely required complex stuff. My concern is more about the code structure and less about the program flow
@FreeMan Yeah, but I wish to contribute. RD is very useful to me but all the performance issues make working with it harder and harder
When I need to export stuff from MS Access, I consitently have to close the DB, not open VBE, and do my exports. I also can't run RD on machines with less than 4 GB of RAM due to the fact it can easily reach 500-600MB of RAM (while Access alone takes only 20-30 MB)
 
@Elcan if you can get us reproducible benchmark results for where the contention is, that should already help a lot
 
Wasn't that already reported in the "Holy RAM, Batman!" issue, @Vogel612 ?
 
It was, same as the "RD parses the code 4 times when exporting" stuff, exponentially increasing RAM usage because it looks like not everything gets GCd
 
The real problem, though is that we can't actually peek inside the unmanaged memory area.
the managed memory allocation is kind of smallish but there's this big black hole of unmanaged allocation for.....
something
 
that's mostly a memory benchmark
 
2:58 PM
@Elcan I don't think it's a GC issue. At least it shouldn't be.
 
Parsing the same code multiple times does increase RAM usage. Since the code didn't change, I'm assuming something that should get cleaned between each parsing isn't
 
@Elcan IIRC, if you do several tasks in rapid succession, it will increase, yes but let it idle and it does eventually drop
because GC is async
 
GC never kicks in after I rescan multiple times, or at least doesn't clear a visible amount of RAM
Every scan adds 80MB to the RAM usage
 
hmm.
last time i checked it drops off after idling for a bit.
let me see.
 
@Elcan What kind of memory pressure do you have when you're doing this? The GC can take a while if it doesn't need to free anything.
 
3:04 PM
Okay, it kicks in when it reaches 800 MB
Looks like the GC only kicks in after reaching a thresold (no idea what it bases itself on)
 
TBH, I'd feel a bit cheated if it didn't behave like that - GC isn't cheap.
 
Problem is that sometimes it doesn't kick in fast enough
I mean, it kicks in at a too high RAM usage
 
the way modern OS are, you shouldn't free what you don't need to free.
 
I often get "Not enough memory" errors on MS Access because it can only handle so many MBs, but the GC refuses to clean anything
 
The only way I can see out of this is to move that into a shared memory area
or into its own process (which is even more bigger PITA)
 
3:08 PM
Or find a way to reduce RAM usage somehow (even if I doubt this is possible)
I'd be curious to see how Declarations are in-memory. Are there duplicate informations ?
 
we really have to understand what is really there in the dark region of the unmanaged memory.
declarations are peanuts.
last time I looked, those added up to what? 20? 30? MB
 
Gah! I just realized that someone scheduled a meeting for 3-4 pm today. I was hoping to slip out of the office for an early start to my Thanksgiving weekend. Very not fun!
 
Damn, what requires 400+ MB of RAM then ?
 
0
Q: How can I make my code run faster (copying columns)?

RavI'm new to vba and wrote the code below to match and copy columns; I'm trying to figure out how to make it more efficient as it takes some time to churn out the output. Any advice or feedback would be much appreciated! I thought of switching screen updating to false but it doesn't really speed th...

 
it's unmanaged memory that I have no clue what is going in there.
 
3:10 PM
If it gets filled, then something is filling it
 
@Elcan weren't we considering trading off RAM usage for speed by having an in-memory SQL DB of some sort? Kinda shoots that idea...
 
WPF's DirectX going nuts? Some 3rd party component? I'm not sure.
 
If hashmaps only take 20MB, then no need for an SQL DB
 
@this there's easily 45,000 of 'em though
 
oops, that was aimed @this, not @Elcan
 
3:11 PM
I assumed duplicate data and stuff in Declarations were adding up to 400MB
 
maybe it's parse trees. I honestly don't know.
I've tried to find a decent memory analysis tool with no luck.
 
@Elcan at the end of the day, whatever you do the compromise is between memory usage and performance
 
the dotPeek only tells us about managed memory and Visual Studio's memory snapshot is.... wonky.
 
"wonky" is being charitable.
 
I know, but for 20MB of RAM, the question isn't even relevant
 
3:12 PM
Do you get the same observed behavior in Excel?
The only thought that I have is that we could potentially be causing Access to leak memory from bound recordsets.
 
No idea, I have no Excel project large enough
 
Yes, I can observe similar behavior in Excel
but not at the scale because like @Elcan I don't have a large Excel project.
 
for a while I thought the 45K declarations and the many copies we cache in the DeclarationFinder was part of the memory problem - it isn't. declarations really are peanuts.
 
a typical Access project can have several forms with code behind, so there's lot of modules with lot of code.
not unusual to see 100+ VBComponents in such project.
 
3:14 PM
or in VB6
 
Yikes
 
My "stress test" Excel workbook has hundreds of modules with a bunch of (largely stoopid) code.
 
that means VB6 may suffer, too.
 
definitely
 
and unlike Access, can't be fixed by just using 64-bit version
 
3:15 PM
VB6 shouldn't be as much of an issue. It isn't holding onto a lot of data.
 
?
 
It doesn't share memory space with, say, a huge database.
 
That's not right. Without RD loaded, Access only typically consume small amount of memory.
 
I.e., when I'm running VB6, VB6 isn't managing 20 worksheets with 100K rows.
 
With Access, the database is almost always disk-bound.
 
3:17 PM
Access DBs usually take in memory what they take in HDD space
(HDD space when compacted of course)
 
I'm not saying RD isn't causing Access to consume memory - I'm saying VB6 isn't as likely to suffer from a host application using unmanaged memory.
VB6 should act like a VBE without a host application vis-a-vis memory use.
 
gotcha
 
@MathieuGuindon yea. a few pointers packed together and pointered to is not much memory
 
This would be worth trying on a large VB6 project to see if a similar-sized overhead is generated
 
I'm not sure that would narrow down the issue much if we're doing something that causes unmanaged references not to be released.
 
3:21 PM
Hence the need to pry open the dark pits of the unmanaged memory.
 
My currently open Excel project was sitting idle for a bit and was at 364M. A parse brought it upwards of 429M. A 2nd parse brought it to 476M, then a spike at >600M with a drop back to 445M stablish. A 3rd parse spiked to 515M, dropped to 470M then climbed to 570M where it's currently sitting. I'll let it idle a bit to see if it drops back to <400M
 
One of the theories was that there's unmanaged COM references
But with the cleanup of COM which may also have been too aggressive, that didn't make a difference.
 
Letting it idle doesn't change anything. C#'s GC only kicks in at certain memory levels
 
browses random SE posts for a while
 
3:22 PM
You mean like transients? Foo.Bar.Baz
 
Oh...
 
That, and the other COM leaks.
we are now guarding against that.
 
Mine usually kicks in when a program reaches 800 MB of RAM. I tried it on one of my projects and the GC is consistent
 
So that leaves the other theory -- that it's WPF's directX
because WPF is too dumb to not page what it's not showing, maybe?
 
I'm all for throwing some hate at WPF.
2
 
3:24 PM
Truth be told, I'm half-tempted just to chuck WPF out and just use a web browser + HTML + CSS + JS.
That's what all kids are doing nowadays, right?
 
Although, if it was WPF, you'd be able to replicate it in an arbitrary application.
 
did we try that in the SlimDucky?
not sure if we tried with a full load of inspection results & code explorer view model items.
 
Is that some bizarre Eminem song?
 
SlimDucky? I guess that was during your hiatus. someone made a spin off of RD to test whether WPF is at fault for the crash at exit issue
but Wayne came along and showed that it was a bug in .NET's Winforms implementation and worked around it.
 
FWIW, we can use UWP controls in WinForms now!
 
3:26 PM
meh. It's still XAML.
 
LOL.
FWIW, I'm NOT helping convert WPF to web stuff.
 
LOL
 
@this "someone" = @MathieuGuindon
 
TBH, I think XAML is $1.00 and web stuff is 4 quarters.
#ScrewedEitherWay
 
3:28 PM
XAML is easier to develop, anyway.
 
At least XAML doesn't involve JS.
 
Or CSS.
 
SlimDucky "A bare-bones VBIDE add-in to troubleshoot Rubberduck's WPF dockable toolwindows "
 
And it has nice grids.
 
@Hosch250 I have my doubts about that.
 
3:28 PM
 
Just for the sake of starring
 
@this I don't.
I've done both. I work with JS/HTML/CSS on a daily basis.
 
@Duga eh?
@FreeMan after about 10 minutes, it's dropped as low as 525M, but it's now back to 544M
 
Is there a way to force the GC ?
 
Yes, but it's generally a bad idea.
 
3:33 PM
^
 
you could attach a debugger then via immediate window do GC.Collect
 
...and, if it's not managed memory, that isn't going to do anything.
 
^
 
Looks like there's 23 blocks of 16 MB waiting for a GC after a RD parsing, in my case
Looking at the memory
 
What isn't clear is if the GC is aware that it's sharing the heap with the host.
 
3:37 PM
hmm... Image is what? literally graphics?
 
I almost wonder if we should explicitly "carve out" some host memory with GC.AddMemoryPressure.
 
No, DLLs and stuff
Looks like function references from a ton of DLLS
 
> If a small managed object allocates a large amount of unmanaged memory, the runtime takes into account only the managed memory, and thus underestimates the urgency of scheduling garbage collection.
 
500K isn't really "tons" though.
 
hmm would need to test it out
 
3:39 PM
@FreeMan lock workstation, bathroom break, get coffee, RAM use down to 465M
 
@Elcan that's 370 MB then ...
@Elcan we do have a few function references explicitly declared, mostly on VBNative stuff
 
Guessing the vast majority of those are SCWs.
 
For a ~500MB RAM usage. That would give Access + RD an actual usage of about 150 MB
This seems reasonable, considering the DB alone is 50 MB
 
Is putting this in a separate shared memory a way to remove the memory pressure on the host?
 
there's also the notion that an IDE add-in like RD is NOT lightweight. this isn't MZ "msgbox assistant" or "toggle line numbers". we're talking in-depth, [mostly] thorough understanding of what the code is doing - reading in some R# comment section that one has a solution opened in VS consuming 400MB, consumes 2GB with R#. Insight doesn't come for free.
3
 
3:47 PM
According to VMMap, the large memory usage comes from memory that should be GCd but isn't
 
"Should be but isn't"? Not sure what that means.
 
@Elcan if that's the SCWs, then VMMap is wrong
sure we could GC them. and brick the VBE.
 
VMMap just gives a "GC" detail on about 30 blocks of 16 MB of RAM in RD+Access's managed heap
 
Most of them should be really short lived anyway.
 
3:50 PM
Not familiar with VMMap - is it a snapshot or a dynamic live view?
 
The managed heap is fully free when RD has not been started
Refreshable snapshot
 
That's expected. There's no managed stuff in the host application, AFAIk.
 
afaik you can't have it dynamically poll the memory, but you can force it to refresh the view with the current memory
 
@MathieuGuindon Fully agree. What does make this problematic is when we're sharing memory with the host and that interferes with host's operations.
 
I'm not convinced storing all our data out-of-process would even help a little
 
3:53 PM
That would also probably be a performance nightmare.
 
and TBH I don't want to deal with inter-process communication.
 
hey how about we ditch WPF and replace it with p/invoke and send message?
 
(what ComIntern said)
 
LOL
 
@MathieuGuindon HHCIB?
 
3:53 PM
?
 
Unparsed VBE+RD has about 40 MB of "GC" blocks, this goes up to 384 MB after a parse.
 
I'm not entirely convinced it's WPF.
 
How hard could it be (replacing WPF w/ SendMessage)
 
Simple.
 
question is, how ugly would it be
 
3:54 PM
SendMessage(WM_SHOW_THE_WPF_THING)
 
I'm having trouble to understand why Access would complain about too little memory when it's not remotely close to 2GB memory usage?
 
lol
 
@Inarion that
 
@Inarion frankly that's a question to ask the A-team.
 
Here it complains when reaching 800 MB, Microsoft maths I guess
Also it often happens when I export and RD does 4 parses in quick succession
 
3:55 PM
They are aware of that, and we know that using 64-bit Access avoids that problem.
My suspicion is that nobody seriously reviewed the memory allocation algorithms that was written 25 years ago
 
It's probably marketing. They make Access whine to increase SQL Server sales.
 
Thing is, SQL Server can't replace Access
 
since Access has had more features added and became bigger... so it's an issue even without RD in the picture.
^^
@Comintern which would be totally moronic.
 
afaik there is no alternative for that kind of easy to make desktop database with forms and a programming language included inside it
 
^
Access the front end + SQL Server :+1: Access the database - meh.
 
3:57 PM
They ditched that idea a while ago.
 
I tried to replicate the "Dynamic Table" thing that exists in Access when viewing queries/tables, both in web languages and desktop languages
Oh boy, this is impossible
 
@Comintern what idea?
 
.adp
 
yeah it was fundamentally flawed.
"let's lock you down into this one product that you can't afford and you can't use anything else!"
No surprise it didn't go anywhere.
(keep in mind that was all before SSEE was even a thing. YOu had to cough up a lot of dough for "desktop version of SQL Server" which was a POS)
 
Yep, that thing was worthless.
 
3:59 PM
One alternative would be to rewrite a proper MS-Access compatible software (LibreOffice Base doesn't count, it sucks)
 

« first day (1629 days earlier)      last day (1551 days later) »