VBA Rubberducking

Duga

00:08

REFRESH!

[Minesweeper] 200 Games Played. 98 Bombs Used. 20078 Moves Performed. 35 New Users

[rubberduck-vba/Rubberduck] 1 issue comments

[Rubberduck] 1 Synchronizations

Mathieu Guindon

00:32

Compile errors resolved, still wiring things up

(all LSP object members have the appropriate JsonPropertyName attributes)

Mathieu Guindon

00:48

I wonder how hard it could be to make RpcServerConsole into a valid NLog target

Meh, distraction

Mathieu Guindon

01:02

Once I get to the DataServer client (/LSP Server), the common bits that the LSP client would reuse are going to be under Rubberduck.RPC.Platform, where the other shared abstractions (JsonRpcServer, JsonRpcClient, etc.) are. That's separated from the shared model under Rubberduck.InternalApi

coffee.

Mathieu Guindon

04:06

hm, a grid might actually be a better idea

also, #OffByOne haha

Mathieu Guindon

04:18

eh, another distraction. #ItWorksMoveOn

just need to pull the status bar into its own control, and then implement the commands, then moving on to the dataserver client

Mathieu Guindon

05:14

much better

narrator: yet still off by one

Mathieu Guindon

09:32

1	1/15/2023 4:24:59 AM	False		Info	Starting server...
2	1/15/2023 4:24:59 AM	False		Trace	Socket server started.	Port: 3031
3	1/15/2023 4:24:59 AM	False		Trace	Uptime stopwatch started.
4	1/15/2023 4:24:59 AM	False		Trace	JsonRpc services configured.	Path(s): '/Rubberduck/DataServer'
5	1/15/2023 4:24:59 AM	False		Trace	JsonRpcServer.Start() completed in 11ms.
6	1/15/2023 4:25:12 AM	False		Info	Executing SetTraceCommand...	Parameter: 'off'
7	1/15/2023 4:25:14 AM	False		Info	Executing SetTraceCommand...

Mathieu Guindon

09:47

all commands work

Mathieu Guindon

10:01

now, the DataServer client

Rubberduck.DataServer command-line args:

public class Options
{
    [Option('p', "port", Required = true, HelpText = "Sets the RPC port for this server (required).")]
    public int Port { get; set; }

    [Option('i', "interactive", Required = false, HelpText = "Whether this server should display a UI.")]
    public bool Interactive { get; set; }

    [Option('v', "verbose", Required = false, HelpText = "Whether this server logs verbose messages.")]
    public bool Verbose { get; set; }

    [Option('s', "silent", Required = false, HelpText = "Whether this trace is disabled on this server.")]

Mathieu Guindon

11:00

Ok, let me pull all of that into a console app, and then have a DataServer.Client WPF app that will have the UI and define the client proxies.

Then Rubberduck.Server.Client (is that confusing?) defines the client proxies for the LSP server - and the add-in references and invokes these proxies.

Vogel612

12:53

@MathieuGuindon Rubberduck.LSP.Client seems less confusing

Mathieu Guindon

13:24

Ok so Rubberduck.DataServer becomes Rubberduck.LocalDbClient, and Rubberduck.Client becomes Rubberduck.LspClient

I'm going to push as a PR this time

not sure about Rubberduck.InternalApi.RPC

also Rubberduck.Server becomes Rubberduck.Server.LSP (console), and then there's Rubberduck.Server.LocalDb (also console)

RPC paths become /Rubberduck.Server/LocalDb and /Rubberduck.Server/LSP

this

I'm confused - why would be a console a "server"? Isn't a console a client?

Mathieu Guindon

tbh IDK what project type to build the server as

this

13:40

oh, I see now, you're referring tot he project type, not the actual console window itself

Mathieu Guindon

I mean it's gotta be some kind of executable, yeah

this

The console project type is probably the best fit anyway esp. for the command line argument handling and we don't want any GUI

Mathieu Guindon

lol I like my "server" GUI :D

narrator: nice client you got there

console plays nicely with stdout/stderr too

plus we get color output for free *almost

this

Nice.

Mathieu Guindon

ok I'm pushing a branch, hold on

github.com/rubberduck-vba/Rubberduck3/pull/1

this

13:54

cool! Only 760 files!

Mathieu Guindon

haha

yeah

well

LSP alone is quite a few

I mean, from a high level I guess, does stuff makes sense? and then if you drill down, do you find there what you would expect there to be?

this

do we have folders at the solution folder?

If not, I suspect that's where it'll help a lot because when I look at the projects names only, they are pretty intermingled so it's hard to follow which project is used when; a folder would go a long way in grouping all the server components, apart from parser components and so forth, I think.

Mathieu Guindon

that's a very, very good idea!

it doesn't even build yet, time to break things!

it'll help immensely

this

yes.

Mathieu Guindon

there's 20 projects and RD isn't halfway there

this

14:09

only need another 20 more and we'll have a Ducky OS!

Mathieu Guindon

LOL!

oh wait lol

more like it

Rubberduck.UI is becoming increasingly useless

this

that helps a lot. Only one small doubt - doesn't the parsing depend on stuff from VBEditor projects?

Mathieu Guindon

the code only needs the string content, so for user code we're golden

there's a part where we're going to have to have the server ask the client to give it some declarations

comms go both ways :)

this

right. I keep falling into the mental trap that it's server -> client, not server <-> client.

Mathieu Guindon

the common stuff like QualifiedModuleName and Selection is already moved to InternalApi

there's a request-response mechanism in place though

this

14:22

cool

and yes, Rubberduck.UI is probably better off as a subfolder under the respective project that needs a UI, with maybe a solution-level folder for shared UI components.

Mathieu Guindon

:)

somewhere in the LSP call chain we're going to have the parser do its thing, and in the end it's going to be issuing declarations and annotations and identifier references, ...and attributes

those will be sent to the database

saved as a transaction, so we could perhaps bunch them by scope or by module

and then we've sent a request to the client (we're still the server in the context of LSP) to respond with the COM-produced declarations, and we save those as well

the rest is basically all about reimplementing DeclarationFinder as JSON-RPC requests

M.Doerner

I think we also have to ask the client for the project meta data.

Mathieu Guindon

yeah

hm, and probably other things too

M.Doerner

There are actually not that many things not in the source code.

Mathieu Guindon

yeah it's still just a baby shell

also kind of broken on the client side ATM

I mean, the editor stuff.. it loads but it's not doing anything anymore

M.Doerner

14:31

I think there is the project meta data, including the references, the secret internal type libraries, which we primarily need for super types and locked/ignored projects, and the controls.

Mathieu Guindon

meta data includes precompiler constants?

M.Doerner

Forgot about those. But yes, that is project meta data.

I think the meta data is whatever is in the project window and the references.

Mathieu Guindon

we can already do a lot with the vbproject and vbcomponent names: QMN are used as lookup keys in plenty of places

need to find a better place for the stuff under InternalApi.RPC.LSP.Client.

that's where the server -> client operations are

the meaning of "client" and "server" in LSP is essentially request-scoped

I mean from a JsonRpc standpoint the "server" is both, and the client is both, depending on the direction of the request

I need to review these attributes, I forgot WCF stuff in there

#LiveAndLearn

RE: QMNs - I think we should package all the metadata as early as possible and notify the server so it knows as much as possible as early as possible.

Should the LSP model be broken up into sub-namespaces? I tried to not go overboard with namespacing...

this

14:57

if not sure, don't.

wait until there's a clear need for it

@MathieuGuindon because it can change and because it might be possible to not have it for some reason, the server should be able to observe that it's working without X metadata and make a record about it. That will help things, I think.

Mathieu Guindon

There's pretty much constant comm between the client and server, I'd have to look it up but I'm sure there's something in LSP that's exactly for this

client is going to need those vbide hooks

For ide level events I mean

this

yeah that wouldn't change, only that we make it a DTO thing rather than throwing around COM bombs

Mathieu Guindon

exactly

this

In other news, the hidden and restricted attributes seems to be working; will work on the attribute to support aliasing of the enum's names. Then we'll see if they'll accept the PR with the new attributes. Will get to it later.

Mathieu Guindon

Also once we have a uri for a "document", it's the server that owns it, so we get to maintain a workspace, ...syncs constantly with RDE, but only on-demand with VBE

@this that's amazing

I'm sure it'll go through!

this

15:10

@MathieuGuindon awesome.

Mathieu Guindon

there's a number of requests that transfer the ownership back

in other news... I did it again, ...laptop plugged in, sleep didn't happen. but hey we got a spiffy localdb client!

I'm thinking of adding "Requests" and "Responses" tabs, and perhaps visualize the JSON with a little tree

So many things...

this

I wondered about the sleep part. :) Keep focused; get it working and we can add gold plates later.

Mathieu Guindon

aye, got a bit more brain-dead attributes removal todo (buh-bye System.ServiceModel)

actually, that means Rubberduck.InternApi can't have the JsonRpcMethods

simple as that

using AustinHarris.JsonRpc; <~ stays under Rubberduck.Server.Something

Mathieu Guindon

15:32

this one is interesting:

window/showMessage can be called the server whenever we'd use a MessageBox; window/showMessageRequest does the same but comes back with a value :)

the workDoneProgress stuff is interesting too

textDocument/publishDiagnostics sends a notification to the client with a PublishDiagnoticsParams object that contains all the "diagnostics" for a document

reading LSP and choosing not to implement LSP for RD would have been insanely dumb

I wonder if we'll get to implement textDocument/codeLens

or rather, how we're going to tweak the editor to make codelens work :D

> 12 references

with a small action link just between a procedure and its annotations

support for ctrl+click navigation is there too

and the best part is that pretty much everything in the protocol is implementable one way or another

except the debugger stuff

Mathieu Guindon

17:23

Got about half the proxies done

After that, RPC is just an abstraction

Moving all the hard-coded string method names/paths to a namespace-scoped JsonRpcMethods static class; there's one under LocalDb and another one under LSP

Makes a nice map of the entire routing

Needs more xmldocs

Hm I think the LSPAny implementation is going to make it hard to interface with another client

Not that it's a concern, but still

C# needs union types to do it right

So, one day we'll build RD in C# 12 and make LSP match closer to spec, I guess

Mathieu Guindon

18:34

Ok, got all the proxies

Hm, having a json visualizer might be pretty easy actually... it's a read-only AvalonEdit with json syntax highlighting and folding.

Also for the localdb server visualizing the SQL queries could be nice

this

18:55

Agreed, that would help in troubleshooting. I anticipate that we may have to deal with bugs with inspections arising from getting too little or too many data from queries.

Mathieu Guindon

19:13

oof that's a good practice run... Rubberduck.Client.LocalDb client needs to start the Rubberduck.Server.LocalDb process... and then connect to the server and... either I messed up, either I need an RPC API for ~~WMI~~ server management instrumentation

..which, makes sense when I write it down like that

it's basically the last piece of the puzzle

hehe.. I'll borrow LSP's ExecuteCommand notification :)

Ok it's not that hard really; shutdown, pause, resume, and settrace all make sense and have a LSP corrolary

...and the others are just client-side UI commands

Damn, there's my "learning to walk" moment. The LSP client is much more complicated than this, I can't stumble here.

Ha, the UI is missing a StartCommand, the power button should spin up a server process if none is running (only one localdb can run)

Localdb does make a great little project to start learning about JSON-RPC though

this

20:15

@MathieuGuindon hm, need to consider how to determine if server is already running, you don't wnat to spawn mulitple server processes for the local db, right?

Mathieu Guindon

solved :)

(mutex)

this

cool

Mathieu Guindon

ugh why can't I help it lol

narrator: because you set yourself up to just like sneeze and poof it's done you dummy

for that part anyway :D

not going to bother with announcing "server capacities" with localdb

or should it be doing that?

this

20:30

Not sure I follow that part - what "server capacities"?

Mathieu Guindon

in LSP the client and server capabilities (woopsie, typo) are advertised on connect

some client may support registering capabilities later

/lazily

that and the client does the same to the server

so on init we're going to tell the LSP server that we don't do the debugger stuff

this

sounds like a debug level stuff

let me pass in a --debug flag and I can then see that kind of extra info

Mathieu Guindon

no no, things like live hints

and call stack IIRC

...things tB can do

doesn't matter, it's still LSP compliant

even the weird strongly-typed LSPAny works

Mathieu Guindon

21:03

man this is going to be so slick

Mathieu Guindon

21:54

TODO: eventually move to Protobuf / binary serialization

35

A: How are protocol-buffers faster than XML and JSON?

String representations of data: require text encode/decode (which can be cheap, but is still an extra step) requires complex parse code, especially if there are human-friendly rules like "must allow whitespace" usually involves more bandwidth - so more actual payload to churn - due to embedding...

I gather tB went somewhere along that line

Won't play nicely with JsonRpc though, so it's a conscious design decision to go with human-readable json here

I guess what I mean is, if we find we need to squeeze more perf out of our RPC ... there are options.

Greedo

22:48

You announce capabilities to say "my language server can do 50% of the commands documented in the Protocol" and the other way too "My dev-tool (client) supports save and keypress events but not hover over"

That's the gist right?

Mathieu Guindon

pretty much yeah

it reduces unnecessary socket traffic

...in a scenario where you're writing a server and you don't know who the client is

Greedo

Isn't "my server uses a local database cache ” an implementation detail rather than a capability to announce?

@MathieuGuindon that makes sense and seems clever

Mathieu Guindon

more like "this server can resolve additional properties on completion items if you ask for them"

Greedo

@MathieuGuindon just not sure what this meant

Mathieu Guindon

so you get the completion items quickly, and then user selects one and you request its description to show a tooltip

Greedo

22:51

Oh I see

Mathieu Guindon

server isn't going to bother sending some notifications it knows the client doesn't handle, etc.

Mathieu Guindon

23:25

Ok initializing localdb isn't quite like initializing the LSP server - the client that starts LSP owns LSP, but a client that connects to localdb doesn't own its process.

So I want a similar interface, but the Exit notification isn't making it there

localdb just exits when the last client disconnects

the process-starting needs to try to get the server process if it's running

no I just need a short timeout on the mutex

process-starting either launched the process, or launched a dupe that didn't get to start

so just fire it up and setup the RPC client

It's that initialization handshake I'm at now

Ugh #ImAnIdiot of course I want the exit notification, that's how the console client app is able to command it to shut down

Hm I'll share the msgbox/prompt API too

It's starting to clear up

I think

JsonRpc client is going to receive an exception probably, if the socket server is closed with something going on. Needs testing.

Duga

23:53

[rubberduck-vba/Rubberduck] 1701 stars vs. [decalage2/oletools] 2307 stars

Transcript for