> > When I have searched the Rubberduck source for references it seems to be an easter egg status (?). As far as I can tell it triggers if there are more than 11 failures in a test, but I might be wrong.
That is correct... the idea being that a useful unit test should `Assert` one thing, i.e. have no more than a single reason to fail.
I'm not against discarding it, but I would think implementing data-driven tests (ref. #3699) would be a better approach than using `Assert` in a loop. Of c
> > When I have searched the Rubberduck source for references it seems to be an easter egg status (?). As far as I can tell it triggers if there are more than 11 failures in a test, but I might be wrong.
That is correct... the idea being that a useful unit test should `Assert` one thing, i.e. have no more than a single reason to fail.
I'm not against discarding it, but I would think implementing data-driven tests (ref. #1229) would be a better approach than using `Assert` in a loop. Of c
> > When I have searched the Rubberduck source for references it seems to be an easter egg status (?). As far as I can tell it triggers if there are more than 11 failures in a test, but I might be wrong.
That is correct... the idea being that a useful unit test should `Assert` one thing, i.e. have no more than a single reason to fail.
I'm not against discarding it, but I would think implementing data-driven tests (ref. #1229) would be a better approach than using `Assert` in a loop. Of c
> > When I have searched the Rubberduck source for references it seems to be an easter egg status (?). As far as I can tell it triggers if there are more than 11 failures in a test, but I might be wrong.
That is correct... the idea being that a useful unit test should `Assert` one thing, i.e. have no more than a single reason to fail.
I'm not against discarding it, but I would think implementing data-driven tests (ref. #1229) would be a better approach than using `Assert` in a loop. Of c
I'm pretty sure he's somewhere in a tux, driving a fast sporty car with 2 women in his arms, evading some evil villain bent on world dominion, and drinking a stirred martini meanwhile.
> Hmm, well, having some sort of way of supporting DDT would obviously help, but it seems to me like you are adding unnecessary restrictions to dictate how people use your tool.
I agree that in a perfect world with perfect code a unit test should test one thing, hence the name, but sometimes, when your code isn't perfect, you rather go for testing one concept that feels small enough that the test makes sense. Say 10 conditions you feel like should be fulfilled in one scenario, and these 10 th
> Hmm, well, having some sort of way of supporting DDT would obviously help, but it seems to me like you are adding unnecessary restrictions to dictate how people use your tool.
I agree that in a perfect world with perfect code a unit test should test one thing, hence the name, but sometimes, when your code isn't perfect, you rather go for testing one concept that feels small enough that the test makes sense. Say 10 conditions you feel like should be fulfilled in one scenario, and these 10 th
- Easter egg test outcome in its current form is harmful and needs to be removed; - `@TestCase` annotation and adding support for parameterized test methods needs to be added; - A new inspection would be flagging tests with more than a single `Assert` call, and/or tests that `Assert` in a loop. The inspection's xml-doc and meta-description would justify the result with the widely-accepted best practice of single test == single assert, and code examples would illustr
- Easter egg test outcome in its current form is harmful and needs to be removed; - `@TestCase` annotation and adding support for parameterized test methods needs to be added; - A new inspection would be flagging tests with more than a single `Assert` call (excluding `Assert.Inconclusive` and `Assert.Fail`), and/or tests that `Assert` in a loop. The inspection's xml-doc and meta-description would justify the result with the widely-accepted best practice of single te
> Sounds great. I did a bit of reading and couldn't find any good arguments for single test == single asserts, isn't it better to test concepts than semantics? Sure, you should test one thing, but that one thing will in many cases be much more readable if it is done with multiple asserts. E.g. testing if a value is between a range is much more readable by testing two less than asserts than one big logic blob. Oh well, this is not the place for these discussions and if you make it an inspection c
> - [Official Microsoft guidelines](https://docs.microsoft.com/en-us/dotnet/core/testing/unit-testing-best-practices#avoid-multiple-asserts) "When writing your tests, try to only include one Assert per test." - [Some stackify blog post](https://stackify.com/unit-testing-basics-best-practices/) (no direct link to the "best practices" section) "I won’t go so far as to say that no test should ever contain a number of assertions other than one. But I will say that your unit test suite should have
> But you also have [this](https://softwareengineering.stackexchange.com/questions/7823/is-it-ok-to-have-multiple-asserts-in-a-single-unit-test) pretty popular stackexchange post where almost every post agrees that they do multiple asserts in an effort to test one feature/behaviour. One assert just seems like a really unnecessary restriction that doesn't necessarily correlate with what you want to achieve, testing one behaviour.
Again, it seems like something to strive for because striving fo
> I think the Stackify article says it best: "Unit testing newbies commonly make a mistake of testing all of the things in one test method." -- basically the "one test, one assert" guideline is there to remind beginners that the *Single Responsibility Principle* also applies to test methods, so that when a test fails, it's immediately apparent why.
The accepted answer on that SE thread also puts it nicely:
> but I do think we should **strive towards** only having single asserts in our test
> But you also have [this](https://softwareengineering.stackexchange.com/questions/7823/is-it-ok-to-have-multiple-asserts-in-a-single-unit-test) pretty popular stackexchange post where almost every post agrees that they do multiple asserts in an effort to test one feature/behaviour. One assert just seems like a really unnecessary restriction that doesn't necessarily correlate with what you want to achieve, testing one behaviour.
Again, it seems like something to strive for because striving fo
> But you also have [this](https://softwareengineering.stackexchange.com/questions/7823/is-it-ok-to-have-multiple-asserts-in-a-single-unit-test) pretty popular stackexchange post where almost every post agrees that they do multiple asserts in an effort to test one feature/behaviour. One assert just seems like a really unnecessary restriction that doesn't necessarily correlate with what you want to achieve, testing one behaviour.
Again, it seems like something to strive for because striving fo
> I agree, sorry, didn't mean to argue, I can get quite passionate on discussing rules for best practices as I feel like they should be guidelines and not rules quite often.
Anyway, reading through what I say I totally agree that the inspection should be there, and I was in essence mostly arguing about whether it should be turned on by default or not. This is obviously not something I should really have strong opinions on as it is very easy for me as an (overly) opinionated user to switch tha
> Rubberduck has always been a pretty opinionated tool. I suspect it will stay that way until it becomes absolutely clear that no one is going to step up and create an alternative. That is life.
@retailcoder in regards to data driven tests, can we think about how we might implement them so the user has compiler assistance? I like the annotation as a first step, but it’s not as friendly as it could be. I’ll be honest, I don’t really know how we’d do that. I recently helped a friend implement
> Rubberduck has always been a pretty opinionated tool. I suspect it will stay that way until it becomes absolutely clear that no one is going to step up and create an alternative. That is life.
@retailcoder in regards to data driven tests, can we think about how we might implement them so the user has compiler assistance? I like the annotation as a first step, but it’s not as friendly as it could be. I’ll be honest, I don’t really know how we’d do that. I recently helped a friend implement
> @rubberduck203 the `@TestMethod` annotation is already parameterized and takes an optional "category" argument, so DDT will require a new annotation; I like `@TestCase` , with the arguments matching the signature of the test method.
It could look like this:
```vb '@TestMethod("test category") '@TestCase(1, "foo") '@TestCase(0, "foo") '@TestCase(-1, "foo") '@TestCase(1, "bar") '@TestCase(0, "bar") '@TestCase(-1, "bar") Public Sub SomeTestMethod(ByVal arg1 As Long, ByVal arg2 As St
do we get compile-time validation of test cases in NUnit?
I think not.
(at least not beyond verifying that the attribute is well-formed)
My hunch is that it's probably easier to just hijack the Compile menu so that RD can "compile" its annotations to offer a compile-time validation of the annotations which would satisfy Chris' concerns.... (if I understood him correctly)
> My initial thought is that what if you have a prime generator and have a lookup for say the first 1000 primes and want the test to just run through all of them as a dummy just to be sure? Or in my case I have a random user input generator to try to catch some scenarios I can't think of myself and want to run this 10000 times to see if I am missing something.
> I would say that the question of a random data generator is a separate question from the question of data driven test. That definitely needs its own issue & discussion, apart from this one here.
> I think allowing parameterized tests with the parameter values supplied by an annotation is a good first step towards enabling data-driven tests. Further enhancements could include adding a `@TestCaseSource` annotation that can be substituted to a plethora of `@TestCase` annotations to load the test cases from a file with a specified format - but the first step is really to just enable parameterized tests.
> *The only way to eat an elephant, is one bite at a time.*
> > @retailcoder in regards to data driven tests, can we think about how we might implement them so the user has compiler assistance?
Looking at NUnit as an example, the only compile-time validation we get is whether the attributes are well-formed. I doubt those will get flagged as a compile-time error:
``` [TestCase("ha ha I am a number!", 42)] ... public void derp(int someNumber, string someString) ```
But I appreciate the desire to have compile-time validation of the `TestCase a
> > @retailcoder in regards to data driven tests, can we think about how we might implement them so the user has compiler assistance?
Looking at NUnit as an example, the only compile-time validation we get is whether the attributes are well-formed. I doubt those will get flagged as a compile-time error:
``` [TestCase("ha ha I am a number!", 42)] ... public void derp(int someNumber, string someString) ```
But I appreciate the desire to have compile-time validation of the `TestCase a
> > @retailcoder in regards to data driven tests, can we think about how we might implement them so the user has compiler assistance?
Looking at NUnit as an example, the only compile-time validation we get is whether the attributes are well-formed. I doubt those will get flagged as a compile-time error:
``` [TestCase("ha ha I am a number!", 42)] ... public void derp(int someNumber, string someString) ```
But I appreciate the desire to have compile-time validation of the `TestCase`
> > I would say that the question of a random data generator is a separate question from the question of data driven test. That definitely needs its own issue & discussion, apart from this one here.
Agreed, but wouldn't it to some extent be useful if you could find a solution that solves both DDT and some sort of data generated testing (not entirely sure what this paradigm is called when you have computer generated input with bounds for validity).
> > I would say that the question of a random data generator is a separate question from the question of data driven test. That definitely needs its own issue & discussion, apart from this one here.
Agreed, but wouldn't it to some extent be useful if you could find a solution that solves both DDT and some sort of computer generated testing (not entirely sure what this paradigm is called when you have computer generated input with bounds for validity).
taking annotation params and turning them into test args is as far as I'm willing to go, and that's only because the alternative is a massive can of worms that will take forever to implement.
I really don't like the idea of executable comments
I'd really go with an inspection - that way if/when we ever get the results into the code panes, we can have red squiggles under the annotation with a "test case mismatches test method signature" tooltip
> Also (sorry for double posting), I am looking at solving the titular issue, i.e. "Unit testing with for loops", and there are generally four scenarios where I find myself doing that:
1. When I have a large lookup table of the results (sure, I could run a regex on it to generate test cases, but that isn't exactly ideal) 2. When I am simply replacing a `SequenceEquals` because it is easier to check the elements individually than create arrays that this method accepts, or if I only want to ch
@IvenBach You've done great, and starting from an earlier spot than me. C# has gotten more complex over the years, when I first used it in v2 there were far fewer abstractions.
But no matter what stage one is at, contributing to a (well-run) OSS project can help one get better.
Question: do we want to expend any effort at all supporting Office Developer Edition? Let me qualify that - ODE is Office + a very small subset of VB6. Support would likely just consist of making some VB6 commands available.
But I ask because it's been discontinued since forever (even though you can hack it back to life with a reg key...)
Actually, let me rephase... If I PR'd some small change that recognised its existence, would it be merged?
Example - ODE lets you add standalone projects. I'm contemplating a feature-request to add 'Add Project' to the VB6 context menu. That could also be exposed to ODE.
ODE adds stuff to the File menu
Another example - CE Add - for ODE standalone projects it can also include "ActiveX designer"
ok cool. Not because I expect many to use it, more because there's bits of the wider OM that we're ignoring atm. Sets off my OCD ;-)
Wonder if a blog post on "How to get Office 2019 Developer Edition" would be appropriate (or legal!) Would basically be 1) Buy ODE 2000 or XP. 2) Install it in a VM 3) Find this reg key 4) Copy it to your Office 2019 machine...
But realistically that might just make people skip step 1, 2 and 3 and google the key value...
Proof that the VBE has been ignored by MS since forever. The "check for license key" would have been in 2000 and XP. So the easiest way to discontinue was just to stop shipping an SKU that set that key.
VBA Rubberducking
VBA Rubberducking