Tests and stuff

Dan Pantry

14:43

So I was speaking to a manager today who said that refactoring is evil and you should never do it because it introduces more bugs :D

Not my manager thankfully

Mast

Managers are evil, refactoring isn't.

6

Mat's Mug

@DanPantry facepalm

Dan Pantry

He said this because an airplane controls ystem (or something) didnt' allow refactoring at all

that he worked on.. or his company worked on

Ismael Miguel

Why would it be bad?

Dan Pantry

Because it can introduce bugs

Which is true, it can

But so can, you know, writing any code

Mast

14:44

High-profile stuff has it's own rules, it's where legacy code is holy.

But if a manager for a regular company would spout stuff like that, I'd leave.

skiwi

Don't break stuff if it ain't fixed

Ismael Miguel

Refactoring may also uncover old bugs

skiwi

I can understand that there's no reason to refactor if things are working as intended and barely require any maintenance

Dan Pantry

@Mast It's a company that is a consultancy for the program we ar working on right now

Said code has plenty of italian-dish-style code

Mast

@IsmaelMiguel Not in those circles, they usually test very thoroughly.

skiwi

14:45

It's mostly that bad code + maintenance results in a hell

Dan Pantry

And zero unit tests

Ismael Miguel

@Mast I guess Dan proved you wrong, a bit

Dan Pantry

And I said "why dont' we refactor this?"

"refactoring is evil, you will just introduce more bugs"

You mean, you want me not to refactor and to copy-paste this url everywhere?

nhgrif

You can't introduce bugs if you write adequate automated tests.

Dan Pantry

And then when we have to change it, copy paste it all over again?

because that won't introduce bugs, right?

his response was, I shit you not

Mast

14:46

@IsmaelMiguel If you don't refactor AND don't unit test, you're creating a disaster.

Dan Pantry

"That's what find and replace is for"

3

jacwah

@nhgrif You can't introduce bugs if you write adequate code

Dan Pantry

@jacwah false

nhgrif

Yes you can, @jacwah

Dan Pantry

every code has bugs in it

Madara Uchiha

14:47

@jacwah Very, very false.

Dan Pantry

You should always assume your code has a bug in it

Madara Uchiha

s/a bug/bugs/

jacwah

I mean, you can have bugs in your unit tests too

Dan Pantry

Yes, you can, @jacwah

Mast

My bugs have plenty of code.

2

Ismael Miguel

14:47

If you can't find a bug, you are doing it wrong

Dan Pantry

However, if you have bugs in your unit tests

Madara Uchiha

@jacwah Yes, but then you need to have an overlapping bug in two systems

skiwi

@DanPantry That... I don't even

Dan Pantry

Then your unit tests are too complex

JohnSnow

@jacwah I think I just spotted another padawan, like me! :D

Madara Uchiha

14:48

That's far more unlikely than one bug you didn't find because you didn't have a test

Mast

@DanPantry I heard you like unit-testing, so I wrote a unit-test to test your unit-tests so you can unit-test while you unit-test.

2

Madara Uchiha

@Mast Oh, I do that.

Most people tend to think that "test code" is just "test", they tend to forget that it's also "code"

I write tests for my testing framework as well

jacwah

What I'm saying is that you can never be sure there are no bugs, even if you have tests

JohnSnow

Something that's fun is spending days debugging a script only to find it was the trigger script that had the bug.

Dan Pantry

And honestly, eschewing unit tests because "you might wrong your unit tests wrong" makes no sense. If you're writing bugs with that frequency you shouldn't be writing production code in the first place.

Mast

14:49

But in the end, your unit tests should boil down to such clear statements it's hard to introduce a bug in it.

Dan Pantry

@jacwah unit tests do not prove the absence of a bug

but neither does manual testing

Madara Uchiha

For example, if I test DOM in JavaScript I have a few base functions I always use

jacwah

@DanPantry That's what I'm saying

Dan Pantry

and unit testing is a hell of a lot faster and cheaper to do and less error prone than manual testing

Madara Uchiha

Those functions must be trusted, so I test them as well

So yeah, testing your test code is a good thing

Eventually, you can't really be 100% sure

But 99% is a whole lot better than 0.

jacwah

14:50

The point being that of course you can introduce bugs when refactoring with tests

Dan Pantry

TLDR though, manager says refactoring is bad because it introduces bugs, his developers don't know how to unit test

@jacwah Again, though, your integration tests should catch any bugs that affect existing functionality

Madara Uchiha

@jacwah Of course you can, but you are almost sure to introduce bugs when refactoring without tests.

skiwi

@jacwah There are ways to ensure that you have no bugs

jacwah

Even though it's less likely

Madara Uchiha

When something breaks, two tests should fail.

One unit test, and one integration test.

Dan Pantry

14:51

Any regressions can be caught by the tests you have written

jacwah

@skiwi How?

Dan Pantry

That's why tests are awesome

skiwi

@jacwah Formal verification

Madara Uchiha

I use unit tests to only cover defined behavior.

Dan Pantry

If a test doesn't break and you still have a bug, then you write a test, fix the bug, and there you go - You'll never find that bug again

Madara Uchiha

14:51

^

skiwi

@DanPantry That is something a lot of companies don't seem to understand

Madara Uchiha

@skiwi Well, to be fair, it's not always that simple.

JohnSnow

Sometimes, manual testing > automated testing

Madara Uchiha

There are structural and architectural things to consider more often than not.

@JohnSnow Why not both?

skiwi

But there are occasions when bugs recur like 5 times in new patches

Madara Uchiha

14:52

What's stopping you from testing things manually, but then also writing automated tests so that you don't have to do the same manual testing over and over again?

Vogel612

@JohnSnow that's when you should think over your architecture

Madara Uchiha

You know that tool that does repetitive and annoying tasks very efficiently, very quickly and without complaining?

JohnSnow

@MadaraUchiha well, in the cases where manual testing > automated testing, the automated testing already run and failed to find the bug :P

Madara Uchiha

Yeah, it's called a Computer, let it do its work for you :P

@JohnSnow Then you need to improve your automated test code.

Dan Pantry

@JohnSnow That's when you find the bug, replicate the steps you took in code, and then fix the bug

JohnSnow

14:53

@Vogel612 Probably,still doesn't change the fact that sometimes, you have to do manual tests because the automated ones can't catch it.

Dan Pantry

If the bug ever appears again, your test will show it

Mast

@MadaraUchiha Undefined behavior should default to a certain state though. Try to catch as much as you can.

Dan Pantry

you should never be writign a program monolithic enough (or badly enough) that you can't test it with automation

Madara Uchiha

@Mast undefined behavior = nasal demons

I don't care, and neither should you.

Vogel612

@JohnSnow true, but strong indicator you should refactor

Dan Pantry

14:54

(key word: should, obviously, you do have to do this sometime)

Madara Uchiha

If it defaults to a certain behavior, it was defined.

JohnSnow

@Vogel612 Agreed

Madara Uchiha

@JohnSnow The only "bugs" an automated test cannot find, is structural problems

Mast

I recently wrote a server. Whatever happens, if it's not defined it should still not hang-up the server.

When in doubt, the connection gets severed.

Madara Uchiha

"This layer leaks abstraction, and cause weirdness with global state"

An automated test can't catch that bug, and even a manual test will take a skilled developer to

JohnSnow

14:55

I'm agreeing with you

Madara Uchiha

But that's not a bug that should happen

JohnSnow

You make it seem like I said manual testing is ALWAYS better than automated

Madara Uchiha

If that's the bug you're having, you may need to make super extensive refactors and maybe even a rewrite.

@JohnSnow On the contrary

I claim that automated tests are nearly always better than manual ones

Dan Pantry

No one said that, @JohnSnow. However, manual testing should never be the only testing you do, and automated testing is almost better in every single aspect to manual testing

Madara Uchiha

Manual testing is great for sanity checks

JohnSnow

14:57

@DanPantry Agreed

Madara Uchiha

Not much more

Dan Pantry

automating testing will follow instructions as you wrote them 100x in a row without making a mistake and will do it in milliseconds

The only way an automated test will ever be wrong is if you gave it the wrong instructions

But hey, you can do that to a human, and a human may just execute them wrong as well :P

Madara Uchiha

I had a function whose input consisted of any of the 127 characters in the standard ASCII first half

Captain Obvious

1

Q: Standard deviation of hourly temperatures of 2 days

It seems that when I program in lisp my brain goes on auto pilot and I end up solving the problem somehow. I don't even think I just do and it works out. That said this is some horrible lisp code that was hacked together in about 20mins just to see if I could do it. The c++ program that this is...

skiwi

@DanPantry Unless you sleep for multithreaded code

Madara Uchiha

14:58

To test it manually, you'd have to sit and insert character by character, whereas in an automated test, I just wrote a for loop :P

Dan Pantry

@skiwi Why the hell are you using sleep()? :P

Madara Uchiha

When would manual tests be OK? For a sanity check to make sure that you don't have stupid errors

You check for one or two characters and move on

skiwi

@DanPantry Well, I'm not, but it's a proven way (tm) to write broken test code that does not validate broken code

Dan Pantry

@Madara or for UX reasons in terms of web dev

Madara Uchiha

But the actual test, the one that fails the build, is automated

Dan Pantry

14:59

@skiwi If you show anyone a test that uses sleep, they will immediately say "Why teh hell are you using sleep()". Then you have the peer test failing :P

Madara Uchiha

@DanPantry Well, those cannot really be quantified in the form of "pass" or "fail"

Dan Pantry

@MadaraUchiha No, but you can write perfectly good automated tests in selenium that look like rubbish in the browser

That's basically the only manual tests I do in my own projects

Transcript for