last day (18 days later) » 

13:01
173
Q: Leaving intentional bugs in code for testers to find

KrishnabhadraWe don't do this at our firm, but one of my friends says that his project manager asked every dev to add intentional bugs just before the product goes to QA. This is how it works: Just before the product goes to QA, the dev team adds some intentional bugs at random places in the code. They prop...

Lol. "my, err, friend would like to know...."
@ScottWhitlock: the other question just asked for a term or name, not for any drawbacks, so it is clearly not a duplicate.
@ScottWhitlock I saw that question. A decoy bug is added to divert management attention from any other section in the product. My question is entirely different.
@ScottWhitlock but it doesn't explain any drawbacks of Red Rabbit Test (thanks for the name). So it is a known practice, and at least some people follow this (so they had to name it)
@ScottWhitlock: your answer does not tell anything about a "red rabbit test" beeing a good or a bad thing. See the answers here below, they are quite different from the answers of the "term" question.
13:01
On the other hand, there is also "defect injection," wherein some number of known defects are inserted in the software, and the rules of conditional probability are used to estimate the number of unknown defects based on the ratio of known defects found to defects injected.
Perhaps the testers should respond by inserting random features in the product?
They properly back up the original, working code to make sure that those bugs aren't shipped with the end product. What could possibly go wrong? ;)
I believe that this works for you friend's company, after all, they tried it, and have evidence that it works and I don't have any to counter that claim. But not all companies are the same. I very much doubt it would work in our company for example, where we view the whole team as a unit and a primary pillar is trust that everyone can and will do their job.
Testers feed on bugs. Not finding any bugs will affect their morale. So giving them an easy one to find will help their morale. - I don't think finding a bug, after being told that bugs were introduced on purpose, helps morale at all.
@DoubleDouble: Not only that, but I would expect that deliberately-introduced bugs would be sufficiently different from accidental ones that people who become more focused on finding the former might be more likely to miss the latter.
13:01
I know I am expected to write a lengthy answer, but the point is, the QA team tests a completely different software, that never gets shipped to the customer.
I envy your friend, working at a place where software quality is so high they worry about not having enough bugs!
Intentionally adding ducks (#5) should be an indicator that there is a problem.
I am imagining stuff like this - it's a bug for the QA team to find, right?
Don't you have enough bugs for testers as-is? If they need softballs to find bugs, then all your QA is good for is finding obvious bugs. If you need to plant obscure bugs to get them to test thoroughly, well, trust me: You already accidentally planted obscure bugs. It sounds like someone is cya-ing some heat they're about to drop on QA. "We had 7 obscure bugs planted, and only one found. What's going on in QA?" Too often, positive change can only occur [in a bad co] if you've got metrics showing what's wrong now, and it's hard to show how many unknown bugs weren't found. Good luck.
"They properly back up the original, working code" Whoa! Aren't they using source control?
13:01
"A test should think like a developer"... interesting. I would have thought it was obvious that a tester should not think like a developer but like a user.
it could be a fun game, it just depends on how the results are handled. an informal approach with learning as the goal, with no middle management involved. in my experience, testers are some of the most thorough and hard working members of the team, and i bet they would find these bugs.
I once knew a 'tester' who was testing high quality software (almost no bugs). In fact he was only filling the form with 'ok', and playing flash games, testing only when a boss was near. I guess he wouldn't like this method...
I'm tempted to tell our testers that we do this, without inserting any bugs. At least we could claim that the first bugs found were intentional, and maybe we could get some other benefits too. Now all we need is to actually have testers for our software...
I think this question falls under "Good Subjective" as described in Good Subjective, Bad Subjective. (But that is a subjective opinion.) The close votes based on this being primarily opinion based or on being a duplicate are off base.
This is just awful, for all of the reasons that have already been mentioned. If there's concern about QA's processes not resulting in an effective test suite, there are techniques like mutation testing that can quantify this.
13:01
Wow. Way to widen the empathy gap between DEV and QA even more... DEV and QA at most companies that I've worked for have had an unhealthy relationship, where QA is not treated as equally important as DEV. This is straight up disrespectful and I would encourage your QA staff to quit until the technical director started showing some basic respect.
Yeah, this sounds like a great way to create an antagonistic relationship between coders and QA, which is the LAST thing you want.
Note that it's entirely possible for an intentional bug to mask an accidental bug, such that it's impossible to find the accidental bug.
Even more evil option: Tell the testers you've introduced intentional bugs, but then don't.
if (environment == Environment.Production) EraseDatabase(); Console.Write(projectManager.GetsItYet);
Sounds like a stupid idea. Bugs are not parametrisable objects that can be used to calibrate some bug finding curve. They can range from coding accidents to fundamental design issues. If I was testing, it would either seem like a waste of effort or a lack of trust in what I am doing.
13:01
"They properly back up the original, working code" And hope that the none of that had an unintentional bug that got replaced by an intentional bug?
Tim
Tim
A long time ago I worked at a company that did this. I'm not there anymore - and neither are most of the developers I worked with.
You should introduce bugs that appear based on System Time so you can drive testers totally crazy which will boost their morale even higher.
I am guessing the turnover rate for QA at that company is extremely high.
leo
leo
Leonardo da Vinci is said to have inserted intentional bugs in many of his war machines, for completely different reasons though
So the code to be installed is not the actual code being tested? (blink) WTF?!?!?!?!?!

  last day (18 days later) »