Re: [eclipse-dev] Bug verification and reopening

[Rodrigo Rocha Gomes e Souza <rodrigorgs@xxxxxxxxx>]

Ok, just so I don't make a mistake when writing about Eclipse... Is it true that there's no testing team in Eclipse/Platform?

[]s

Rodrigo

On Sun, Sep 29, 2013 at 11:31 AM, Rodrigo Rocha Gomes e Souza <rodrigorgs@xxxxxxxxx> wrote:

Hello,

My name is Rodrigo Souza and I'm a PhD student at the Federal University of Bahia, Brazil (and also a Java programmer that uses Eclipse every day).

In my PhD, I'm analyzing Eclipse/Platform bug reports to try to measure the impacts of reopened bugs and discover how verification can be made more effective, so bugs are reopened less often. I've found some preliminary results, but I'd like some feedback from the community to know if the results make sense and try to raise some explanations.

First, regarding the verification process, I've found that...

* ... there's no testing team; instead, developers contribute both with fixes and verifications;

* ... often, a developer asks a colleague to verify his/her fix;

* ... verification is more intense a week before the next milestone build.

Are these observations correct?

Also, the Bugzilla HOWTO states that "it is important that the verifier be a different person than the fixer because the fixer is too close to the code and thus may not be as diligent at testing the corner cases". However...

* ... in about 40% of the cases, the fixer also marks the bug as verified. 

Why is that? In these cases, is the verifier a different person, that approves the fix using a different tool? Or is fixer = verifier, because the fixer regarded the bug as trivial and decided that verification was not needed?

Regarding bug reopening, I've found that...

* ... there's no evidence that bugs in which verifier != fixer are less likely to be reopened;

* ... verified bugs are 4x less likely to be reopened when the verifier states that he/she used code review, code inspection etc.

* ... verified bugs are 29% less likely to be reopened if the verification occurred in a day with intense verification activity.

Do these results make sense?

Please, let me know what you think about the results. It'll be a pleasure to share with you more facts extracted from Eclipse's bug reports, but I need to understand better what happens behind the scenes (e.g., why some bugs are verified by the fixer?) so results can become more meaningful.

Thanks,

Rodrigo