Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [eclipse-dev] Bug verification and reopening

Thanks for your feedback, Pascal and John!

On Tue, Oct 8, 2013 at 10:54 AM, John Arthorne <John_Arthorne@xxxxxxxxxx> wrote:
Correct.  I suspect dedicated testing teams are rare in open source projects - you won't find volunteers to do this all the time and it's not something that companies want to pay for.
Yeah, sure. I've found two large open source projects with dedicated testing teams (one of them uses an outsourced team), but I don't have data to support any conclusions about the cost-effectiveness of this approach.

Yes, the entire team does at least one dedicated testing day in the week of a milestone, and this is likely where the majority of verification happens.

Hmm, interesting. Are the dates of the testing days documented anywhere (mailing list, wiki)?

> * ... verified bugs are 4x less likely to be reopened when the 
> verifier states that he/she used code review, code inspection etc.

This is an interesting finding. How did you determine this - just based on the comment that was made at the time the bug was marked verified? Did you manually categorize these based on comments or have some heuristic? 

Yes, I used the comments made when a bug is marked verified. I matched them against regular expressions for manual testing, automated testing, code review, and automated static analysis. However, only 4% of the comments could be matched (most comments are like "Verified." or "Verified in I201310..."), which seriously threatens the results. 

Also, it could be that code reviews are performed mostly for specific types of issues (e.g., API change, changes to XML and config files) that are less prone to reopening. What do you think?

I'm assuming that most verifications correspond to manual testing, and that code review is not frequent. Is this a reasonable assumption?


Back to the top