[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
[wtp-dev] How can we improve our weekly declared build process?

> Long ago, we had a rule that we'd declare at deadline even if teams had not reported, but we

> had a few "bad builds" slip through that way, and as a group we decided to use the "everyone must

> vote" rule, before declaring.

 

I think that the current process could use some tweaking. It locks the build for a good portion of the week. That is detrimental to progress (especially during functional milestones) and can actually lead to increased stability problems (people can’t release code, so everyone is keeping their changes private, then there is a rush to release code on Monday and something always breaks as the result). In the long term, I am in agreement with Dave, that we should strive to eliminate manual smoke testing completely. We should ultimately be able to go by the junit results alone when picking a build to declare. I haven’t tried SWTBot yet, so I don’t know how reliable it is in terms of not producing a lot of false negatives.

 

In the short term…

 

I propose that we limit the actual lock period to no more than 24 hours. Let’s pick the start of the 24 hour period such that it falls during biz hours for most teams and they can start testing. Teams can only release critical changes for the benefit of the smoke test during that 24 hour period. If testing hasn’t completed by the end of the period, the code line opens back up, but smoke testing continues. The only difference is that you can’t pickup a new build as declaration candidate. You are limited to builds that completed in that 24 hour period. Once all teams report, we look at the results. If a bad regression is found that affects a lot of functional area, we just don’t declare a weekly build. The relevant teams would take p1 items for next week to correct the issues in time for that week’s integration build.

 

I further propose that we move the start of the lock period to Friday from Thursday. We would evaluate the results on Monday morning and decide if we want to declare the build or skip that week. This would give us 4 solid work days to release code normally instead of the current situation where we have 3 and maybe a few hours on Friday afternoon if we are lucky.

 

- Konstantin

 

Oracle
Konstantin Komissarchik | Consulting Member of Technical Staff
Phone: +1 425 945 8445 | Mobile: +1 206 898 0611
Oracle Eclipse Tooling
411 108th Ave NE, Suite 800 | Bellevue, WA 98004

 


From: David M Williams [mailto:david_williams@xxxxxxxxxx]
Sent: Saturday, January 31, 2009 11:28 PM
To: konstantin.komissarchik@xxxxxxxxxx; Webtools releng discussion list
Subject: RE: [wtp-releng] Status of weekly I build

 

Yes, we set the deadline of 3:00 on Friday, then moved it to 3:00 on Saturday due to re-spin request from Dali team, ... so I can see why you'd think we'd be done by now ... but the JEE Team has not met even that deadline. As indicated on
http://wiki.eclipse.org/WTP_Smoke_Test_Results_R31_013009
and, the shorter, central summary you and others requested, at
http://build.eclipse.org/webtools/status/

Long ago, we had a rule that we'd declare at deadline even if teams had not reported, but we had a few "bad builds" slip through that way, and as a group we decided to use the "everyone must vote" rule, before declaring.

Suggestions welcome,

And an explanation from JEE team would help ... I've asked that if a test can't be done in time, that you at least keep us informed here on this list, so we all know what's going on and when to expect the test. I think this type of unexplained delay happens frequently, and its no fun for me to try and run everyone down, send mail that's unanswered, and repeatedly have to check to see if it is done yet since people are waiting on it.

Let me know if there's ways we can improve the predictability of our builds .. we are always within a few days, but never by 3:00 on Friday, like we say we will be.





Inactive hide details for "Konstantin Komissarchik" ---02/01/2009 01:12:38 AM---Hmm. I was under impression that a deadline for"Konstantin Komissarchik" ---02/01/2009 01:12:38 AM---Hmm. I was under impression that a deadline for smoke tests was mentioned the other day so we have o


From:


"Konstantin Komissarchik" <konstantin.komissarchik@xxxxxxxxxx>


To:


"Webtools releng discussion list" <wtp-releng@xxxxxxxxxxx>


Date:


02/01/2009 01:12 AM


Subject:


RE: [wtp-releng] Status of weekly I build


Sent by:


wtp-releng-bounces@xxxxxxxxxxx





Hmm… I was under impression that a deadline for smoke tests was mentioned the other day so we have our weekly build at this point. Did I misunderstand something? The changes that I released are not related to this past week’s integration build, so I wouldn’t move the declared build number.

Oracle
Konstantin Komissarchik | Consulting Member of Technical Staff
Phone: +1 425 945 8445 | Mobile: +1 206 898 0611

Oracle
Eclipse Tooling
411 108th Ave NE, Suite 800 | Bellevue, WA 98004


From: wtp-releng-bounces@xxxxxxxxxxx [mailto:wtp-releng-bounces@xxxxxxxxxxx] On Behalf Of David M Williams
Sent:
Saturday, January 31, 2009 9:42 PM
To:
wtp-releng@xxxxxxxxxxx
Subject:
[wtp-releng] Status of weekly I build

We are still waiting to hear from JEE Team before declaring the weekly build.

http://wiki.eclipse.org/WTP_Smoke_Test_Results_R31_013009

I noticed a build kicked off, I think due to changes to
common-fproj.map

I assume this was an "accident"? but please let us know here on this list,
if anything was released that would merit re-testing ... since normally
anything released during the weekly testing days is to fix a bad bug.
_______________________________________________
wtp-releng mailing list
wtp-releng@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/wtp-releng

GIF image

GIF image

GIF image

GIF image