> Long ago, we had a rule that we'd declare at deadline even if
teams had not reported, but we
> had a few "bad builds" slip through that way, and as a
group we decided to use the "everyone must
> vote" rule, before declaring.
I think that the current process could use some tweaking. It locks the
build for a good portion of the week. That is detrimental to progress (especially
during functional milestones) and can actually lead to increased stability
problems (people can’t release code, so everyone is keeping their changes
private, then there is a rush to release code on Monday and something always
breaks as the result). In the long term, I am in agreement with Dave, that we
should strive to eliminate manual smoke testing completely. We should ultimately
be able to go by the junit results alone when picking a build to declare. I
haven’t tried SWTBot yet, so I don’t know how reliable it is in
terms of not producing a lot of false negatives.
In the short term…
I propose that we limit the actual lock period to no more than 24
hours. Let’s pick the start of the 24 hour period such that it falls
during biz hours for most teams and they can start testing. Teams can only
release critical changes for the benefit of the smoke test during that 24 hour
period. If testing hasn’t completed by the end of the period, the code
line opens back up, but smoke testing continues. The only difference is that
you can’t pickup a new build as declaration candidate. You are limited to
builds that completed in that 24 hour period. Once all teams report, we look at
the results. If a bad regression is found that affects a lot of functional
area, we just don’t declare a weekly build. The relevant teams would take
p1 items for next week to correct the issues in time for that week’s
integration build.
I further propose that we move the start of the lock period to Friday
from Thursday. We would evaluate the results on Monday morning and decide if we
want to declare the build or skip that week. This would give us 4 solid work
days to release code normally instead of the current situation where we have 3
and maybe a few hours on Friday afternoon if we are lucky.
- Konstantin
Konstantin Komissarchik | Consulting
Member of Technical Staff
Phone: +1 425 945 8445 | Mobile: +1 206 898 0611
Oracle
Eclipse Tooling
411 108th Ave NE, Suite 800
| Bellevue, WA 98004
From: David M Williams
[mailto:david_williams@xxxxxxxxxx]
Sent: Saturday, January 31, 2009
11:28 PM
To:
konstantin.komissarchik@xxxxxxxxxx; Webtools releng discussion list
Subject: RE: [wtp-releng] Status
of weekly I build
Yes, we set the deadline of 3:00 on Friday, then moved
it to 3:00 on Saturday due to re-spin request from Dali team, ... so I can see
why you'd think we'd be done by now ... but the JEE Team has not met even that
deadline. As indicated on
http://wiki.eclipse.org/WTP_Smoke_Test_Results_R31_013009
and, the shorter, central summary you and others requested, at
http://build.eclipse.org/webtools/status/
Long ago, we had a rule that we'd declare at deadline even if teams had not
reported, but we had a few "bad builds" slip through that way, and as
a group we decided to use the "everyone must vote" rule, before
declaring.
Suggestions welcome,
And an explanation from JEE team would help ... I've asked that if a test can't
be done in time, that you at least keep us informed here on this list, so we
all know what's going on and when to expect the test. I think this type of unexplained
delay happens frequently, and its no fun for me to try and run everyone down,
send mail that's unanswered, and repeatedly have to check to see if it is done
yet since people are waiting on it.
Let me know if there's ways we can improve the predictability of our builds ..
we are always within a few days, but never by 3:00 on Friday, like we say we
will be.
"Konstantin Komissarchik"
---02/01/2009 01:12:38 AM---Hmm. I was under impression that a deadline for
smoke tests was mentioned the other day so we have o
|
From:
|
"Konstantin
Komissarchik" <konstantin.komissarchik@xxxxxxxxxx>
|
|
To:
|
"Webtools
releng discussion list" <wtp-releng@xxxxxxxxxxx>
|
|
Date:
|
02/01/2009 01:12 AM
|
|
Subject:
|
RE: [wtp-releng]
Status of weekly I build
|
|
Sent by:
|
wtp-releng-bounces@xxxxxxxxxxx
|
Hmm… I was under impression that a deadline for smoke tests
was mentioned the other day so we have our weekly build at this point. Did I
misunderstand something? The changes that I released are not related to this
past week’s integration build, so I wouldn’t move the declared
build number.
Konstantin Komissarchik | Consulting Member of Technical Staff
Phone: +1 425 945 8445 | Mobile: +1 206 898 0611
Oracle Eclipse Tooling
411 108th Ave NE, Suite 800
| Bellevue, WA 98004
From: wtp-releng-bounces@xxxxxxxxxxx [mailto:wtp-releng-bounces@xxxxxxxxxxx]
On Behalf Of David M Williams
Sent: Saturday, January 31, 2009 9:42 PM
To: wtp-releng@xxxxxxxxxxx
Subject: [wtp-releng] Status of weekly I build
We are
still waiting to hear from JEE Team before declaring the weekly build.
http://wiki.eclipse.org/WTP_Smoke_Test_Results_R31_013009
I noticed a build kicked off, I think due to changes to
common-fproj.map
I assume this was an "accident"? but please let us know here on this
list,
if anything was released that would merit re-testing ... since normally
anything released during the weekly testing days is to fix a bad bug. _______________________________________________
wtp-releng mailing list
wtp-releng@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/wtp-releng
