From:
wtp-dev-bounces@xxxxxxxxxxx [mailto:wtp-dev-bounces@xxxxxxxxxxx] On Behalf Of Timothy Deboer
Sent: Wednesday, June 29, 2005
5:50 AM
To: naci.dai@xxxxxxxxxxxxx; General discussion of project-wide or architectural issues.
Subject: Re: [wtp-dev] Stability
of I-builds during release shutdown
Hi Naci,
I
agree about N-builds - they are only useful for getting the very latest
changes, with absolutely no stability guarantee. I do not think we should
change our N-builds at all.
What
I think both Thomas and I are getting at is that for external teams, weekly
I-builds are not enough. If somebody is waiting for a change or a bug fix and
it is done incorrectly or misses the next I-build, you can easily wait two
weeks to get a build that you can use. During early milestones we might be able
to slow down the I-builds; during intermediate milestones once a week is ok;
but during release shutdown this lag in the cycle is too much - our turnaround
time for builds will keep people from verifying bugs or getting through a
complete scenario with WTP.
The
Eclipse platform does an RC-build, stops producing I-builds for a couple days
so that people can test, and then does an I-build every few days (spinning
until clean) until the next RC. All I'm suggesting is that we do something
similar to ensure that we have a stable RC- or I-build at least once a week,
and preferrably more like twice. Since we'll be in shutdown, it should be
easier (and safer) for component leads to release their map files more often.
Thanks,
Tim
deBoer
WebSphere Tools - IBM Canada Ltd.
(905) 413-3503 (tieline 969)
deboer@xxxxxxxxxx
Naci Dai
<naci.dai@xxxxxxxxxxxxx>
Sent
by: wtp-dev-bounces@xxxxxxxxxxx
06/29/2005 08:02 AM
Please
respond to
naci.dai and "General discussion of
project-wide or architectural issues."
|
|
To
|
"General
discussion of project-wide or architectural issues."
<wtp-dev@xxxxxxxxxxx>
|
cc
|
|
Subject
|
Re: [wtp-dev] Stability of I-builds during
release shutdown
|
|
I will have to add to the discussion as some of the suggestions are not
workable.
N-Build, or a nightly build, is a build constructed from the cvs head
directly. This build at the best of the times indicate the status of the
latest changes. It is not meant for adoption and/or testing, but it will
give an indication of the current status of the code. The fundamental
property is that this builds configuration is *not* managed. An n-build
is not reproducible. Current N-Build frequency is once every 12 hours if
there are any changes. However, N-Builds have not been used very
successfully in the last 10 months. N-Builds do not offer good feedback
to produce I-Builds.
I-Build, the integration build is configuration managed build. The
component owners release their cvs tagged code into this build. These are
maintained in a set, called the build map. Each integration build has a
unique cvs tagged mapped. It is reproducible. An explicit
cross-component effort and testing takes place to declare an
I-Build. Only way to get feedback on the quality and status of an I-Build
is to try to produce that I-Build repeatedly until satisfied.
Back to the main point, we should and do have only one Integration build
per week. The fact that there are many trial I-Builds produced in the
open confuses the issue, we will start restricting access to these warm
up/trial builds, as they are only meant as a feedback to the committers to fix
the problems before the I-Build is declared.
One I-Build per week maybe too frequent in itself, it gives little breathing
room. However, this was necessary because of the loose coordination
between the teams, and number of changes accumulated in two weeks made the
integration harder and often impossible. We can switch to a longer
frequency after we are convinced that the code base has stabilized to afford a
longer integration period. Currently the number of changes each week are
massive to keep the time short. Remember the M1 & M2 horror, they
were the integration builds due to the lack of integration builds.
Post R0.7, wtp adopters can use M-builds, or communicate the I-Build they are
based on so that we continue maintaining it to assist less painful adoption.
Finally, the quality of an I-Build can only be good (all tests pass,
smoke testing approved, component owners approve, etc), and, it should
not regress wrt to a prior build. We had hard time achieving this goal
due to test quality issues and api changes, and relaxed our policy during the
M5 cycle. Relaxed means we declared some builds with test failures.
Post M5, we should activate the strict policy again.
My experience trying to improve I-Build quality by suggesting good practices
over email has been pretty much useless, we will have to enforce the practice
locally within each team. Peer-pressure and good quality feedback
is the mechanisms for achieving the level of quality we seek. WTP is the
second largest eclipse project (platform being the first), and probably the
largest geographically in terms of number of teams on how they are distributed
so we will have to find the different ways to manage it ourselves.
In terms of build frequency, can I have people declare opinions on the
following survey:
a) Produce an I-Build once a week (Thursday - as usual)
b) Produce an I-Build once every two weeks and more only when needed.
I also support frequent *trial* I-Builds (4 hours etc. more if needed),
prior to declaring each build. Doing this the day before is too short as
the team distribution crosses the day-night barrier so we will have to extend
it to at least 48 hours.
Yes. We observed similar pattern.
Adopt an I-Build is a very costly process for us. It is
measured in man-days.
Every time, we adopt I-Build that doesn’t go as far as
the previous one, we take big hit on productive and we would not able to
proceed with some implementation. Without proceed with implementation, we
won’t discover critical bug that need to discover before 0.7.
Because we only have 2 weeks, I would think that we want to
go one level further. We should respin nightly build every 4 hours, and make it
the first priority to have all compile error and JUnit test fixed once it is
discovered in nightly build. And, all JUnit test should be deemed critical.
This suggestion might appear to be costly. But, I believe the
saving from the unstable build’s cascade effect is beyond the cost. After
all, compile error and JUnit tests need to be fixed anyway. Fixing it early
enable people/team in the downstream to start working on that earlier.
In short, I counter propose:
1/ We move the expectation of a nighlty build up, as what we
would had expected from a I-Build
2/ We respin the nightly build every 4 hours.
3/ We fix all compile error and all JUnit error within a
I-Build.
4/ We produce a warm-up I-Build on Thursday.
Thomas
From: wtp-dev-bounces@xxxxxxxxxxx [mailto:wtp-dev-bounces@xxxxxxxxxxx] On Behalf Of Timothy Deboer
Sent: Tuesday, June 28, 2005 6:19 PM
To: wtp-dev@xxxxxxxxxxx
Subject: [wtp-dev] Stability of I-builds during release shutdown
Hi everyone,
The 0.7 release candidate builds are proposed for the 13th and 22nd, with the
release at the end of the month. What I haven't seen is what happens between
these times - presumably we are going to run nightly I-builds or something
similar. My concern is that with constant I-builds typically comes constant
churn - every team ends up taking a turn at breaking the build and we have
brown-outs that are nobody's fault - most of the build is good, but there are
some less stable parts which change from build to build. Our past I-builds are
a good example - although they are fairly stable on Tuesday, we typically have
a series of build and JUnit failures until sometime Thursday when everyone
starts focusing on it.
To ensure that we have frequent, high-quality builds that can be used for
testing and external teams building on top of WTP, I propose one of the
following:
1. We do less frequent I-builds (e.g. every 3-4 days), but use warmups like we
have done with previous I-builds. For example, we could start spinning I-builds
on every Monday morning and spin every 4 hours until we get it clean, then do
it again on Thursday.
2. We stay vigilant - every nightly I-build should be as good as the previous
milestone/release candidate. Any compile errors or JUnit failures are deemed
critical and must be fixed asap by a respin.
Thought? Opinions? Other suggestions?
Thanks,
Tim deBoer
WebSphere Tools - IBM Canada Ltd.
(905) 413-3503 (tieline 969)
deboer@xxxxxxxxxx
_______________________________________________
wtp-dev mailing list
wtp-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/wtp-dev
--
Naci Dai,
eteration a.s.
Inonu cad. Sumer sok. Zitas D1-15
Kozyatagi, Istanbul
34742
+90 (533) 580 2393 (cell)
+90 (216) 361 5434 (phone)
+90 (216) 361 2034 (fax)
http://www.eteration.com/
mailto:nacidai@xxxxxxx
mailto:naci@xxxxxxxxxxxxx_______________________________________________
wtp-dev mailing list
wtp-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/wtp-dev