I think the issue here is that adopters of WTP need a two things:
1) they need stability so they can develop new function instead of
catching up with interface changes
2) they need fast turnaround time for critical fixes so they can
proceed with testing
These are valid requirements and we need to start satisfying them at
this point in the cycle. We need to stop making interface changes and
focus exclusively on quality. If we do that then our I-builds will
have high quality and adopters will be able to move to them quickly.
I suggest we continue with weekly I-builds. I believe no one is
planning any interface changes at this time. Let's monitor the
situation and take a checkpoint a week after M5 is released. I'd like
to ask Tim and Thomas to immediately flag any regressions during this
time so we can understand the cause.
Arthur Ryman,
Rational Desktop Tools Development
phone: +1-905-413-3077, TL 969-3077
assistant: +1-905-413-2411, TL 969-2411
fax: +1-905-413-4920, TL 969-4920
mobile: +1-416-939-5063, text: 4169395063@xxxxxxx
intranet: http://labweb.torolab.ibm.com/DRY6/
*Timothy Deboer/Toronto/IBM@IBMCA*
Sent by: wtp-dev-bounces@xxxxxxxxxxx
06/29/2005 08:50 AM
Please respond to
"General discussion of project-wide or architectural issues."
To
naci.dai@xxxxxxxxxxxxx, "General discussion of project-wide or
architectural issues." <wtp-dev@xxxxxxxxxxx>
cc
Subject
Re: [wtp-dev] Stability of I-builds during release shutdown
Hi Naci,
I agree about N-builds - they are only useful for getting the very
latest changes, with absolutely no stability guarantee. I do not think
we should change our N-builds at all.
What I think both Thomas and I are getting at is that for external
teams, weekly I-builds are not enough. If somebody is waiting for a
change or a bug fix and it is done incorrectly or misses the next
I-build, you can easily wait two weeks to get a build that you can
use. During early milestones we might be able to slow down the
I-builds; during intermediate milestones once a week is ok; but during
release shutdown this lag in the cycle is too much - our turnaround
time for builds will keep people from verifying bugs or getting
through a complete scenario with WTP.
The Eclipse platform does an RC-build, stops producing I-builds for a
couple days so that people can test, and then does an I-build every
few days (spinning until clean) until the next RC. All I'm suggesting
is that we do something similar to ensure that we have a stable RC- or
I-build at least once a week, and preferrably more like twice. Since
we'll be in shutdown, it should be easier (and safer) for component
leads to release their map files more often.
Thanks,
Tim deBoer
WebSphere Tools - IBM Canada Ltd.
(905) 413-3503 (tieline 969)
deboer@xxxxxxxxxx
*Naci Dai <naci.dai@xxxxxxxxxxxxx>*
Sent by: wtp-dev-bounces@xxxxxxxxxxx
06/29/2005 08:02 AM
Please respond to
naci.dai and "General discussion of project-wide or architectural issues."
To
"General discussion of project-wide or architectural issues."
<wtp-dev@xxxxxxxxxxx>
cc
Subject
Re: [wtp-dev] Stability of I-builds during release shutdown
I will have to add to the discussion as some of the suggestions are
not workable.
N-Build, or a nightly build, is a build constructed from the cvs head
directly. This build at the best of the times indicate the status of
the latest changes. It is not meant for adoption and/or testing, but
it will give an indication of the current status of the code. The
fundamental property is that this builds configuration is *not*
managed. An n-build is not reproducible. Current N-Build frequency
is once every 12 hours if there are any changes. However, N-Builds
have not been used very successfully in the last 10 months. N-Builds
do not offer good feedback to produce I-Builds.
I-Build, the integration build is configuration managed build. The
component owners release their cvs tagged code into this build. These
are maintained in a set, called the build map. Each integration build
has a unique cvs tagged mapped. It is reproducible. An explicit
cross-component effort and testing takes place to declare an I-Build.
Only way to get feedback on the quality and status of an I-Build is
to try to produce that I-Build repeatedly until satisfied.
Back to the main point, we should and do have only one Integration
build per week. The fact that there are many trial I-Builds produced
in the open confuses the issue, we will start restricting access to
these warm up/trial builds, as they are only meant as a feedback to
the committers to fix the problems before the I-Build is declared.
One I-Build per week maybe too frequent in itself, it gives little
breathing room. However, this was necessary because of the loose
coordination between the teams, and number of changes accumulated in
two weeks made the integration harder and often impossible. We can
switch to a longer frequency after we are convinced that the code base
has stabilized to afford a longer integration period. Currently the
number of changes each week are massive to keep the time short.
Remember the M1 & M2 horror, they were the integration builds due to
the lack of integration builds.
Post R0.7, wtp adopters can use M-builds, or communicate the I-Build
they are based on so that we continue maintaining it to assist less
painful adoption.
Finally, the quality of an I-Build can only be good (all tests pass,
smoke testing approved, component owners approve, etc), and, it
should not regress wrt to a prior build. We had hard time achieving
this goal due to test quality issues and api changes, and relaxed our
policy during the M5 cycle. Relaxed means we declared some builds
with test failures. Post M5, we should activate the strict policy again.
My experience trying to improve I-Build quality by suggesting good
practices over email has been pretty much useless, we will have to
enforce the practice locally within each team. Peer-pressure and good
quality feedback is the mechanisms for achieving the level of quality
we seek. WTP is the second largest eclipse project (platform being
the first), and probably the largest geographically in terms of number
of teams on how they are distributed so we will have to find the
different ways to manage it ourselves.
In terms of build frequency, can I have people declare opinions on the
following survey:
a) Produce an I-Build once a week (Thursday - as usual)
b) Produce an I-Build once every two weeks and more only when needed.
I also support frequent *trial* I-Builds (4 hours etc. more if
needed), prior to declaring each build. Doing this the day before is
too short as the team distribution crosses the day-night barrier so we
will have to extend it to at least 48 hours.
Yes. We observed similar pattern.
Adopt an I-Build is a very costly process for us. It is measured in
man-days.
Every time, we adopt I-Build that doesn’t go as far as the previous
one, we take big hit on productive and we would not able to proceed
with some implementation. Without proceed with implementation, we
won’t discover critical bug that need to discover before 0.7.
Because we only have 2 weeks, I would think that we want to go one
level further. We should respin nightly build every 4 hours, and make
it the first priority to have all compile error and JUnit test fixed
once it is discovered in nightly build. And, all JUnit test should be
deemed critical.
This suggestion might appear to be costly. But, I believe the saving
from the unstable build’s cascade effect is beyond the cost. After
all, compile error and JUnit tests need to be fixed anyway. Fixing it
early enable people/team in the downstream to start working on that
earlier.
In short, I counter propose:
1/ We move the expectation of a nighlty build up, as what we would had
expected from a I-Build
2/ We respin the nightly build every 4 hours.
3/ We fix all compile error and all JUnit error within a I-Build.
4/ We produce a warm-up I-Build on Thursday.
Thomas
------------------------------------------------------------------------
*
From:* _wtp-dev-bounces@eclipse.org_
<mailto:wtp-dev-bounces@xxxxxxxxxxx>
[_mailto:wtp-dev-bounces@eclipse.org_] *On Behalf Of *Timothy Deboer*
Sent:* Tuesday, June 28, 2005 6:19 PM*
To:* _wtp-dev@eclipse.org_ <mailto:wtp-dev@xxxxxxxxxxx>*
Subject:* [wtp-dev] Stability of I-builds during release shutdown
Hi everyone,
The 0.7 release candidate builds are proposed for the 13th and 22nd,
with the release at the end of the month. What I haven't seen is what
happens between these times - presumably we are going to run nightly
I-builds or something similar. My concern is that with constant
I-builds typically comes constant churn - every team ends up taking a
turn at breaking the build and we have brown-outs that are nobody's
fault - most of the build is good, but there are some less stable
parts which change from build to build. Our past I-builds are a good
example - although they are fairly stable on Tuesday, we typically
have a series of build and JUnit failures until sometime Thursday when
everyone starts focusing on it.
To ensure that we have frequent, high-quality builds that can be used
for testing and external teams building on top of WTP, I propose one
of the following:
1. We do less frequent I-builds (e.g. every 3-4 days), but use warmups
like we have done with previous I-builds. For example, we could start
spinning I-builds on every Monday morning and spin every 4 hours until
we get it clean, then do it again on Thursday.
2. We stay vigilant - every nightly I-build should be as good as the
previous milestone/release candidate. Any compile errors or JUnit
failures are deemed critical and must be fixed asap by a respin.
Thought? Opinions? Other suggestions?
Thanks,
Tim deBoer
WebSphere Tools - IBM Canada Ltd.
(905) 413-3503 (tieline 969)_
__deboer@xxxxxx.com_ <mailto:deboer@xxxxxxxxxx>
------------------------------------------------------------------------
_______________________________________________
wtp-dev mailing list_
__wtp-dev@eclipse.org_ <mailto:wtp-dev@xxxxxxxxxxx>_
__https://dev.eclipse.org/mailman/listinfo/wtp-dev_
--
Naci Dai,
eteration a.s.
Inonu cad. Sumer sok. Zitas D1-15
Kozyatagi, Istanbul 34742
+90 (533) 580 2393 (cell)
+90 (216) 361 5434 (phone)
+90 (216) 361 2034 (fax)_
__http://www.eteration.com/__
__mailto:nacidai@acm.org__
__mailto:naci@eteration.com________________________________________________
wtp-dev mailing list
wtp-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/wtp-dev
_______________________________________________
wtp-dev mailing list
wtp-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/wtp-dev
------------------------------------------------------------------------
_______________________________________________
wtp-dev mailing list
wtp-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/wtp-dev
