[
Date Prev][
Date Next][
Thread Prev][
Thread Next][
Date Index][
Thread Index]
[
List Home]
RE: [cross-project-issues-dev] endgame for the Europa Winter Maintenance
|
Yes, it is scary and has been a problem for cross-project/EPP testing. For
example, on Mylyn we had a very small change that we would have pushed on
Monday evening or Tuesday. But we opted against doing it because any time
that we have updated our features the build has been inexplicably broken for
24+ hours. For this final Europa release we are fortunate that EPP testing
from last week held up and that nothing has needed to be changed since
Monday. The EPP packages that we tested are Java, JEE, and RCP (i.e., all
the ones with Mylyn).
Doug, Markus: I'll stop spamming the list now, but I'm not sure where we're
at with the overall build failure. A new build is now in progress so
hopefully CDT fix itself in the 2-3 hours that it usually takes to run.
Since none of us has permissions to run the build, if it fails again you
could consider asking Denis to manually move the file that's failing to
copy.
Mik
> -----Original Message-----
> From: cross-project-issues-dev-bounces@xxxxxxxxxxx [mailto:cross-
> project-issues-dev-bounces@xxxxxxxxxxx] On Behalf Of Schaefer, Doug
> Sent: Thursday, February 28, 2008 7:08 AM
> To: Cross project issues
> Subject: RE: [cross-project-issues-dev] endgame for the Europa Winter
> Maintenance
>
> Yeah, the issue seems to be more with updating the site.xml file in the
> staging area than anything.
>
> Kinda scary that we have intermittent failures like this.
>
> Doug.
>
>
> -----Original Message-----
> From: Mik Kersten [mailto:beatmik@xxxxxxx]
> Sent: Thursday, February 28, 2008 1:43 AM
> To: 'Cross project issues'; Schaefer, Doug
> Subject: RE: [cross-project-issues-dev] endgame for the Europa Winter
> Maintenance
>
> We tested all the EPP packages with Mylyn and all looks good. Thanks
> again Markus. The 8pm EST build is green for Mylyn, although it's
> still
> not clear what was causing the failures and our bits have not changed
> since Monday.
>
> The build is now failing for CDT with the same "File Not Found" error.
>
> http://dash.eclipse.org/~bfreeman/europa/
>
> Doug: fyi, for us this resolved itself after a couple more builds
> (24hrs).
> During that timeframe we also made sure there was no pack200 stuff in
> our site.xml and no .gz files, but don't know how that could be
> related.
>
> Mik
>
> > -----Original Message-----
> > From: cross-project-issues-dev-bounces@xxxxxxxxxxx [mailto:cross-
> > project-issues-dev-bounces@xxxxxxxxxxx] On Behalf Of Mik Kersten
> > Sent: Wednesday, February 27, 2008 1:47 PM
> > To: 'Cross project issues'
> > Subject: RE: [cross-project-issues-dev] endgame for the Europa Winter
> > Maintenance
> >
> > Thanks for saving the day Markus. We have done some quick testing of
> > the JEE and Java EPP packages so far and all looks good. We will
> > continue with testing this afternoon.
> >
> > The Europa build failure is still a mystery.
> >
> > http://dash.eclipse.org/~bfreeman/europa/
> >
> > Mik
> >
> > > -----Original Message-----
> > > From: cross-project-issues-dev-bounces@xxxxxxxxxxx [mailto:cross-
> > > project-issues-dev-bounces@xxxxxxxxxxx] On Behalf Of Markus Knauer
> > > Sent: Wednesday, February 27, 2008 1:09 PM
> > > To: Cross project issues
> > > Subject: Re: [cross-project-issues-dev] endgame for the Europa
> > > Winter Maintenance
> > >
> > > This workaround seems to work: I was able to build the EPP
> packages,
> > so
> > > I
> > > assume that the error happens during the Europa build.
> > >
> > > EPP package download with the features pulled from the Mylyn update
> > > site (instead of the Europa update site) available as 20080227-2022
> > > build from http://www.eclipse.org/epp/download.php or as a direct
> > > link from
> > >
> > >
> >
> http://build.eclipse.org/technology/epp/epp_build/33/download/20080227
> > -
> > > 2022/index.html
> > >
> > > Regards
> > > Markus
> > >
> > >
> > >
> > >
> > > On Wednesday 27 February 2008, Markus Knauer wrote:
> > > > As a workaround and to see if the EPP build is running without
> the
> > > Mylyn
> > > > features from the Europa update site, I've added (as a temporary
> > > solution)
> > > > the Mylyn update site to the EPP configuration.
> > > >
> > > > That way, EPP should pull the features from Mylyn and *not* from
> > the
> > > Europa
> > > > staging update site.
> > > >
> > > > Let's see what happens next :-)
> > > >
> > > > Markus
> > > >
> > > >
> > > > BTW: I looked into the feature.xml's on the Mylyn update site and
> > > checked
> > > > the availability of the provided plugins, and everything seems to
> > be
> > > fine.
> > > > I have no idea why the Europa build fails here.
> > > >
> > > > On Wednesday 27 February 2008, Mik Kersten wrote:
> > > > > The problem is still there. The only way we can reproduce the
> > > failure
> > > > > when running the mirror command is to explicitly remove the
> > > > > org.eclipse.mylyn_2.3.0.v20080225-2200.jar file, so we can't
> see
> > > anything
> > > > > to fix on our end. We re-copied all of the files in case there
> > was
> > > a
> > > > > filesystem problem or something related to the NFS problem
> > > > > listed
> > > on bug
> > > > > 220526.
> > > > >
> > > > > Bjorn: Could you re-run the build?
> > > > >
> > > > > Others: I'm not sure if Bjorn is around, is there anyone else
> > > > > who
> > > can run
> > > > > the build?
> > > > >
> > > > > Markus: In the event that we aren't able to get the build
> > > > > re-run,
> > > is it
> > > > > possible for you to copy the file manually and run the EPP
> build?
> > > If not
> > > > > perhaps we should ask Denis to do it.
> > > > >
> > > > > Mik
> > > > >
> > > > > > -----Original Message-----
> > > > > > From: cross-project-issues-dev-bounces@xxxxxxxxxxx
> > [mailto:cross-
> > > > > > project-issues-dev-bounces@xxxxxxxxxxx] On Behalf Of Markus
> > > Knauer
> > > > > > Sent: Wednesday, February 27, 2008 1:47 AM
> > > > > > To: Cross project issues
> > > > > > Subject: Re: [cross-project-issues-dev] endgame for the
> Europa
> > > Winter
> > > > > > Maintenance
> > > > > >
> > > > > > Mik, I can report that I had many I/O and 'file not found'
> > errors
> > > on
> > > > > > the same
> > > > > > server during the last few days.
> > > > > >
> > > > > > E.g. I had to restart the EPP build this morning (20080227-
> > 0740)
> > > > > > because the
> > > > > > nightly build (20080227-0440) failed for the above reason.
> The
> > > same
> > > > > > build
> > > > > > with the same configuration and the same content was
> > successful.
> > > > > >
> > > > > > The other example is the Ganymede build. If you are looking
> at
>
> > > > > > http://build.eclipse.org/ganymede/ you will see two failed
> > builds
> > > on
> > > > > > Feb 27
> > > > > > and one successful:
> > > > > >
> > > > > > * 2008_58_01_02_15 (Feb 27, 2008 01:02:15 GMT) - fail (some
> > > other
> > > > > > reasons)
> > > > > > * 2008_58_02_07_40 (Feb 27, 2008 02:07:40 GMT) - fail
> > > (Input/output
> > > > > > error)
> > > > > > * 2008_58_07_17_47 (Feb 27, 2008 07:17:47 GMT) - success
> > > > > >
> > > > > > The third one was triggered by myself (thanks to Cruise
> > > > > > Control
> > > this is
> > > > > > now
> > > > > > possible!) without changing an .sc file or anything else, but
> > > this
> > > > > > build
> > > > > > finished successfully.
> > > > > >
> > > > > > Markus
> > > > > >
> > > > > > On Wednesday 27 February 2008, Mik Kersten wrote:
> > > > > > > The build is breaking due to a "File Not Found" error on
> one
> > of
> > > the
> > > > > >
> > > > > > Mylyn
> > > > > >
> > > > > > > features:
> > > > > > >
> > > > > > > http://dash.eclipse.org/~bfreeman/europa/#mylyn
> > > > > > >
> > > > > > > This is a bit confusing because the build started at 8pm
> EST
> > > and we
> > > > > >
> > > > > > updated
> > > > > >
> > > > > > > the Mylyn features well before that, on Tuesday at 12:14pm
> > EST.
> > > > > >
> > > > > > After
> > > > > >
> > > > > > > investigation everything looks fine on our end, so unless
> we
> > > are
> > > > > >
> > > > > > missing
> > > > > >
> > > > > > > something those who are considering updating their features
> > > tomorrow
> > > > > >
> > > > > > might
> > > > > >
> > > > > > > need to watch for similar failures. We will wait for the
> > next
> > > build
> > > > > >
> > > > > > to see
> > > > > >
> > > > > > > if the Mylyn problem fixes itself and report back.
> > > > > > >
> > > > > > > Mik
> > > > > > >
> > > > > > > > -----Original Message-----
> > > > > > > > From: cross-project-issues-dev-bounces@xxxxxxxxxxx
> > > [mailto:cross-
> > > > > > > > project-issues-dev-bounces@xxxxxxxxxxx] On Behalf Of Mik
> > > Kersten
> > > > > > > > Sent: Tuesday, February 26, 2008 1:50 PM
> > > > > > > > To: 'Cross project issues'
> > > > > > > > Cc: 'Mylyn developer discussions'
> > > > > > > > Subject: RE: [cross-project-issues-dev] endgame for the
> > > Europa
> > > > > >
> > > > > > Winter
> > > > > >
> > > > > > > > Maintenance
> > > > > > > >
> > > > > > > > Thanks for scheduling those builds Markus, that's
> helpful.
> > > > > > > >
> > > > > > > > It looks like the suggested 2pm time was a no go because
> > the
> > > build
> > > > > > > > would
> > > > > > > > have needed to be manually invoked.
> > > > > > > >
> > > > > > > > Unless a new build is spun, on Mylyn we'll wait for the
> > > regularly
> > > > > > > > scheduled
> > > > > > > > 5pm EST build, test the Europa packages from that, and
> > > > > > > > then
> > > test
> > > > > >
> > > > > > the
> > > > > >
> > > > > > > > 8:40pm
> > > > > > > > EST EPP build that Markus has scheduled.
> > > > > > > >
> > > > > > > > Mik
> > > > > > > >
> > > > > > > > > -----Original Message-----
> > > > > > > > > From: cross-project-issues-dev-bounces@xxxxxxxxxxx
> > > [mailto:cross-
> > > > > > > > > project-issues-dev-bounces@xxxxxxxxxxx] On Behalf Of
> > Markus
> > > > > >
> > > > > > Knauer
> > > > > >
> > > > > > > > > Sent: Tuesday, February 26, 2008 1:37 PM
> > > > > > > > > To: Cross project issues
> > > > > > > > > Subject: Re: [cross-project-issues-dev] endgame for the
> > > Europa
> > > > > >
> > > > > > Winter
> > > > > >
> > > > > > > > > Maintenance
> > > > > > > > >
> > > > > > > > > Here an update:
> > > > > > > > >
> > > > > > > > > There was a Europa update site build started started
> > > February 26,
> > > > > > > >
> > > > > > > > 2008
> > > > > > > >
> > > > > > > > > 11:20:18 AM EST
> > > > > > > > > http://dash.eclipse.org/~bfreeman/europa/
> > > > > > > > >
> > > > > > > > > Based on this build is the EPP Europa build '20080226-
> > 2035'
> > > > > > > > > http://www.eclipse.org/epp/download.php
> > > > > > > > >
> > > > > > > > > I've added two additional EPP builds in my crontab:
> > > > > > > > > * 8.40pm EST
> > > > > > > > > * 1.40am EST
> > > > > > > > >
> > > > > > > > > That said, it should be possible to run an additional
> > > Europa
> > > > > >
> > > > > > update
> > > > > >
> > > > > > > > > site build
> > > > > > > > > at 6pm (should be enough time to be used by the 8.40pm
> > EPP
> > > > > >
> > > > > > build),
> > > > > >
> > > > > > > > and
> > > > > > > >
> > > > > > > > > another one at 10pm.
> > > > > > > > >
> > > > > > > > > That way, it could be possible to see additional
> results
> > > today.
> > > > > >
> > > > > > And
> > > > > >
> > > > > > > > if
> > > > > > > >
> > > > > > > > > this is
> > > > > > > > > not the case, we should see them tomorrow. :-)
> > > > > > > > >
> > > > > > > > > I hope this helps.
> > > > > > > > >
> > > > > > > > > Markus
> > > > > > > > >
> > > > > > > > > On Tuesday 26 February 2008, Mik Kersten wrote:
> > > > > > > > > > The current build is failing, probably due to new
> BIRT
> > > and TPTP
> > > > > > > > >
> > > > > > > > > features
> > > > > > > > >
> > > > > > > > > > being updated while the build is running. At least
> > > that's what
> > > > > > > > >
> > > > > > > > > happened
> > > > > > > > >
> > > > > > > > > > with Mylyn last week when we got the "No Matching
> > > Features"
> > > > > >
> > > > > > with
> > > > > >
> > > > > > > > > Mylyn. On
> > > > > > > > >
> > > > > > > > > > Mylyn we haven't updated our features to yesterday's
> > > final
> > > > > >
> > > > > > build in
> > > > > >
> > > > > > > > > trying
> > > > > > > > >
> > > > > > > > > > to avoid the break, but since are broken already are
> > > updating
> > > > > >
> > > > > > our
> > > > > >
> > > > > > > > > features
> > > > > > > > >
> > > > > > > > > > right now.
> > > > > > > > > >
> > > > > > > > > > http://dash.eclipse.org/~bfreeman/europa/
> > > > > > > > > >
> > > > > > > > > > Could we make an end-game plan something along these
> > > lines to
> > > > > > > > >
> > > > > > > > > minimize
> > > > > > > > >
> > > > > > > > > > build breakage? This just boils down to batching our
> > re-
> > > spin
> > > > > > > >
> > > > > > > > requests
> > > > > > > >
> > > > > > > > > so
> > > > > > > > >
> > > > > > > > > > that those like Mylyn that need to do cross-project
> > > testing of
> > > > > > > > >
> > > > > > > > > packagings
> > > > > > > > >
> > > > > > > > > > can know when to do so. Proposed times are in build
> > > server
> > > > > >
> > > > > > time
> > > > > >
> > > > > > > > > (EST).
> > > > > > > > >
> > > > > > > > > > Today, 2pm EST: first freeze (manually triggered
> > > > > > > > > > build)
> > > > > > > > > > * Everyone has submitted their bits and won't do so
> > again
> > > until
> > > > > >
> > > > > > the
> > > > > >
> > > > > > > > > build
> > > > > > > > >
> > > > > > > > > > goes green.
> > > > > > > > > > * Markus: after the build has run, and hopefully gone
> > > green, I
> > > > > > > >
> > > > > > > > assume
> > > > > > > >
> > > > > > > > > that
> > > > > > > > >
> > > > > > > > > > some number of hours later we can expect the EPP
> > > packaging of
> > > > > >
> > > > > > it at
> > > > > >
> > > > > > > > > the
> > > > > > > > >
> > > > > > > > > > usual URL? How long does it usually take?
> > > > > > > > > > http://www.eclipse.org/epp/download.php
> > > > > > > > > >
> > > > > > > > > > Today, 8pm EST: second freeze (usual time the build
> > runs)
> > > > > > > > > > * Same as above, but given build times testing of the
> > > result
> > > > > >
> > > > > > waits
> > > > > >
> > > > > > > > > until
> > > > > > > > >
> > > > > > > > > > tomorrow.
> > > > > > > > > >
> > > > > > > > > > Tomorrow: something like the above two freezes for
> > anyone
> > > who
> > > > > >
> > > > > > needs
> > > > > >
> > > > > > > > > last
> > > > > > > > >
> > > > > > > > > > minute changes.
> > > > > > > > > >
> > > > > > > > > > Mik
> > > > > > > > > >
> > > _______________________________________________
> > > cross-project-issues-dev mailing list
> > > cross-project-issues-dev@xxxxxxxxxxx
> > > https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev
> >
> > _______________________________________________
> > cross-project-issues-dev mailing list
> > cross-project-issues-dev@xxxxxxxxxxx
> > https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev
>
> _______________________________________________
> cross-project-issues-dev mailing list
> cross-project-issues-dev@xxxxxxxxxxx
> https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev