Re: [jetty-dev] Fixing our TERRIBLE eclipse oriented build

[Jesse McConnell <jesse.mcconnell@xxxxxxxxx>]

On Mon, Aug 16, 2010 at 12:08, Jeff McAffer <jeff@xxxxxxxxxxxxxxxxx> wrote:
> Seems like at least one of the underlying problems is the lack of clear durability statements around the various eclipse repos.  If there was a strong statement in that area, would that solve your problems? Make it easier?  There has been discussion around retention policies etc for some time.  Perhaps this is an opportunity to make that real?

well, I am not sure that its a lack of clear statements about
durability, i get it that these things don't stay around forever...its
that apparently a dependency goes through a lifecycle of repositories
and can flat out disappear at a particular version along the way.
Take the ejc dependency we have for jsp....that thing was 3.6M# for a
while in some repositories and then changed to 3.6 in another
repository that is a more durable location.  Over the course of weeks
or months, a dependency on some thing in orbit or against the actual
eclipse versions can take quite a while until it appears in a
'durable' repository and in that interim we have to make
releases...and those builds ought to be reproducible out of the
box...not after tracking down what releases have flowed around through
the orbit or eclipse distribution workflow.

tycho will likely address much of this with its p2 support and I hope
that is enough down the road...but for now it is incredibly annoying
to try and build only to then spend 20 minutes or an hour tracking
down where orbit repo's have advanced on to location, version wise and
what artifacts might have been renamed to some new version underneath
you...it doesn't really instill much confidence in the system, feels
more like a big house of cards.

cheers,
jesse

> Jeff
>
> On 2010-08-16, at 1:04 PM, Jesse McConnell wrote:
>
>> only that we couldn't actually release with this in place since the
>> requirement is that all binaries in our jetty-distribution be
>> downloaded from eclipse.org websites....which is why I was proposing
>> to put them someplace safe that _we_ won't rip from underneath our own
>> build..
>>
>> joakim was considering how a orbit-maven-plugin could potentially
>> determine how to update this aspect of the build and pull the correct
>> items automatically as part of the build...that might be worth looking
>> at or seeing if tycho is planning on solving the issue now they are
>> part of eclipse and will presumably have to deal with these inane
>> policies as well
>>
>> jesse
>>
>> --
>> jesse mcconnell
>> jesse.mcconnell@xxxxxxxxx
>>
>>
>>
>> On Sun, Aug 15, 2010 at 23:41, Greg Wilkins <gregw@xxxxxxxxxxx> wrote:
>>> Jesse,
>>>
>>> sorry for the late response.
>>>
>>> I agree that the lack or reproducibility is a huge problem.  Also our
>>> dependency traceability is really badly damaged by the current system
>>> (as evidenced by the cut and past of orbit versions between
>>> jetty-distribution and jetty-osgi/test-jetty-osgi
>>>
>>> My suggestion is that we create a separate project called
>>> jetty-orbit-dependencies, whose entire function would be to pull down
>>> orbit jars and publish them to maven central repository in group id
>>> like: org.eclipse.jetty.orbit.javax.sevlet
>>>
>>> This project itself would not have a reproducible build (as it would
>>> fail once the orbit URLs had changed yet again), but at least it's
>>> artefacts would be durable in maven central and thus jetty releases
>>> would also be durable.
>>>
>>> thoughts?
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> On 7 August 2010 01:25, Jesse McConnell <jesse.mcconnell@xxxxxxxxx> wrote:
>>>> This is just flat out ridiculous how our build has degraded to this point.
>>>>
>>>> IMO we can not release AT ALL based on eclipse repositories in our
>>>> build to obtain artifacts.  They are far far too transitory with their
>>>> retention policy and it absolutely devastates our build
>>>> reproducibility...less then a week or two after one of our releases we
>>>> can't build the tag without making adjustments to the ant download
>>>> portions of the build.
>>>>
>>>> I am open to suggestions on ways to deal with this...and ideally
>>>> eclipse will someday grow up and have a proper artifact repository
>>>> that is durable but until then I propose we do the following.
>>>>
>>>> * pull every artifact that we rely upon out of orbit and put it into
>>>> our downloads.eclipse.org site
>>>>
>>>> we still need to use this ant goop to build since we can't release
>>>> with external maven repositories but this way we will at least have
>>>> reproducible builds for longer then a week based on a tag
>>>>
>>>> I'll hold off on making this change to solicit feedback...for a little while...
>>>>
>>>> jesse
>>>>
>>>> --
>>>> jesse mcconnell
>>>> jesse.mcconnell@xxxxxxxxx
>>>> _______________________________________________
>>>> jetty-dev mailing list
>>>> jetty-dev@xxxxxxxxxxx
>>>> https://dev.eclipse.org/mailman/listinfo/jetty-dev
>>>>
>>> _______________________________________________
>>> jetty-dev mailing list
>>> jetty-dev@xxxxxxxxxxx
>>> https://dev.eclipse.org/mailman/listinfo/jetty-dev
>>>
>> _______________________________________________
>> jetty-dev mailing list
>> jetty-dev@xxxxxxxxxxx
>> https://dev.eclipse.org/mailman/listinfo/jetty-dev
>
> _______________________________________________
> jetty-dev mailing list
> jetty-dev@xxxxxxxxxxx
> https://dev.eclipse.org/mailman/listinfo/jetty-dev
>