Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Eclipse Projects » Buckminster dev » Build system now checked into SVN
Build system now checked into SVN [message #24807] Tue, 12 August 2008 22:47 Go to next message
Adrian Skehill is currently offline Adrian Skehill
Messages: 55
Registered: July 2009
Member
Hi Folks,

Just to let you know, the files that are being used by the Hudson build
are now checked into the Buckminster svn repository in [1]. These build
files now also build the update with the addition of an extra
"buckminster.build.site" target.

If you have any suggestions for improvement, or ideas to share feel free
to bring them up and I'll try to roll them into the build files. In
particular, I'm sure there must be a better way for me to specify the
properties file to produce an agnostic update site.

Next plan of attack is to complete out the delivery of the update site
and to wiki'fy this information. Any volunteers to review the content,
and to make some suggestions as to what they'd like to see in it?

Regards,
Adrian.

[1]
http://dev.eclipse.org/svnroot/tools/org.eclipse.buckminster /trunk/build/
Re: Build system now checked into SVN [message #24846 is a reply to message #24807] Wed, 13 August 2008 07:38 Go to previous messageGo to next message
Thomas Hallgren is currently offline Thomas Hallgren
Messages: 3229
Registered: July 2009
Senior Member
Hi Adrian,

Adrian Skehill wrote:
> Hi Folks,
>
> Just to let you know, the files that are being used by the Hudson build
> are now checked into the Buckminster svn repository in [1]. These build
> files now also build the update with the addition of an extra
> "buckminster.build.site" target.
>
I checked it out and ran it on my machine. It just works. Beautiful.

I discovered two minor flaws.
1. The platform download picks the wrong zip on an x86_64 machine since
that zip contains the os.arch at the end (not that it matters now that
you use the delta pack, but anyway).
2. The check for installed features fails since the
${buckminster.version} will vary between features. I think you will need
some concept where you check for things like
"org.eclipse.buckminster.core.headless.feature_*" but I'm not sure how
to do that. Perhaps you can convert a fileset into a property
(pathconvert) and then check if the property is empty?


> If you have any suggestions for improvement, or ideas to share feel free
> to bring them up and I'll try to roll them into the build files. In
> particular, I'm sure there must be a better way for me to specify the
> properties file to produce an agnostic update site.
>
I would suggest that you provide the needed properties to the
perform command using the --properties <propfile> option, i.e.

<target name="buckminster.build.site">
<echo message="Building Buckminster Update Site"/>
<property name="buckminster.properties"
location="buckminster.properties"/>
<buckminster command="perform">
<args>
<arg value="-Dbuckminster.output.root=${build.directory}"/>
<arg value="--properties"/>
<arg value="${buckminster.properties}"/>
<arg value="org.eclipse.buckminster#normalize.site"/>
</args>
</buckminster>
</target>

> Next plan of attack is to complete out the delivery of the update site
> and to wiki'fy this information. Any volunteers to review the content,
> and to make some suggestions as to what they'd like to see in it?
>
Count me in :-)

I have a wish list on what I'd like to see the automatic build produce.
I'm however not a very typical Buckminster user so it's not unlikely
that my ideas can be improved a great deal. In any case, here's what I'd
like to see:

1. A 'bleeding edge' update site. This site would contain the output of
the last build that was automatically triggered from check-ins to our
SVN. I like this concept better then 'nightly builds' since a) there's
no reason to rebuild unless there indeed has been a check-in, and b)
there might be reasons to test check-ins going the full circle, fix
problems, test again, etc. during the day.

2. Integration update site. A more reliable site where we publish things
that we feel is semi-stable (similar to our update site today). This
update site must also be present as a zipped archived site that we
maintain a history of. I think a reasonable "pulse" for these builds
would be once a week.

3. Release site. An update site guaranteed not to move between releases
of the Eclipse platform.

It would be really awesome if a build could automatically extract all
affected bugzillas since the last archived build and somehow make them
available as a link next to the new archive link.

Regards,
Thomas Hallgren
Re: Build system now checked into SVN [message #24878 is a reply to message #24846] Wed, 13 August 2008 08:15 Go to previous messageGo to next message
Adrian Skehill is currently offline Adrian Skehill
Messages: 55
Registered: July 2009
Member
Thomas Hallgren wrote:

>>
> I checked it out and ran it on my machine. It just works. Beautiful.

Excellent, that's good to hear.
>
> I discovered two minor flaws.
> 1. The platform download picks the wrong zip on an x86_64 machine since
> that zip contains the os.arch at the end (not that it matters now that
> you use the delta pack, but anyway).
> 2. The check for installed features fails since the
> ${buckminster.version} will vary between features. I think you will need
> some concept where you check for things like
> "org.eclipse.buckminster.core.headless.feature_*" but I'm not sure how
> to do that. Perhaps you can convert a fileset into a property
> (pathconvert) and then check if the property is empty?
>
Grand job, I'll take a look at these and get them fixed up.


>
>> If you have any suggestions for improvement, or ideas to share feel
>> free to bring them up and I'll try to roll them into the build files.
>> In particular, I'm sure there must be a better way for me to specify
>> the properties file to produce an agnostic update site.
>>
> I would suggest that you provide the needed properties to the
> perform command using the --properties <propfile> option, i.e.
>
> <target name="buckminster.build.site">
> <echo message="Building Buckminster Update Site"/>
> <property name="buckminster.properties"
> location="buckminster.properties"/>
> <buckminster command="perform">
> <args>
> <arg value="-Dbuckminster.output.root=${build.directory}"/>
> <arg value="--properties"/>
> <arg value="${buckminster.properties}"/>
> <arg value="org.eclipse.buckminster#normalize.site"/>
> </args>
> </buckminster>
> </target>


Yip, much better than what I did. Late night and out of ideas tends to
lead to some sub optimal solutions. Will roll this in.

>
>> Next plan of attack is to complete out the delivery of the update site
>> and to wiki'fy this information. Any volunteers to review the content,
>> and to make some suggestions as to what they'd like to see in it?
>>
> Count me in :-)
>
> I have a wish list on what I'd like to see the automatic build produce.
> I'm however not a very typical Buckminster user so it's not unlikely
> that my ideas can be improved a great deal. In any case, here's what I'd
> like to see:
>
> 1. A 'bleeding edge' update site. This site would contain the output of
> the last build that was automatically triggered from check-ins to our
> SVN. I like this concept better then 'nightly builds' since a) there's
> no reason to rebuild unless there indeed has been a check-in, and b)
> there might be reasons to test check-ins going the full circle, fix
> problems, test again, etc. during the day.
>
> 2. Integration update site. A more reliable site where we publish things
> that we feel is semi-stable (similar to our update site today). This
> update site must also be present as a zipped archived site that we
> maintain a history of. I think a reasonable "pulse" for these builds
> would be once a week.
>
> 3. Release site. An update site guaranteed not to move between releases
> of the Eclipse platform.


Despite not being a typical Buckminster user, these requirements are
right up there with people who are using Buckminster.

So #1 is easily done, I can modify the per-commit job in Hudson to build
the update site and extract it. Though one thing that Buckminster
developers should start getting their head around for this to be
successful is the provision of unit tests. It's nice to have the
confidence that stuff that gets published to a bleeding edge site will
have tests run on it (and that this test coverage is constantly
improving). Expect to hear more from me on this in the future :-)

On #2 and #3, I've created a job on our Hudson instance called
"Buckminster_Update_Site". This is a manually run job and can only be
kicked off by administrators (for now Thomas & I). Also within Hudson
I've installed the "Promoted Builds" plugin which allows us to promote
builds that we see fit. This needs further investigation to figure out
the optimal path, but I certainly think what you're looking for there is
feasible with this mechanism.


>
> It would be really awesome if a build could automatically extract all
> affected bugzillas since the last archived build and somehow make them
> available as a link next to the new archive link.


It would alright, but I haven't a clue if this is possible. I'll do some
thinking on it. :-)

Adrian
Re: Build system now checked into SVN [message #24974 is a reply to message #24878] Wed, 13 August 2008 08:38 Go to previous messageGo to next message
Thomas Hallgren is currently offline Thomas Hallgren
Messages: 3229
Registered: July 2009
Senior Member
I just added a new component to the Buckminster bugzilla named
'internal-build' where we can maintain issues related to the new build
system.

- thomas

Adrian Skehill wrote:
> Thomas Hallgren wrote:
>
>>>
>> I checked it out and ran it on my machine. It just works. Beautiful.
>
> Excellent, that's good to hear.
>>
>> I discovered two minor flaws.
>> 1. The platform download picks the wrong zip on an x86_64 machine
>> since that zip contains the os.arch at the end (not that it matters
>> now that you use the delta pack, but anyway).
>> 2. The check for installed features fails since the
>> ${buckminster.version} will vary between features. I think you will
>> need some concept where you check for things like
>> "org.eclipse.buckminster.core.headless.feature_*" but I'm not sure how
>> to do that. Perhaps you can convert a fileset into a property
>> (pathconvert) and then check if the property is empty?
>>
> Grand job, I'll take a look at these and get them fixed up.
>
>
>>
>>> If you have any suggestions for improvement, or ideas to share feel
>>> free to bring them up and I'll try to roll them into the build files.
>>> In particular, I'm sure there must be a better way for me to specify
>>> the properties file to produce an agnostic update site.
>>>
>> I would suggest that you provide the needed properties to the
>> perform command using the --properties <propfile> option, i.e.
>>
>> <target name="buckminster.build.site">
>> <echo message="Building Buckminster Update Site"/>
>> <property name="buckminster.properties"
>> location="buckminster.properties"/>
>> <buckminster command="perform">
>> <args>
>> <arg value="-Dbuckminster.output.root=${build.directory}"/>
>> <arg value="--properties"/>
>> <arg value="${buckminster.properties}"/>
>> <arg value="org.eclipse.buckminster#normalize.site"/>
>> </args>
>> </buckminster>
>> </target>
>
>
> Yip, much better than what I did. Late night and out of ideas tends to
> lead to some sub optimal solutions. Will roll this in.
>
>>
>>> Next plan of attack is to complete out the delivery of the update
>>> site and to wiki'fy this information. Any volunteers to review the
>>> content, and to make some suggestions as to what they'd like to see
>>> in it?
>>>
>> Count me in :-)
>>
>> I have a wish list on what I'd like to see the automatic build
>> produce. I'm however not a very typical Buckminster user so it's not
>> unlikely that my ideas can be improved a great deal. In any case,
>> here's what I'd like to see:
>>
>> 1. A 'bleeding edge' update site. This site would contain the output
>> of the last build that was automatically triggered from check-ins to
>> our SVN. I like this concept better then 'nightly builds' since a)
>> there's no reason to rebuild unless there indeed has been a check-in,
>> and b) there might be reasons to test check-ins going the full circle,
>> fix problems, test again, etc. during the day.
>>
>> 2. Integration update site. A more reliable site where we publish
>> things that we feel is semi-stable (similar to our update site today).
>> This update site must also be present as a zipped archived site that
>> we maintain a history of. I think a reasonable "pulse" for these
>> builds would be once a week.
>>
>> 3. Release site. An update site guaranteed not to move between
>> releases of the Eclipse platform.
>
>
> Despite not being a typical Buckminster user, these requirements are
> right up there with people who are using Buckminster.
>
> So #1 is easily done, I can modify the per-commit job in Hudson to build
> the update site and extract it. Though one thing that Buckminster
> developers should start getting their head around for this to be
> successful is the provision of unit tests. It's nice to have the
> confidence that stuff that gets published to a bleeding edge site will
> have tests run on it (and that this test coverage is constantly
> improving). Expect to hear more from me on this in the future :-)
>
> On #2 and #3, I've created a job on our Hudson instance called
> "Buckminster_Update_Site". This is a manually run job and can only be
> kicked off by administrators (for now Thomas & I). Also within Hudson
> I've installed the "Promoted Builds" plugin which allows us to promote
> builds that we see fit. This needs further investigation to figure out
> the optimal path, but I certainly think what you're looking for there is
> feasible with this mechanism.
>
>
>>
>> It would be really awesome if a build could automatically extract all
>> affected bugzillas since the last archived build and somehow make them
>> available as a link next to the new archive link.
>
>
> It would alright, but I haven't a clue if this is possible. I'll do some
> thinking on it. :-)
>
> Adrian
Re: Build system now checked into SVN [message #25128 is a reply to message #24807] Fri, 29 August 2008 19:03 Go to previous messageGo to next message
Scott Hendrickson is currently offline Scott Hendrickson
Messages: 69
Registered: July 2009
Member
Adrian Skehill wrote:
> Hi Folks,
>
> Just to let you know, the files that are being used by the Hudson build
> are now checked into the Buckminster svn repository in [1]. These build
> files now also build the update with the addition of an extra
> "buckminster.build.site" target.
>
> If you have any suggestions for improvement, or ideas to share feel free
> to bring them up and I'll try to roll them into the build files. In
> particular, I'm sure there must be a better way for me to specify the
> properties file to produce an agnostic update site.
>


Thanks for the build script, Adrian.

I've tried it out on our Solaris system, and it breaks when trying to
download the eclipse platform. The Solaris platform download is:

http://download.eclipse.org/eclipse/downloads/drops/R-3.4-20 0806172000/download.php?dropFile=eclipse-SDK-3.4-solaris-gtk .zip

Note that it's a ".zip" file, but the script treats it like a ".tar.gz"
file.

I'd love to get this working on our system. Could you fix the bug?

Thanks,
--Scott


> Next plan of attack is to complete out the delivery of the update site
> and to wiki'fy this information. Any volunteers to review the content,
> and to make some suggestions as to what they'd like to see in it?
>
> Regards,
> Adrian.
>
> [1]
> http://dev.eclipse.org/svnroot/tools/org.eclipse.buckminster /trunk/build/
Re: Build system now checked into SVN [message #25372 is a reply to message #25128] Wed, 10 September 2008 10:18 Go to previous message
Adrian Skehill is currently offline Adrian Skehill
Messages: 55
Registered: July 2009
Member
>
> Thanks for the build script, Adrian.
>
> I've tried it out on our Solaris system, and it breaks when trying to
> download the eclipse platform. The Solaris platform download is:
>
> http://download.eclipse.org/eclipse/downloads/drops/R-3.4-20 0806172000/download.php?dropFile=eclipse-SDK-3.4-solaris-gtk .zip
>
>
> Note that it's a ".zip" file, but the script treats it like a ".tar.gz"
> file.
>
> I'd love to get this working on our system. Could you fix the bug?
>
> Thanks,
> --Scott
>


Hi Folks, my apologies for late follow up on this. I did reply to Scott
last week but forgot to copy the newsgroup.

I made some changes to the build.xml and Scott very kindly tested it
and did the rest of the updates. I've raised a bug [1] and will now
check the changes into SVN.

A.

[1] https://bugs.eclipse.org/bugs/show_bug.cgi?id=246841
Previous Topic:[buckminster-dev] Project meta data is out of date for tools.buckminster
Next Topic:SVN access for new commiters
Goto Forum:
  


Current Time: Fri Oct 24 18:20:37 GMT 2014

Powered by FUDForum. Page generated in 0.02222 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software