Home » Archived » Buckminster » lacking features and some small problems.
| lacking features and some small problems. [message #502273] |
Mon, 07 December 2009 15:01  |
 David 
Messages: 88
Registered: July 2009 |
Member |
|
|
After spent time with the aggregator (editor / headless) for some time i feel its good to share what i have found so far. so sorry for a way to long post, if you want bugzilla issues on some i would be happy to provide em.
some machine details:
eclipse.buildId=M20090917-0800
java.version=1.5.0_14
java.vendor=Sun Microsystems Inc.
BootLoader constants: OS=win32, ARCH=x86, WS=win32, NL=sv_SE
Buckminster Aggregator Editor 1.0.0.r10994 org.eclipse.buckminster.aggregator.editor.feature.feature.gr oup
some more stuff:
* We are behind a firewall
* Around 600 machines have Eclipse
* Developers are not allowed to install plugins themselves
* Developers *should* not go out on internet and download plugins via the update manager ( devs are cleaver they always break this)
* We have many different developer languages (Java, C/C++, Cobol, Perl, VB etc)
Or current update site was built as an "updatesite project" it was basically handcrafted after going through an mirror of galileo and picking the stuff we needed and discarding the rest. Using 17 different update sites to build the internal updatesites, all from code analysis programs to small helpers etc.
Took a long time to handcraft, ant scripts to mirror down changes, install into an IDE then build and test the update site worked as it should. Much of this was manual labour, with me not liking the internal proxy that cuts downloads after 5 minutes, forces ntlm authentification but seem to drop the line anyways after awhile all this makes any updates to the internal site hard to manage and not easy to teach any other.
The "update site project" editor almost encourages you to make mistakes. Adding features with out understanding their relationship is easy, adding an top node feature like "Eclipse web developer tools" for instance then also adding features from org.eclipse.wst.* into the same category is possible it does not complain because the top feature already has these features as required and will happily build. Which i did 
You get into problems when you will update that site for instance with the SR1 Galileo releases. Since the users are default getting Eclipse 3.5.0 (as an pushed out package via some fancy install mechanism) those org.eclipse.wst.* features was installed by users as the users have no clue they just check the entire "Web, XML, and Java EE Development" categori and go on.
Now trying to update a 3.5.0 client with an Galileo SR1 release will not work!
you get into problems like:
Cannot complete the install because of a conflicting dependency.
Software being installed: Eclipse Web Developer Tools 3.1.1.v200908120400-7R77FSpEVw2xXR0CtKvX6bG6a2qU (org.eclipse.wst.web_ui.feature.feature.group 3.1.1.v200908120400-7R77FSpEVw2xXR0CtKvX6bG6a2qU)
Software currently installed: WST Server UI 3.1.0.v200901260254-7B3DLAtJZNyxfIUbDVOO4256 (org.eclipse.wst.server_ui.feature.feature.group 3.1.0.v200901260254-7B3DLAtJZNyxfIUbDVOO4256)
Only one of the following can be installed at once:
Server UI 1.1.102.v20090515 (org.eclipse.wst.server.ui 1.1.102.v20090515)
Server UI 1.1.103.v20090909 (org.eclipse.wst.server.ui 1.1.103.v20090909)
Cannot satisfy dependency:
From: WST Server UI 3.1.0.v200901260254-7B3DLAtJZNyxfIUbDVOO4256 (org.eclipse.wst.server_ui.feature.feature.group 3.1.0.v200901260254-7B3DLAtJZNyxfIUbDVOO4256)
To: org.eclipse.wst.server.ui [1.1.102.v20090515]
Cannot satisfy dependency:
From: WST Server UI 3.1.1.v20090727b-7B3DOAtJZOyxlIZDDYPO4256 (org.eclipse.wst.server_ui.feature.feature.group 3.1.1.v20090727b-7B3DOAtJZOyxlIZDDYPO4256)
To: org.eclipse.wst.server.ui [1.1.103.v20090909]
Cannot satisfy dependency:
From: Eclipse Web Developer Tools 3.1.1.v200908120400-7R77FSpEVw2xXR0CtKvX6bG6a2qU (org.eclipse.wst.web_ui.feature.feature.group 3.1.1.v200908120400-7R77FSpEVw2xXR0CtKvX6bG6a2qU)
To: org.eclipse.wst.server_ui.feature.feature.group [3.1.1.v20090727b-7B3DOAtJZOyxlIZDDYPO4256]
So why did the update site project allow me to make these kind of mistakes? You need to revert config and then install from scratch and you are fine. Fun for all devs to do? no.
The aggregator also allows this kind of behavior, and it must, since you might just need "org.eclipse.wst.server_ui.feature.feature.group" and nothing else.
How can you detect that there is an feature "above" you that is dependent on you? This kind of metadata is not available to that feature at least what i know.
Maybe some kind of analysis could be done on the site you are trying to aggregate and spit out an warning/error saying "org.eclipse.wst.server_ui.feature.feature.group is required by org.eclipse.wst.web_ui.feature.feature.group already will not add it as duplicate metadata"
enough ranting on that problem. Maybe i have missed something?
Some more problems/lacking features i have found
1. Copying and pasting IUs, say i would like to add all org.eclipse.wst.* to an specific homemade category, will not work. Marking them all in the editor ctrl+c will work it seems, but pasting does not ( just greayed out) i get no stacktrace, etc.
2. Drag and dropping of IUs to categories in big models works but is cumbersome, since all the IUs is all the way down that i need now copy paste does not work as a mentioned before so i need to drag it upwards to the category i need and it just jumps 1 node at like every 2 secs and i have alot of nodes to go up! Will be solved if you fix the copy paste problem of course.
3. Starting the aggregator with a big model and many mapped repositories seems for me at least always start with that the model ends up with a lot of red crosses, and i need to manually reload the repos. Even though that all repos are internal and fast accessible. I can't select many nodes (ctrl+ click on mapped repositories) and select "reload repository" either so i need to right click each one.
4. version on mirroring "products", i mirrored the eclipse 3.5.1 release area (http://download.eclipse.org/eclipse/updates/3.5) , so firstly i listed it in the aggregator UI and saw there is several versions available both the 3.5.0 and the 3.5.1 SDK are available from this site, i.e org.eclipse.sdk.ide / 3.5.0 and 3.5.1, thought i need em both and tried to mirror. I got an error in the error log saying:
Error
Mon Dec 07 15:40:48 CET 2009
org.eclipse.sdk.ide is explicitly mapped using both version 3.5.0.I20090611-1540 and 3.5.1.M20090917-0800
so here i am not allowed to mirror both versions, i can solve that by breaking it apart but how did this site get built then in the first place if mirroring of two versions is not allowed?
5. listing headless with versions?
buckminster.bat listsite http://download.eclipse.org/eclipse/updates/3.5
gets:
com.ibm.icu.base master-ecf (build utility feature for ecf plugins)
master-equinox-p2 (build utility feature for equinox p2 plugins)
master-equinox (build utility feature for equinox plugins)
master-jetty (build utility feature for equinox http jetty plugins)
org.eclipse.cvs (%featureName)
org.eclipse.cvs.source (%featureName)
org.eclipse.equinox.compendium.sdk (%featureName)
org.eclipse.equinox.core.sdk (%featureName)
org.eclipse.equinox.executable (%featureName)
org.eclipse.equinox.jmx.client.feature (Resource Monitoring Client Feature (Incubation))
org.eclipse.equinox.jmx.common.feature (Resource Monitoring Common Feature (Incubation))
org.eclipse.equinox.jmx.server.feature (Resource Monitoring Server Feature (Incubation))
org.eclipse.equinox.p2.agent.feature (Equinox p2 Agent)
org.eclipse.equinox.p2.director.feature (Equinox p2 Director Application)
org.eclipse.equinox.p2.generator.feature (Equinox p2 Generator Application)
org.eclipse.equinox.p2.sdk (%featureName)
....
where are the versions? if i headless would like to find out the versions on the update site then take actions, to either update or not? seems to not be possible. ( this use case worked in the old update manager)
Testing the aggregator was like to come to another world where it "just worked".
Now our internal update sites is provisioned through an headless director, that headless gets buckminster headless, that headless gets the headless aggregator that gets just 1 small update site, then we loop this and have an .build file for all update sites so we mirror in all changes from the external update sites. We then use an "master" .build file to build the entire update site from all the local provisioned update sites. The final site now build in around 5 minutes headless with confidence! Keep the good work up 
cheers
Dave an happy tester!
|
|
| |
| Re: lacking features and some small problems. [message #502291 is a reply to message #502273] |
Mon, 07 December 2009 15:38  |
|
Hi David,
On 2009-12-07 16:01, David wrote:
> Maybe some kind of analysis could be done on the site you are trying to
> aggregate and spit out an warning/error saying
> "org.eclipse.wst.server_ui.feature.feature.group is required by
> org.eclipse.wst.web_ui.feature.feature.group already will not add it as
> duplicate metadata"
>
> enough ranting on that problem. Maybe i have missed something?
>
I guess you'd like to see that once the org.eclipse.wst.web_ui.feature.feature.group is mapped, everything that it in
turn will bring with it would be marked as 'already mapped'? I guess that would be possible. It's a matter of creating a
slice using the known roots.
In any case, mapping it again, inconvenient as it might be, will be a no-op. The aggregator will not add the same
feature twice. There will never be duplicate meta-data in a repository created by the aggregator.
> Some more problems/lacking features i have found
>
> 1. Copying and pasting IUs, say i would like to add all
> org.eclipse.wst.* to an specific homemade category, will not work.
> Marking them all in the editor ctrl+c will work it seems, but pasting
> does not ( just greayed out) i get no stacktrace, etc.
>
Sounds like a deficiency that deserves a bugzilla.
> 3. Starting the aggregator with a big model and many mapped repositories
> seems for me at least always start with that the model ends up with a
> lot of red crosses, and i need to manually reload the repos. Even though
> that all repos are internal and fast accessible. I can't select many
> nodes (ctrl+ click on mapped repositories) and select "reload
> repository" either so i need to right click each one.
>
I don't like the sound of that. Please enter a bugzilla for that as well where we can continue examining and discussing.
> 4. version on mirroring "products", i mirrored the eclipse 3.5.1 release
> area (http://download.eclipse.org/eclipse/updates/3.5) , so firstly i
> listed it in the aggregator UI and saw there is several versions
> available both the 3.5.0 and the 3.5.1 SDK are available from this site,
> i.e org.eclipse.sdk.ide / 3.5.0 and 3.5.1, thought i need em both and
> tried to mirror. I got an error in the error log saying:
>
> Error
> Mon Dec 07 15:40:48 CET 2009
> org.eclipse.sdk.ide is explicitly mapped using both version
> 3.5.0.I20090611-1540 and 3.5.1.M20090917-0800
>
> so here i am not allowed to mirror both versions, i can solve that by
> breaking it apart but how did this site get built then in the first
> place if mirroring of two versions is not allowed?
>
Mirroring both versions means that you create a repository that is internally inconsistent. That in turn, is a potential
problem for anyone using that repository.
A solution could be to create two mirrors, one for each product, and then bring them together using a composite. The
composite will then of course be inconsistent in itself but in some use-cases that is the preferred way of doing things.
>
> 5. listing headless with versions?
>
> buckminster.bat listsite http://download.eclipse.org/eclipse/updates/3.5
>
> gets:
> com.ibm.icu.base master-ecf (build utility feature for ecf plugins)
> master-equinox-p2 (build utility feature for equinox p2 plugins)
> master-equinox (build utility feature for equinox plugins)
> master-jetty (build utility feature for equinox http jetty plugins)
> org.eclipse.cvs (%featureName)
> org.eclipse.cvs.source (%featureName)
> org.eclipse.equinox.compendium.sdk (%featureName)
> org.eclipse.equinox.core.sdk (%featureName)
> org.eclipse.equinox.executable (%featureName)
> org.eclipse.equinox.jmx.client.feature (Resource Monitoring Client
> Feature (Incubation))
> org.eclipse.equinox.jmx.common.feature (Resource Monitoring Common
> Feature (Incubation))
> org.eclipse.equinox.jmx.server.feature (Resource Monitoring Server
> Feature (Incubation))
> org.eclipse.equinox.p2.agent.feature (Equinox p2 Agent)
> org.eclipse.equinox.p2.director.feature (Equinox p2 Director Application)
> org.eclipse.equinox.p2.generator.feature (Equinox p2 Generator Application)
> org.eclipse.equinox.p2.sdk (%featureName)
> ...
>
> where are the versions? if i headless would like to find out the
> versions on the update site then take actions, to either update or not?
> seems to not be possible. ( this use case worked in the old update manager)
>
I agree. The versions should definitely be included (and the %featureName should be translated). Can you please enter an
enhancement request for this in our bugzilla?
> Testing the aggregator was like to come to another world where it "just
> worked".
> Now our internal update sites is provisioned through an headless
> director, that headless gets buckminster headless, that headless gets
> the headless aggregator that gets just 1 small update site, then we loop
> this and have an .build file for all update sites so we mirror in all
> changes from the external update sites. We then use an "master" .build
> file to build the entire update site from all the local provisioned
> update sites. The final site now build in around 5 minutes headless with
> confidence! Keep the good work up :)
>
> cheers
>
> Dave an happy tester!
That sounds great! We appreciate your feedback very much. I'm sorry I have to bug you with requests to enter bugzillas
but that's mostly because we want to keep track of who helps out with what. I.e. you get the kudos for your own
feedback. You will of course also get automatic feedback from us when we act on the bugzilla.
Kind Regards,
Thomas Hallgren
|
|
| | |
| Re: lacking features and some small problems. [message #502465 is a reply to message #502273] |
Tue, 08 December 2009 12:17 |
David
Messages: 88
Registered: July 2009 |
Member |
|
|
There updated to last release.
created:
https://bugs.eclipse.org/bugs/show_bug.cgi?id=297165 (copy paste issue)
https://bugs.eclipse.org/bugs/show_bug.cgi?id=297180 (no version information in headless )
I can't reproduce the red crosses anymore, so that seems to been fixed.
| Quote: | I guess you'd like to see that once the org.eclipse.wst.web_ui.feature.feature.group is mapped, everything that it in turn will bring with it would be marked as 'already mapped'? I guess that would be possible. It's a matter of creating a slice using the known roots.
|
Yes, that's what i meant. Either it's provided when you try to validate the model or as an error when building it.
| Quote: | In any case, mapping it again, inconvenient as it might be, will be a no-op. The aggregator will not add the same
feature twice. There will never be duplicate meta-data in a repository created by the aggregator.
|
Today it allows this behavior, or rather it allows to add an feature that is dependent on a feature below it and you can still add that feature to the same category even.
One solution might be to grey out those IUs that already was dependent by some feature.
I totally forgot about the composite repos and that's of course how the site is build (http://download.eclipse.org/eclipse/updates/3.5)
Another thing that i have seen is that many repos keep old versions lying there. That means when i aggregate these that version is always there, and thus the bigger build will always find that version and never fail ( if i have pointed out feature and version). But they have upgraded their versions on that site but i don't know since all builds will work, and then i have old versions if this was totally automated.
I guess this is more of a process issue then a tool issue, how often do i need to go through the nodes and see if there is any updates. One thing that would be neat is some kind of mechanism that alerted you that changes had been done in one of the mirrors and you need to act upon it basically. Notify on changes from current.
Really an nice to have feature though.
cheers
Dave
|
|
| | |
Goto Forum:
Current Time: Thu Apr 25 21:38:34 GMT 2024
Powered by FUDForum. Page generated in 0.02971 seconds |
] 
Search
Help
Register
Login
Home
