Home » Archived » Buckminster » Repositories updated from rmap
|
| Re: Repositories updated from rmap [message #546103 is a reply to message #546099] |
Sun, 11 July 2010 04:02   |
 Henrik Lindberg
Messages: 2509
Registered: July 2009 |
Senior Member |
|
|
Hi, no, BM materializes the projects using the respective SCM, but does
not perform SCM update, merge, etc. Also, there are no checks if
projects have changed association to some other branch/tag. If BM finds
material that has meta data that satisfies the requirements, then
nothing new is materialized for those.
The idea is that once material is in the IDE, the developer is in
control.
In b3 we have expanded on the model to include specification of what
should happen in the various situations (under what conditions should
the checked out material be accepted? Eg some types if builds may
require that source is checked in, always updated to head of branch,
source from tags should be untampered, etc
Some of this (if nor all) will end up in buckminster.
Regards
- henrik
Miles Parker <milesparker@gmail.com> wrote:
> This is a question that is probably really obvious, but i have been
> assuming that importing a cspec automatically caused any source
> repositories in the workspace to be updated? Is this not the case?
--
- henrik
|
|
|
| Re: Repositories updated from rmap [message #546105 is a reply to message #546103] |
Sun, 11 July 2010 04:55 |
Miles Parker
Messages: 1341
Registered: July 2009 |
Senior Member |
|
|
| Henrik Lindberg wrote on Sun, 11 July 2010 00:02 | Hi, no, BM materializes the projects using the respective SCM, but does
not perform SCM update, merge, etc.
|
Ah, ok thanks Henrik, that sort of explains a lot of my fumbling around actually. #-) My thinking was that the design intent was that if you took a manifestation in one place and a manifestation from another location at the same time with the same CSPEC that you would wind up with the same contents in both places. I don't know why I made that assumption..
It sounds like to have that you need to actually wipe out and re-materialize the workplace every time you do a build based on a VCS change. But if your project has a lot of generated code, graphic images for docs, etc.. that can be a lot of stuff to pull down every time one makes a change. Or is there a better pattern for dealing with this situation?
One other option that I gather that a lot of people use it to use your build environment (Hudson) to handle the CVS for your changed code and just have the rmap to refer to local. You've got to do that anyway, if you want to trigger on VCS changes. But the disadvantage of that is that then you end up having to maintain two separate rmaps which is the kind of thing we're trying to avoid with the Buckminster approach in the first place.
Does it make any sense to look at using B3 at this point? My impression was that its not really quite ready for use as a general purpose build tool but I would like to try it especially if I can take all of the B3 knowledge along with me.
cheers,
Miles
|
|
|
| Re: Repositories updated from rmap [message #546125 is a reply to message #546105] |
Sun, 11 July 2010 12:31 |
Henrik Lindberg
Messages: 2509
Registered: July 2009 |
Senior Member |
|
|
On 7/11/10 6:55 AM, Miles Parker wrote:
> Henrik Lindberg wrote on Sun, 11 July 2010 00:02
>> Hi, no, BM materializes the projects using the respective SCM, but does
>> not perform SCM update, merge, etc.
I should have stated that more clearly. Buckminster can perform update,
but not when the workspace is used for resolution (see below).
>
> It sounds like to have that you need to actually wipe out and
> re-materialize the workplace every time you do a build based on a VCS
> change. But if your project has a lot of generated code, graphic images
> for docs, etc.. that can be a lot of stuff to pull down every time one
> makes a change. Or is there a better pattern for dealing with this
> situation?
>
There are policies in the MSPEC that can help with this - i.e. what to
do when materialization detects that there is content already available
(keep, replace, etc.). To make this happen in the first place, you can
naturally not resolve against the workspace itself because the already
checked out (and possible modified components) will satisfy the
resolution requests.
Did you try running without using the workspace in the resolution?
(I think you can even do it selectively, as you may have lots of
components that are stable and you want to speed up the process by
trusting what you already have checked out for those without even
checking against the SCM - for instance having fetched material based on
a tag).
> One other option that I gather that a lot of people use it to use your
> build environment (Hudson) to handle the CVS for your changed code and
> just have the rmap to refer to local. You've got to do that anyway, if
> you want to trigger on VCS changes. But the disadvantage of that is that
> then you end up having to maintain two separate rmaps which is the kind
> of thing we're trying to avoid with the Buckminster approach in the
> first place.
>
You are right. Don't know the best practices here, if it is possible to
do something clever - can someone else fill in?
> Does it make any sense to look at using B3 at this point? My impression
> was that its not really quite ready for use as a general purpose build
> tool but I would like to try it especially if I can take all of the B3
> knowledge along with me.
>
You are right, not ready to be used for general building at this point.
What you will see is improvements in buckminster to enable b3 to use
buckminster as an execution engine. Work has already begun on
implementing buckminster support for the repositories/components design
that we came up with for b3.
You can follow some of the discussions about this feature in the b3
newsgroup - this post
news://news.eclipse.org:119/hurh8r$ken$1@build.eclipse.org has a
description sums up the grammar/syntax for this in b3, and shows a
couple of examples. There will be something similar in Buckminster.
As you can see, the idea is to use different policies at different times
while still using the same description and mechanisms. I.e. developers
typically want a different setup (trust dirty workspace), whereas on a
server for a more official build a dirty workspace is forbidden.
With this separation it should also be possible to drive the Hudson
"watch the SCM" feature from a single description.
- henrik
|
|
|
| Re: Repositories updated from rmap [message #546155 is a reply to message #546125] |
Sun, 11 July 2010 20:51 |
Miles Parker
Messages: 1341
Registered: July 2009 |
Senior Member |
|
|
| Henrik Lindberg wrote on Sun, 11 July 2010 08:31 |
Did you try running without using the workspace in the resolution?
(I think you can even do it selectively, as you may have lots of
components that are stable and you want to speed up the process by
trusting what you already have checked out for those without even
checking against the SCM - for instance having fetched material based on
a tag).
|
If I understand what you meant, that's basically what I'm doing, I think. That is, I don't refer to the local workspace anywhere in the rmap. But of course the plugins and feature are copied into the workspace and that's where the build occurs.
Also, I have everything that I can that I know that is stable in a P2 site from my targetdef. That way I can chain builds by sending to a p2.site, then building from that p2 site, and so on. BTW, I took the excellent suggestion -- I think from Martin Taal -- of building the target platfrom in a seperate build step, so I don't need to materialize that every time I make a change.
| Quote: |
> But the disadvantage of that is that
> then you end up having to maintain two separate rmaps which is the kind
> of thing we're trying to avoid with the Buckminster approach in the
> first place.
>
You are right. Don't know the best practices here, if it is possible to
do something clever - can someone else fill in?
|
Well since you can import rmaps, you could at least take isolate just the parts that are different..have one rmap for the user workspace materialization with the local stuff and the other for the build materialization. But as I say above, I'm not sure I quite get what impact is any the searchpath based on local vs. cvs provider would have.
| Quote: | As you can see, the idea is to use different policies at different times
while still using the same description and mechanisms. I.e. developers
typically want a different setup (trust dirty workspace), whereas on a
server for a more official build a dirty workspace is forbidden.
With this separation it should also be possible to drive the Hudson
"watch the SCM" feature from a single description.
|
Yes, that is *exactly* what I think we need. IOTW, the one thing that has always bugged me about trying to do Eclipse builds is that after all you are dealing with the same artifacts in the workspace and on the build server -- why shouldn't you be able to come up with different usages from the same artifact mappings by just declaring where, in what context and for what reason you are doing it? My sense is that that's a major part of the overall usage story for you guys.
|
|
|
| Re: Repositories updated from rmap [message #546181 is a reply to message #546155] |
Mon, 12 July 2010 02:52 |
Henrik Lindberg
Messages: 2509
Registered: July 2009 |
Senior Member |
|
|
On 7/11/10 10:51 PM, Miles Parker wrote:
> Henrik Lindberg wrote on Sun, 11 July 2010 08:31
>> Did you try running without using the workspace in the resolution?
>> (I think you can even do it selectively, as you may have lots of
>> components that are stable and you want to speed up the process by
>> trusting what you already have checked out for those without even
>> checking against the SCM - for instance having fetched material based
>> on a tag).
>
>
> If I understand what you meant, that's basically what I'm doing, I
> think. That is, I don't refer to the local workspace anywhere in the
> rmap. But of course the plugins and feature are copied into the
> workspace and that's where the build occurs.
>
no, building takes place later. When *resolving* - the query treats the
workspace and the active TP as special repositories - if it can resolve
from one of these places it does not go to the RMAP. (Unless you tell it
to not use the workspace and TP). I beleive this can be turned off in
the properties, but is also available as advisor nodes in the query so
it can be done selectively to speed things up.
Sounds like you want to turn them off, and use conflict policy settings
on materialization. That will ensure that you are building what you have
specified.
- henrik
|
|
|
| Re: Repositories updated from rmap [message #546225 is a reply to message #546181] |
Mon, 12 July 2010 08:23 |
|
On 07/12/2010 04:52 AM, Henrik Lindberg wrote:
> On 7/11/10 10:51 PM, Miles Parker wrote:
>> Henrik Lindberg wrote on Sun, 11 July 2010 08:31
>>> Did you try running without using the workspace in the resolution?
>>> (I think you can even do it selectively, as you may have lots of
>>> components that are stable and you want to speed up the process by
>>> trusting what you already have checked out for those without even
>>> checking against the SCM - for instance having fetched material based
>>> on a tag).
>>
>>
>> If I understand what you meant, that's basically what I'm doing, I
>> think. That is, I don't refer to the local workspace anywhere in the
>> rmap. But of course the plugins and feature are copied into the
>> workspace and that's where the build occurs.
>>
> no, building takes place later. When *resolving* - the query treats the
> workspace and the active TP as special repositories - if it can resolve
> from one of these places it does not go to the RMAP. (Unless you tell it
> to not use the workspace and TP). I beleive this can be turned off in
> the properties, but is also available as advisor nodes in the query so
> it can be done selectively to speed things up.
>
> Sounds like you want to turn them off, and use conflict policy settings
> on materialization. That will ensure that you are building what you have
> specified.
>
The only way to turn WS/TP resolution off is by using advisor nodes in the cquery. It makes a lot of sense to be able to
control this with properties so an enhancement request for that would be good.
- thomas
|
|
|
| Re: Repositories updated from rmap [message #546239 is a reply to message #546105] |
Mon, 12 July 2010 08:59 |
Johannes Utzig
Messages: 329
Registered: July 2009 |
Senior Member |
|
|
Hi Miles,
>
> One other option that I gather that a lot of people use it to use your
> build environment (Hudson) to handle the CVS for your changed code and
> just have the rmap to refer to local. You've got to do that anyway, if
> you want to trigger on VCS changes. But the disadvantage of that is that
> then you end up having to maintain two separate rmaps which is the kind
> of thing we're trying to avoid with the Buckminster approach in the
> first place.
>
You do not need to maintain two rmaps, you just need two alternative
providers in your search path. See the example:
<searchPath name="bicng">
<provider readerType="local"
componentTypes="osgi.bundle,eclipse.feature" mutable="true" source="true">
<uri format="file:///{0}/{1}/">
<bc:propertyRef key="workspace.root" />
<bc:propertyRef key="buckminster.component" />
</uri>
</provider>
<provider
readerType="cvs"
componentTypes="eclipse.feature,osgi.bundle,buckminster"
source="true">
<uri
format=":extssh:{0}@yourcvsserver:24/var/cvs,your/modules/{1} ">
<bc:propertyRef key="cvs.user" />
<bc:propertyRef key="buckminster.component" />
</uri>
</provider>
</searchPath>
The first provider will fail on developer side because it expects the
materials to be already present in the workspace root.
Buckminster will fall back to the second provider and find the materials
there.
In Hudson the first provider will match and even if for some reason
(wrong name of a component, or so) the local provider does not find its
materials, Buckminster will be unable to use the second provider because
the cvs.user property is not set. So it will fail as expected.
Hope that helps.
Best regards,
Johannes
|
|
| |
Goto Forum:
Current Time: Tue Apr 16 21:01:32 GMT 2024
Powered by FUDForum. Page generated in 0.32398 seconds |
] 
Search
Help
Register
Login
Home
