Eclipse Community Forums: Buckminster » SVN materialization issues in 3.5

Help

Home

Home » Archived » Buckminster » SVN materialization issues in 3.5

Show: Today's Messages :: Show Polls :: Message Navigator

SVN materialization issues in 3.5 [message #387653]

Tue, 07 July 2009 19:11

Nick Veys

Messages: 52
Registered: July 2009

Member

Working on upgrading Buckminster from 3.4 to 3.5, and I'm running into
some troubles materializing our workspace. The exact location (in terms
of which plugin) of the error is fairly intermittent but the issue in
general is pretty consistent. I think the workspace materialized
successfully twice out of tens of tries.

This is with Buckminster 1.1.350.r10356, built from update sites
yesterday. Using subversive 1.0.0.r10301.

I'm seeing the following exception: http://pastebin.com/m2e655477

It complains that the working directory of the plugin it is trying to
checkout is locked. It seems like some of the parallel materialization
jobs are trying to check out the same plugins at the same time leading to
obvious problems. It then tries to recursively delete the directory since
it failed, which fails, presumably because it's still in use.

I'm debugging to try and figure out what exactly is going on but I figured
I'd ask and see if there's any guidance on where to look. I've managed to
trap multiple instances of SubversiveRemoteFile trying to run
innerMaterialize where their m_session URL's are identical. And these are
the same URLs that will eventually kill the materialization.

Ideas? Should it be possible to have 2 MaterializerJob instances acting
on the same component?

Report message to a moderator

Re: SVN materialization issues in 3.5 [message #387654 is a reply to message #387653]

Tue, 07 July 2009 19:53

Nick Veys

Messages: 52
Registered: July 2009

Member

I've dumped all 'Resolutions' being created and handed to the individual
MaterializerJobs. I have duplicates of 4 different plugins, and hopefully
not coincidentally those plugins are all typically listed as
'resolution:=optional' by most of our other plugins.

Report message to a moderator

Re: SVN materialization issues in 3.5 [message #387655 is a reply to message #387653]

Tue, 07 July 2009 22:22

Thomas Hallgren

Messages: 3240
Registered: July 2009

Senior Member

Nick Veys wrote:
> Ideas? Should it be possible to have 2 MaterializerJob instances acting
> on the same component?
>
No, buckminster should never make an attempt to materialize the same component twice. Is it possible that the resolution
contains two non-overlapping versions of the same source? Say from different branches?

- thomas

Report message to a moderator

Re: SVN materialization issues in 3.5 [message #387656 is a reply to message #387653]

Tue, 07 July 2009 22:43

Thomas Hallgren

Messages: 3240
Registered: July 2009

Senior Member

Nick Veys wrote:
> This is with Buckminster 1.1.350.r10356, built from update sites
> yesterday. Using subversive 1.0.0.r10301.
>
The version seems a bit old though. Looking at the Galileo release, Buckminster core is at 1.1.350.r10382 and the
subversive feature is at 1.0.0.r10356. So from what update site did you get this yesterday?

- thomas

Report message to a moderator

Re: SVN materialization issues in 3.5 [message #387720 is a reply to message #387656]

Wed, 08 July 2009 15:10

Nick Veys

Messages: 52
Registered: July 2009

Member

My mistake, looking @ the plugins I see core_1.1.350.r10382, but
subversive is still 1.0.0.r10301. These are the plugin versions. I'm not
sure where features live in this new p2 world.

But it doesn't look like Subversive is related at all. During the
BOM.findMaterializationCandidates, the empty list is created and filled by
walking the tree, an empty HashSet is seeded as a 'perused' set to (I
would assume) prevent duplicates being entered into the list. But the
Resolution class inherits a final hashCode/equals from UUIDKeyed that
simply rely on raw XML content describing the resolution. This doesn't
seem to be correct for all Resolutions.

I've dumped all the resolutions being added in
MaterializationJob.prepareJobs, and here is an example of a duplicate
entry (first the Resolution toString, then the XML image for it):

workspace: Name: com.junitRunner, 2.0.0.qualifier
<?xml version="1.0" encoding="UTF-8"?>
<md:resolution xmlns:md="http://www.eclipse.org/buckminster/MetaData-1.0"
cspecId="36fe0779-e16a-3bd2-9e4b-913ee772754b" materializable="true"
providerId="c7ccc97e-1c9d-318a-a28f-ecae37428d24"
repository="http://svn/svnroot/core/trunk/plugins/com.junitRunner"
componentType="osgi.bundle">
<md:request name="com.junitRunner" componentType="osgi.bundle"/>
<md:versionMatch version="2.0.0.qualifier"/>
</md:resolution>

workspace: Name: com.junitRunner, 2.0.0.qualifier
<?xml version="1.0" encoding="UTF-8"?>
<md:resolution xmlns:md="http://www.eclipse.org/buckminster/MetaData-1.0"
cspecId="36fe0779-e16a-3bd2-9e4b-913ee772754b" materializable="true"
providerId="c7ccc97e-1c9d-318a-a28f-ecae37428d24"
repository="http://svn/svnroot/core/trunk/plugins/com.junitRunner"
componentType="osgi.bundle">
<md:request name="com.junitRunner" componentType="osgi.bundle"
filter="(!(eclipse.p2.optional=false))"/>
<md:versionMatch version="2.0.0.qualifier"/>
</md:resolution>

So the first one is required, as it is listed in some of our plugins as a
required dependency, however it is also listed as optional in some other
plugins. So the second one seems to be that optional dependency with the
not-not-optional filter applied. This makes the XML images different,
therefore the Resolutions are different and bam, duplicate resolutions.

Thomas Hallgren wrote:

> Nick Veys wrote:
>> This is with Buckminster 1.1.350.r10356, built from update sites
>> yesterday. Using subversive 1.0.0.r10301.
>>
> The version seems a bit old though. Looking at the Galileo release,
Buckminster core is at 1.1.350.r10382 and the
> subversive feature is at 1.0.0.r10356. So from what update site did you get
this yesterday?

> - thomas

Report message to a moderator

Re: SVN materialization issues in 3.5 [message #387721 is a reply to message #387720]

Wed, 08 July 2009 15:14

Thomas Hallgren

Messages: 3240
Registered: July 2009

Senior Member

Hi Nick,
Thanks for tracking this down. The perused HashSet should not use the full as key resolution at this stage. Can you
please enter a bugzilla for this?

Thanks,
- thomas

Nick Veys wrote:
> My mistake, looking @ the plugins I see core_1.1.350.r10382, but
> subversive is still 1.0.0.r10301. These are the plugin versions. I'm
> not sure where features live in this new p2 world.
>
> But it doesn't look like Subversive is related at all. During the
> BOM.findMaterializationCandidates, the empty list is created and filled
> by walking the tree, an empty HashSet is seeded as a 'perused' set to (I
> would assume) prevent duplicates being entered into the list. But the
> Resolution class inherits a final hashCode/equals from UUIDKeyed that
> simply rely on raw XML content describing the resolution. This doesn't
> seem to be correct for all Resolutions.
>
> I've dumped all the resolutions being added in
> MaterializationJob.prepareJobs, and here is an example of a duplicate
> entry (first the Resolution toString, then the XML image for it):
>
> workspace: Name: com.junitRunner, 2.0.0.qualifier
> <?xml version="1.0" encoding="UTF-8"?>
> <md:resolution xmlns:md="http://www.eclipse.org/buckminster/MetaData-1.0"
> cspecId="36fe0779-e16a-3bd2-9e4b-913ee772754b" materializable="true"
> providerId="c7ccc97e-1c9d-318a-a28f-ecae37428d24"
> repository="http://svn/svnroot/core/trunk/plugins/com.junitRunner"
> componentType="osgi.bundle">
> <md:request name="com.junitRunner" componentType="osgi.bundle"/>
> <md:versionMatch version="2.0.0.qualifier"/>
> </md:resolution>
> workspace: Name: com.junitRunner, 2.0.0.qualifier
> <?xml version="1.0" encoding="UTF-8"?>
> <md:resolution xmlns:md="http://www.eclipse.org/buckminster/MetaData-1.0"
> cspecId="36fe0779-e16a-3bd2-9e4b-913ee772754b" materializable="true"
> providerId="c7ccc97e-1c9d-318a-a28f-ecae37428d24"
> repository="http://svn/svnroot/core/trunk/plugins/com.junitRunner"
> componentType="osgi.bundle">
> <md:request name="com.junitRunner" componentType="osgi.bundle"
> filter="(!(eclipse.p2.optional=false))"/>
> <md:versionMatch version="2.0.0.qualifier"/>
> </md:resolution>
>
> So the first one is required, as it is listed in some of our plugins as
> a required dependency, however it is also listed as optional in some
> other plugins. So the second one seems to be that optional dependency
> with the not-not-optional filter applied. This makes the XML images
> different, therefore the Resolutions are different and bam, duplicate
> resolutions.
>
> Thomas Hallgren wrote:
>
>> Nick Veys wrote:
>>> This is with Buckminster 1.1.350.r10356, built from update sites
>>> yesterday. Using subversive 1.0.0.r10301.
>>>
>> The version seems a bit old though. Looking at the Galileo release,
> Buckminster core is at 1.1.350.r10382 and the
>> subversive feature is at 1.0.0.r10356. So from what update site did
>> you get
> this yesterday?
>
>> - thomas
>
>

Report message to a moderator