Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » EMF » [CDO] CDOChangeSetData with CDOCollectionLoadingPolicy set
[CDO] CDOChangeSetData with CDOCollectionLoadingPolicy set [message #653882] Fri, 11 February 2011 17:28 Go to next message
Szabolcs Bardy is currently offline Szabolcs BardyFriend
Messages: 9
Registered: July 2009
Junior Member
Hi,

in our CDO model we have to handle very large lists. As I read at http://wiki.eclipse.org/Tweaking_CDO_Performance, collection handling can be optimized by fetching collections partially using an optimized CDOCollectionLoadingPolicy. If we use CollectionLoading.Default loading of objects containing large lists take a while, probably because all of the CDOIDs will be loaded when accessing the object. This issue can be solved by using the suggested CDOUtil.createCollectionLoadingPolicy(0, 300) policy.

Another CDO feature that we would like to use in our application is CDO branches, and telling the difference between two branches using CDOChangeSetData. Our problem is that there is a difference between the change sets, when using the different collection loading policies.

The model is the following: there are concepts, a concept can have descriptions, and relationships to other concepts.

When using the default collection loading policy the CDOChangeSetData computed between 2 branch points is the following:

ChangeSet content: ChangeSetData[newObjects=3, changedObjects=2, detachedObjects=0]
--- Changed objects ---
CDORevisionDelta[CDOResource@OID2:0v1 --> [CDOFeatureDelta[contents, LIST, list=[CDOFeatureDelta[contents, ADD, value=OID6409, index=1 404]]]]]
CDORevisionDelta[Concept@OID76:0v1 --> [CDOFeatureDelta[inboundRelationships, LIST, list=[CDOFeatureDelta[inboundRelationships, ADD, value=OID6411, index=3]]]]]
--- New objects ---
Concept@OID6409:6v1
Description@OID6410:6v1
Relationship@OID6411:6v1

That's ok, a new concept with 1 description and 1 relationship has been added to a CDOResource.

However when using the non-default collection loading policy the ChangeSet will contain lots of unnecessary elements:

ChangeSet content: ChangeSetData[newObjects=3, changedObjects=2, detachedObjects=0]
--- Changed objects ---
CDORevisionDelta[CDOResource@OID2:0v1 --> [CDOFeatureDelta[contents, LIST, list=[CDOFeatureDelta[contents, ADD, value=CDOElementProxy[0], index=0],....]
CDORevisionDelta[Concept@OID76:0v1 -->
[CDOFeatureDelta[descriptions, LIST, list=[
CDOFeatureDelta[descriptions, ADD, value=CDOElementProxy[0], index=0],
CDOFeatureDelta[descriptions, ADD, value=CDOElementProxy[1], index=1],
CDOFeatureDelta[descriptions, ADD, value=CDOElementProxy[2], index=2],
CDOFeatureDelta[descriptions, ADD, value=CDOElementProxy[3], index=3],..
CDOFeatureDelta[descriptions, REMOVE, value=CDOElementProxy[29], index=59],
CDOFeatureDelta[descriptions, REMOVE, value=CDOElementProxy[28], index=58],
CDOFeatureDelta[descriptions, REMOVE, value=CDOElementProxy[27], index=57],...,
CDOFeatureDelta[inboundRelationships, LIST, list=[
CDOFeatureDelta[inboundRelationships, ADD, value=CDOElementProxy[0], index=0],
CDOFeatureDelta[inboundRelationships, ADD, value=CDOElementProxy[1], index=1],
CDOFeatureDelta[inboundRelationships, ADD, value=CDOElementProxy[2], index=2],
CDOFeatureDelta[inboundRelationships, ADD, value=OID6408, index=3],
CDOFeatureDelta[inboundRelationships, REMOVE, value=CDOElementProxy[2], index=6],
CDOFeatureDelta[inboundRelationships, REMOVE, value=CDOElementProxy[1], index=5],
CDOFeatureDelta[inboundRelationships, REMOVE, value=CDOElementProxy[0], index=4]]]]]
--- New objects ---
Relationship@OID6408:5v1
Concept@OID6406:5v1
Description@OID6407:5v1

It seems that each 'ADD' CDOFeatureDelta has a corresponding opposite 'REMOVE' CDOFeatureDelta containing CDOElementProxys as values. I suppose this is in connection with the loading of objects.
Is it correct that these elements appear in the change set?


Another problem appears when we try to merge back the branch to the MAIN branch. Again the change set contains unwanted CDOProxy elements, and the following exception is thrown, when trying to commit the changes to the MAIN branch after the merge:

Caused by: java.lang.IllegalStateException: Unable to provideCDOID: org.eclipse.emf.internal.cdo.revision.CDOElementProxyImpl
at org.eclipse.emf.internal.cdo.view.AbstractCDOView.provideCDO ID(AbstractCDOView.java:858)
at org.eclipse.emf.internal.cdo.transaction.CDOTransactionImpl. provideCDOID(CDOTransactionImpl.java:1985)
at org.eclipse.emf.cdo.internal.common.revision.delta.CDOSingle ValueFeatureDeltaImpl.writeValue(CDOSingleValueFeatureDeltaI mpl.java:86)
at org.eclipse.emf.cdo.internal.common.revision.delta.CDOSingle ValueFeatureDeltaImpl.write(CDOSingleValueFeatureDeltaImpl.j ava:62)
at org.eclipse.emf.cdo.internal.common.protocol.CDODataOutputIm pl.writeCDOFeatureDelta(CDODataOutputImpl.java:379)
at org.eclipse.emf.cdo.internal.common.revision.delta.CDOListFe atureDeltaImpl.write(CDOListFeatureDeltaImpl.java:136)
at org.eclipse.emf.cdo.internal.common.protocol.CDODataOutputIm pl.writeCDOFeatureDelta(CDODataOutputImpl.java:379)
at org.eclipse.emf.cdo.internal.common.revision.delta.CDORevisi onDeltaImpl.write(CDORevisionDeltaImpl.java:158)
at org.eclipse.emf.cdo.internal.common.protocol.CDODataOutputIm pl.writeCDORevisionDelta(CDODataOutputImpl.java:374)
at org.eclipse.emf.cdo.internal.net4j.protocol.CommitTransactio nRequest.requestingCommit(CommitTransactionRequest.java:161)
at org.eclipse.emf.cdo.internal.net4j.protocol.CommitTransactio nRequest.requesting(CommitTransactionRequest.java:112)
at org.eclipse.emf.cdo.internal.net4j.protocol.CDOClientRequest WithMonitoring.requesting(CDOClientRequestWithMonitoring.jav a:92)
at org.eclipse.net4j.signal.RequestWithMonitoring.requesting(Re questWithMonitoring.java:163)
at org.eclipse.net4j.signal.RequestWithConfirmation.doExtendedO utput(RequestWithConfirmation.java:117)
at org.eclipse.net4j.signal.Signal.doOutput(Signal.java:296)
at org.eclipse.net4j.signal.RequestWithConfirmation.doExecute(R equestWithConfirmation.java:102)
at org.eclipse.net4j.signal.RequestWithMonitoring.doExecute(Req uestWithMonitoring.java:233)
at org.eclipse.net4j.signal.SignalActor.execute(SignalActor.jav a:51)
at org.eclipse.net4j.signal.Signal.runSync(Signal.java:251)
at org.eclipse.net4j.signal.SignalProtocol.startSignal(SignalPr otocol.java:396)
at org.eclipse.net4j.signal.RequestWithConfirmation.doSend(Requ estWithConfirmation.java:87)
at org.eclipse.net4j.signal.RequestWithConfirmation.send(Reques tWithConfirmation.java:73)
at org.eclipse.net4j.signal.RequestWithMonitoring.send(RequestW ithMonitoring.java:108)
at org.eclipse.emf.cdo.internal.net4j.protocol.CDOClientProtoco l.send(CDOClientProtocol.java:413)
at org.eclipse.emf.cdo.internal.net4j.protocol.CDOClientProtoco l.commitTransaction(CDOClientProtocol.java:295)
at org.eclipse.emf.internal.cdo.transaction.CDOSingleTransactio nStrategyImpl.commit(CDOSingleTransactionStrategyImpl.java:8 0)
at org.eclipse.emf.internal.cdo.transaction.CDOTransactionImpl. commit(CDOTransactionImpl.java:1027)
... 26 more


Is it perhaps a bug in CDO? Or is there any option we can set so that change sets do not contain the unwanted CDOElementProxy values?

Thanks,
Szabolcs
Re: [CDO] CDOChangeSetData with CDOCollectionLoadingPolicy set [message #653924 is a reply to message #653882] Fri, 11 February 2011 21:33 Go to previous messageGo to next message
Eike Stepper is currently offline Eike StepperFriend
Messages: 6450
Registered: July 2009
Senior Member
Am 11.02.2011 18:28, schrieb Szabolcs Bardy:
> Hi,
>
> in our CDO model we have to handle very large lists. As I read at http://wiki.eclipse.org/Tweaking_CDO_Performance, collection handling can be optimized by fetching collections partially using an optimized CDOCollectionLoadingPolicy. If we use CollectionLoading.Default loading of objects containing large lists take a while, probably because all of the CDOIDs will be loaded when accessing the object. This issue can be solved by using the suggested CDOUtil.createCollectionLoadingPolicy(0, 300) policy.
>
> Another CDO feature that we would like to use in our application is CDO branches, and telling the difference between two branches using CDOChangeSetData. Our problem is that there is a difference between the change sets, when using the different collection loading policies.
>
> The model is the following: there are concepts, a concept can have descriptions, and relationships to other concepts.
>
> When using the default collection loading policy the CDOChangeSetData computed between 2 branch points is the following:
>
> ChangeSet content: ChangeSetData[newObjects=3, changedObjects=2, detachedObjects=0]
> --- Changed objects ---
> CDORevisionDelta[mailto:CDOResource@OID2:0v1 --> [CDOFeatureDelta[contents, LIST, list=[CDOFeatureDelta[contents, ADD, value=OID6409, index=1 404]]]]]
> CDORevisionDelta[mailto:Concept@OID76:0v1 --> [CDOFeatureDelta[inboundRelationships, LIST, list=[CDOFeatureDelta[inboundRelationships, ADD, value=OID6411, index=3]]]]]
> --- New objects ---
> mailto:Concept@OID6409:6v1
> mailto:Description@OID6410:6v1
> mailto:Relationship@OID6411:6v1
>
> That's ok, a new concept with 1 description and 1 relationship has been added to a CDOResource.
>
> However when using the non-default collection loading policy the ChangeSet will contain lots of unnecessary elements:
>
> ChangeSet content: ChangeSetData[newObjects=3, changedObjects=2, detachedObjects=0]
> --- Changed objects ---
> CDORevisionDelta[mailto:CDOResource@OID2:0v1 --> [CDOFeatureDelta[contents, LIST, list=[CDOFeatureDelta[contents, ADD, value=CDOElementProxy[0], index=0],....]
> CDORevisionDelta[mailto:Concept@OID76:0v1 --> [CDOFeatureDelta[descriptions, LIST, list=[
> CDOFeatureDelta[descriptions, ADD, value=CDOElementProxy[0], index=0], CDOFeatureDelta[descriptions, ADD, value=CDOElementProxy[1], index=1], CDOFeatureDelta[descriptions, ADD, value=CDOElementProxy[2], index=2], CDOFeatureDelta[descriptions, ADD, value=CDOElementProxy[3], index=3],.. CDOFeatureDelta[descriptions, REMOVE, value=CDOElementProxy[29], index=59], CDOFeatureDelta[descriptions, REMOVE, value=CDOElementProxy[28], index=58], CDOFeatureDelta[descriptions, REMOVE, value=CDOElementProxy[27], index=57],..., CDOFeatureDelta[inboundRelationships, LIST, list=[
> CDOFeatureDelta[inboundRelationships, ADD, value=CDOElementProxy[0], index=0], CDOFeatureDelta[inboundRelationships, ADD, value=CDOElementProxy[1], index=1], CDOFeatureDelta[inboundRelationships, ADD, value=CDOElementProxy[2], index=2], CDOFeatureDelta[inboundRelationships, ADD, value=OID6408, index=3], CDOFeatureDelta[inboundRelationships, REMOVE, value=CDOElementProxy[2], index=6], CDOFeatureDelta[inboundRelationships, REMOVE, value=CDOElementProxy[1], index=5], CDOFeatureDelta[inboundRelationships, REMOVE, value=CDOElementProxy[0], index=4]]]]]
> --- New objects ---
> mailto:Relationship@OID6408:5v1
> mailto:Concept@OID6406:5v1
> mailto:Description@OID6407:5v1
>
> It seems that each 'ADD' CDOFeatureDelta has a corresponding opposite 'REMOVE' CDOFeatureDelta containing CDOElementProxys as values. I suppose this is in connection with the loading of objects. Is it correct that these elements appear in the change set?
I doubt it. It's well possible that the new change sets do not properly work in combination with partial collection loading.

> Another problem appears when we try to merge back the branch to the MAIN branch. Again the change set contains unwanted CDOProxy elements, and the following exception is thrown, when trying to commit the changes to the MAIN branch after the merge:
> Caused by: java.lang.IllegalStateException: Unable to provideCDOID: org.eclipse.emf.internal.cdo.revision.CDOElementProxyImpl
> at org.eclipse.emf.internal.cdo.view.AbstractCDOView.provideCDO ID(AbstractCDOView.java:858)
> at org.eclipse.emf.internal.cdo.transaction.CDOTransactionImpl. provideCDOID(CDOTransactionImpl.java:1985)
> at org.eclipse.emf.cdo.internal.common.revision.delta.CDOSingle ValueFeatureDeltaImpl.writeValue(CDOSingleValueFeatureDeltaI mpl.java:86)
> at org.eclipse.emf.cdo.internal.common.revision.delta.CDOSingle ValueFeatureDeltaImpl.write(CDOSingleValueFeatureDeltaImpl.j ava:62)
> at org.eclipse.emf.cdo.internal.common.protocol.CDODataOutputIm pl.writeCDOFeatureDelta(CDODataOutputImpl.java:379)
> at org.eclipse.emf.cdo.internal.common.revision.delta.CDOListFe atureDeltaImpl.write(CDOListFeatureDeltaImpl.java:136)
> at org.eclipse.emf.cdo.internal.common.protocol.CDODataOutputIm pl.writeCDOFeatureDelta(CDODataOutputImpl.java:379)
> at org.eclipse.emf.cdo.internal.common.revision.delta.CDORevisi onDeltaImpl.write(CDORevisionDeltaImpl.java:158)
> at org.eclipse.emf.cdo.internal.common.protocol.CDODataOutputIm pl.writeCDORevisionDelta(CDODataOutputImpl.java:374)
> at org.eclipse.emf.cdo.internal.net4j.protocol.CommitTransactio nRequest.requestingCommit(CommitTransactionRequest.java:161)
> at org.eclipse.emf.cdo.internal.net4j.protocol.CommitTransactio nRequest.requesting(CommitTransactionRequest.java:112)
> at org.eclipse.emf.cdo.internal.net4j.protocol.CDOClientRequest WithMonitoring.requesting(CDOClientRequestWithMonitoring.jav a:92)
> at org.eclipse.net4j.signal.RequestWithMonitoring.requesting(Re questWithMonitoring.java:163)
> at org.eclipse.net4j.signal.RequestWithConfirmation.doExtendedO utput(RequestWithConfirmation.java:117)
> at org.eclipse.net4j.signal.Signal.doOutput(Signal.java:296)
> at org.eclipse.net4j.signal.RequestWithConfirmation.doExecute(R equestWithConfirmation.java:102)
> at org.eclipse.net4j.signal.RequestWithMonitoring.doExecute(Req uestWithMonitoring.java:233)
> at org.eclipse.net4j.signal.SignalActor.execute(SignalActor.jav a:51)
> at org.eclipse.net4j.signal.Signal.runSync(Signal.java:251)
> at org.eclipse.net4j.signal.SignalProtocol.startSignal(SignalPr otocol.java:396)
> at org.eclipse.net4j.signal.RequestWithConfirmation.doSend(Requ estWithConfirmation.java:87)
> at org.eclipse.net4j.signal.RequestWithConfirmation.send(Reques tWithConfirmation.java:73)
> at org.eclipse.net4j.signal.RequestWithMonitoring.send(RequestW ithMonitoring.java:108)
> at org.eclipse.emf.cdo.internal.net4j.protocol.CDOClientProtoco l.send(CDOClientProtocol.java:413)
> at org.eclipse.emf.cdo.internal.net4j.protocol.CDOClientProtoco l.commitTransaction(CDOClientProtocol.java:295)
> at org.eclipse.emf.internal.cdo.transaction.CDOSingleTransactio nStrategyImpl.commit(CDOSingleTransactionStrategyImpl.java:8 0)
> at org.eclipse.emf.internal.cdo.transaction.CDOTransactionImpl. commit(CDOTransactionImpl.java:1027)
> ... 26 more
>
>
> Is it perhaps a bug in CDO? Or is there any option we can set so that change sets do not contain the unwanted CDOElementProxy values?
I suspect that it's a bug. Please file a bugzilla. Unfortunately I don't have much time until the EclipseCon. If you dare to track down the issue and attach a patch, I promise to review it quickly ;-)

Cheers
/Eike


Re: [CDO] CDOChangeSetData with CDOCollectionLoadingPolicy set [message #654031 is a reply to message #653924] Sun, 13 February 2011 09:51 Go to previous message
Szabolcs Bardy is currently offline Szabolcs BardyFriend
Messages: 9
Registered: July 2009
Junior Member
Hi Eike,

thanks for your answer!

I have submitted a new bug report (https://bugs.eclipse.org/bugs/show_bug.cgi?id=337054), and will dig into the issue.

Regards,
Szabolcs
Previous Topic:CDOSetFeatureDelta.getOldValue is always UNSPECIFIED
Next Topic:EClass loading parameters dynamically from an XML file
Goto Forum:
  


Current Time: Wed Nov 13 20:20:27 GMT 2019

Powered by FUDForum. Page generated in 0.02173 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top