Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » EMF » [CDO] Adding different models to the same ResourceNode concurrently
[CDO] Adding different models to the same ResourceNode concurrently [message #504777] Tue, 22 December 2009 11:00 Go to next message
Oliver Wong is currently offline Oliver WongFriend
Messages: 47
Registered: July 2009
Member
Hi,

I'm using CDO to save my model instances concurrently on a certain path. Each model instance is created in a separate Job and saved onto CDO within its own CDO transaction by using a path like /myModels/fileName.

The jobs create the transaction and commits concurrently.
This is causing a ConcurrentModificationException : " java.util.ConcurrentModificationException: Trying to update object OID741 that was already modified".

In order to avoid the concurrent modification, I used a lock in the ResourceNode "/stm" in the example code provided, which would be /myModels according to the explanation.

Using the locks causes recurring timeouts. Is this a good way to use locks and is there any pattern to try and prevent or reduce timeouts ?

Any help on how to deal with this issue would be greatly appreciated.

Thanks,

Cheers,
public class PureCDOConcurrentModification {
	@Test
	public void runTest() {

		int counter = 0;
		ExecutorService threadPool = Executors.newFixedThreadPool(100);

		final IProgressMonitor group = Job.getJobManager().createProgressGroup();
		group.beginTask("Commiting", 100);
		while (counter < 100) {
			Segment seg = buildFakeModel(counter);
			CommitJob commitJob = new CommitJob("suntrust" + counter, seg);
			threadPool.execute(commitJob);
			counter++;
		}
		waitForThreads(threadPool);

	}

	public Segment buildFakeModel(int index) {
		Segment root = KdmFactory.eINSTANCE.createSegment();
		root.setName("Root" + index);
		for (int i = 0; i < 100; i++) {
			Segment child = KdmFactory.eINSTANCE.createSegment();
			child.setName("Segment_" + index + "_" + i);
			root.getSegment().add(child);
		}
		return root;
	}

	private void waitForThreads(ExecutorService threadPool) {
		try {
			threadPool.shutdown();
			threadPool.awaitTermination(1, TimeUnit.HOURS);
		}
		catch (final InterruptedException e) {
			e.printStackTrace();
		}
	}

	class CommitJob implements Runnable {
		final private Segment seg;
		final private String name;

		public CommitJob(String name, Segment seg) {
			this.name = name;
			this.seg = seg;
		}

		@Override
		public void run() {
			final CDOConnector cdoConnector = CDOConnectorFactory.getDefault().getCDOConnector();
			CDOTransaction trans = cdoConnector.openTransaction();
			trans.options().setAutoReleaseLocksEnabled(true);
			CDOResourceNode rezNode = trans.getResourceNode("/stm");
			rezNode.cdoWriteLock().lock();
			CDOResource rez = trans.getOrCreateResource("/stm/" + name);
			rez.getContents().add(seg);
			try {
				trans.commit();
				System.out.print("Commited " + name);
			}
			catch (Exception e) {
				// TODO Auto-generated catch block
				e.printStackTrace();
			}
			trans.close();
		}
	}

}
Re: [CDO] Adding different models to the same ResourceNode concurrently [message #504792 is a reply to message #504777] Tue, 22 December 2009 12:20 Go to previous messageGo to next message
Eike Stepper is currently offline Eike StepperFriend
Messages: 6682
Registered: July 2009
Senior Member
Hi Oliver,

Which CDO version are you using? A race condition in the protocol was
recently fixed that could cause CMEs.

Can you paste in the stack trace of the timeouts while trying to acquire
a lock?

If I understand your code snippet correctly you're using CDO server-side
synchronization (write locks) to protect a whole CommitJob sequence. As
a consequence your threads will execute sequentially. Wouldn't it be
easier, faster and more straight forward to just execute them in the
main thread? If you want to increase concurrency and you know that no
concurrent write access will happen except to your /stm resource folder
node, you should consider to commit twice per CommitJob. Once after
adding just the root segment (with a lock on /stm), and once for the
rest of the tree (without any locks). That might increase the chance
that no timeout on signal level occurs.

That said, if your timeout stack trace shows that the timeout happens on
signal level, we should consider to solve this issue in CDO. Either by
making the signal timeout for LockObjectsRequest configurable or by
adding an IProgressMonitor similar to long running commits.

Cheers
/Eike

----
http://thegordian.blogspot.com
http://twitter.com/eikestepper



Oliver Wong schrieb:
> Hi,
>
> I'm using CDO to save my model instances concurrently on a certain
> path. Each model instance is created in a separate Job and saved onto
> CDO within its own CDO transaction by using a path like
> /myModels/fileName.
>
> The jobs create the transaction and commits concurrently.
> This is causing a ConcurrentModificationException : "
> java.util.ConcurrentModificationException: Trying to update object
> OID741 that was already modified".
>
> In order to avoid the concurrent modification, I used a lock in the
> ResourceNode "/stm" in the example code provided, which would be
> /myModels according to the explanation.
>
> Using the locks causes recurring timeouts. Is this a good way to use
> locks and is there any pattern to try and prevent or reduce timeouts ?
>
> Any help on how to deal with this issue would be greatly appreciated.
> Thanks,
>
> Cheers,
>
> public class PureCDOConcurrentModification {
> @Test
> public void runTest() {
>
> int counter = 0;
> ExecutorService threadPool = Executors.newFixedThreadPool(100);
>
> final IProgressMonitor group =
> Job.getJobManager().createProgressGroup();
> group.beginTask("Commiting", 100);
> while (counter < 100) {
> Segment seg = buildFakeModel(counter);
> CommitJob commitJob = new CommitJob("suntrust" + counter,
> seg);
> threadPool.execute(commitJob);
> counter++;
> }
> waitForThreads(threadPool);
>
> }
>
> public Segment buildFakeModel(int index) {
> Segment root = KdmFactory.eINSTANCE.createSegment();
> root.setName("Root" + index);
> for (int i = 0; i < 100; i++) {
> Segment child = KdmFactory.eINSTANCE.createSegment();
> child.setName("Segment_" + index + "_" + i);
> root.getSegment().add(child);
> }
> return root;
> }
>
> private void waitForThreads(ExecutorService threadPool) {
> try {
> threadPool.shutdown();
> threadPool.awaitTermination(1, TimeUnit.HOURS);
> }
> catch (final InterruptedException e) {
> e.printStackTrace();
> }
> }
>
> class CommitJob implements Runnable {
> final private Segment seg;
> final private String name;
>
> public CommitJob(String name, Segment seg) {
> this.name = name;
> this.seg = seg;
> }
>
> @Override
> public void run() {
> final CDOConnector cdoConnector =
> CDOConnectorFactory.getDefault().getCDOConnector();
> CDOTransaction trans = cdoConnector.openTransaction();
> trans.options().setAutoReleaseLocksEnabled(true);
> CDOResourceNode rezNode = trans.getResourceNode("/stm");
> rezNode.cdoWriteLock().lock();
> CDOResource rez = trans.getOrCreateResource("/stm/" + name);
> rez.getContents().add(seg);
> try {
> trans.commit();
> System.out.print("Commited " + name);
> }
> catch (Exception e) {
> // TODO Auto-generated catch block
> e.printStackTrace();
> }
> trans.close();
> }
> }
>
> }
>


Re: [CDO] Adding different models to the same ResourceNode concurrently [message #504922 is a reply to message #504792] Wed, 23 December 2009 09:56 Go to previous messageGo to next message
kaab  is currently offline kaab Friend
Messages: 43
Registered: July 2009
Member
Hi Eike,

I had big trouble posting on the newgroup yesterday so I asked Oliver to post this one.
It seems that my messages were simply ignored, did I do something wrong ? I love eclipse, please don't ban me Smile

You are right about the sequential execution resulting from this lock, I was trying out cdo locks for the first time and was not really cautious when setting the test case.
I think that there's one thing which is not stated here, that was really reducing possiblilities. Usually we open transactions trough an internal API and don't expose CDOTransaction, our API expose only one and unique way to open a transaction, ie : openXXXModelTransaction(final Path path) with the path beeing the cdo path minus the first segment that is appended automatically.
I think that the best way to go is to extend that api and do 2 commit per job as you suggested.
The timeout is not at signal level and was caused by the other commits waiting too long for the lock.

Thanks for the quick and useful reply,
Re: [CDO] Adding different models to the same ResourceNode concurrently [message #504951 is a reply to message #504922] Wed, 23 December 2009 13:40 Go to previous messageGo to next message
Eike Stepper is currently offline Eike StepperFriend
Messages: 6682
Registered: July 2009
Senior Member
Hi Kaab,

Comments below...


kaab schrieb:
> Hi Eike,
>
> I had big trouble posting on the newgroup yesterday so I asked Oliver
> to post this one. It seems that my messages were simply ignored, did I
> do something wrong ? I love eclipse, please don't ban me :)
While it can be possible that I overlook messages inrare cases I'm not
aware of any recent ones. Which one exactly are you referring to?

>
> You are right about the sequential execution resulting from this lock,
> I was trying out cdo locks for the first time and was not really
> cautious when setting the test case.
> I think that there's one thing which is not stated here, that was
> really reducing possiblilities. Usually we open transactions trough an
> internal API and don't expose CDOTransaction
Ok, in your test case you seem to hide CDOSession behind the
CDOConnector facade.

> , our API expose only one and unique way to open a transaction, ie :
> openXXXModelTransaction(final Path path) with the path beeing the cdo
> path minus the first segment that is appended automatically. I think
> that the best way to go is to extend that api and do 2 commit per job
> as you suggested.
Not necessarily API. The implementation of that method could detach the
root object from the transient resource, i.e. before committing the
first time. After the first commit you re-attach the root object and
commit a second time.

To further reduce lock contention for your /stm resource you could add
additional folder levels between /stm and the resource of your root objects.

> The timeout is not at signal level and was caused by the other commits
> waiting too long for the lock.
Then you can use the java.util.concurrent.locks.Lock.tryLock(long,
TimeUnit) method to set a timeout of your choice.

>
> Thanks for the quick and useful reply,
You're welcome ;-)

Cheers
/Eike

----
http://thegordian.blogspot.com
http://twitter.com/eikestepper


Re: [CDO] Adding different models to the same ResourceNode concurrently [message #504964 is a reply to message #504922] Wed, 23 December 2009 20:12 Go to previous messageGo to next message
Joel Rosi-Schwartz is currently offline Joel Rosi-SchwartzFriend
Messages: 624
Registered: July 2009
Location: London. England
Senior Member
Hi Kaab,

If I understand you correctly you are saying that sometimes your messages simply do not get posted to the forum. Is that correct? I also have that problem frequently. I now copy my message/reply before trying to post it, this way if/when it does not get accepted, I simply try again and paste the message in. Saves me *a lot* of retyping Wink

hth,
Joel


kaab wrote on Wed, 23 December 2009 09:56
Hi Eike,

I had big trouble posting on the newgroup yesterday so I asked Oliver to post this one.
It seems that my messages were simply ignored, did I do something wrong ? I love eclipse, please don't ban me Smile


Re: [CDO] Adding different models to the same ResourceNode concurrently [message #504995 is a reply to message #504964] Thu, 24 December 2009 03:34 Go to previous message
Eike Stepper is currently offline Eike StepperFriend
Messages: 6682
Registered: July 2009
Senior Member
Joel Rosi-Schwartz schrieb:
> Hi Kaab,
>
> If I understand you correctly you are saying that sometimes your
> messages simply do not get posted to the forum. Is that correct?
No. I wanted to say that in very rare cases I overlook a question. If I
send an answer I always see it ending up in the forum.

> I also have that problem frequently. I now copy my message/reply
> before trying to post it, this way if/when it does not get accepted, I
> simply try again and paste the message in. Saves me *a lot* of
> retyping ;)
That sounds odd, but IIRC I never experienced that.

Cheers
/Eike

----
http://thegordian.blogspot.com
http://twitter.com/eikestepper


>
> hth,
> Joel
>
>
> kaab wrote on Wed, 23 December 2009 09:56
>> Hi Eike,
>>
>> I had big trouble posting on the newgroup yesterday so I asked Oliver
>> to post this one. It seems that my messages were simply ignored, did
>> I do something wrong ? I love eclipse, please don't ban me :)
>
>


Previous Topic:persisting unchangeable attributes
Next Topic:[CDO] Problem setting up target platform
Goto Forum:
  


Current Time: Fri Apr 19 13:36:00 GMT 2024

Powered by FUDForum. Page generated in 0.02044 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top