Eclipse Community Forums: EMF » Sanity check: algorithm for thread safe commits in stand-alone applications

Home » Modeling » EMF » Sanity check: algorithm for thread safe commits in stand-alone applications(Transactions without org.eclipse.emf.transaction)

Show: Today's Messages :: Show Polls :: Message Navigator

Sanity check: algorithm for thread safe commits in stand-alone applications [message #693625]

Wed, 06 July 2011 22:35

Kenneth Gulbrandsoy

Messages: 4
Registered: June 2011
Location: Norway

Junior Member

Hi EMF community!

EMF Transaction is only available on the Eclipse platform (see message #68148).

This is too restrictive for my project named EFeature, which is to add spatial support to EMF using Geotools (a general purpose library for spatial data).

A alternative to EMF Transaction is to combine GeoTools (GT) feature locking capabilities with a EMF ChangeRecorder. This however, will only be thread safe if all access to locked EObjects are performed from GT transaction aware client code. Hence, locked EObjects are still accessible (read and write) by EMF dependent client code not aware of (or don't care about) the GT transaction. As long as EFeature client code is aware of these assumptions, thread safe transactions should be feasible.

The algorithm could be something like this:

1) On begin:

- lock all features to prevent access from other transactions
- create a temporary transaction resource
- add this resource to a ChangeRecorder

2) On write:

- add EObject to the transaction resource
(removes EObject from it's current resource)
- cache removed resource in a "EObject-TO-Reference" map

*** Every change is recorded by the ChangeRecorder ***

4) On Commit:

- apply and revert changes to transaction resource
(this produces a forward delta ChangeDescription)
- add all changed EObjects to old resource
- apply forward delta

My question is: does this approach honor the "EMF way"? Are there any other considerations to take into account?

I am going to add EMF transaction support to EFeature eventually, probably when EFeature is added to uDIG, a GIS framework for Eclipse.

Best regards,
Kenneth Gulbrandsøy
=====================
Geotools EFeature module maintainer
- Norway

Report message to a moderator

Re: Sanity check: algorithm for thread safe commits in stand-alone applications [message #693677 is a reply to message #693625]

Thu, 07 July 2011 01:22

Ed Merks

Messages: 33133
Registered: July 2009

Senior Member

Kenneth,

Comments below.

On 06/07/2011 3:35 PM, Kenneth wrote:
> Hi EMF community!
>
> EMF Transaction is only available on the Eclipse platform (see message
> #68148).
> This is too restrictive for my project named EFeature, which is to add
> spatial support to EMF using Geotools (a general purpose library for
> spatial data).
>
> A alternative to EMF Transaction is to combine GeoTools (GT) feature
> locking capabilities with a EMF ChangeRecorder. This however, will
> only be thread safe if all access to locked EObjects are performed
> from GT transaction aware client code. Hence, locked EObjects are
> still accessible (read and write) by EMF dependent client code not
> aware of (or don't care about) the GT transaction. As long as EFeature
> client code is aware of these assumptions, thread safe transactions
> should be feasible.
Can't you just have a simple high level policy for your threads, i.e., a
single lock that must be held to do anything with EFeature data...
>
> The algorithm could be something like this:
>
> 1) On begin:
> - lock all features to prevent access from other
> transactions - create a temporary transaction resource
> - add this resource to a ChangeRecorder
> 2) On write: - add EObject to the
> transaction resource (removes EObject from it's current
> resource)
This removal thing sounds scary.
> - cache removed resource in a "EObject-TO-Reference" map
> *** Every change is recorded by the ChangeRecorder ***
> 4) On Commit:
>
> - apply and revert changes to transaction resource
> (this produces a forward delta ChangeDescription)
> - add all changed EObjects to old resource
> - apply forward delta
If you want a forward delta, there's new support for that:
http://ed-merks.blogspot.com/2011/06/so-much-to-do-so-little-time.html
>
> My question is: does this approach honor the "EMF way"? Are there any
> other considerations to take into account?
I wonder if CDO wouldn't be a good fit...
> I am going to add EMF transaction support to EFeature eventually,
> probably when EFeature is added to uDIG, a GIS framework for Eclipse.
>
> Best regards,
> Kenneth Gulbrandsøy
> =====================
> Geotools EFeature module maintainer
> - Norway

Ed Merks
Professional Support: https://www.macromodeling.com/

Report message to a moderator

Re: Sanity check: algorithm for thread safe commits in stand-alone applications [message #693734 is a reply to message #693677]

Thu, 07 July 2011 06:07

Kenneth Gulbrandsoy

Messages: 4
Registered: June 2011
Location: Norway

Junior Member

Thanks Ed.

Your input is very much appreciated Razz

I have some follow-up questions which you find below.

Quote:

> A alternative to EMF Transaction is to combine GeoTools (GT) feature
> locking capabilities with a EMF ChangeRecorder. This however, will
> only be thread safe if all access to locked EObjects are performed
> from GT transaction aware client code. Hence, locked EObjects are
> still accessible (read and write) by EMF dependent client code not
> aware of (or don't care about) the GT transaction. As long as EFeature
> client code is aware of these assumptions, thread safe transactions
> should be feasible.
Can't you just have a simple high level policy for your threads, i.e., a
single lock that must be held to do anything with EFeature data...

That is a simple solution, but does not fit well with Geotools' transaction and locking patterns. The goal is to make EFeature a first class citizen of Geotools.

Quote:

>
> The algorithm could be something like this:
>
> 1) On begin:
> - lock all features to prevent access from other
> transactions - create a temporary transaction resource
> - add this resource to a ChangeRecorder
> 2) On write: - add EObject to the
> transaction resource (removes EObject from it's current
> resource)
This removal thing sounds scary.

Just curious. What makes this scary? I didn't mention this earlier, but EFeature is only working on EAttribute values. Another solution could therefore be to do a ECoreUtil.copy() on write, adding it to the transaction resource, then do a ECoreUtil.replace() on commit. This will yield the same result but with a higher memory footprint.

Quote:

> 4) On Commit:
>
> - apply and revert changes to transaction resource
> (this produces a forward delta ChangeDescription)
> - add all changed EObjects to old resource
> - apply forward delta
If you want a forward delta, there's new support for that: <url>

I have noticed the forward delta capabilites in 2.7. However, my goal is to support as much legacy EMF models as possible. Restricting EFeature to to 2.7 might make this goal harder to achieve because of classpath conflicts in stand-alone applications. What do you think?

Quote:

> My question is: does this approach honor the "EMF way"? Are there any
> other considerations to take into account?
I wonder if CDO wouldn't be a good fit...

CDO is definitly on my radar already. However, expecting all EFeature clients to use CDO is a bit restrictive also. I think I have to leave the choice of persistence framework to the EFeature user.

Cheers,
Kenneth

Report message to a moderator

Re: Sanity check: algorithm for thread safe commits in stand-alone applications [message #693880 is a reply to message #693625]

Thu, 07 July 2011 12:06

Eike Stepper

Messages: 6682
Registered: July 2009

Senior Member

Hi Kenneth,

I think the topic of concurrent model access is hot, so I've just written an article about it. Maybe you find it interesting: http://thegordian.blogspot.com/2011/07/concurrent-access-to-models.html

Cheers
/Eike

----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper

Am 07.07.2011 00:35, schrieb Kenneth:
> Hi EMF community!
>
> EMF Transaction is only available on the Eclipse platform (see message #68148).
> This is too restrictive for my project named EFeature, which is to add spatial support to EMF using Geotools (a general purpose library for spatial data).
>
> A alternative to EMF Transaction is to combine GeoTools (GT) feature locking capabilities with a EMF ChangeRecorder. This however, will only be thread safe if all access to locked EObjects are performed from GT transaction aware client code. Hence, locked EObjects are still accessible (read and write) by EMF dependent client code not aware of (or don't care about) the GT transaction. As long as EFeature client code is aware of these assumptions, thread safe transactions should be feasible.
>
> The algorithm could be something like this:
>
> 1) On begin:
> - lock all features to prevent access from other transactions - create a temporary transaction resource
> - add this resource to a ChangeRecorder
> 2) On write: - add EObject to the transaction resource (removes EObject from it's current resource)
> - cache removed resource in a "EObject-TO-Reference" map
> *** Every change is recorded by the ChangeRecorder ***
> 4) On Commit:
>
> - apply and revert changes to transaction resource
> (this produces a forward delta ChangeDescription)
> - add all changed EObjects to old resource
> - apply forward delta
>
> My question is: does this approach honor the "EMF way"? Are there any other considerations to take into account?
> I am going to add EMF transaction support to EFeature eventually, probably when EFeature is added to uDIG, a GIS framework for Eclipse.
>
> Best regards,
> Kenneth Gulbrandsøy
> =====================
> Geotools EFeature module maintainer
> - Norway

Cheers
/Eike

----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper

Report message to a moderator

Re: Sanity check: algorithm for thread safe commits in stand-alone applications [message #693900 is a reply to message #693880]

Thu, 07 July 2011 13:12

Kenneth Gulbrandsoy

Messages: 4
Registered: June 2011
Location: Norway

Junior Member

Thanks Eike, this was a very clear article on how concurrent access can be done using CDO!

As I said, CDO is one solution that solves my problem. The challenge for me however, is that I also want to support legacy EMF models which can not be regenereded for CDO. As of now, EFeature support 1) dynamic (at runtime) mapping from any EObject containing geometry information (limited to JTS Simple Feature specification) to EFeature by mapping EAttributes to EFeature, and 2) extension of the EFeature model (like the ExtendedLibrary example).

I believe CDO is a obvious choice when working on large models, which spatial models often are (10 000-100 000 objects are typical numbers). However, I still want to supply some support for models which can not be persisted with CDO. I have to admitt that I do not know how big a problem this really is (number of models that can not use CDO, or client code that can not change the persistence framework to CDO) though.

Eike Stepper wrote on Thu, 07 July 2011 14:06

Hi Kenneth,

I think the topic of concurrent model access is hot, so I've just written an article about it.

Report message to a moderator

Re: Sanity check: algorithm for thread safe commits in stand-alone applications [message #694056 is a reply to message #693734]

Thu, 07 July 2011 17:36

Ed Merks

Messages: 33133
Registered: July 2009

Senior Member

Kenneth,

Comments below.

On 06/07/2011 11:07 PM, Kenneth wrote:
> Thanks Ed.
> Your input is very much appreciated :p
> I have some follow-up questions which you find below.
>
>
> Quote:
>> > A alternative to EMF Transaction is to combine GeoTools (GT)
>> feature > locking capabilities with a EMF ChangeRecorder. This
>> however, will > only be thread safe if all access to locked EObjects
>> are performed > from GT transaction aware client code. Hence, locked
>> EObjects are > still accessible (read and write) by EMF dependent
>> client code not > aware of (or don't care about) the GT transaction.
>> As long as EFeature > client code is aware of these assumptions,
>> thread safe transactions > should be feasible.
>> Can't you just have a simple high level policy for your threads,
>> i.e., a single lock that must be held to do anything with EFeature
>> data...
>
> That is a simple solution, but does not fit well with Geotools'
> transaction and locking patterns. The goal is to make EFeature a first
> class citizen of Geotools.
I see. I know nothing about those details though...
>
> Quote:
>> >
>> > The algorithm could be something like this:
>> >
>> > 1) On begin:
>> > - lock all features to prevent access from other > transactions -
>> create a temporary transaction resource
>> > - add this resource to a ChangeRecorder
>> > 2) On write: - add EObject to the > transaction resource (removes
>> EObject from it's current > resource)
>> This removal thing sounds scary.
>
> Just curious. What makes this scary?
Because that removal too is a modification that could be recorded.
> I didn't mention this earlier, but EFeature is only working on
> EAttribute values. Another solution could therefore be to do a
> ECoreUtil.copy() on write, adding it to the transaction resource, then
> do a ECoreUtil.replace() on commit. This will yield the same result
> but with a higher memory footprint.
> Quote:
>> > 4) On Commit:
>> >
>> > - apply and revert changes to transaction resource
>> > (this produces a forward delta ChangeDescription)
>> > - add all changed EObjects to old resource
>> > - apply forward delta
>> If you want a forward delta, there's new support for that: <url>
>
> I have noticed the forward delta capabilites in 2.7. However, my goal
> is to support as much legacy EMF models as possible. Restricting
> EFeature to to 2.7 might make this goal harder to achieve because of
> classpath conflicts in stand-alone applications. What do you think?
2.7 supports all versions that have come before so I'm not sure there
would be any barrier created by using it.
>
> Quote:
>> > My question is: does this approach honor the "EMF way"? Are there
>> any > other considerations to take into account?
>> I wonder if CDO wouldn't be a good fit...
>
> CDO is definitly on my radar already. However, expecting all EFeature
> clients to use CDO is a bit restrictive also. I think I have to leave
> the choice of persistence framework to the EFeature user.
>
> Cheers,
> Kenneth

Ed Merks
Professional Support: https://www.macromodeling.com/

Report message to a moderator

Re: Sanity check: algorithm for thread safe commits in stand-alone applications [message #694272 is a reply to message #694056]

Fri, 08 July 2011 08:42

Kenneth Gulbrandsoy

Messages: 4
Registered: June 2011
Location: Norway

Junior Member

Thanks Ed.

I think I have sufficient feedback to make a decision.

Cheers,
Kenneth

Report message to a moderator

Re: Sanity check: algorithm for thread safe commits in stand-alone applications [message #694319 is a reply to message #693880]

Fri, 08 July 2011 10:30

ModelGeek Mising name

Messages: 550
Registered: June 2011

Senior Member

Hi,

I am trying to follow your article but i am unable to find org.eclipse.emf.cdo.transaction package. How can i get that package?

Regards,

Report message to a moderator

Re: Sanity check: algorithm for thread safe commits in stand-alone applications [message #694329 is a reply to message #693880]

Fri, 08 July 2011 10:52

ModelGeek Mising name

Messages: 550
Registered: June 2011

Senior Member

Hi,

I am following your article but i am unable to fing Util Class... Can you please tell which package it belongs to?

Warmest Regards,

Report message to a moderator

Re: Sanity check: algorithm for thread safe commits in stand-alone applications [message #694331 is a reply to message #694319]

Fri, 08 July 2011 10:48

Eike Stepper

Messages: 6682
Registered: July 2009

Senior Member

Am 08.07.2011 12:30, schrieb forums-noreply@eclipse.org:
> Hi,
>
> I am trying to follow your article but i am unable to find org.eclipse.emf.cdo.transaction package. How can i get that package?
This *package* is in the org.eclipse.emf.cdo *plugin* (jar). Here's the package's contents: http://dev.eclipse.org/svnroot/modeling/org.eclipse.emf.cdo/trunk/plugins/org.eclipse.emf.cdo/src/org/eclipse/emf/cdo/transaction/

Cheers
/Eike

----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper

Cheers
/Eike

----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper

Report message to a moderator

Re: Sanity check: algorithm for thread safe commits in stand-alone applications [message #694339 is a reply to message #694331]

Fri, 08 July 2011 11:25

ModelGeek Mising name

Messages: 550
Registered: June 2011

Senior Member

Hi,

i have got the package but i am still confused about Util.openSession(). Please guide me about Util.openSession().

Regards,

Report message to a moderator

Re: Sanity check: algorithm for thread safe commits in stand-alone applications [message #694346 is a reply to message #694329]

Fri, 08 July 2011 10:57

Eike Stepper

Messages: 6682
Registered: July 2009

Senior Member

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html; charset=UTF-8" http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
Am 08.07.2011 12:52, schrieb <a class="moz-txt-link-abbreviated" href="mailto:forums-noreply@eclipse.org:">forums-noreply@eclipse.org:</a>
<blockquote cite="mid:iv6n00$n39$1@news.eclipse.org" type="cite">Hi,
 
 
I am following your article but i am unable to fing Util Class...
Can you please tell which package it belongs to?
 
</blockquote>
Sorry, that guy is a fake to keep the networking details out of the
article. Here's an example for embedded transport: 
 
<title></title>
<style type="text/css">

</style>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">




<div class="java" align="left">
<table bgcolor="#ffffff" border="0" cellpadding="3"
cellspacing="0">
<tbody>
<tr>
 <td nowrap="nowrap" align="left"
valign="top"> <code>
 acceptor = Net4jUtil.getAcceptor(IPluginContainer.INSTANCE, "jvm", repositoryName); 
 connector = Net4jUtil.getConnector(IPluginContainer.INSTANCE, "jvm", repositoryName); 
 
 CDONet4jSessionConfiguration config = CDONet4jUtil.createNet4jSessionConfiguration(); 
 config.setConnector(connector); 
 config.setRepositoryName(repositoryName); 
 
 CDONet4jSession session = config.openNet4jSession();</code>
</td>
 </tr>
</tbody>
</table>
</div>


 
This could would open a session to a remote repository: 
 
<title></title>
<style type="text/css">

</style>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">




<div class="java" align="left">
<table bgcolor="#ffffff" border="0" cellpadding="3"
cellspacing="0">
<tbody>
<tr>
 <td nowrap="nowrap" align="left"
valign="top"> <code>
 connector = Net4jUtil.getConnector(IPluginContainer.INSTANCE, "tcp", "localhost"); 
 
 CDONet4jSessionConfiguration config = CDONet4jUtil.createNet4jSessionConfiguration(); 
 config.setConnector(connector); 
 config.setRepositoryName(repositoryName); 
 
 CDONet4jSession session = config.openNet4jSession();</code>
</td>
 </tr>
</tbody>
</table>
</div>


 
Does that help? 
 
Cheers 
/Eike 
 
---- 
<a class="moz-txt-link-freetext" href="http://www.esc-net.de">http://www.esc-net.de</a> 
<a class="moz-txt-link-freetext" href="http://thegordian.blogspot.com">http://thegordian.blogspot.com</a> 
<a class="moz-txt-link-freetext" href="http://twitter.com/eikestepper">http://twitter.com/eikestepper</a> 
 
 
</body>
</html>

Cheers
/Eike

----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper

Report message to a moderator

Re: Sanity check: algorithm for thread safe commits in stand-alone applications [message #694355 is a reply to message #694346]

Fri, 08 July 2011 12:12

ModelGeek Mising name

Messages: 550
Registered: June 2011

Senior Member

Hi,

Actually i donot want to access remote repository but i want to create and evaluate 1000 concrete model concurrently in standalone application.

or i can also create 1 concrete model and perform 1000 mutations concurrently... is CDOTransaction helpful for me?

Please guide.

Warmest Regards,

Report message to a moderator

Re: Sanity check: algorithm for thread safe commits in stand-alone applications [message #694422 is a reply to message #694355]

Fri, 08 July 2011 13:59

Eike Stepper

Messages: 6682
Registered: July 2009

Senior Member

Am 08.07.2011 14:12, schrieb forums-noreply@eclipse.org:
> Hi,
>
> Actually i donot want to access remote repository but i want to create and evaluate 1000 concrete model concurrently in standalone application.
>
> or i can also create 1 concrete model and perform 1000 mutations concurrently... is CDOTransaction helpful for me?
If this discussion gets longer we should probably not be hijacking this thread that was originally about something different ;-)

With CDO your model always "lives" in a repository.
This repository doesn't have to be remote, it can be embedded into the client (see my first example about embedded transport in the previous post).
And it doesn't have to be persistent, it can also be setup with a MEMStore. In this case you may also want to switch off revision caching in the client and the repository.

Cheers
/Eike

----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper

Cheers
/Eike

----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper

Report message to a moderator

Previous Topic:	[CDO] Auto-Refresh broken in 4.0
Next Topic:	Link between 2 dynamic ecore

Goto Forum:

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

]

Current Time: Tue Apr 16 04:17:53 GMT 2024

.:: Contact :: Home ::.

Breadcrumbs

Sign up to our Newsletter