Home » Modeling » EMF » Sanity check: algorithm for thread safe commits in stand-alone applications(Transactions without org.eclipse.emf.transaction)
Sanity check: algorithm for thread safe commits in stand-alone applications [message #693625] |
Wed, 06 July 2011 22:35 |
|
Hi EMF community!
EMF Transaction is only available on the Eclipse platform (see message #68148).
This is too restrictive for my project named EFeature, which is to add spatial support to EMF using Geotools (a general purpose library for spatial data).
A alternative to EMF Transaction is to combine GeoTools (GT) feature locking capabilities with a EMF ChangeRecorder. This however, will only be thread safe if all access to locked EObjects are performed from GT transaction aware client code. Hence, locked EObjects are still accessible (read and write) by EMF dependent client code not aware of (or don't care about) the GT transaction. As long as EFeature client code is aware of these assumptions, thread safe transactions should be feasible.
The algorithm could be something like this:
1) On begin:
- lock all features to prevent access from other transactions
- create a temporary transaction resource
- add this resource to a ChangeRecorder
2) On write:
- add EObject to the transaction resource
(removes EObject from it's current resource)
- cache removed resource in a "EObject-TO-Reference" map
*** Every change is recorded by the ChangeRecorder ***
4) On Commit:
- apply and revert changes to transaction resource
(this produces a forward delta ChangeDescription)
- add all changed EObjects to old resource
- apply forward delta
My question is: does this approach honor the "EMF way"? Are there any other considerations to take into account?
I am going to add EMF transaction support to EFeature eventually, probably when EFeature is added to uDIG, a GIS framework for Eclipse.
Best regards,
Kenneth Gulbrandsøy
=====================
Geotools EFeature module maintainer
- Norway
|
|
|
Re: Sanity check: algorithm for thread safe commits in stand-alone applications [message #693677 is a reply to message #693625] |
Thu, 07 July 2011 01:22 |
Ed Merks Messages: 33148 Registered: July 2009 |
Senior Member |
|
|
Kenneth,
Comments below.
On 06/07/2011 3:35 PM, Kenneth wrote:
> Hi EMF community!
>
> EMF Transaction is only available on the Eclipse platform (see message
> #68148).
> This is too restrictive for my project named EFeature, which is to add
> spatial support to EMF using Geotools (a general purpose library for
> spatial data).
>
> A alternative to EMF Transaction is to combine GeoTools (GT) feature
> locking capabilities with a EMF ChangeRecorder. This however, will
> only be thread safe if all access to locked EObjects are performed
> from GT transaction aware client code. Hence, locked EObjects are
> still accessible (read and write) by EMF dependent client code not
> aware of (or don't care about) the GT transaction. As long as EFeature
> client code is aware of these assumptions, thread safe transactions
> should be feasible.
Can't you just have a simple high level policy for your threads, i.e., a
single lock that must be held to do anything with EFeature data...
>
> The algorithm could be something like this:
>
> 1) On begin:
> - lock all features to prevent access from other
> transactions - create a temporary transaction resource
> - add this resource to a ChangeRecorder
> 2) On write: - add EObject to the
> transaction resource (removes EObject from it's current
> resource)
This removal thing sounds scary.
> - cache removed resource in a "EObject-TO-Reference" map
> *** Every change is recorded by the ChangeRecorder ***
> 4) On Commit:
>
> - apply and revert changes to transaction resource
> (this produces a forward delta ChangeDescription)
> - add all changed EObjects to old resource
> - apply forward delta
If you want a forward delta, there's new support for that:
http://ed-merks.blogspot.com/2011/06/so-much-to-do-so-little-time.html
>
> My question is: does this approach honor the "EMF way"? Are there any
> other considerations to take into account?
I wonder if CDO wouldn't be a good fit...
> I am going to add EMF transaction support to EFeature eventually,
> probably when EFeature is added to uDIG, a GIS framework for Eclipse.
>
> Best regards,
> Kenneth Gulbrandsøy
> =====================
> Geotools EFeature module maintainer
> - Norway
Ed Merks
Professional Support: https://www.macromodeling.com/
|
|
|
Re: Sanity check: algorithm for thread safe commits in stand-alone applications [message #693734 is a reply to message #693677] |
Thu, 07 July 2011 06:07 |
|
Thanks Ed.
Your input is very much appreciated
I have some follow-up questions which you find below.
Quote:> A alternative to EMF Transaction is to combine GeoTools (GT) feature
> locking capabilities with a EMF ChangeRecorder. This however, will
> only be thread safe if all access to locked EObjects are performed
> from GT transaction aware client code. Hence, locked EObjects are
> still accessible (read and write) by EMF dependent client code not
> aware of (or don't care about) the GT transaction. As long as EFeature
> client code is aware of these assumptions, thread safe transactions
> should be feasible.
Can't you just have a simple high level policy for your threads, i.e., a
single lock that must be held to do anything with EFeature data...
That is a simple solution, but does not fit well with Geotools' transaction and locking patterns. The goal is to make EFeature a first class citizen of Geotools.
Quote:>
> The algorithm could be something like this:
>
> 1) On begin:
> - lock all features to prevent access from other
> transactions - create a temporary transaction resource
> - add this resource to a ChangeRecorder
> 2) On write: - add EObject to the
> transaction resource (removes EObject from it's current
> resource)
This removal thing sounds scary.
Just curious. What makes this scary? I didn't mention this earlier, but EFeature is only working on EAttribute values. Another solution could therefore be to do a ECoreUtil.copy() on write, adding it to the transaction resource, then do a ECoreUtil.replace() on commit. This will yield the same result but with a higher memory footprint.
Quote:> 4) On Commit:
>
> - apply and revert changes to transaction resource
> (this produces a forward delta ChangeDescription)
> - add all changed EObjects to old resource
> - apply forward delta
If you want a forward delta, there's new support for that: <url>
I have noticed the forward delta capabilites in 2.7. However, my goal is to support as much legacy EMF models as possible. Restricting EFeature to to 2.7 might make this goal harder to achieve because of classpath conflicts in stand-alone applications. What do you think?
Quote:> My question is: does this approach honor the "EMF way"? Are there any
> other considerations to take into account?
I wonder if CDO wouldn't be a good fit...
CDO is definitly on my radar already. However, expecting all EFeature clients to use CDO is a bit restrictive also. I think I have to leave the choice of persistence framework to the EFeature user.
Cheers,
Kenneth
|
|
|
Re: Sanity check: algorithm for thread safe commits in stand-alone applications [message #693880 is a reply to message #693625] |
Thu, 07 July 2011 12:06 |
|
Hi Kenneth,
I think the topic of concurrent model access is hot, so I've just written an article about it. Maybe you find it interesting: http://thegordian.blogspot.com/2011/07/concurrent-access-to-models.html
Cheers
/Eike
----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper
Am 07.07.2011 00:35, schrieb Kenneth:
> Hi EMF community!
>
> EMF Transaction is only available on the Eclipse platform (see message #68148).
> This is too restrictive for my project named EFeature, which is to add spatial support to EMF using Geotools (a general purpose library for spatial data).
>
> A alternative to EMF Transaction is to combine GeoTools (GT) feature locking capabilities with a EMF ChangeRecorder. This however, will only be thread safe if all access to locked EObjects are performed from GT transaction aware client code. Hence, locked EObjects are still accessible (read and write) by EMF dependent client code not aware of (or don't care about) the GT transaction. As long as EFeature client code is aware of these assumptions, thread safe transactions should be feasible.
>
> The algorithm could be something like this:
>
> 1) On begin:
> - lock all features to prevent access from other transactions - create a temporary transaction resource
> - add this resource to a ChangeRecorder
> 2) On write: - add EObject to the transaction resource (removes EObject from it's current resource)
> - cache removed resource in a "EObject-TO-Reference" map
> *** Every change is recorded by the ChangeRecorder ***
> 4) On Commit:
>
> - apply and revert changes to transaction resource
> (this produces a forward delta ChangeDescription)
> - add all changed EObjects to old resource
> - apply forward delta
>
> My question is: does this approach honor the "EMF way"? Are there any other considerations to take into account?
> I am going to add EMF transaction support to EFeature eventually, probably when EFeature is added to uDIG, a GIS framework for Eclipse.
>
> Best regards,
> Kenneth Gulbrandsøy
> =====================
> Geotools EFeature module maintainer
> - Norway
Cheers
/Eike
----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper
|
|
|
Re: Sanity check: algorithm for thread safe commits in stand-alone applications [message #693900 is a reply to message #693880] |
Thu, 07 July 2011 13:12 |
|
Thanks Eike, this was a very clear article on how concurrent access can be done using CDO!
As I said, CDO is one solution that solves my problem. The challenge for me however, is that I also want to support legacy EMF models which can not be regenereded for CDO. As of now, EFeature support 1) dynamic (at runtime) mapping from any EObject containing geometry information (limited to JTS Simple Feature specification) to EFeature by mapping EAttributes to EFeature, and 2) extension of the EFeature model (like the ExtendedLibrary example).
I believe CDO is a obvious choice when working on large models, which spatial models often are (10 000-100 000 objects are typical numbers). However, I still want to supply some support for models which can not be persisted with CDO. I have to admitt that I do not know how big a problem this really is (number of models that can not use CDO, or client code that can not change the persistence framework to CDO) though.
Eike Stepper wrote on Thu, 07 July 2011 14:06Hi Kenneth,
I think the topic of concurrent model access is hot, so I've just written an article about it.
|
|
|
Re: Sanity check: algorithm for thread safe commits in stand-alone applications [message #694056 is a reply to message #693734] |
Thu, 07 July 2011 17:36 |
Ed Merks Messages: 33148 Registered: July 2009 |
Senior Member |
|
|
Kenneth,
Comments below.
On 06/07/2011 11:07 PM, Kenneth wrote:
> Thanks Ed.
> Your input is very much appreciated :p
> I have some follow-up questions which you find below.
>
>
> Quote:
>> > A alternative to EMF Transaction is to combine GeoTools (GT)
>> feature > locking capabilities with a EMF ChangeRecorder. This
>> however, will > only be thread safe if all access to locked EObjects
>> are performed > from GT transaction aware client code. Hence, locked
>> EObjects are > still accessible (read and write) by EMF dependent
>> client code not > aware of (or don't care about) the GT transaction.
>> As long as EFeature > client code is aware of these assumptions,
>> thread safe transactions > should be feasible.
>> Can't you just have a simple high level policy for your threads,
>> i.e., a single lock that must be held to do anything with EFeature
>> data...
>
> That is a simple solution, but does not fit well with Geotools'
> transaction and locking patterns. The goal is to make EFeature a first
> class citizen of Geotools.
I see. I know nothing about those details though...
>
> Quote:
>> >
>> > The algorithm could be something like this:
>> >
>> > 1) On begin:
>> > - lock all features to prevent access from other > transactions -
>> create a temporary transaction resource
>> > - add this resource to a ChangeRecorder
>> > 2) On write: - add EObject to the > transaction resource (removes
>> EObject from it's current > resource)
>> This removal thing sounds scary.
>
> Just curious. What makes this scary?
Because that removal too is a modification that could be recorded.
> I didn't mention this earlier, but EFeature is only working on
> EAttribute values. Another solution could therefore be to do a
> ECoreUtil.copy() on write, adding it to the transaction resource, then
> do a ECoreUtil.replace() on commit. This will yield the same result
> but with a higher memory footprint.
> Quote:
>> > 4) On Commit:
>> >
>> > - apply and revert changes to transaction resource
>> > (this produces a forward delta ChangeDescription)
>> > - add all changed EObjects to old resource
>> > - apply forward delta
>> If you want a forward delta, there's new support for that: <url>
>
> I have noticed the forward delta capabilites in 2.7. However, my goal
> is to support as much legacy EMF models as possible. Restricting
> EFeature to to 2.7 might make this goal harder to achieve because of
> classpath conflicts in stand-alone applications. What do you think?
2.7 supports all versions that have come before so I'm not sure there
would be any barrier created by using it.
>
> Quote:
>> > My question is: does this approach honor the "EMF way"? Are there
>> any > other considerations to take into account?
>> I wonder if CDO wouldn't be a good fit...
>
> CDO is definitly on my radar already. However, expecting all EFeature
> clients to use CDO is a bit restrictive also. I think I have to leave
> the choice of persistence framework to the EFeature user.
>
> Cheers,
> Kenneth
Ed Merks
Professional Support: https://www.macromodeling.com/
|
|
| | | | | |
Re: Sanity check: algorithm for thread safe commits in stand-alone applications [message #694346 is a reply to message #694329] |
Fri, 08 July 2011 10:57 |
|
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html; charset=UTF-8" http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
Am 08.07.2011 12:52, schrieb <a class="moz-txt-link-abbreviated" href="mailto:forums-noreply@eclipse.org:">forums-noreply@eclipse.org:</a>
<blockquote cite="mid:iv6n00$n39$1@news.eclipse.org" type="cite">Hi,
<br>
<br>
I am following your article but i am unable to fing Util Class...
Can you please tell which package it belongs to?
<br>
</blockquote>
Sorry, that guy is a fake to keep the networking details out of the
article. Here's an example for embedded transport:<br>
<br>
<title></title>
<style type="text/css">
<!--code { font-family: Courier New, Courier; font-size: 10pt; margin: 0px; }-->
</style>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<!-- ======================================================== -->
<!-- = Java Sourcecode to HTML automatically converted code = -->
<!-- = Java2Html Converter 5.0 [2006-02-26] by Markus Gebhard markus@jave.de = -->
<!-- = Further information: http://www.java2html.de = -->
<div class="java" align="left">
<table bgcolor="#ffffff" border="0" cellpadding="3"
cellspacing="0">
<tbody>
<tr>
<!-- start source code --> <td nowrap="nowrap" align="left"
valign="top"> <code>
<font color="#ffffff"> </font><font color="#000000">acceptor = Net4jUtil.getAcceptor</font><font
color="#000000">(</font><font color="#000000">IPluginContainer.INSTANCE, </font><font
color="#2a00ff">"jvm"</font><font color="#000000">, repositoryName</font><font
color="#000000">)</font><font color="#000000">;</font><br>
<font color="#ffffff"> </font><font color="#000000">connector = Net4jUtil.getConnector</font><font
color="#000000">(</font><font color="#000000">IPluginContainer.INSTANCE, </font><font
color="#2a00ff">"jvm"</font><font color="#000000">, repositoryName</font><font
color="#000000">)</font><font color="#000000">;</font><br>
<font color="#ffffff"></font><br>
<font color="#ffffff"> </font><font color="#000000">CDONet4jSessionConfiguration config = CDONet4jUtil.createNet4jSessionConfiguration</font><font
color="#000000">()</font><font color="#000000">;</font><br>
<font color="#ffffff"> </font><font color="#000000">config.setConnector</font><font
color="#000000">(</font><font color="#000000">connector</font><font
color="#000000">)</font><font color="#000000">;</font><br>
<font color="#ffffff"> </font><font color="#000000">config.setRepositoryName</font><font
color="#000000">(</font><font color="#000000">repositoryName</font><font
color="#000000">)</font><font color="#000000">;</font><br>
<font color="#ffffff"></font><br>
<font color="#ffffff"> </font><font color="#000000">CDONet4jSession session = config.openNet4jSession</font><font
color="#000000">()</font><font color="#000000">;</font></code>
</td>
<!-- end source code --> </tr>
</tbody>
</table>
</div>
<!-- = END of automatically generated HTML code = -->
<!-- ======================================================== -->
<br>
This could would open a session to a remote repository:<br>
<br>
<title></title>
<style type="text/css">
<!--code { font-family: Courier New, Courier; font-size: 10pt; margin: 0px; }-->
</style>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<!-- ======================================================== -->
<!-- = Java Sourcecode to HTML automatically converted code = -->
<!-- = Java2Html Converter 5.0 [2006-02-26] by Markus Gebhard markus@jave.de = -->
<!-- = Further information: http://www.java2html.de = -->
<div class="java" align="left">
<table bgcolor="#ffffff" border="0" cellpadding="3"
cellspacing="0">
<tbody>
<tr>
<!-- start source code --> <td nowrap="nowrap" align="left"
valign="top"> <code>
<font color="#ffffff"> </font><font color="#000000">connector = Net4jUtil.getConnector</font><font
color="#000000">(</font><font color="#000000">IPluginContainer.INSTANCE, </font><font
color="#2a00ff">"tcp"</font><font color="#000000">, </font><font
color="#2a00ff">"localhost"</font><font
color="#000000">)</font><font color="#000000">;</font><br>
<font color="#ffffff"></font><br>
<font color="#ffffff"> </font><font color="#000000">CDONet4jSessionConfiguration config = CDONet4jUtil.createNet4jSessionConfiguration</font><font
color="#000000">()</font><font color="#000000">;</font><br>
<font color="#ffffff"> </font><font color="#000000">config.setConnector</font><font
color="#000000">(</font><font color="#000000">connector</font><font
color="#000000">)</font><font color="#000000">;</font><br>
<font color="#ffffff"> </font><font color="#000000">config.setRepositoryName</font><font
color="#000000">(</font><font color="#000000">repositoryName</font><font
color="#000000">)</font><font color="#000000">;</font><br>
<font color="#ffffff"></font><br>
<font color="#ffffff"> </font><font color="#000000">CDONet4jSession session = config.openNet4jSession</font><font
color="#000000">()</font><font color="#000000">;</font></code>
</td>
<!-- end source code --> </tr>
</tbody>
</table>
</div>
<!-- = END of automatically generated HTML code = -->
<!-- ======================================================== -->
<br>
Does that help?<br>
<br>
Cheers<br>
/Eike<br>
<br>
----<br>
<a class="moz-txt-link-freetext" href="http://www.esc-net.de">http://www.esc-net.de</a><br>
<a class="moz-txt-link-freetext" href="http://thegordian.blogspot.com">http://thegordian.blogspot.com</a><br>
<a class="moz-txt-link-freetext" href="http://twitter.com/eikestepper">http://twitter.com/eikestepper</a><br>
<br>
<br>
</body>
</html>
Cheers
/Eike
----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper
|
|
| | |
Goto Forum:
Current Time: Thu May 16 17:17:17 GMT 2024
Powered by FUDForum. Page generated in 0.05521 seconds
|