Home » Modeling » EMF » Concurrent access on EMF model
|
| Re: Concurrent access on EMF model [message #1465226 is a reply to message #1464731] |
Sat, 08 November 2014 06:31   |
 Ed Merks
Messages: 33138
Registered: July 2009 |
Senior Member |
|
|
Alan,
Comments below.
On 07/11/2014 9:48 PM, Alan DW wrote:
> Hello everyone,
>
> we are building a web application that uses Ecore for its data model.
> All clients operate on the same data model instance that is held
> in-memory on the server.
>
> Since we have multiple clients, we also have parallel access to the
> model. In our case, it will be 90% read access - so reads need to be
> really fast. We have our own locking mechanisms for write operations.
> However, if one client acquires the write lock on an element, we do
> not want to prevent all other clients from reading it (our domain does
> not permit this) - i.e. we have to permit "dirty reads".
Well, dirty reads is one thing, but without any kind of atomicity in the
writes relative to the reads, this still seems problematic. E.g., any
update to any containment list or bidirectional reference involves
toggling the state of several objects so even if dirty reads are okay,
reads that see a partially modified state will always be problematic...
>
> The problem is as follows: imagine a multiplicity-many EReference
> somewhere in the model. A client "A" is currently reading (i.e.
> iterating over) it. A second client "B" acquires the write lock and
> changes the value of the EReference. This will inevitably throw a
> ConcurrentModificationException on client "A".
Suppose the client is expecting there to be a next element, but that one
is removed, you'd get an index out of bounds exception...
>
> Long story short: what's the simplest way of preventing that
> ConcurrentModificationException?
I think that's just one symptom of many you might see. And it's one
that generally helps you realize there are likely to be many other
problems that crop up...
> In a normal Java model, I would replace the collections with
> "Copy-On-Write" collections,
But even this, without any kind of synchronization will be problematic.
And even single valued containment/bidirectional references will be
problematic...
> but Ecore does not allow that.
As long as you return the right list implementation, that's okay (though
not entirely expected). You can even use
org.eclipse.emf.ecore.util.EcoreEList.UnmodifiableEList.UnmodifiableEList(InternalEObject,
EStructuralFeature, int, Object[]) to prevent the client from modifying
the list.
> Is it really necessary to switch to a full-fledged repository like CDO
> just to prevent this ConcurrentModificationException?
>
>
> I'd be very grateful for any advice.
Depending on whether the writers are doing very restricted/limited kinds
of updates, I don't think you can assuming that other than CMEs, the
readers will generally work fine in the face of multi-threaded updates.
>
>
> Alan
Ed Merks
Professional Support: https://www.macromodeling.com/
|
|
| |
| Re: Concurrent access on EMF model [message #1466619 is a reply to message #1465296] |
Sun, 09 November 2014 10:42 |
Alan DW
Messages: 119
Registered: March 2012 |
Senior Member |
|
|
Hi,
first of all: thank you for your responses.
@Ed Merks: I can clearly see the issue with dirty reads where an object reads an inconsistent state. This is precisely what is causing my nightmares at the moment. I'm trying hard to think of cases where this can actually happen. Our current solution is to wrap each and every EObject such that its methods return a Copy-On-Write wrapper around the actual EList. That wrapper is thread-safe in all of its operations, it automatically wraps all contained EObjects it returns in its methods (such that they in turn can again return wrapped ELists), and furthermore allows clients to acquire a lock on the entire list, if absolutely required. That being said, the individual elements of our model are actually quite simple. We do not have (or need) any opposite references, and containment references are not as problematic either, because essentially we have one big root container object and a lot of objects that are contained - but that's it. We might have additions and removals from that container, but we never have things like moving one EObject to a different parent. Generally speaking, our "object graph" is rather simple: a container with containment references to its elements, and the elements have normal (non-opposite) references to each other.
@Ed Willink: You do have a point here. We could certainly take Ecore out of our data model implementation and roll our own. From a certain perspective, this would make total sense and we were discussing it internally as well. However, there is one thing about Ecore that we really need and cannot do without: dynamic ecore. We need the capabilities to create EClasses at runtime and instantiate them on the fly without having to do any code generation or Java compilation in between. And that's something that Java on its own just is not good at doing. You could do some nasty bytecode manipulation stuff, but we really do not want to go that way. If we were to switch from Ecore to our own data model implementation, we would basically have to re-implement the entire reflective layer (EClass, EStructuralFeature, eGet(), eSet() and so on).
Thank you both for your input, it certainly gave me more food for thought.
|
|
| |
| Re: Concurrent access on EMF model [message #1467729 is a reply to message #1467705] |
Mon, 10 November 2014 08:58 |
Ed Merks
Messages: 33138
Registered: July 2009 |
Senior Member |
|
|
Ed,
Comments below.
On 10/11/2014 9:30 AM, Ed Willink wrote:
> Hi
>
> On 09/11/2014 10:42, Alan DW wrote:
>> We need the
>> capabilities to create EClasses at runtime and instantiate them on the
>> fly without having to do any code generation or Java compilation in
>> between.
>
> Certainly more challenging, but the equivalent of DynamicEObject is
> not that hard.
>
> Even more challenging to make Ecore thread safe.
That isn't Alan's goal. His goal is to make instances thread safe,
although the exactly meaning of that isn't entirely clear. That's a
major problem with thread safety discussions, particularly in the face
of data structure modifications. It's typically the case that thread
safe primitive data structures don't result in thread safe course
grained code, e.g., the following isn't thread safe even if the list is:
int index = list.indexOf(x);
if (index != -1)
{
list.remove(index):
}
> Until recently the Ecore philosophy was that metamodel changes at
> run-time were illegal, so any multi-threaded creation of EClasses
> requires more detailed thread-safe consideration of how superclass
> relationships are being maintained and of the stability of feature ids.
That's still the general limitation. It's not so different from Java's
hot-replace where you can't add/remove fields/methods from the
already-created instances.
> Ecore does now permit some metamodel evolution but I rather doubt that
> those changes were made with thread safety in mind.
>
> Regards
>
> Ed Willink
>
Ed Merks
Professional Support: https://www.macromodeling.com/
|
|
|
| Re: Concurrent access on EMF model [message #1468499 is a reply to message #1467729] |
Mon, 10 November 2014 22:27 |
Alan DW
Messages: 119
Registered: March 2012 |
Senior Member |
|
|
Hello again,
@Ed Willink: We had a discussion in our team today about implementing our own data model. Right now, it would be simply too much effort, mainly due to the tight integration of Ecore structures in our UI code. But we will keep that option in mind for the future.
@Ed Merks: You nailed the problem with your example code snippet. "Check then act" is the bane of thread safety. We do have a classical 3-layer-model in our application architecture: UI, business and persistence. In the business layer, we hold the shared data model. Now, the thing is: my feeling tells me that access synchronization to the data model really is no job for the UI layer, wouldn't you agree? At least I would not expect to find "synchronized" blocks or try-finally blocks with java.util.concurrent.Lock in clean UI code. But then again, where else? Any level lower than the UI and you wind up with code snippets such as yours, which you just *can't* synchronize properly without knowing the entire code block that requires access, and that would clearly be in the UI. I can see that in a "normal" scenario, one would employ some kind of data store or repository to lazily fetch the elements the UI currently needs (providing each user thread with its own copy) and let that data store handle the heavy lifting about synchronization. However, our application demands that we execute queries that work globally on the entire model, and with 300.000+ model elements, you just can't instantiate it 50 times for 50 users on a central server, that's just not feasible in terms of RAM. On some occasions, we even need ALL elements, which pretty much defeats the purpose of lazy loading. Do you have any ideas that come to your mind?
On a personal note: I'm a young software developer and in the business for about 2 years now. I have never ever seen any library as awesome as EMF and Ecore. As far as I know, you have been one of the masterminds behind it - I just would like to say a big, heartfelt "thanks" 
|
|
|
| Re: Concurrent access on EMF model [message #1469004 is a reply to message #1468499] |
Tue, 11 November 2014 08:05 |
Ed Merks
Messages: 33138
Registered: July 2009 |
Senior Member |
|
|
Alan,
Comments below
On 10/11/2014 11:27 PM, Alan DW wrote:
> Hello again,
>
>
> @Ed Willink: We had a discussion in our team today about implementing
> our own data model. Right now, it would be simply too much effort,
> mainly due to the tight integration of Ecore structures in our UI
> code. But we will keep that option in mind for the future.
>
> @Ed Merks: You nailed the problem with your example code snippet.
> "Check then act" is the bane of thread safety.
Yes, it's my standard example back from my corporate days where there
were armies of misguided people who firmly believed that sprinkling the
EMF framework with some magical synchronization dust would solve their
high level threading problems.
> We do have a classical 3-layer-model in our application architecture:
> UI, business and persistence. In the business layer, we hold the
> shared data model. Now, the thing is: my feeling tells me that access
> synchronization to the data model really is no job for the UI layer,
> wouldn't you agree?
The EMF transaction framework tries to provide a kind of shared
read/exclusive write model. The Xtext framework takes a different but
somewhat analogous approach. Certainly there needs to be some type of
high-level policy for how threads should interact...
> At least I would not expect to find "synchronized" blocks or
> try-finally blocks with java.util.concurrent.Lock in clean UI code.
Well, hopefully not logs of hand-written such things.
> But then again, where else?
Given that overall thread safety cannot be achieved simply by using
thread safe primitives, on generally does need a thread-based policy
with course grained locking...
> Any level lower than the UI and you wind up with code snippets such as
> yours, which you just *can't* synchronize properly without knowing the
> entire code block that requires access, and that would clearly be in
> the UI. I can see that in a "normal" scenario, one would employ some
> kind of data store or repository to lazily fetch the elements the UI
> currently needs (providing each user thread with its own copy) and let
> that data store handle the heavy lifting about synchronization.
> However, our application demands that we execute queries that work
> globally on the entire model, and with 300.000+ model elements, you
> just can't instantiate it 50 times for 50 users on a central server,
> that's just not feasible in terms of RAM.
No.
> On some occasions, we even need ALL elements, which pretty much
> defeats the purpose of lazy loading.
Yes, life is a tradeoff.
> Do you have any ideas that come to your mind?
The only type of approach that seems generally consistent in this sort
of case is a multiple-readers exclusive writers model. To use such an
approach with a general viewer, I could imagine delegating content/label
providers that do the high level read-locking and that via the command
stack one does the high-level write locking...
> On a personal note: I'm a young software developer and in the business
> for about 2 years now. I have never ever seen any library as awesome
> as EMF and Ecore.
Flattery gets you far in life. :-P
> As far as I know, you have been one of the masterminds behind it - I
> just would like to say a big, heartfelt "thanks" :)
Ed Merks
Professional Support: https://www.macromodeling.com/
|
|
| | | |
| Re: Concurrent access on EMF model [message #1470603 is a reply to message #1470333] |
Wed, 12 November 2014 13:24 |
|
Hi, Felix,
Yes, read access is exclusive because reading EMF objects can produce
concrete side-effects (proxy resolution, resource loading, etc.)
However, readers can cooperatively yield access to other readers that
may be waiting, if they know that they are doing a lot of work in a
read-only transaction. It's not automatic/preemptive, a throw-back to
the 1980s, but it's there.
Christian
On 2014-11-12 08:56:45 +0000, Felix Dorner said:
> On 11/11/2014 09:05, Ed Merks wrote
>> The EMF transaction framework tries to provide a kind of shared
>> read/exclusive write model.
>
> I don't think it's shared read. Transactional access is entirely
> serial, be it read or write. There may only be one reader or writer at
> a given time, so there's 0 concurrency.
>
> Felix
|
|
|
Goto Forum:
Current Time: Sat Apr 20 07:24:22 GMT 2024
Powered by FUDForum. Page generated in 0.04095 seconds |
] 
Search
Help
Register
Login
Home
