Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » EMF » [EMF][CDO] Scalability and the recommended use of ItemProviderAdapters when using CDO
[EMF][CDO] Scalability and the recommended use of ItemProviderAdapters when using CDO [message #687555] Wed, 22 June 2011 18:13 Go to next message
Josh M is currently offline Josh MFriend
Messages: 22
Registered: July 2009
Junior Member
[Sorry for the repost, my message seemed to disappear from this forum]

I am experimenting with migrating from the default EMF implementation into CDO. Most of our desire for making this change is to take advantage of the scalability CDO offers with respect to the memory usage of our application (http://thegordian.blogspot.com/2008/11/how-scalable-are-my-models.html).

So far, the migration has gone well, but when running our app, I am still noticing that the memory usage grows. Tracing this down, one of the components that seems to be holding onto objects is the 'targets' collection of the org.eclipse.emf.edit.provider.ItemProviderAdapter class. Although this is expected (we are using the Singleton Provider type, and based on the docs it states "one adapter object for every type of object in the model (recommended when possible)"), I am concerned this does not lend itself well to the scalability we are seeking. There is the choice of a Stateful Provider type, but I don't have experience with using this yet to understand what the impacts of such a change would be.

Specifically, our application can generate many of the same types of objects per second, and can be executed for many hours at a time (obviously generating hundreds of thousand objects or more). These objects are added to the model through use of AddCommands created via the AdapterFactoryEditingDomain, and persisted. Our hope is that they could be GC'd at that point. This does not seem to be happening based on the reference still held by the corresponding ItemProviderAdapter mentioned before.

I was hoping there might be some thought's from the community as to the best way of handling this kind of situation. Any help or guidance you might have would be greatly appreciated. Thanks!
Re: [EMF][CDO] Scalability and the recommended use of ItemProviderAdapters when using CDO [message #687569 is a reply to message #687555] Wed, 22 June 2011 18:38 Go to previous messageGo to next message
Eike Stepper is currently offline Eike StepperFriend
Messages: 6444
Registered: July 2009
Senior Member
Am 22.06.2011 20:13, schrieb Sed:
> [Sorry for the repost, my message seemed to disappear from this forum]
I've heard about a forum software problem: https://bugs.eclipse.org/bugs/show_bug.cgi?id=348449

>
> I am experimenting with migrating from the default EMF implementation into CDO. Most of our desire for making this change is to take advantage of the scalability CDO offers with respect to the memory usage of our application (http://thegordian.blogspot.com/2008/11/how-scalable-are-my-models.html).
>
> So far, the migration has gone well, but when running our app, I am still noticing that the memory usage grows. Tracing this down, one of the components that seems to be holding onto objects is the 'targets' collection of the org.eclipse.emf.edit.provider.ItemProviderAdapter class. Although this is expected (we are using the Singleton Provider type, and based on the docs it states "one adapter object for every type of object in the model (recommended when possible)"), I am concerned this does not lend itself well to the scalability we are seeking. There is the choice of a Stateful Provider type, but I don't have experience with using this yet to understand what the impacts of such a change would be.
>
> Specifically, our application can generate many of the same types of objects per second, and can be executed for many hours at a time (obviously generating hundreds of thousand objects or more). These objects are added to the model through use of AddCommands created via the AdapterFactoryEditingDomain, and persisted. Our hope is that they could be GC'd at that point. This does not seem to be happening based on the reference still held by the corresponding ItemProviderAdapter mentioned before.
>
> I was hoping there might be some thought's from the community as to the best way of handling this kind of situation. Any help or guidance you might have would be greatly appreciated. Thanks!
I've heard about similar issues with JFace viewers, too. But I must admit that I'm not an expert with ItemProviderAdapters. I've cc'ed Ed, perhaps he has an idea...

Cheers
/Eike

----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper


Re: [EMF][CDO] Scalability and the recommended use of ItemProviderAdapters when using CDO [message #687578 is a reply to message #687569] Wed, 22 June 2011 19:01 Go to previous messageGo to next message
Ed Merks is currently offline Ed MerksFriend
Messages: 30450
Registered: July 2009
Senior Member
Yes, adapters stay attached to their objects until some action is taken
to remove them. Even stateful adapters will generally be managed in a
way that they can all be disposed when the viewer no longer needs them,
i.e., they're kept in a list that itself can't be garbage collected.

Are your new objects actually being viewed?


Eike Stepper wrote:
> Am 22.06.2011 20:13, schrieb Sed:
>> [Sorry for the repost, my message seemed to disappear from this forum]
> I've heard about a forum software problem:
> https://bugs.eclipse.org/bugs/show_bug.cgi?id=348449
>
>>
>> I am experimenting with migrating from the default EMF implementation
>> into CDO. Most of our desire for making this change is to take
>> advantage of the scalability CDO offers with respect to the memory
>> usage of our application
>> (http://thegordian.blogspot.com/2008/11/how-scalable-are-my-models.html).
>>
>>
>> So far, the migration has gone well, but when running our app, I am
>> still noticing that the memory usage grows. Tracing this down, one
>> of the components that seems to be holding onto objects is the
>> 'targets' collection of the
>> org.eclipse.emf.edit.provider.ItemProviderAdapter class. Although
>> this is expected (we are using the Singleton Provider type, and based
>> on the docs it states "one adapter object for every type of object in
>> the model (recommended when possible)"), I am concerned this does not
>> lend itself well to the scalability we are seeking. There is the
>> choice of a Stateful Provider type, but I don't have experience with
>> using this yet to understand what the impacts of such a change would be.
>>
>> Specifically, our application can generate many of the same types of
>> objects per second, and can be executed for many hours at a time
>> (obviously generating hundreds of thousand objects or more). These
>> objects are added to the model through use of AddCommands created via
>> the AdapterFactoryEditingDomain, and persisted. Our hope is that
>> they could be GC'd at that point. This does not seem to be happening
>> based on the reference still held by the corresponding
>> ItemProviderAdapter mentioned before.
>>
>> I was hoping there might be some thought's from the community as to
>> the best way of handling this kind of situation. Any help or
>> guidance you might have would be greatly appreciated. Thanks!
> I've heard about similar issues with JFace viewers, too. But I must
> admit that I'm not an expert with ItemProviderAdapters. I've cc'ed Ed,
> perhaps he has an idea...
>
> Cheers
> /Eike
>
> ----
> http://www.esc-net.de
> http://thegordian.blogspot.com
> http://twitter.com/eikestepper
>
>
Re: [EMF][CDO] Scalability and the recommended use of ItemProviderAdapters when using CDO [message #687587 is a reply to message #687578] Wed, 22 June 2011 19:37 Go to previous messageGo to next message
Josh M is currently offline Josh MFriend
Messages: 22
Registered: July 2009
Junior Member
No they are not (at least not at that time). The code path seems to attach them to the ItemProviderAdapter when the object(s) is/are added to the owner list, or set against an owner attribute. The debug stack looks like:

MyItemProvider(ItemProviderAdapter).setTarget(Notifier) line: 1535
CDOObjectImpl$1(BasicNotifierImpl$EAdapterList<E>).didAdd(int, E) line: 127	
CDOObjectImpl$1.didAdd(int, Adapter) line: 436	
CDOObjectImpl$1.didAdd(int, Object) line: 1	
CDOObjectImpl$1(BasicEList<E>).addUnique(E) line: 425	
CDOObjectImpl$1(AbstractEList<E>).add(E) line: 307	
CDOObjectImpl$1(BasicNotifierImpl$EAdapterList<E>).add(E) line: 199	
MyItemProviderAdapterFactory(AdapterFactoryImpl).associate(Adapter, Notifier) line: 156	
MyItemProviderAdapterFactory(AdapterFactoryImpl).adaptNew(Notifier, Object) line: 108	
MyItemProviderAdapterFactory(AdapterFactoryImpl).adapt(Notifier, Object) line: 93	
MyItemProviderAdapterFactory.adapt(Notifier, Object) line: 255	
ComposedAdapterFactory.adapt(Notifier, Object, boolean) line: 300	
ComposedAdapterFactory.adapt(Notifier, Object) line: 277	
ComposedAdapterFactory.adapt(Object, Object) line: 264	
AdapterFactoryEditingDomain.createCommand(Class<Command>, CommandParameter) line: 460
...


It is at line 1535 that they are added to the targets collection of the respective ItemProviderAdapter.
Re: [EMF][CDO] Scalability and the recommended use of ItemProviderAdapters when using CDO [message #687699 is a reply to message #687587] Thu, 23 June 2011 02:42 Go to previous messageGo to next message
Ed Merks is currently offline Ed MerksFriend
Messages: 30450
Registered: July 2009
Senior Member
Sed,

Yes, the item providers act as command factories. Perhaps you could
just clear the adapters after you've executed the command.


Sed wrote:
> No they are not (at least not at that time). The code path seems to
> attach them to the ItemProviderAdapter when the object(s) is/are added
> to the owner list, or set against an owner attribute. The debug stack
> looks like:
>
>
> MyItemProvider(ItemProviderAdapter).setTarget(Notifier) line: 1535
> CDOObjectImpl$1(BasicNotifierImpl$EAdapterList<E>).didAdd(int, E)
> line: 127
> CDOObjectImpl$1.didAdd(int, Adapter) line: 436
> CDOObjectImpl$1.didAdd(int, Object) line: 1
> CDOObjectImpl$1(BasicEList<E>).addUnique(E) line: 425
> CDOObjectImpl$1(AbstractEList<E>).add(E) line: 307
> CDOObjectImpl$1(BasicNotifierImpl$EAdapterList<E>).add(E) line: 199
> MyItemProviderAdapterFactory(AdapterFactoryImpl).associate(Adapter,
> Notifier) line: 156
> MyItemProviderAdapterFactory(AdapterFactoryImpl).adaptNew(Notifier,
> Object) line: 108
> MyItemProviderAdapterFactory(AdapterFactoryImpl).adapt(Notifier,
> Object) line: 93
> MyItemProviderAdapterFactory.adapt(Notifier, Object) line: 255
> ComposedAdapterFactory.adapt(Notifier, Object, boolean) line: 300
> ComposedAdapterFactory.adapt(Notifier, Object) line: 277
> ComposedAdapterFactory.adapt(Object, Object) line: 264
> AdapterFactoryEditingDomain.createCommand(Class<Command>,
> CommandParameter) line: 460
> ..
>
>
> It is at line 1535 that they are added to the targets collection of
> the respective ItemProviderAdapter.
>
Re: [EMF][CDO] Scalability and the recommended use of ItemProviderAdapters when using CDO [message #687706 is a reply to message #687699] Thu, 23 June 2011 03:52 Go to previous messageGo to next message
Josh M is currently offline Josh MFriend
Messages: 22
Registered: July 2009
Junior Member
Thanks for the quick reply!

Is there a recommend way, or best practice, for clearing the adapters?

At the point in the code where this situation occurs, I usually just have access to the editing domain and the adapter factory (and of course any of the public contents they offer). It seems that I would I would need emulate the converse of the ComposedAdapterFacroty.adapt method (unadapt?) but such a behavior does not seem to exist.

I can see three possible ways to do this:

[Option 1]
1. Create the command via the AdapterFactoryEditingDomain.createCommand() method
2. Execute the command
3. Iterate over the owner (and possibly the value or collection of values of the command) and access their eAdapters, i.e. myEObject.eAdapters()
4. For each object, iterate over Adapters given by eAdapters and find instances of ItemProviderAdapter
5. Then call ItemProviderAdapter.unsetTarget(myEObject)

[Option 2]
Similar to above but at step five, just calling ItemProviderAdapter.dispose().

[Option 3]
// Retrieving the ePackage of myEObject
EPackage myEPackage = myEObject.eClass().getEPackage();
// Asking the ComposedAdapterFactory for the appropriate ItemProviderAdapterFactory
AdapterFactory myAdapterFactory = composedAdapterFactory.getFactoryForTypes(Collections.singletonList(myEPackage);
// Retrieve (hopefully) the ItemProviderAdapter
Adapter myAdapter = myAdapterFactory.createAdapter(myEObject);
// Double check that that the adapter is an ItemProviderAdapter
if (null != myAdapter && myAdapter instanceof ItemProviderAdapter) {

    if (ed_says_to_unsetTarget) {
        ((ItemProviderAdapter)myAdapter).unsetTarget(myEObject);
    }
    else if (ed_says_to_dispose_itemprovideradapter) {
        ((ItemProviderAdapter)myAdapter).dispose();
    }
    else {
        throw new NeedHelpException();
    }
}

Are there other approaches that I am not seeing? Also, if you recommend doing either of the processes outlined above, do you foresee any undesired consequences when the object is adapted later (either by being viewed, or having another command executed against it)? I also start to question thread safety and being able to synchronize unsetTarget/dispose with adapts/setTargets that may be happening on other threads (I could be becoming paranoid :] )

On a related note, would the solution discussed here also be applicable for items viewed in JFace viewers (we stay with the AdapterFactoryContentProviders and AdapterFactoryLabelProviders as often as we can)? I would enjoy being able to unload objects from their ItemProviderAdapters when views are closed/disposed to lessen memory "leaks" as much as possible.

[Updated on: Thu, 23 June 2011 05:01]

Report message to a moderator

Re: [EMF][CDO] Scalability and the recommended use of ItemProviderAdapters when using CDO [message #687718 is a reply to message #687706] Thu, 23 June 2011 04:48 Go to previous messageGo to next message
Ed Merks is currently offline Ed MerksFriend
Messages: 30450
Registered: July 2009
Senior Member
Sed,

Comments below.

Sed wrote:
> Thanks for the quick reply!
> Is there a recommend way, or best practice, for clearing the adapters?
Not really.
> At the point in the code where this situation occurs, I usually just
> have access to the editing domain and the adapter factory (and of
> course any of the public contents they offer). It seems that I would
> I would need emulate the converse of the ComposedAdapterFacroty.adapt
> method (unadapt?) but such a behavior does not seem to exist.
Nope.
>
> The next possibility I see would be steps similar to the following:
> 1. Create the command via the
> AdapterFactoryEditingDomain.createCommand() method
> 2. Execute the command
> 3. Iterate over the owner (and possibly the value or collection of
> values of the command) and access their eAdapters, i.e.
> myEObject.eAdapters()
I'm not sure it all needs to be so complicated. It seems to me you know
you're adding some object using an AddCommand and executing that so
that's a good place to say, hey, I know this thing adds adapters, so
I'll just clear them...
> 4. For each object, iterate over Adapters given by eAdapters and find
> instances of ItemProviderAdapter
> 5. Then call ItemProviderAdapter.unsetTarget(myEObject)
No, remove the adapters from the eAdapters list...
>
> Are there other approaches that I am not seeing? Also, if you
> recommend doing to process outlined above, do you foresee any
> undesired consequences when the object is adapted later (either by
> being viewed, or having another command executed against it)?
No, adapters are added on demand. If the object isn't being viewed,
there's no really need to be listening for notifications...
>
> On a related note, would the solution discussed here also be
> applicable for items viewed in JFace viewers (we stay with the
> AdapterFactoryContentProviders and AdapterFactoryLabelProviders as
> often as we can)? I would enjoy being able to unload objects from
> their ItemProviderAdapters when views are closed/disposed to lessen
> memory "leaks" as much as possible.
Calling unload will clear all adapters of all objects contained by the
resource.
Re: [EMF][CDO] Scalability and the recommended use of ItemProviderAdapters when using CDO [message #687720 is a reply to message #687706] Thu, 23 June 2011 04:48 Go to previous messageGo to next message
Ed Merks is currently offline Ed MerksFriend
Messages: 30450
Registered: July 2009
Senior Member
Sed,

Comments below.

Sed wrote:
> Thanks for the quick reply!
> Is there a recommend way, or best practice, for clearing the adapters?
Not really.
> At the point in the code where this situation occurs, I usually just
> have access to the editing domain and the adapter factory (and of
> course any of the public contents they offer). It seems that I would
> I would need emulate the converse of the ComposedAdapterFacroty.adapt
> method (unadapt?) but such a behavior does not seem to exist.
Nope.
>
> The next possibility I see would be steps similar to the following:
> 1. Create the command via the
> AdapterFactoryEditingDomain.createCommand() method
> 2. Execute the command
> 3. Iterate over the owner (and possibly the value or collection of
> values of the command) and access their eAdapters, i.e.
> myEObject.eAdapters()
I'm not sure it all needs to be so complicated. It seems to me you know
you're adding some object using an AddCommand and executing that so
that's a good place to say, hey, I know this thing adds adapters, so
I'll just clear them...
> 4. For each object, iterate over Adapters given by eAdapters and find
> instances of ItemProviderAdapter
> 5. Then call ItemProviderAdapter.unsetTarget(myEObject)
No, remove the adapters from the eAdapters list...
>
> Are there other approaches that I am not seeing? Also, if you
> recommend doing to process outlined above, do you foresee any
> undesired consequences when the object is adapted later (either by
> being viewed, or having another command executed against it)?
No, adapters are added on demand. If the object isn't being viewed,
there's no really need to be listening for notifications...
>
> On a related note, would the solution discussed here also be
> applicable for items viewed in JFace viewers (we stay with the
> AdapterFactoryContentProviders and AdapterFactoryLabelProviders as
> often as we can)? I would enjoy being able to unload objects from
> their ItemProviderAdapters when views are closed/disposed to lessen
> memory "leaks" as much as possible.
Calling unload will clear all adapters of all objects contained by the
resource.
Re: [EMF][CDO] Scalability and the recommended use of ItemProviderAdapters when using CDO [message #687724 is a reply to message #687718] Thu, 23 June 2011 05:08 Go to previous message
Josh M is currently offline Josh MFriend
Messages: 22
Registered: July 2009
Junior Member
Thanks so much for the help! I really appreciate the time you offer by making yourself so available. Thanks again!
Previous Topic:[CDO] Recovering from offline mode
Next Topic:EMF update site is outdated?
Goto Forum:
  


Current Time: Wed Sep 18 11:14:08 GMT 2019

Powered by FUDForum. Page generated in 0.02849 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top