Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » EMF » HibernateResource vs XmiResourceImpl performances ?
HibernateResource vs XmiResourceImpl performances ? [message #425073] Fri, 14 November 2008 14:28 Go to next message
Stephane  fournier is currently offline Stephane fournierFriend
Messages: 340
Registered: July 2009
Senior Member
Hi,

Testing Teneo, I've made some performances comparisons against a standard
XmiResource.

Scenario : I'm using the ExtendedLibrary meta-model. I have initialized a
mysql database (installed on my computer) with a library with 10 000
children : sub-libraries, books, writers, employees...

As a library can be contained by another library, Library meta-class is
not considered as a top class by Teneo.
So, loading my resource gives me an empty resource content. I query the
database to retrieve my root library (I tried to load the resource from an
uri that specifies the initial query, but the open editor is in read-only).

Some figures :
Empty resource loaded in : 11012ms
Root Library retrieved in : 16ms
Root library added in the empty resource : 15225ms.

I doesn't understand why it's so long to add the root library in the
HibernateResource.

Does anyone have an idea ?

I've made some additional tests against XmiResource.
If I add a sub-tree with 10 000 children to a library already contained in
a resource, it took 1s.
If I add a sub-tree with 10 000 children to a temporary library not
attached to a resource yet, it took 200ms + 63ms to attach the temporary
library to a library contained in a resource.
Does anyone have an idea to explain the performance differences ?

I export the database content as a xml file.
Export resource loaded in : 736ms

Retrieving the root library in the database is straight forward (16ms),
the library children are resolved (checked with the debugger just before
adding the root library to the resource content).

What is the Teneo option to avoid library children auto resolution when
querying for it.
I would like to perform the children resolution when expanding the
TreeItem only. (I set containment proxy to true in the genmodel). I tested
PersistenceOptions.SET_PROXY but I didn't speed up resource loading.

Stephane.
Re: HibernateResource vs XmiResourceImpl performances ? [message #425075 is a reply to message #425073] Fri, 14 November 2008 14:35 Go to previous messageGo to next message
Ed Merks is currently offline Ed MerksFriend
Messages: 33113
Registered: July 2009
Senior Member
Stéphane,

Comments below.


Stéphane Fournier wrote:
> Hi,
>
> Testing Teneo, I've made some performances comparisons against a
> standard XmiResource.
>
> Scenario : I'm using the ExtendedLibrary meta-model. I have
> initialized a mysql database (installed on my computer) with a library
> with 10 000 children : sub-libraries, books, writers, employees...
>
> As a library can be contained by another library, Library meta-class
> is not considered as a top class by Teneo. So, loading my resource
> gives me an empty resource content. I query the database to retrieve
> my root library (I tried to load the resource from an uri that
> specifies the initial query, but the open editor is in read-only).
>
> Some figures :
> Empty resource loaded in : 11012ms
> Root Library retrieved in : 16ms
> Root library added in the empty resource : 15225ms.
>
> I doesn't understand why it's so long to add the root library in the
> HibernateResource.
>
> Does anyone have an idea ?
>
> I've made some additional tests against XmiResource.
> If I add a sub-tree with 10 000 children to a library already
> contained in a resource, it took 1s.
> If I add a sub-tree with 10 000 children to a temporary library not
> attached to a resource yet, it took 200ms + 63ms to attach the
> temporary library to a library contained in a resource.
> Does anyone have an idea to explain the performance differences ?
How much work does ResourceImpl.attached doing? Is it walking the whole
tree and computing UUIDs or something like that?
>
> I export the database content as a xml file.
> Export resource loaded in : 736ms
>
> Retrieving the root library in the database is straight forward
> (16ms), the library children are resolved (checked with the debugger
> just before adding the root library to the resource content).
>
> What is the Teneo option to avoid library children auto resolution
> when querying for it.
> I would like to perform the children resolution when expanding the
> TreeItem only. (I set containment proxy to true in the genmodel). I
> tested PersistenceOptions.SET_PROXY but I didn't speed up resource
> loading.
>
> Stephane.
>
>
>
>
>


Ed Merks
Professional Support: https://www.macromodeling.com/
Re: HibernateResource vs XmiResourceImpl performances ? [message #425122 is a reply to message #425075] Fri, 14 November 2008 15:06 Go to previous messageGo to next message
Stephane  fournier is currently offline Stephane fournierFriend
Messages: 340
Registered: July 2009
Senior Member
Ed Merks wrote:

> Stéphane,

> Comments below.


> Stéphane Fournier wrote:
>> Hi,
>>
>> Testing Teneo, I've made some performances comparisons against a
>> standard XmiResource.
>>
>> Scenario : I'm using the ExtendedLibrary meta-model. I have
>> initialized a mysql database (installed on my computer) with a library
>> with 10 000 children : sub-libraries, books, writers, employees...
>>
>> As a library can be contained by another library, Library meta-class
>> is not considered as a top class by Teneo. So, loading my resource
>> gives me an empty resource content. I query the database to retrieve
>> my root library (I tried to load the resource from an uri that
>> specifies the initial query, but the open editor is in read-only).
>>
>> Some figures :
>> Empty resource loaded in : 11012ms
>> Root Library retrieved in : 16ms
>> Root library added in the empty resource : 15225ms.
>>
>> I doesn't understand why it's so long to add the root library in the
>> HibernateResource.
>>
>> Does anyone have an idea ?
>>
>> I've made some additional tests against XmiResource.
>> If I add a sub-tree with 10 000 children to a library already
>> contained in a resource, it took 1s.
>> If I add a sub-tree with 10 000 children to a temporary library not
>> attached to a resource yet, it took 200ms + 63ms to attach the
>> temporary library to a library contained in a resource.
>> Does anyone have an idea to explain the performance differences ?
> How much work does ResourceImpl.attached doing? Is it walking the whole
> tree and computing UUIDs or something like that?
>>
I'm not using UUIDs, I use the default behavior based on positions.
When I add 10 000 children to library contained in a resource,
ResourceImpl.attached is called for each object but that code is never
executed :
for (TreeIterator<EObject> tree = getAllProperContents(eObject);
tree.hasNext(); )
{
attachedHelper(tree.next());
}

When adding a sub tree containing 10 000 objects to a not attached
library, the ResourceImpl.attached is called once when the temporary lib
is added to a resource contained one but the this time, the TreeIterator
is walked to attach every contained objects.


>> I export the database content as a xml file.
>> Export resource loaded in : 736ms
>>
>> Retrieving the root library in the database is straight forward
>> (16ms), the library children are resolved (checked with the debugger
>> just before adding the root library to the resource content).
>>
>> What is the Teneo option to avoid library children auto resolution
>> when querying for it.
>> I would like to perform the children resolution when expanding the
>> TreeItem only. (I set containment proxy to true in the genmodel). I
>> tested PersistenceOptions.SET_PROXY but I didn't speed up resource
>> loading.
>>
>> Stephane.
>>
>>
>>
>>
>>
Re: HibernateResource vs XmiResourceImpl performances ? [message #425123 is a reply to message #425122] Fri, 14 November 2008 15:17 Go to previous messageGo to next message
Ed Merks is currently offline Ed MerksFriend
Messages: 33113
Registered: July 2009
Senior Member
Stéphane,

So the tree walk is the answer. Can you tell why it's needing to walk
the tree? It's guarded such that the tree walk is avoided in most cases...


Stéphane Fournier wrote:
> Ed Merks wrote:
>
>> Stéphane,
>
>> Comments below.
>
>
>> Stéphane Fournier wrote:
>>> Hi,
>>>
>>> Testing Teneo, I've made some performances comparisons against a
>>> standard XmiResource.
>>>
>>> Scenario : I'm using the ExtendedLibrary meta-model. I have
>>> initialized a mysql database (installed on my computer) with a
>>> library with 10 000 children : sub-libraries, books, writers,
>>> employees...
>>>
>>> As a library can be contained by another library, Library meta-class
>>> is not considered as a top class by Teneo. So, loading my resource
>>> gives me an empty resource content. I query the database to retrieve
>>> my root library (I tried to load the resource from an uri that
>>> specifies the initial query, but the open editor is in read-only).
>>>
>>> Some figures :
>>> Empty resource loaded in : 11012ms
>>> Root Library retrieved in : 16ms
>>> Root library added in the empty resource : 15225ms.
>>>
>>> I doesn't understand why it's so long to add the root library in the
>>> HibernateResource.
>>>
>>> Does anyone have an idea ?
>>>
>>> I've made some additional tests against XmiResource.
>>> If I add a sub-tree with 10 000 children to a library already
>>> contained in a resource, it took 1s.
>>> If I add a sub-tree with 10 000 children to a temporary library not
>>> attached to a resource yet, it took 200ms + 63ms to attach the
>>> temporary library to a library contained in a resource.
>>> Does anyone have an idea to explain the performance differences ?
>> How much work does ResourceImpl.attached doing? Is it walking the
>> whole tree and computing UUIDs or something like that?
>>>
> I'm not using UUIDs, I use the default behavior based on positions.
> When I add 10 000 children to library contained in a resource,
> ResourceImpl.attached is called for each object but that code is never
> executed :
> for (TreeIterator<EObject> tree = getAllProperContents(eObject);
> tree.hasNext(); )
> {
> attachedHelper(tree.next());
> }
>
> When adding a sub tree containing 10 000 objects to a not attached
> library, the ResourceImpl.attached is called once when the temporary
> lib is added to a resource contained one but the this time, the
> TreeIterator is walked to attach every contained objects.
>
>
>>> I export the database content as a xml file.
>>> Export resource loaded in : 736ms
>>>
>>> Retrieving the root library in the database is straight forward
>>> (16ms), the library children are resolved (checked with the debugger
>>> just before adding the root library to the resource content).
>>>
>>> What is the Teneo option to avoid library children auto resolution
>>> when querying for it.
>>> I would like to perform the children resolution when expanding the
>>> TreeItem only. (I set containment proxy to true in the genmodel). I
>>> tested PersistenceOptions.SET_PROXY but I didn't speed up resource
>>> loading.
>>>
>>> Stephane.
>>>
>>>
>>>
>>>
>>>
>
>


Ed Merks
Professional Support: https://www.macromodeling.com/
Re: HibernateResource vs XmiResourceImpl performances ? [message #425126 is a reply to message #425123] Fri, 14 November 2008 15:41 Go to previous messageGo to next message
Stephane  fournier is currently offline Stephane fournierFriend
Messages: 340
Registered: July 2009
Senior Member
Ed Merks wrote:

> Stéphane,

> So the tree walk is the answer. Can you tell why it's needing to walk
> the tree? It's guarded such that the tree walk is avoided in most cases...

Here is my analysis :
The tree is walked (on the added sub tree) because when I add a temp lib
with a sub tree containing my 10 000 children, no one is attached to a
resource.
When adding the temp lib in a lib attached to a resource, the
EObjectContainmentWithInverseEList.Resolving#add calls the opposite
relation i.e setParentBranch(..) that calls eBasicSetContainer(..) that
calls ResourceImpl.attached(..).
ResourceImpl.getAllProperContents returns the added sub tree objects that
are not attached to the parent library resource yet.

Nevertheless, my tests show that's worth calling once
ResourceImpl.attached() with a tree walk on children rather than calling x
times this methods without tree walk.


> Stéphane Fournier wrote:
>> Ed Merks wrote:
>>
>>> Stéphane,
>>
>>> Comments below.
>>
>>
>>> Stéphane Fournier wrote:
>>>> Hi,
>>>>
>>>> Testing Teneo, I've made some performances comparisons against a
>>>> standard XmiResource.
>>>>
>>>> Scenario : I'm using the ExtendedLibrary meta-model. I have
>>>> initialized a mysql database (installed on my computer) with a
>>>> library with 10 000 children : sub-libraries, books, writers,
>>>> employees...
>>>>
>>>> As a library can be contained by another library, Library meta-class
>>>> is not considered as a top class by Teneo. So, loading my resource
>>>> gives me an empty resource content. I query the database to retrieve
>>>> my root library (I tried to load the resource from an uri that
>>>> specifies the initial query, but the open editor is in read-only).
>>>>
>>>> Some figures :
>>>> Empty resource loaded in : 11012ms
>>>> Root Library retrieved in : 16ms
>>>> Root library added in the empty resource : 15225ms.
>>>>
>>>> I doesn't understand why it's so long to add the root library in the
>>>> HibernateResource.
>>>>
>>>> Does anyone have an idea ?
>>>>
>>>> I've made some additional tests against XmiResource.
>>>> If I add a sub-tree with 10 000 children to a library already
>>>> contained in a resource, it took 1s.
>>>> If I add a sub-tree with 10 000 children to a temporary library not
>>>> attached to a resource yet, it took 200ms + 63ms to attach the
>>>> temporary library to a library contained in a resource.
>>>> Does anyone have an idea to explain the performance differences ?
>>> How much work does ResourceImpl.attached doing? Is it walking the
>>> whole tree and computing UUIDs or something like that?
>>>>
>> I'm not using UUIDs, I use the default behavior based on positions.
>> When I add 10 000 children to library contained in a resource,
>> ResourceImpl.attached is called for each object but that code is never
>> executed :
>> for (TreeIterator<EObject> tree = getAllProperContents(eObject);
>> tree.hasNext(); )
>> {
>> attachedHelper(tree.next());
>> }
>>
>> When adding a sub tree containing 10 000 objects to a not attached
>> library, the ResourceImpl.attached is called once when the temporary
>> lib is added to a resource contained one but the this time, the
>> TreeIterator is walked to attach every contained objects.
>>
>>
>>>> I export the database content as a xml file.
>>>> Export resource loaded in : 736ms
>>>>
>>>> Retrieving the root library in the database is straight forward
>>>> (16ms), the library children are resolved (checked with the debugger
>>>> just before adding the root library to the resource content).
>>>>
>>>> What is the Teneo option to avoid library children auto resolution
>>>> when querying for it.
>>>> I would like to perform the children resolution when expanding the
>>>> TreeItem only. (I set containment proxy to true in the genmodel). I
>>>> tested PersistenceOptions.SET_PROXY but I didn't speed up resource
>>>> loading.
>>>>
>>>> Stephane.
>>>>
>>>>
>>>>
>>>>
>>>>
>>
>>
Re: HibernateResource vs XmiResourceImpl performances ? [message #425129 is a reply to message #425126] Fri, 14 November 2008 16:03 Go to previous messageGo to next message
Ed Merks is currently offline Ed MerksFriend
Messages: 33113
Registered: July 2009
Senior Member
This is a multi-part message in MIME format.
--------------040709090002050605050804
Content-Type: text/plain; charset=ISO-8859-15; format=flowed
Content-Transfer-Encoding: 8bit

St


Ed Merks
Professional Support: https://www.macromodeling.com/
Re: HibernateResource vs XmiResourceImpl performances ? [message #425130 is a reply to message #425129] Fri, 14 November 2008 16:19 Go to previous messageGo to next message
Stephane  fournier is currently offline Stephane fournierFriend
Messages: 340
Registered: July 2009
Senior Member
because in
protected boolean isAttachedDetachedHelperRequired()
{
return useIDs() || super.isAttachedDetachedHelperRequired();
}

useIDs() always returns true in XmlResourceImpl. In the genmodel,
ResourceType is set to NONE.

Ed Merks wrote:

> Stéphane,

> Sorry my point is it's guarded like this:

> public void attached(EObject eObject)
> {
> if (isAttachedDetachedHelperRequired())
> {
> attachedHelper(eObject);
> for (TreeIterator<EObject> tree =
> getAllProperContents(eObject); tree.hasNext(); )
> {
> attachedHelper(tree.next());
> }
> }
> }


> protected boolean isAttachedDetachedHelperRequired()
> {
> return useIDs() || super.isAttachedDetachedHelperRequired();
> }

> protected boolean useIDs()
> {
> return eObjectToIDMap != null || idToEObjectMap != null || useUUIDs();
> }

> protected boolean useIDAttributes()
> {
> return true;
> }

> protected boolean useUUIDs()
> {
> return false;
> }

> protected boolean isAttachedDetachedHelperRequired()
> {
> return isTrackingModification() || getIntrinsicIDToEObjectMap() != null;
> }


> So generally I would expect the guard to return false by default. So
> I'm wondering why it's returning true.



> Stéphane Fournier wrote:
>> Ed Merks wrote:
>>
>>> Stéphane,
>>
>>> So the tree walk is the answer. Can you tell why it's needing to
>>> walk the tree? It's guarded such that the tree walk is avoided in
>>> most cases...
>>
>> Here is my analysis :
>> The tree is walked (on the added sub tree) because when I add a temp
>> lib with a sub tree containing my 10 000 children, no one is attached
>> to a resource.
>> When adding the temp lib in a lib attached to a resource, the
>> EObjectContainmentWithInverseEList.Resolving#add calls the opposite
>> relation i.e setParentBranch(..) that calls eBasicSetContainer(..)
>> that calls ResourceImpl.attached(..).
>> ResourceImpl.getAllProperContents returns the added sub tree objects
>> that are not attached to the parent library resource yet.
>>
>> Nevertheless, my tests show that's worth calling once
>> ResourceImpl.attached() with a tree walk on children rather than
>> calling x times this methods without tree walk.
>>
>>
>>> Stéphane Fournier wrote:
>>>> Ed Merks wrote:
>>>>
>>>>> Stéphane,
>>>>
>>>>> Comments below.
>>>>
>>>>
>>>>> Stéphane Fournier wrote:
>>>>>> Hi,
>>>>>>
>>>>>> Testing Teneo, I've made some performances comparisons against a
>>>>>> standard XmiResource.
>>>>>>
>>>>>> Scenario : I'm using the ExtendedLibrary meta-model. I have
>>>>>> initialized a mysql database (installed on my computer) with a
>>>>>> library with 10 000 children : sub-libraries, books, writers,
>>>>>> employees...
>>>>>>
>>>>>> As a library can be contained by another library, Library
>>>>>> meta-class is not considered as a top class by Teneo. So, loading
>>>>>> my resource gives me an empty resource content. I query the
>>>>>> database to retrieve my root library (I tried to load the resource
>>>>>> from an uri that specifies the initial query, but the open editor
>>>>>> is in read-only).
>>>>>>
>>>>>> Some figures :
>>>>>> Empty resource loaded in : 11012ms
>>>>>> Root Library retrieved in : 16ms
>>>>>> Root library added in the empty resource : 15225ms.
>>>>>>
>>>>>> I doesn't understand why it's so long to add the root library in
>>>>>> the HibernateResource.
>>>>>>
>>>>>> Does anyone have an idea ?
>>>>>>
>>>>>> I've made some additional tests against XmiResource.
>>>>>> If I add a sub-tree with 10 000 children to a library already
>>>>>> contained in a resource, it took 1s.
>>>>>> If I add a sub-tree with 10 000 children to a temporary library
>>>>>> not attached to a resource yet, it took 200ms + 63ms to attach the
>>>>>> temporary library to a library contained in a resource.
>>>>>> Does anyone have an idea to explain the performance differences ?
>>>>> How much work does ResourceImpl.attached doing? Is it walking the
>>>>> whole tree and computing UUIDs or something like that?
>>>>>>
>>>> I'm not using UUIDs, I use the default behavior based on positions.
>>>> When I add 10 000 children to library contained in a resource,
>>>> ResourceImpl.attached is called for each object but that code is
>>>> never executed :
>>>> for (TreeIterator<EObject> tree = getAllProperContents(eObject);
>>>> tree.hasNext(); )
>>>> {
>>>> attachedHelper(tree.next());
>>>> }
>>>>
>>>> When adding a sub tree containing 10 000 objects to a not attached
>>>> library, the ResourceImpl.attached is called once when the temporary
>>>> lib is added to a resource contained one but the this time, the
>>>> TreeIterator is walked to attach every contained objects.
>>>>
>>>>
>>>>>> I export the database content as a xml file.
>>>>>> Export resource loaded in : 736ms
>>>>>>
>>>>>> Retrieving the root library in the database is straight forward
>>>>>> (16ms), the library children are resolved (checked with the
>>>>>> debugger just before adding the root library to the resource
>>>>>> content).
>>>>>>
>>>>>> What is the Teneo option to avoid library children auto resolution
>>>>>> when querying for it.
>>>>>> I would like to perform the children resolution when expanding the
>>>>>> TreeItem only. (I set containment proxy to true in the genmodel).
>>>>>> I tested PersistenceOptions.SET_PROXY but I didn't speed up
>>>>>> resource loading.
>>>>>>
>>>>>> Stephane.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>
>>>>
>>
>>
Re: HibernateResource vs XmiResourceImpl performances ? [message #425131 is a reply to message #425130] Fri, 14 November 2008 16:27 Go to previous messageGo to next message
Ed Merks is currently offline Ed MerksFriend
Messages: 33113
Registered: July 2009
Senior Member
Stéphane,

The assumption being that XMI resources will generally use xmi:id which
in strict conformance to the XMI specification is required. In any
case, this is clearly something you can specialize. I.e., you might
just use XMLResourceImpl instead...


Stéphane Fournier wrote:
> because in
> protected boolean isAttachedDetachedHelperRequired()
> {
> return useIDs() || super.isAttachedDetachedHelperRequired();
> }
>
> useIDs() always returns true in XmlResourceImpl. In the genmodel,
> ResourceType is set to NONE.
>
> Ed Merks wrote:
>
>> Stéphane,
>
>> Sorry my point is it's guarded like this:
>
>> public void attached(EObject eObject)
>> {
>> if (isAttachedDetachedHelperRequired())
>> {
>> attachedHelper(eObject);
>> for (TreeIterator<EObject> tree =
>> getAllProperContents(eObject); tree.hasNext(); )
>> {
>> attachedHelper(tree.next());
>> }
>> }
>> }
>
>
>> protected boolean isAttachedDetachedHelperRequired()
>> {
>> return useIDs() || super.isAttachedDetachedHelperRequired();
>> }
>
>> protected boolean useIDs()
>> {
>> return eObjectToIDMap != null || idToEObjectMap != null ||
>> useUUIDs();
>> }
>
>> protected boolean useIDAttributes()
>> {
>> return true;
>> }
>
>> protected boolean useUUIDs()
>> {
>> return false;
>> }
>
>> protected boolean isAttachedDetachedHelperRequired()
>> {
>> return isTrackingModification() || getIntrinsicIDToEObjectMap()
>> != null;
>> }
>
>
>> So generally I would expect the guard to return false by default. So
>> I'm wondering why it's returning true.
>
>
>
>> Stéphane Fournier wrote:
>>> Ed Merks wrote:
>>>
>>>> Stéphane,
>>>
>>>> So the tree walk is the answer. Can you tell why it's needing to
>>>> walk the tree? It's guarded such that the tree walk is avoided in
>>>> most cases...
>>>
>>> Here is my analysis :
>>> The tree is walked (on the added sub tree) because when I add a temp
>>> lib with a sub tree containing my 10 000 children, no one is
>>> attached to a resource.
>>> When adding the temp lib in a lib attached to a resource, the
>>> EObjectContainmentWithInverseEList.Resolving#add calls the opposite
>>> relation i.e setParentBranch(..) that calls eBasicSetContainer(..)
>>> that calls ResourceImpl.attached(..).
>>> ResourceImpl.getAllProperContents returns the added sub tree objects
>>> that are not attached to the parent library resource yet.
>>>
>>> Nevertheless, my tests show that's worth calling once
>>> ResourceImpl.attached() with a tree walk on children rather than
>>> calling x times this methods without tree walk.
>>>
>>>
>>>> Stéphane Fournier wrote:
>>>>> Ed Merks wrote:
>>>>>
>>>>>> Stéphane,
>>>>>
>>>>>> Comments below.
>>>>>
>>>>>
>>>>>> Stéphane Fournier wrote:
>>>>>>> Hi,
>>>>>>>
>>>>>>> Testing Teneo, I've made some performances comparisons against a
>>>>>>> standard XmiResource.
>>>>>>>
>>>>>>> Scenario : I'm using the ExtendedLibrary meta-model. I have
>>>>>>> initialized a mysql database (installed on my computer) with a
>>>>>>> library with 10 000 children : sub-libraries, books, writers,
>>>>>>> employees...
>>>>>>>
>>>>>>> As a library can be contained by another library, Library
>>>>>>> meta-class is not considered as a top class by Teneo. So,
>>>>>>> loading my resource gives me an empty resource content. I query
>>>>>>> the database to retrieve my root library (I tried to load the
>>>>>>> resource from an uri that specifies the initial query, but the
>>>>>>> open editor is in read-only).
>>>>>>>
>>>>>>> Some figures :
>>>>>>> Empty resource loaded in : 11012ms
>>>>>>> Root Library retrieved in : 16ms
>>>>>>> Root library added in the empty resource : 15225ms.
>>>>>>>
>>>>>>> I doesn't understand why it's so long to add the root library in
>>>>>>> the HibernateResource.
>>>>>>>
>>>>>>> Does anyone have an idea ?
>>>>>>>
>>>>>>> I've made some additional tests against XmiResource.
>>>>>>> If I add a sub-tree with 10 000 children to a library already
>>>>>>> contained in a resource, it took 1s.
>>>>>>> If I add a sub-tree with 10 000 children to a temporary library
>>>>>>> not attached to a resource yet, it took 200ms + 63ms to attach
>>>>>>> the temporary library to a library contained in a resource.
>>>>>>> Does anyone have an idea to explain the performance differences ?
>>>>>> How much work does ResourceImpl.attached doing? Is it walking
>>>>>> the whole tree and computing UUIDs or something like that?
>>>>>>>
>>>>> I'm not using UUIDs, I use the default behavior based on positions.
>>>>> When I add 10 000 children to library contained in a resource,
>>>>> ResourceImpl.attached is called for each object but that code is
>>>>> never executed :
>>>>> for (TreeIterator<EObject> tree = getAllProperContents(eObject);
>>>>> tree.hasNext(); )
>>>>> {
>>>>> attachedHelper(tree.next());
>>>>> }
>>>>>
>>>>> When adding a sub tree containing 10 000 objects to a not attached
>>>>> library, the ResourceImpl.attached is called once when the
>>>>> temporary lib is added to a resource contained one but the this
>>>>> time, the TreeIterator is walked to attach every contained objects.
>>>>>
>>>>>
>>>>>>> I export the database content as a xml file.
>>>>>>> Export resource loaded in : 736ms
>>>>>>>
>>>>>>> Retrieving the root library in the database is straight forward
>>>>>>> (16ms), the library children are resolved (checked with the
>>>>>>> debugger just before adding the root library to the resource
>>>>>>> content).
>>>>>>>
>>>>>>> What is the Teneo option to avoid library children auto
>>>>>>> resolution when querying for it.
>>>>>>> I would like to perform the children resolution when expanding
>>>>>>> the TreeItem only. (I set containment proxy to true in the
>>>>>>> genmodel). I tested PersistenceOptions.SET_PROXY but I didn't
>>>>>>> speed up resource loading.
>>>>>>>
>>>>>>> Stephane.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>
>>>>>
>>>
>>>
>
>


Ed Merks
Professional Support: https://www.macromodeling.com/
Re: HibernateResource vs XmiResourceImpl performances ? [message #425132 is a reply to message #425131] Fri, 14 November 2008 17:16 Go to previous messageGo to next message
Stephane  fournier is currently offline Stephane fournierFriend
Messages: 340
Registered: July 2009
Senior Member
Ed Merks wrote:

> Stéphane,

> The assumption being that XMI resources will generally use xmi:id which
> in strict conformance to the XMI specification is required. In any
> case, this is clearly something you can specialize. I.e., you might
> just use XMLResourceImpl instead...

I've made the change, now the guard does its job i.e the code in the guard
block is never executed.
I've ran my tests again, it does not change a lot the overall performances.
Creating a sub tree and adding it at creation end to a library attached to
a resource is always the fastest way.

Nevertheless, performances with standard resources either XMI or XML are
great to me :)

My issue is regarding HibernateResource. I do hope that Martin could give
a hint to improve that !

Stephane.

> Stéphane Fournier wrote:
>> because in
>> protected boolean isAttachedDetachedHelperRequired()
>> {
>> return useIDs() || super.isAttachedDetachedHelperRequired();
>> }
>>
>> useIDs() always returns true in XmlResourceImpl. In the genmodel,
>> ResourceType is set to NONE.
>>
>> Ed Merks wrote:
>>
>>> Stéphane,
>>
>>> Sorry my point is it's guarded like this:
>>
>>> public void attached(EObject eObject)
>>> {
>>> if (isAttachedDetachedHelperRequired())
>>> {
>>> attachedHelper(eObject);
>>> for (TreeIterator<EObject> tree =
>>> getAllProperContents(eObject); tree.hasNext(); )
>>> {
>>> attachedHelper(tree.next());
>>> }
>>> }
>>> }
>>
>>
>>> protected boolean isAttachedDetachedHelperRequired()
>>> {
>>> return useIDs() || super.isAttachedDetachedHelperRequired();
>>> }
>>
>>> protected boolean useIDs()
>>> {
>>> return eObjectToIDMap != null || idToEObjectMap != null ||
>>> useUUIDs();
>>> }
>>
>>> protected boolean useIDAttributes()
>>> {
>>> return true;
>>> }
>>
>>> protected boolean useUUIDs()
>>> {
>>> return false;
>>> }
>>
>>> protected boolean isAttachedDetachedHelperRequired()
>>> {
>>> return isTrackingModification() || getIntrinsicIDToEObjectMap()
>>> != null;
>>> }
>>
>>
>>> So generally I would expect the guard to return false by default. So
>>> I'm wondering why it's returning true.
>>
>>
>>
>>> Stéphane Fournier wrote:
>>>> Ed Merks wrote:
>>>>
>>>>> Stéphane,
>>>>
>>>>> So the tree walk is the answer. Can you tell why it's needing to
>>>>> walk the tree? It's guarded such that the tree walk is avoided in
>>>>> most cases...
>>>>
>>>> Here is my analysis :
>>>> The tree is walked (on the added sub tree) because when I add a temp
>>>> lib with a sub tree containing my 10 000 children, no one is
>>>> attached to a resource.
>>>> When adding the temp lib in a lib attached to a resource, the
>>>> EObjectContainmentWithInverseEList.Resolving#add calls the opposite
>>>> relation i.e setParentBranch(..) that calls eBasicSetContainer(..)
>>>> that calls ResourceImpl.attached(..).
>>>> ResourceImpl.getAllProperContents returns the added sub tree objects
>>>> that are not attached to the parent library resource yet.
>>>>
>>>> Nevertheless, my tests show that's worth calling once
>>>> ResourceImpl.attached() with a tree walk on children rather than
>>>> calling x times this methods without tree walk.
>>>>
>>>>
>>>>> Stéphane Fournier wrote:
>>>>>> Ed Merks wrote:
>>>>>>
>>>>>>> Stéphane,
>>>>>>
>>>>>>> Comments below.
>>>>>>
>>>>>>
>>>>>>> Stéphane Fournier wrote:
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> Testing Teneo, I've made some performances comparisons against a
>>>>>>>> standard XmiResource.
>>>>>>>>
>>>>>>>> Scenario : I'm using the ExtendedLibrary meta-model. I have
>>>>>>>> initialized a mysql database (installed on my computer) with a
>>>>>>>> library with 10 000 children : sub-libraries, books, writers,
>>>>>>>> employees...
>>>>>>>>
>>>>>>>> As a library can be contained by another library, Library
>>>>>>>> meta-class is not considered as a top class by Teneo. So,
>>>>>>>> loading my resource gives me an empty resource content. I query
>>>>>>>> the database to retrieve my root library (I tried to load the
>>>>>>>> resource from an uri that specifies the initial query, but the
>>>>>>>> open editor is in read-only).
>>>>>>>>
>>>>>>>> Some figures :
>>>>>>>> Empty resource loaded in : 11012ms
>>>>>>>> Root Library retrieved in : 16ms
>>>>>>>> Root library added in the empty resource : 15225ms.
>>>>>>>>
>>>>>>>> I doesn't understand why it's so long to add the root library in
>>>>>>>> the HibernateResource.
>>>>>>>>
>>>>>>>> Does anyone have an idea ?
>>>>>>>>
>>>>>>>> I've made some additional tests against XmiResource.
>>>>>>>> If I add a sub-tree with 10 000 children to a library already
>>>>>>>> contained in a resource, it took 1s.
>>>>>>>> If I add a sub-tree with 10 000 children to a temporary library
>>>>>>>> not attached to a resource yet, it took 200ms + 63ms to attach
>>>>>>>> the temporary library to a library contained in a resource.
>>>>>>>> Does anyone have an idea to explain the performance differences ?
>>>>>>> How much work does ResourceImpl.attached doing? Is it walking
>>>>>>> the whole tree and computing UUIDs or something like that?
>>>>>>>>
>>>>>> I'm not using UUIDs, I use the default behavior based on positions.
>>>>>> When I add 10 000 children to library contained in a resource,
>>>>>> ResourceImpl.attached is called for each object but that code is
>>>>>> never executed :
>>>>>> for (TreeIterator<EObject> tree = getAllProperContents(eObject);
>>>>>> tree.hasNext(); )
>>>>>> {
>>>>>> attachedHelper(tree.next());
>>>>>> }
>>>>>>
>>>>>> When adding a sub tree containing 10 000 objects to a not attached
>>>>>> library, the ResourceImpl.attached is called once when the
>>>>>> temporary lib is added to a resource contained one but the this
>>>>>> time, the TreeIterator is walked to attach every contained objects.
>>>>>>
>>>>>>
>>>>>>>> I export the database content as a xml file.
>>>>>>>> Export resource loaded in : 736ms
>>>>>>>>
>>>>>>>> Retrieving the root library in the database is straight forward
>>>>>>>> (16ms), the library children are resolved (checked with the
>>>>>>>> debugger just before adding the root library to the resource
>>>>>>>> content).
>>>>>>>>
>>>>>>>> What is the Teneo option to avoid library children auto
>>>>>>>> resolution when querying for it.
>>>>>>>> I would like to perform the children resolution when expanding
>>>>>>>> the TreeItem only. (I set containment proxy to true in the
>>>>>>>> genmodel). I tested PersistenceOptions.SET_PROXY but I didn't
>>>>>>>> speed up resource loading.
>>>>>>>>
>>>>>>>> Stephane.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>>
>>>>
>>>>
>>
>>
Re: HibernateResource vs XmiResourceImpl performances ? [message #425134 is a reply to message #425132] Fri, 14 November 2008 21:43 Go to previous messageGo to next message
Martin Taal is currently offline Martin TaalFriend
Messages: 5468
Registered: July 2009
Senior Member
Hi Stephane,
The use case is not completely clear for me yet. I understand this:
1) you create a HibernateResource and load it, it remains empty
2) you use a separate hibernate session to read a library from the db and add it to the hibernate
resource?

Then you have two scenarios:
A- read a library and add it to the resource, then read a second library (with the 10000 children)
and add it to the library which is already present in the resource?, or
B- read a library (with the 10000 children) from the db and add it to the resource directly

or did I miss anything?

In which scenario do you see which tree-walking behavior?

gr. Martin

Stéphane Fournier wrote:
> Ed Merks wrote:
>
>> Stéphane,
>
>> The assumption being that XMI resources will generally use xmi:id
>> which in strict conformance to the XMI specification is required. In
>> any case, this is clearly something you can specialize. I.e., you
>> might just use XMLResourceImpl instead...
>
> I've made the change, now the guard does its job i.e the code in the
> guard block is never executed.
> I've ran my tests again, it does not change a lot the overall performances.
> Creating a sub tree and adding it at creation end to a library attached
> to a resource is always the fastest way.
>
> Nevertheless, performances with standard resources either XMI or XML are
> great to me :)
>
> My issue is regarding HibernateResource. I do hope that Martin could
> give a hint to improve that !
>
> Stephane.
>
>> Stéphane Fournier wrote:
>>> because in
>>> protected boolean isAttachedDetachedHelperRequired()
>>> {
>>> return useIDs() || super.isAttachedDetachedHelperRequired();
>>> }
>>>
>>> useIDs() always returns true in XmlResourceImpl. In the genmodel,
>>> ResourceType is set to NONE.
>>>
>>> Ed Merks wrote:
>>>
>>>> Stéphane,
>>>
>>>> Sorry my point is it's guarded like this:
>>>
>>>> public void attached(EObject eObject)
>>>> {
>>>> if (isAttachedDetachedHelperRequired())
>>>> {
>>>> attachedHelper(eObject);
>>>> for (TreeIterator<EObject> tree =
>>>> getAllProperContents(eObject); tree.hasNext(); )
>>>> {
>>>> attachedHelper(tree.next());
>>>> }
>>>> }
>>>> }
>>>
>>>
>>>> protected boolean isAttachedDetachedHelperRequired()
>>>> {
>>>> return useIDs() || super.isAttachedDetachedHelperRequired();
>>>> }
>>>
>>>> protected boolean useIDs()
>>>> {
>>>> return eObjectToIDMap != null || idToEObjectMap != null ||
>>>> useUUIDs();
>>>> }
>>>
>>>> protected boolean useIDAttributes()
>>>> {
>>>> return true;
>>>> }
>>>
>>>> protected boolean useUUIDs()
>>>> {
>>>> return false;
>>>> }
>>>
>>>> protected boolean isAttachedDetachedHelperRequired()
>>>> {
>>>> return isTrackingModification() || getIntrinsicIDToEObjectMap()
>>>> != null;
>>>> }
>>>
>>>
>>>> So generally I would expect the guard to return false by default.
>>>> So I'm wondering why it's returning true.
>>>
>>>
>>>
>>>> Stéphane Fournier wrote:
>>>>> Ed Merks wrote:
>>>>>
>>>>>> Stéphane,
>>>>>
>>>>>> So the tree walk is the answer. Can you tell why it's needing to
>>>>>> walk the tree? It's guarded such that the tree walk is avoided in
>>>>>> most cases...
>>>>>
>>>>> Here is my analysis :
>>>>> The tree is walked (on the added sub tree) because when I add a
>>>>> temp lib with a sub tree containing my 10 000 children, no one is
>>>>> attached to a resource.
>>>>> When adding the temp lib in a lib attached to a resource, the
>>>>> EObjectContainmentWithInverseEList.Resolving#add calls the opposite
>>>>> relation i.e setParentBranch(..) that calls eBasicSetContainer(..)
>>>>> that calls ResourceImpl.attached(..).
>>>>> ResourceImpl.getAllProperContents returns the added sub tree
>>>>> objects that are not attached to the parent library resource yet.
>>>>>
>>>>> Nevertheless, my tests show that's worth calling once
>>>>> ResourceImpl.attached() with a tree walk on children rather than
>>>>> calling x times this methods without tree walk.
>>>>>
>>>>>
>>>>>> Stéphane Fournier wrote:
>>>>>>> Ed Merks wrote:
>>>>>>>
>>>>>>>> Stéphane,
>>>>>>>
>>>>>>>> Comments below.
>>>>>>>
>>>>>>>
>>>>>>>> Stéphane Fournier wrote:
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> Testing Teneo, I've made some performances comparisons against
>>>>>>>>> a standard XmiResource.
>>>>>>>>>
>>>>>>>>> Scenario : I'm using the ExtendedLibrary meta-model. I have
>>>>>>>>> initialized a mysql database (installed on my computer) with a
>>>>>>>>> library with 10 000 children : sub-libraries, books, writers,
>>>>>>>>> employees...
>>>>>>>>>
>>>>>>>>> As a library can be contained by another library, Library
>>>>>>>>> meta-class is not considered as a top class by Teneo. So,
>>>>>>>>> loading my resource gives me an empty resource content. I query
>>>>>>>>> the database to retrieve my root library (I tried to load the
>>>>>>>>> resource from an uri that specifies the initial query, but the
>>>>>>>>> open editor is in read-only).
>>>>>>>>>
>>>>>>>>> Some figures :
>>>>>>>>> Empty resource loaded in : 11012ms
>>>>>>>>> Root Library retrieved in : 16ms
>>>>>>>>> Root library added in the empty resource : 15225ms.
>>>>>>>>>
>>>>>>>>> I doesn't understand why it's so long to add the root library
>>>>>>>>> in the HibernateResource.
>>>>>>>>>
>>>>>>>>> Does anyone have an idea ?
>>>>>>>>>
>>>>>>>>> I've made some additional tests against XmiResource.
>>>>>>>>> If I add a sub-tree with 10 000 children to a library already
>>>>>>>>> contained in a resource, it took 1s.
>>>>>>>>> If I add a sub-tree with 10 000 children to a temporary library
>>>>>>>>> not attached to a resource yet, it took 200ms + 63ms to attach
>>>>>>>>> the temporary library to a library contained in a resource.
>>>>>>>>> Does anyone have an idea to explain the performance differences ?
>>>>>>>> How much work does ResourceImpl.attached doing? Is it walking
>>>>>>>> the whole tree and computing UUIDs or something like that?
>>>>>>>>>
>>>>>>> I'm not using UUIDs, I use the default behavior based on positions.
>>>>>>> When I add 10 000 children to library contained in a resource,
>>>>>>> ResourceImpl.attached is called for each object but that code is
>>>>>>> never executed :
>>>>>>> for (TreeIterator<EObject> tree = getAllProperContents(eObject);
>>>>>>> tree.hasNext(); )
>>>>>>> {
>>>>>>> attachedHelper(tree.next());
>>>>>>> }
>>>>>>>
>>>>>>> When adding a sub tree containing 10 000 objects to a not
>>>>>>> attached library, the ResourceImpl.attached is called once when
>>>>>>> the temporary lib is added to a resource contained one but the
>>>>>>> this time, the TreeIterator is walked to attach every contained
>>>>>>> objects.
>>>>>>>
>>>>>>>
>>>>>>>>> I export the database content as a xml file.
>>>>>>>>> Export resource loaded in : 736ms
>>>>>>>>>
>>>>>>>>> Retrieving the root library in the database is straight forward
>>>>>>>>> (16ms), the library children are resolved (checked with the
>>>>>>>>> debugger just before adding the root library to the resource
>>>>>>>>> content).
>>>>>>>>>
>>>>>>>>> What is the Teneo option to avoid library children auto
>>>>>>>>> resolution when querying for it.
>>>>>>>>> I would like to perform the children resolution when expanding
>>>>>>>>> the TreeItem only. (I set containment proxy to true in the
>>>>>>>>> genmodel). I tested PersistenceOptions.SET_PROXY but I didn't
>>>>>>>>> speed up resource loading.
>>>>>>>>>
>>>>>>>>> Stephane.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>
>>>>>
>>>
>>>
>
>


--

With Regards, Martin Taal

Springsite/Elver.org
Office: Hardwareweg 4, 3821 BV Amersfoort
Postal: Nassaulaan 7, 3941 EC Doorn
The Netherlands
Cell: +31 (0)6 288 48 943
Tel: +31 (0)84 420 2397
Fax: +31 (0)84 225 9307
Mail: mtaal@springsite.com - mtaal@elver.org
Web: www.springsite.com - www.elver.org
Re: HibernateResource vs XmiResourceImpl performances ? [message #425139 is a reply to message #425134] Sat, 15 November 2008 17:53 Go to previous messageGo to next message
Stephane  fournier is currently offline Stephane fournierFriend
Messages: 340
Registered: July 2009
Senior Member
Hi Martin,

1st scenario :

1) I create a HibernateResource that remains empty due to my meta-model
(ExtendedLibrary). I have no top class because a Library can contain
another one.
The elapsed time to load my empty HibernateResource is long from my
point of view : 11s more or less.

2) On the SessionWrapper provided by the loaded resource, I open a
transaction.
I query the db to find my root library (the db contains a root library
with 10 000 children).
The query is really fast to retrieve the library : 16ms.

3) Then, I add this root library to the empty loaded HibernateResource
content.
It took 15s more or less. It's long because all library children objects
are already resolved (checked with the debugger). I haven't understood
where time is wasted.

4) I close the transaction.
5)My editor finishes to open with this resource.


2nd scenario (the editor is open with the root library ant its 10 000
children):
1) I select a library either a child of the root one or the root one itself.
2) I create 10 000 children on the selected library.
3) I save the HibernateResource.

As performances were not good (for me), I made some tests with standard
EMF resources to have an idea of standard performances with XML files.

In doing theses tests with XML resources, I got better performances if I
create objects as children of an object not attached to a resource. At
the end, I add the not attached object to an object contained in a resource.
Ed and I discussed this point in this post.

Stephane.


Martin Taal wrote:
> Hi Stephane,
> The use case is not completely clear for me yet. I understand this:
> 1) you create a HibernateResource and load it, it remains empty
> 2) you use a separate hibernate session to read a library from the db
> and add it to the hibernate resource?
>
> Then you have two scenarios:
> A- read a library and add it to the resource, then read a second library
> (with the 10000 children) and add it to the library which is already
> present in the resource?, or
> B- read a library (with the 10000 children) from the db and add it to
> the resource directly
>
> or did I miss anything?
>
> In which scenario do you see which tree-walking behavior?
>
> gr. Martin
>
> Stéphane Fournier wrote:
>> Ed Merks wrote:
>>
>>> Stéphane,
>>
>>> The assumption being that XMI resources will generally use xmi:id
>>> which in strict conformance to the XMI specification is required. In
>>> any case, this is clearly something you can specialize. I.e., you
>>> might just use XMLResourceImpl instead...
>>
>> I've made the change, now the guard does its job i.e the code in the
>> guard block is never executed.
>> I've ran my tests again, it does not change a lot the overall
>> performances.
>> Creating a sub tree and adding it at creation end to a library
>> attached to a resource is always the fastest way.
>>
>> Nevertheless, performances with standard resources either XMI or XML
>> are great to me :)
>>
>> My issue is regarding HibernateResource. I do hope that Martin could
>> give a hint to improve that !
>>
>> Stephane.
>>
>>> Stéphane Fournier wrote:
>>>> because in
>>>> protected boolean isAttachedDetachedHelperRequired()
>>>> {
>>>> return useIDs() || super.isAttachedDetachedHelperRequired();
>>>> }
>>>>
>>>> useIDs() always returns true in XmlResourceImpl. In the genmodel,
>>>> ResourceType is set to NONE.
>>>>
>>>> Ed Merks wrote:
>>>>
>>>>> Stéphane,
>>>>
>>>>> Sorry my point is it's guarded like this:
>>>>
>>>>> public void attached(EObject eObject)
>>>>> {
>>>>> if (isAttachedDetachedHelperRequired())
>>>>> {
>>>>> attachedHelper(eObject);
>>>>> for (TreeIterator<EObject> tree =
>>>>> getAllProperContents(eObject); tree.hasNext(); )
>>>>> {
>>>>> attachedHelper(tree.next());
>>>>> }
>>>>> }
>>>>> }
>>>>
>>>>
>>>>> protected boolean isAttachedDetachedHelperRequired()
>>>>> {
>>>>> return useIDs() || super.isAttachedDetachedHelperRequired();
>>>>> }
>>>>
>>>>> protected boolean useIDs()
>>>>> {
>>>>> return eObjectToIDMap != null || idToEObjectMap != null ||
>>>>> useUUIDs();
>>>>> }
>>>>
>>>>> protected boolean useIDAttributes()
>>>>> {
>>>>> return true;
>>>>> }
>>>>
>>>>> protected boolean useUUIDs()
>>>>> {
>>>>> return false;
>>>>> }
>>>>
>>>>> protected boolean isAttachedDetachedHelperRequired()
>>>>> {
>>>>> return isTrackingModification() || getIntrinsicIDToEObjectMap()
>>>>> != null;
>>>>> }
>>>>
>>>>
>>>>> So generally I would expect the guard to return false by default.
>>>>> So I'm wondering why it's returning true.
>>>>
>>>>
>>>>
>>>>> Stéphane Fournier wrote:
>>>>>> Ed Merks wrote:
>>>>>>
>>>>>>> Stéphane,
>>>>>>
>>>>>>> So the tree walk is the answer. Can you tell why it's needing
>>>>>>> to walk the tree? It's guarded such that the tree walk is
>>>>>>> avoided in most cases...
>>>>>>
>>>>>> Here is my analysis :
>>>>>> The tree is walked (on the added sub tree) because when I add a
>>>>>> temp lib with a sub tree containing my 10 000 children, no one is
>>>>>> attached to a resource.
>>>>>> When adding the temp lib in a lib attached to a resource, the
>>>>>> EObjectContainmentWithInverseEList.Resolving#add calls the
>>>>>> opposite relation i.e setParentBranch(..) that calls
>>>>>> eBasicSetContainer(..) that calls ResourceImpl.attached(..).
>>>>>> ResourceImpl.getAllProperContents returns the added sub tree
>>>>>> objects that are not attached to the parent library resource yet.
>>>>>>
>>>>>> Nevertheless, my tests show that's worth calling once
>>>>>> ResourceImpl.attached() with a tree walk on children rather than
>>>>>> calling x times this methods without tree walk.
>>>>>>
>>>>>>
>>>>>>> Stéphane Fournier wrote:
>>>>>>>> Ed Merks wrote:
>>>>>>>>
>>>>>>>>> Stéphane,
>>>>>>>>
>>>>>>>>> Comments below.
>>>>>>>>
>>>>>>>>
>>>>>>>>> Stéphane Fournier wrote:
>>>>>>>>>> Hi,
>>>>>>>>>>
>>>>>>>>>> Testing Teneo, I've made some performances comparisons against
>>>>>>>>>> a standard XmiResource.
>>>>>>>>>>
>>>>>>>>>> Scenario : I'm using the ExtendedLibrary meta-model. I have
>>>>>>>>>> initialized a mysql database (installed on my computer) with a
>>>>>>>>>> library with 10 000 children : sub-libraries, books, writers,
>>>>>>>>>> employees...
>>>>>>>>>>
>>>>>>>>>> As a library can be contained by another library, Library
>>>>>>>>>> meta-class is not considered as a top class by Teneo. So,
>>>>>>>>>> loading my resource gives me an empty resource content. I
>>>>>>>>>> query the database to retrieve my root library (I tried to
>>>>>>>>>> load the resource from an uri that specifies the initial
>>>>>>>>>> query, but the open editor is in read-only).
>>>>>>>>>>
>>>>>>>>>> Some figures :
>>>>>>>>>> Empty resource loaded in : 11012ms
>>>>>>>>>> Root Library retrieved in : 16ms
>>>>>>>>>> Root library added in the empty resource : 15225ms.
>>>>>>>>>>
>>>>>>>>>> I doesn't understand why it's so long to add the root library
>>>>>>>>>> in the HibernateResource.
>>>>>>>>>>
>>>>>>>>>> Does anyone have an idea ?
>>>>>>>>>>
>>>>>>>>>> I've made some additional tests against XmiResource.
>>>>>>>>>> If I add a sub-tree with 10 000 children to a library already
>>>>>>>>>> contained in a resource, it took 1s.
>>>>>>>>>> If I add a sub-tree with 10 000 children to a temporary
>>>>>>>>>> library not attached to a resource yet, it took 200ms + 63ms
>>>>>>>>>> to attach the temporary library to a library contained in a
>>>>>>>>>> resource.
>>>>>>>>>> Does anyone have an idea to explain the performance differences ?
>>>>>>>>> How much work does ResourceImpl.attached doing? Is it walking
>>>>>>>>> the whole tree and computing UUIDs or something like that?
>>>>>>>>>>
>>>>>>>> I'm not using UUIDs, I use the default behavior based on positions.
>>>>>>>> When I add 10 000 children to library contained in a resource,
>>>>>>>> ResourceImpl.attached is called for each object but that code is
>>>>>>>> never executed :
>>>>>>>> for (TreeIterator<EObject> tree = getAllProperContents(eObject);
>>>>>>>> tree.hasNext(); )
>>>>>>>> {
>>>>>>>> attachedHelper(tree.next());
>>>>>>>> }
>>>>>>>>
>>>>>>>> When adding a sub tree containing 10 000 objects to a not
>>>>>>>> attached library, the ResourceImpl.attached is called once when
>>>>>>>> the temporary lib is added to a resource contained one but the
>>>>>>>> this time, the TreeIterator is walked to attach every contained
>>>>>>>> objects.
>>>>>>>>
>>>>>>>>
>>>>>>>>>> I export the database content as a xml file.
>>>>>>>>>> Export resource loaded in : 736ms
>>>>>>>>>>
>>>>>>>>>> Retrieving the root library in the database is straight
>>>>>>>>>> forward (16ms), the library children are resolved (checked
>>>>>>>>>> with the debugger just before adding the root library to the
>>>>>>>>>> resource content).
>>>>>>>>>>
>>>>>>>>>> What is the Teneo option to avoid library children auto
>>>>>>>>>> resolution when querying for it.
>>>>>>>>>> I would like to perform the children resolution when expanding
>>>>>>>>>> the TreeItem only. (I set containment proxy to true in the
>>>>>>>>>> genmodel). I tested PersistenceOptions.SET_PROXY but I didn't
>>>>>>>>>> speed up resource loading.
>>>>>>>>>>
>>>>>>>>>> Stephane.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>>
>>>>
>>>>
>>
>>
>
>

--
Re: HibernateResource vs XmiResourceImpl performances ? [message #425161 is a reply to message #425139] Mon, 17 November 2008 20:19 Go to previous messageGo to next message
Martin Taal is currently offline Martin TaalFriend
Messages: 5468
Registered: July 2009
Senior Member
Hi Stéphane,
See questions inline.

gr. Martin

Stéphane Fournier wrote:
> Hi Martin,
>
> 1st scenario :
>
> 1) I create a HibernateResource that remains empty due to my meta-model
> (ExtendedLibrary). I have no top class because a Library can contain
> another one.
> The elapsed time to load my empty HibernateResource is long from my
> point of view : 11s more or less.
MT>> This is long indeed, can you enable hibernate logging (using log4j) to see what hibernate is doing?

>
> 2) On the SessionWrapper provided by the loaded resource, I open a
> transaction.
> I query the db to find my root library (the db contains a root library
> with 10 000 children).
> The query is really fast to retrieve the library : 16ms.
MT>> Yes probably only one record is read

>
> 3) Then, I add this root library to the empty loaded HibernateResource
> content.
> It took 15s more or less. It's long because all library children objects
> are already resolved (checked with the debugger). I haven't understood
> where time is wasted.
MT>> You mean it takes long because the 10000 children are read from the db? I would be surprised
that the children are read in step 2 (depends on your cascading options) in the 16ms.
Maybe the collection of children contains proxy objects, which are resolved?


>
> 4) I close the transaction.
> 5)My editor finishes to open with this resource.
>
>
> 2nd scenario (the editor is open with the root library ant its 10 000
> children):
> 1) I select a library either a child of the root one or the root one
> itself.
> 2) I create 10 000 children on the selected library.
> 3) I save the HibernateResource.
MT>> And what performance do you get here?

>
> As performances were not good (for me), I made some tests with standard
> EMF resources to have an idea of standard performances with XML files.
>
> In doing theses tests with XML resources, I got better performances if I
> create objects as children of an object not attached to a resource. At
> the end, I add the not attached object to an object contained in a
> resource.
> Ed and I discussed this point in this post.
>
> Stephane.
>
>
> Martin Taal wrote:
>> Hi Stephane,
>> The use case is not completely clear for me yet. I understand this:
>> 1) you create a HibernateResource and load it, it remains empty
>> 2) you use a separate hibernate session to read a library from the db
>> and add it to the hibernate resource?
>>
>> Then you have two scenarios:
>> A- read a library and add it to the resource, then read a second
>> library (with the 10000 children) and add it to the library which is
>> already present in the resource?, or
>> B- read a library (with the 10000 children) from the db and add it to
>> the resource directly
>>
>> or did I miss anything?
>>
>> In which scenario do you see which tree-walking behavior?
>>
>> gr. Martin
>>
>> Stéphane Fournier wrote:
>>> Ed Merks wrote:
>>>
>>>> Stéphane,
>>>
>>>> The assumption being that XMI resources will generally use xmi:id
>>>> which in strict conformance to the XMI specification is required.
>>>> In any case, this is clearly something you can specialize. I.e.,
>>>> you might just use XMLResourceImpl instead...
>>>
>>> I've made the change, now the guard does its job i.e the code in the
>>> guard block is never executed.
>>> I've ran my tests again, it does not change a lot the overall
>>> performances.
>>> Creating a sub tree and adding it at creation end to a library
>>> attached to a resource is always the fastest way.
>>>
>>> Nevertheless, performances with standard resources either XMI or XML
>>> are great to me :)
>>>
>>> My issue is regarding HibernateResource. I do hope that Martin could
>>> give a hint to improve that !
>>>
>>> Stephane.
>>>
>>>> Stéphane Fournier wrote:
>>>>> because in
>>>>> protected boolean isAttachedDetachedHelperRequired()
>>>>> {
>>>>> return useIDs() || super.isAttachedDetachedHelperRequired();
>>>>> }
>>>>>
>>>>> useIDs() always returns true in XmlResourceImpl. In the genmodel,
>>>>> ResourceType is set to NONE.
>>>>>
>>>>> Ed Merks wrote:
>>>>>
>>>>>> Stéphane,
>>>>>
>>>>>> Sorry my point is it's guarded like this:
>>>>>
>>>>>> public void attached(EObject eObject)
>>>>>> {
>>>>>> if (isAttachedDetachedHelperRequired())
>>>>>> {
>>>>>> attachedHelper(eObject);
>>>>>> for (TreeIterator<EObject> tree =
>>>>>> getAllProperContents(eObject); tree.hasNext(); )
>>>>>> {
>>>>>> attachedHelper(tree.next());
>>>>>> }
>>>>>> }
>>>>>> }
>>>>>
>>>>>
>>>>>> protected boolean isAttachedDetachedHelperRequired()
>>>>>> {
>>>>>> return useIDs() || super.isAttachedDetachedHelperRequired();
>>>>>> }
>>>>>
>>>>>> protected boolean useIDs()
>>>>>> {
>>>>>> return eObjectToIDMap != null || idToEObjectMap != null ||
>>>>>> useUUIDs();
>>>>>> }
>>>>>
>>>>>> protected boolean useIDAttributes()
>>>>>> {
>>>>>> return true;
>>>>>> }
>>>>>
>>>>>> protected boolean useUUIDs()
>>>>>> {
>>>>>> return false;
>>>>>> }
>>>>>
>>>>>> protected boolean isAttachedDetachedHelperRequired()
>>>>>> {
>>>>>> return isTrackingModification() ||
>>>>>> getIntrinsicIDToEObjectMap() != null;
>>>>>> }
>>>>>
>>>>>
>>>>>> So generally I would expect the guard to return false by default.
>>>>>> So I'm wondering why it's returning true.
>>>>>
>>>>>
>>>>>
>>>>>> Stéphane Fournier wrote:
>>>>>>> Ed Merks wrote:
>>>>>>>
>>>>>>>> Stéphane,
>>>>>>>
>>>>>>>> So the tree walk is the answer. Can you tell why it's needing
>>>>>>>> to walk the tree? It's guarded such that the tree walk is
>>>>>>>> avoided in most cases...
>>>>>>>
>>>>>>> Here is my analysis :
>>>>>>> The tree is walked (on the added sub tree) because when I add a
>>>>>>> temp lib with a sub tree containing my 10 000 children, no one is
>>>>>>> attached to a resource.
>>>>>>> When adding the temp lib in a lib attached to a resource, the
>>>>>>> EObjectContainmentWithInverseEList.Resolving#add calls the
>>>>>>> opposite relation i.e setParentBranch(..) that calls
>>>>>>> eBasicSetContainer(..) that calls ResourceImpl.attached(..).
>>>>>>> ResourceImpl.getAllProperContents returns the added sub tree
>>>>>>> objects that are not attached to the parent library resource yet.
>>>>>>>
>>>>>>> Nevertheless, my tests show that's worth calling once
>>>>>>> ResourceImpl.attached() with a tree walk on children rather than
>>>>>>> calling x times this methods without tree walk.
>>>>>>>
>>>>>>>
>>>>>>>> Stéphane Fournier wrote:
>>>>>>>>> Ed Merks wrote:
>>>>>>>>>
>>>>>>>>>> Stéphane,
>>>>>>>>>
>>>>>>>>>> Comments below.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> Stéphane Fournier wrote:
>>>>>>>>>>> Hi,
>>>>>>>>>>>
>>>>>>>>>>> Testing Teneo, I've made some performances comparisons
>>>>>>>>>>> against a standard XmiResource.
>>>>>>>>>>>
>>>>>>>>>>> Scenario : I'm using the ExtendedLibrary meta-model. I have
>>>>>>>>>>> initialized a mysql database (installed on my computer) with
>>>>>>>>>>> a library with 10 000 children : sub-libraries, books,
>>>>>>>>>>> writers, employees...
>>>>>>>>>>>
>>>>>>>>>>> As a library can be contained by another library, Library
>>>>>>>>>>> meta-class is not considered as a top class by Teneo. So,
>>>>>>>>>>> loading my resource gives me an empty resource content. I
>>>>>>>>>>> query the database to retrieve my root library (I tried to
>>>>>>>>>>> load the resource from an uri that specifies the initial
>>>>>>>>>>> query, but the open editor is in read-only).
>>>>>>>>>>>
>>>>>>>>>>> Some figures :
>>>>>>>>>>> Empty resource loaded in : 11012ms
>>>>>>>>>>> Root Library retrieved in : 16ms
>>>>>>>>>>> Root library added in the empty resource : 15225ms.
>>>>>>>>>>>
>>>>>>>>>>> I doesn't understand why it's so long to add the root library
>>>>>>>>>>> in the HibernateResource.
>>>>>>>>>>>
>>>>>>>>>>> Does anyone have an idea ?
>>>>>>>>>>>
>>>>>>>>>>> I've made some additional tests against XmiResource.
>>>>>>>>>>> If I add a sub-tree with 10 000 children to a library already
>>>>>>>>>>> contained in a resource, it took 1s.
>>>>>>>>>>> If I add a sub-tree with 10 000 children to a temporary
>>>>>>>>>>> library not attached to a resource yet, it took 200ms + 63ms
>>>>>>>>>>> to attach the temporary library to a library contained in a
>>>>>>>>>>> resource.
>>>>>>>>>>> Does anyone have an idea to explain the performance
>>>>>>>>>>> differences ?
>>>>>>>>>> How much work does ResourceImpl.attached doing? Is it walking
>>>>>>>>>> the whole tree and computing UUIDs or something like that?
>>>>>>>>>>>
>>>>>>>>> I'm not using UUIDs, I use the default behavior based on
>>>>>>>>> positions.
>>>>>>>>> When I add 10 000 children to library contained in a resource,
>>>>>>>>> ResourceImpl.attached is called for each object but that code
>>>>>>>>> is never executed :
>>>>>>>>> for (TreeIterator<EObject> tree =
>>>>>>>>> getAllProperContents(eObject); tree.hasNext(); )
>>>>>>>>> {
>>>>>>>>> attachedHelper(tree.next());
>>>>>>>>> }
>>>>>>>>>
>>>>>>>>> When adding a sub tree containing 10 000 objects to a not
>>>>>>>>> attached library, the ResourceImpl.attached is called once when
>>>>>>>>> the temporary lib is added to a resource contained one but the
>>>>>>>>> this time, the TreeIterator is walked to attach every contained
>>>>>>>>> objects.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>> I export the database content as a xml file.
>>>>>>>>>>> Export resource loaded in : 736ms
>>>>>>>>>>>
>>>>>>>>>>> Retrieving the root library in the database is straight
>>>>>>>>>>> forward (16ms), the library children are resolved (checked
>>>>>>>>>>> with the debugger just before adding the root library to the
>>>>>>>>>>> resource content).
>>>>>>>>>>>
>>>>>>>>>>> What is the Teneo option to avoid library children auto
>>>>>>>>>>> resolution when querying for it.
>>>>>>>>>>> I would like to perform the children resolution when
>>>>>>>>>>> expanding the TreeItem only. (I set containment proxy to true
>>>>>>>>>>> in the genmodel). I tested PersistenceOptions.SET_PROXY but I
>>>>>>>>>>> didn't speed up resource loading.
>>>>>>>>>>>
>>>>>>>>>>> Stephane.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>
>>>>>
>>>
>>>
>>
>>
>


--

With Regards, Martin Taal

Springsite/Elver.org
Office: Hardwareweg 4, 3821 BV Amersfoort
Postal: Nassaulaan 7, 3941 EC Doorn
The Netherlands
Cell: +31 (0)6 288 48 943
Tel: +31 (0)84 420 2397
Fax: +31 (0)84 225 9307
Mail: mtaal@springsite.com - mtaal@elver.org
Web: www.springsite.com - www.elver.org
Re: HibernateResource vs XmiResourceImpl performances ? [message #425177 is a reply to message #425161] Tue, 18 November 2008 12:30 Go to previous messageGo to next message
Stephane  fournier is currently offline Stephane fournierFriend
Messages: 340
Registered: July 2009
Senior Member
Martin Taal wrote:

> Hi Stéphane,
> See questions inline.

> gr. Martin

> Stéphane Fournier wrote:
>> Hi Martin,
>>
>> 1st scenario :
>>
>> 1) I create a HibernateResource that remains empty due to my meta-model
>> (ExtendedLibrary). I have no top class because a Library can contain
>> another one.
>> The elapsed time to load my empty HibernateResource is long from my
>> point of view : 11s more or less.
> MT>> This is long indeed, can you enable hibernate logging (using log4j) to
see what hibernate is doing?

SF :
Hibernate is loading data "from Person" query due to
loadUsingTopClasses(SessionWrapper) call. Indeed, Person is the only one
top class in the ExtendedLibrary meta-model. Writer extends Person thus,
writers are loaded, and I think objects involved in Writer relationships
are loaded too. Hence, that loads the entire model.

I didn't use a file input uri with a query inside, because such an uri
gives an open editor in read-only i.e contextual menu items provided by
Edit are grayed.
I do not want to modify my Editing domain to fix that.
I have overridden the HibernateResource to change the loadFromStore method
to use a query that suits my needs i.e retrieving my root library.

Now, the resource is loaded in :
first time : 30s.
second time (editor closed and re-open): 22s.
third time (editor closed and re-open) : 30s.
fourth time (editor closed and re-open) : 30s

All root library children are loaded, I use the default
CASCADE_POLICY_ON_CONTAINMENT option. Which are the option values to
select to get children loaded when I will expand the related TreeItem
only? I'm a bit lost with MERGE, PERSIST, REFRESH, REMOVE values.

>>
>> 2) On the SessionWrapper provided by the loaded resource, I open a
>> transaction.
>> I query the db to find my root library (the db contains a root library
>> with 10 000 children).
>> The query is really fast to retrieve the library : 16ms.
> MT>> Yes probably only one record is read

SF : I agree with that.

>>
>> 3) Then, I add this root library to the empty loaded HibernateResource
>> content.
>> It took 15s more or less. It's long because all library children objects
>> are already resolved (checked with the debugger). I haven't understood
>> where time is wasted.
> MT>> You mean it takes long because the 10000 children are read from the db?
I would be surprised
> that the children are read in step 2 (depends on your cascading options) in
the 16ms.
> Maybe the collection of children contains proxy objects, which are resolved?

SF: How do I recognize resolved proxy objects against resolved ones ?
I would like to get a behavior closed to the EMF one when using proxy
containment set to true.

>>
>> 4) I close the transaction.
>> 5)My editor finishes to open with this resource.
>>
>>
>> 2nd scenario (the editor is open with the root library ant its 10 000
>> children):
>> 1) I select a library either a child of the root one or the root one
>> itself.
>> 2) I create 10 000 children on the selected library.
>> 3) I save the HibernateResource.
> MT>> And what performance do you get here?

SF: 6 to 7s to create objects directly added to a library contained in the
HibernateResource and 43s to save the resource with these 10 000 objects
added.
It is faster to create objects attached to standard resources (XMI, XMl)
than the hibernate one.

What performance do you expect ?

>>
>> As performances were not good (for me), I made some tests with standard
>> EMF resources to have an idea of standard performances with XML files.
>>
>> In doing theses tests with XML resources, I got better performances if I
>> create objects as children of an object not attached to a resource. At
>> the end, I add the not attached object to an object contained in a
>> resource.
>> Ed and I discussed this point in this post.
>>
>> Stephane.
>>
>>
>> Martin Taal wrote:
>>> Hi Stephane,
>>> The use case is not completely clear for me yet. I understand this:
>>> 1) you create a HibernateResource and load it, it remains empty
>>> 2) you use a separate hibernate session to read a library from the db
>>> and add it to the hibernate resource?
>>>
>>> Then you have two scenarios:
>>> A- read a library and add it to the resource, then read a second
>>> library (with the 10000 children) and add it to the library which is
>>> already present in the resource?, or
>>> B- read a library (with the 10000 children) from the db and add it to
>>> the resource directly
>>>
>>> or did I miss anything?
>>>
>>> In which scenario do you see which tree-walking behavior?
>>>
>>> gr. Martin
>>>
>>> Stéphane Fournier wrote:
>>>> Ed Merks wrote:
>>>>
>>>>> Stéphane,
>>>>
>>>>> The assumption being that XMI resources will generally use xmi:id
>>>>> which in strict conformance to the XMI specification is required.
>>>>> In any case, this is clearly something you can specialize. I.e.,
>>>>> you might just use XMLResourceImpl instead...
>>>>
>>>> I've made the change, now the guard does its job i.e the code in the
>>>> guard block is never executed.
>>>> I've ran my tests again, it does not change a lot the overall
>>>> performances.
>>>> Creating a sub tree and adding it at creation end to a library
>>>> attached to a resource is always the fastest way.
>>>>
>>>> Nevertheless, performances with standard resources either XMI or XML
>>>> are great to me :)
>>>>
>>>> My issue is regarding HibernateResource. I do hope that Martin could
>>>> give a hint to improve that !
>>>>
>>>> Stephane.
>>>>
>>>>> Stéphane Fournier wrote:
>>>>>> because in
>>>>>> protected boolean isAttachedDetachedHelperRequired()
>>>>>> {
>>>>>> return useIDs() || super.isAttachedDetachedHelperRequired();
>>>>>> }
>>>>>>
>>>>>> useIDs() always returns true in XmlResourceImpl. In the genmodel,
>>>>>> ResourceType is set to NONE.
>>>>>>
>>>>>> Ed Merks wrote:
>>>>>>
>>>>>>> Stéphane,
>>>>>>
>>>>>>> Sorry my point is it's guarded like this:
>>>>>>
>>>>>>> public void attached(EObject eObject)
>>>>>>> {
>>>>>>> if (isAttachedDetachedHelperRequired())
>>>>>>> {
>>>>>>> attachedHelper(eObject);
>>>>>>> for (TreeIterator<EObject> tree =
>>>>>>> getAllProperContents(eObject); tree.hasNext(); )
>>>>>>> {
>>>>>>> attachedHelper(tree.next());
>>>>>>> }
>>>>>>> }
>>>>>>> }
>>>>>>
>>>>>>
>>>>>>> protected boolean isAttachedDetachedHelperRequired()
>>>>>>> {
>>>>>>> return useIDs() || super.isAttachedDetachedHelperRequired();
>>>>>>> }
>>>>>>
>>>>>>> protected boolean useIDs()
>>>>>>> {
>>>>>>> return eObjectToIDMap != null || idToEObjectMap != null ||
>>>>>>> useUUIDs();
>>>>>>> }
>>>>>>
>>>>>>> protected boolean useIDAttributes()
>>>>>>> {
>>>>>>> return true;
>>>>>>> }
>>>>>>
>>>>>>> protected boolean useUUIDs()
>>>>>>> {
>>>>>>> return false;
>>>>>>> }
>>>>>>
>>>>>>> protected boolean isAttachedDetachedHelperRequired()
>>>>>>> {
>>>>>>> return isTrackingModification() ||
>>>>>>> getIntrinsicIDToEObjectMap() != null;
>>>>>>> }
>>>>>>
>>>>>>
>>>>>>> So generally I would expect the guard to return false by default.
>>>>>>> So I'm wondering why it's returning true.
>>>>>>
>>>>>>
>>>>>>
>>>>>>> Stéphane Fournier wrote:
>>>>>>>> Ed Merks wrote:
>>>>>>>>
>>>>>>>>> Stéphane,
>>>>>>>>
>>>>>>>>> So the tree walk is the answer. Can you tell why it's needing
>>>>>>>>> to walk the tree? It's guarded such that the tree walk is
>>>>>>>>> avoided in most cases...
>>>>>>>>
>>>>>>>> Here is my analysis :
>>>>>>>> The tree is walked (on the added sub tree) because when I add a
>>>>>>>> temp lib with a sub tree containing my 10 000 children, no one is
>>>>>>>> attached to a resource.
>>>>>>>> When adding the temp lib in a lib attached to a resource, the
>>>>>>>> EObjectContainmentWithInverseEList.Resolving#add calls the
>>>>>>>> opposite relation i.e setParentBranch(..) that calls
>>>>>>>> eBasicSetContainer(..) that calls ResourceImpl.attached(..).
>>>>>>>> ResourceImpl.getAllProperContents returns the added sub tree
>>>>>>>> objects that are not attached to the parent library resource yet.
>>>>>>>>
>>>>>>>> Nevertheless, my tests show that's worth calling once
>>>>>>>> ResourceImpl.attached() with a tree walk on children rather than
>>>>>>>> calling x times this methods without tree walk.
>>>>>>>>
>>>>>>>>
>>>>>>>>> Stéphane Fournier wrote:
>>>>>>>>>> Ed Merks wrote:
>>>>>>>>>>
>>>>>>>>>>> Stéphane,
>>>>>>>>>>
>>>>>>>>>>> Comments below.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> Stéphane Fournier wrote:
>>>>>>>>>>>> Hi,
>>>>>>>>>>>>
>>>>>>>>>>>> Testing Teneo, I've made some performances comparisons
>>>>>>>>>>>> against a standard XmiResource.
>>>>>>>>>>>>
>>>>>>>>>>>> Scenario : I'm using the ExtendedLibrary meta-model. I have
>>>>>>>>>>>> initialized a mysql database (installed on my computer) with
>>>>>>>>>>>> a library with 10 000 children : sub-libraries, books,
>>>>>>>>>>>> writers, employees...
>>>>>>>>>>>>
>>>>>>>>>>>> As a library can be contained by another library, Library
>>>>>>>>>>>> meta-class is not considered as a top class by Teneo. So,
>>>>>>>>>>>> loading my resource gives me an empty resource content. I
>>>>>>>>>>>> query the database to retrieve my root library (I tried to
>>>>>>>>>>>> load the resource from an uri that specifies the initial
>>>>>>>>>>>> query, but the open editor is in read-only).
>>>>>>>>>>>>
>>>>>>>>>>>> Some figures :
>>>>>>>>>>>> Empty resource loaded in : 11012ms
>>>>>>>>>>>> Root Library retrieved in : 16ms
>>>>>>>>>>>> Root library added in the empty resource : 15225ms.
>>>>>>>>>>>>
>>>>>>>>>>>> I doesn't understand why it's so long to add the root library
>>>>>>>>>>>> in the HibernateResource.
>>>>>>>>>>>>
>>>>>>>>>>>> Does anyone have an idea ?
>>>>>>>>>>>>
>>>>>>>>>>>> I've made some additional tests against XmiResource.
>>>>>>>>>>>> If I add a sub-tree with 10 000 children to a library already
>>>>>>>>>>>> contained in a resource, it took 1s.
>>>>>>>>>>>> If I add a sub-tree with 10 000 children to a temporary
>>>>>>>>>>>> library not attached to a resource yet, it took 200ms + 63ms
>>>>>>>>>>>> to attach the temporary library to a library contained in a
>>>>>>>>>>>> resource.
>>>>>>>>>>>> Does anyone have an idea to explain the performance
>>>>>>>>>>>> differences ?
>>>>>>>>>>> How much work does ResourceImpl.attached doing? Is it walking
>>>>>>>>>>> the whole tree and computing UUIDs or something like that?
>>>>>>>>>>>>
>>>>>>>>>> I'm not using UUIDs, I use the default behavior based on
>>>>>>>>>> positions.
>>>>>>>>>> When I add 10 000 children to library contained in a resource,
>>>>>>>>>> ResourceImpl.attached is called for each object but that code
>>>>>>>>>> is never executed :
>>>>>>>>>> for (TreeIterator<EObject> tree =
>>>>>>>>>> getAllProperContents(eObject); tree.hasNext(); )
>>>>>>>>>> {
>>>>>>>>>> attachedHelper(tree.next());
>>>>>>>>>> }
>>>>>>>>>>
>>>>>>>>>> When adding a sub tree containing 10 000 objects to a not
>>>>>>>>>> attached library, the ResourceImpl.attached is called once when
>>>>>>>>>> the temporary lib is added to a resource contained one but the
>>>>>>>>>> this time, the TreeIterator is walked to attach every contained
>>>>>>>>>> objects.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>> I export the database content as a xml file.
>>>>>>>>>>>> Export resource loaded in : 736ms
>>>>>>>>>>>>
>>>>>>>>>>>> Retrieving the root library in the database is straight
>>>>>>>>>>>> forward (16ms), the library children are resolved (checked
>>>>>>>>>>>> with the debugger just before adding the root library to the
>>>>>>>>>>>> resource content).
>>>>>>>>>>>>
>>>>>>>>>>>> What is the Teneo option to avoid library children auto
>>>>>>>>>>>> resolution when querying for it.
>>>>>>>>>>>> I would like to perform the children resolution when
>>>>>>>>>>>> expanding the TreeItem only. (I set containment proxy to true
>>>>>>>>>>>> in the genmodel). I tested PersistenceOptions.SET_PROXY but I
>>>>>>>>>>>> didn't speed up resource loading.
>>>>>>>>>>>>
>>>>>>>>>>>> Stephane.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>>
>>>>
>>>>
>>>
>>>
>>
Re: HibernateResource vs XmiResourceImpl performances ? [message #425178 is a reply to message #425177] Tue, 18 November 2008 13:34 Go to previous messageGo to next message
Martin Taal is currently offline Martin TaalFriend
Messages: 5468
Registered: July 2009
Senior Member
Hi Stéphane,
The hibernate resource will be significantly slower than the xml/xmi store. On a standard
development system with mysql you can count on inserting 1-2000 records per second. This without
optimizing the db.
Hibernate is able to persist complex object graphs. To do this often multiple statements are used to
persist one object. For example what can happen in hibernate is that first an object is inserted in
the db with all foreign keys set to null. At that point all id's are known. Then update statements
are used to update the assocation between two objects.

The cascade option does not influence loading behavior, but persist/save behavior.

You can set the option PersistenceOptions.SET_PROXY to true. This will enable hibernate proxying and
make many-to-one relations lazy.

Regarding the load of the resource, it takes far too long. I guess everything is loaded because of
some traversal through the object graph which loads collections which should not be loaded. I think
this what you discussed with Ed. I did not completely follow that discussion. Can you check this and
summarize the behavior you see for me?

gr. Martin

Stéphane Fournier wrote:
> Martin Taal wrote:
>
>> Hi Stéphane,
>> See questions inline.
>
>> gr. Martin
>
>> Stéphane Fournier wrote:
>>> Hi Martin,
>>>
>>> 1st scenario :
>>>
>>> 1) I create a HibernateResource that remains empty due to my
>>> meta-model (ExtendedLibrary). I have no top class because a Library
>>> can contain another one.
>>> The elapsed time to load my empty HibernateResource is long from my
>>> point of view : 11s more or less.
>> MT>> This is long indeed, can you enable hibernate logging (using
>> log4j) to
> see what hibernate is doing?
>
> SF : Hibernate is loading data "from Person" query due to
> loadUsingTopClasses(SessionWrapper) call. Indeed, Person is the only one
> top class in the ExtendedLibrary meta-model. Writer extends Person thus,
> writers are loaded, and I think objects involved in Writer relationships
> are loaded too. Hence, that loads the entire model.
>
> I didn't use a file input uri with a query inside, because such an uri
> gives an open editor in read-only i.e contextual menu items provided by
> Edit are grayed.
> I do not want to modify my Editing domain to fix that.
> I have overridden the HibernateResource to change the loadFromStore
> method to use a query that suits my needs i.e retrieving my root library.
>
> Now, the resource is loaded in :
> first time : 30s.
> second time (editor closed and re-open): 22s.
> third time (editor closed and re-open) : 30s.
> fourth time (editor closed and re-open) : 30s
>
> All root library children are loaded, I use the default
> CASCADE_POLICY_ON_CONTAINMENT option. Which are the option values to
> select to get children loaded when I will expand the related TreeItem
> only? I'm a bit lost with MERGE, PERSIST, REFRESH, REMOVE values.
>
>>>
>>> 2) On the SessionWrapper provided by the loaded resource, I open a
>>> transaction.
>>> I query the db to find my root library (the db contains a root
>>> library with 10 000 children).
>>> The query is really fast to retrieve the library : 16ms.
>> MT>> Yes probably only one record is read
>
> SF : I agree with that.
>
>>>
>>> 3) Then, I add this root library to the empty loaded
>>> HibernateResource content.
>>> It took 15s more or less. It's long because all library children
>>> objects are already resolved (checked with the debugger). I haven't
>>> understood where time is wasted.
>> MT>> You mean it takes long because the 10000 children are read from
>> the db?
> I would be surprised
>> that the children are read in step 2 (depends on your cascading
>> options) in
> the 16ms.
>> Maybe the collection of children contains proxy objects, which are
>> resolved?
>
> SF: How do I recognize resolved proxy objects against resolved ones ?
> I would like to get a behavior closed to the EMF one when using proxy
> containment set to true.
>
>>>
>>> 4) I close the transaction.
>>> 5)My editor finishes to open with this resource.
>>>
>>>
>>> 2nd scenario (the editor is open with the root library ant its 10 000
>>> children):
>>> 1) I select a library either a child of the root one or the root one
>>> itself.
>>> 2) I create 10 000 children on the selected library.
>>> 3) I save the HibernateResource.
>> MT>> And what performance do you get here?
>
> SF: 6 to 7s to create objects directly added to a library contained in
> the HibernateResource and 43s to save the resource with these 10 000
> objects added.
> It is faster to create objects attached to standard resources (XMI, XMl)
> than the hibernate one.
>
> What performance do you expect ?
>>>
>>> As performances were not good (for me), I made some tests with
>>> standard EMF resources to have an idea of standard performances with
>>> XML files.
>>>
>>> In doing theses tests with XML resources, I got better performances
>>> if I create objects as children of an object not attached to a
>>> resource. At the end, I add the not attached object to an object
>>> contained in a resource.
>>> Ed and I discussed this point in this post.
>>>
>>> Stephane.
>>>
>>>
>>> Martin Taal wrote:
>>>> Hi Stephane,
>>>> The use case is not completely clear for me yet. I understand this:
>>>> 1) you create a HibernateResource and load it, it remains empty
>>>> 2) you use a separate hibernate session to read a library from the
>>>> db and add it to the hibernate resource?
>>>>
>>>> Then you have two scenarios:
>>>> A- read a library and add it to the resource, then read a second
>>>> library (with the 10000 children) and add it to the library which is
>>>> already present in the resource?, or
>>>> B- read a library (with the 10000 children) from the db and add it
>>>> to the resource directly
>>>>
>>>> or did I miss anything?
>>>>
>>>> In which scenario do you see which tree-walking behavior?
>>>>
>>>> gr. Martin
>>>>
>>>> Stéphane Fournier wrote:
>>>>> Ed Merks wrote:
>>>>>
>>>>>> Stéphane,
>>>>>
>>>>>> The assumption being that XMI resources will generally use xmi:id
>>>>>> which in strict conformance to the XMI specification is required.
>>>>>> In any case, this is clearly something you can specialize. I.e.,
>>>>>> you might just use XMLResourceImpl instead...
>>>>>
>>>>> I've made the change, now the guard does its job i.e the code in
>>>>> the guard block is never executed.
>>>>> I've ran my tests again, it does not change a lot the overall
>>>>> performances.
>>>>> Creating a sub tree and adding it at creation end to a library
>>>>> attached to a resource is always the fastest way.
>>>>>
>>>>> Nevertheless, performances with standard resources either XMI or
>>>>> XML are great to me :)
>>>>>
>>>>> My issue is regarding HibernateResource. I do hope that Martin
>>>>> could give a hint to improve that !
>>>>>
>>>>> Stephane.
>>>>>
>>>>>> Stéphane Fournier wrote:
>>>>>>> because in
>>>>>>> protected boolean isAttachedDetachedHelperRequired()
>>>>>>> {
>>>>>>> return useIDs() || super.isAttachedDetachedHelperRequired();
>>>>>>> }
>>>>>>>
>>>>>>> useIDs() always returns true in XmlResourceImpl. In the genmodel,
>>>>>>> ResourceType is set to NONE.
>>>>>>>
>>>>>>> Ed Merks wrote:
>>>>>>>
>>>>>>>> Stéphane,
>>>>>>>
>>>>>>>> Sorry my point is it's guarded like this:
>>>>>>>
>>>>>>>> public void attached(EObject eObject)
>>>>>>>> {
>>>>>>>> if (isAttachedDetachedHelperRequired())
>>>>>>>> {
>>>>>>>> attachedHelper(eObject);
>>>>>>>> for (TreeIterator<EObject> tree =
>>>>>>>> getAllProperContents(eObject); tree.hasNext(); )
>>>>>>>> {
>>>>>>>> attachedHelper(tree.next());
>>>>>>>> }
>>>>>>>> }
>>>>>>>> }
>>>>>>>
>>>>>>>
>>>>>>>> protected boolean isAttachedDetachedHelperRequired()
>>>>>>>> {
>>>>>>>> return useIDs() || super.isAttachedDetachedHelperRequired();
>>>>>>>> }
>>>>>>>
>>>>>>>> protected boolean useIDs()
>>>>>>>> {
>>>>>>>> return eObjectToIDMap != null || idToEObjectMap != null ||
>>>>>>>> useUUIDs();
>>>>>>>> }
>>>>>>>
>>>>>>>> protected boolean useIDAttributes()
>>>>>>>> {
>>>>>>>> return true;
>>>>>>>> }
>>>>>>>
>>>>>>>> protected boolean useUUIDs()
>>>>>>>> {
>>>>>>>> return false;
>>>>>>>> }
>>>>>>>
>>>>>>>> protected boolean isAttachedDetachedHelperRequired()
>>>>>>>> {
>>>>>>>> return isTrackingModification() ||
>>>>>>>> getIntrinsicIDToEObjectMap() != null;
>>>>>>>> }
>>>>>>>
>>>>>>>
>>>>>>>> So generally I would expect the guard to return false by
>>>>>>>> default. So I'm wondering why it's returning true.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> Stéphane Fournier wrote:
>>>>>>>>> Ed Merks wrote:
>>>>>>>>>
>>>>>>>>>> Stéphane,
>>>>>>>>>
>>>>>>>>>> So the tree walk is the answer. Can you tell why it's
>>>>>>>>>> needing to walk the tree? It's guarded such that the tree
>>>>>>>>>> walk is avoided in most cases...
>>>>>>>>>
>>>>>>>>> Here is my analysis :
>>>>>>>>> The tree is walked (on the added sub tree) because when I add a
>>>>>>>>> temp lib with a sub tree containing my 10 000 children, no one
>>>>>>>>> is attached to a resource.
>>>>>>>>> When adding the temp lib in a lib attached to a resource, the
>>>>>>>>> EObjectContainmentWithInverseEList.Resolving#add calls the
>>>>>>>>> opposite relation i.e setParentBranch(..) that calls
>>>>>>>>> eBasicSetContainer(..) that calls ResourceImpl.attached(..).
>>>>>>>>> ResourceImpl.getAllProperContents returns the added sub tree
>>>>>>>>> objects that are not attached to the parent library resource yet.
>>>>>>>>>
>>>>>>>>> Nevertheless, my tests show that's worth calling once
>>>>>>>>> ResourceImpl.attached() with a tree walk on children rather
>>>>>>>>> than calling x times this methods without tree walk.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> Stéphane Fournier wrote:
>>>>>>>>>>> Ed Merks wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Stéphane,
>>>>>>>>>>>
>>>>>>>>>>>> Comments below.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> Stéphane Fournier wrote:
>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Testing Teneo, I've made some performances comparisons
>>>>>>>>>>>>> against a standard XmiResource.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Scenario : I'm using the ExtendedLibrary meta-model. I have
>>>>>>>>>>>>> initialized a mysql database (installed on my computer)
>>>>>>>>>>>>> with a library with 10 000 children : sub-libraries, books,
>>>>>>>>>>>>> writers, employees...
>>>>>>>>>>>>>
>>>>>>>>>>>>> As a library can be contained by another library, Library
>>>>>>>>>>>>> meta-class is not considered as a top class by Teneo. So,
>>>>>>>>>>>>> loading my resource gives me an empty resource content. I
>>>>>>>>>>>>> query the database to retrieve my root library (I tried to
>>>>>>>>>>>>> load the resource from an uri that specifies the initial
>>>>>>>>>>>>> query, but the open editor is in read-only).
>>>>>>>>>>>>>
>>>>>>>>>>>>> Some figures :
>>>>>>>>>>>>> Empty resource loaded in : 11012ms
>>>>>>>>>>>>> Root Library retrieved in : 16ms
>>>>>>>>>>>>> Root library added in the empty resource : 15225ms.
>>>>>>>>>>>>>
>>>>>>>>>>>>> I doesn't understand why it's so long to add the root
>>>>>>>>>>>>> library in the HibernateResource.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Does anyone have an idea ?
>>>>>>>>>>>>>
>>>>>>>>>>>>> I've made some additional tests against XmiResource.
>>>>>>>>>>>>> If I add a sub-tree with 10 000 children to a library
>>>>>>>>>>>>> already contained in a resource, it took 1s.
>>>>>>>>>>>>> If I add a sub-tree with 10 000 children to a temporary
>>>>>>>>>>>>> library not attached to a resource yet, it took 200ms +
>>>>>>>>>>>>> 63ms to attach the temporary library to a library contained
>>>>>>>>>>>>> in a resource.
>>>>>>>>>>>>> Does anyone have an idea to explain the performance
>>>>>>>>>>>>> differences ?
>>>>>>>>>>>> How much work does ResourceImpl.attached doing? Is it
>>>>>>>>>>>> walking the whole tree and computing UUIDs or something like
>>>>>>>>>>>> that?
>>>>>>>>>>>>>
>>>>>>>>>>> I'm not using UUIDs, I use the default behavior based on
>>>>>>>>>>> positions.
>>>>>>>>>>> When I add 10 000 children to library contained in a
>>>>>>>>>>> resource, ResourceImpl.attached is called for each object but
>>>>>>>>>>> that code is never executed :
>>>>>>>>>>> for (TreeIterator<EObject> tree =
>>>>>>>>>>> getAllProperContents(eObject); tree.hasNext(); )
>>>>>>>>>>> {
>>>>>>>>>>> attachedHelper(tree.next());
>>>>>>>>>>> }
>>>>>>>>>>>
>>>>>>>>>>> When adding a sub tree containing 10 000 objects to a not
>>>>>>>>>>> attached library, the ResourceImpl.attached is called once
>>>>>>>>>>> when the temporary lib is added to a resource contained one
>>>>>>>>>>> but the this time, the TreeIterator is walked to attach every
>>>>>>>>>>> contained objects.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>>> I export the database content as a xml file.
>>>>>>>>>>>>> Export resource loaded in : 736ms
>>>>>>>>>>>>>
>>>>>>>>>>>>> Retrieving the root library in the database is straight
>>>>>>>>>>>>> forward (16ms), the library children are resolved (checked
>>>>>>>>>>>>> with the debugger just before adding the root library to
>>>>>>>>>>>>> the resource content).
>>>>>>>>>>>>>
>>>>>>>>>>>>> What is the Teneo option to avoid library children auto
>>>>>>>>>>>>> resolution when querying for it.
>>>>>>>>>>>>> I would like to perform the children resolution when
>>>>>>>>>>>>> expanding the TreeItem only. (I set containment proxy to
>>>>>>>>>>>>> true in the genmodel). I tested
>>>>>>>>>>>>> PersistenceOptions.SET_PROXY but I didn't speed up resource
>>>>>>>>>>>>> loading.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Stephane.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>
>
>
>


--

With Regards, Martin Taal

Springsite/Elver.org
Office: Hardwareweg 4, 3821 BV Amersfoort
Postal: Nassaulaan 7, 3941 EC Doorn
The Netherlands
Cell: +31 (0)6 288 48 943
Tel: +31 (0)84 420 2397
Fax: +31 (0)84 225 9307
Mail: mtaal@springsite.com - mtaal@elver.org
Web: www.springsite.com - www.elver.org
Re: HibernateResource vs XmiResourceImpl performances ? [message #425184 is a reply to message #425178] Tue, 18 November 2008 15:51 Go to previous messageGo to next message
Stephane  fournier is currently offline Stephane fournierFriend
Messages: 340
Registered: July 2009
Senior Member
Hi Martin,
First of all, thanks for your answers ! :)
My comments inline.

Stephane.

Martin Taal wrote:

> Hi Stéphane,
> The hibernate resource will be significantly slower than the xml/xmi store.
On a standard
> development system with mysql you can count on inserting 1-2000 records per
second. This without
> optimizing the db.
> Hibernate is able to persist complex object graphs. To do this often
multiple statements are used to
> persist one object. For example what can happen in hibernate is that first
an object is inserted in
> the db with all foreign keys set to null. At that point all id's are known.
Then update statements
> are used to update the assocation between two objects.

> The cascade option does not influence loading behavior, but persist/save
behavior.

> You can set the option PersistenceOptions.SET_PROXY to true. This will
enable hibernate proxying and
> make many-to-one relations lazy.

I have tried that and I got ClassNotFoundException. My editor is based on
two meta-models:
1) ExtendedLibrary
2) ExtendedExtendedLibrary that only defines one class ExtendedBook ->
Book defined in ExtendedLibrary.

The data store registers both meta-model root packages i.e one for
ExtendedBook and another one for ExtendedExtendedBook.
When opening my editor, I got exceptions due to the ExtendedBook class
that is not found by a Teneo/Hibernate classloader...

Do you have an idea ? Is it linked to buddy policy ?

I have checked all packages are exported in the plug-in that hosts
ExtendedBook (library.ext). This plug-in depends on the library one which
depends and re-exports hibernate & teneo plug-ins. Finally, the
library.editor plug-in depends on library plug-in and library.ext plug-in.

Here is the exception I got when activating SET_PROXY option :

Caused by: net.sf.cglib.core.CodeGenerationException:
java.lang.reflect.InvocationTargetException-->null
at
net.sf.cglib.core.AbstractClassGenerator.create(AbstractClas sGenerator.java:237)
at net.sf.cglib.proxy.Enhancer.createHelper(Enhancer.java:377)
at net.sf.cglib.proxy.Enhancer.createClass(Enhancer.java:317)
at
org.hibernate.proxy.pojo.cglib.CGLIBLazyInitializer.getProxy Factory(CGLIBLazyInitializer.java:127)
at
org.hibernate.proxy.pojo.cglib.CGLIBProxyFactory.postInstant iate(CGLIBProxyFactory.java:43)
at
org.eclipse.emf.teneo.hibernate.tuplizer.EMFTuplizer.buildPr oxyFactory(EMFTuplizer.java:214)
at
org.hibernate.tuple.entity.AbstractEntityTuplizer.<init>(AbstractEntityTuplizer.java:135)
at
org.eclipse.emf.teneo.hibernate.tuplizer.EMFTuplizer.<init>(EMFTuplizer.java:76)
... 119 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce ssorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe thodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at net.sf.cglib.core.ReflectUtils.defineClass(ReflectUtils.java :384)
at
net.sf.cglib.core.AbstractClassGenerator.create(AbstractClas sGenerator.java:219)
... 126 more
Caused by: java.lang.NoClassDefFoundError:
com/thalesgroup/mde/tig/ext/extlibrary/extendedlibrary/Exten dedBook
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:620)
... 132 more




> Regarding the load of the resource, it takes far too long. I guess
everything is loaded because of
> some traversal through the object graph which loads collections which should
not be loaded. I think
> this what you discussed with Ed. I did not completely follow that
discussion. Can you check this and
> summarize the behavior you see for me?

Ok, I will check that and I let you know the result of my analysis.

> gr. Martin

> Stéphane Fournier wrote:
>> Martin Taal wrote:
>>
>>> Hi Stéphane,
>>> See questions inline.
>>
>>> gr. Martin
>>
>>> Stéphane Fournier wrote:
>>>> Hi Martin,
>>>>
>>>> 1st scenario :
>>>>
>>>> 1) I create a HibernateResource that remains empty due to my
>>>> meta-model (ExtendedLibrary). I have no top class because a Library
>>>> can contain another one.
>>>> The elapsed time to load my empty HibernateResource is long from my
>>>> point of view : 11s more or less.
>>> MT>> This is long indeed, can you enable hibernate logging (using
>>> log4j) to
>> see what hibernate is doing?
>>
>> SF : Hibernate is loading data "from Person" query due to
>> loadUsingTopClasses(SessionWrapper) call. Indeed, Person is the only one
>> top class in the ExtendedLibrary meta-model. Writer extends Person thus,
>> writers are loaded, and I think objects involved in Writer relationships
>> are loaded too. Hence, that loads the entire model.
>>
>> I didn't use a file input uri with a query inside, because such an uri
>> gives an open editor in read-only i.e contextual menu items provided by
>> Edit are grayed.
>> I do not want to modify my Editing domain to fix that.
>> I have overridden the HibernateResource to change the loadFromStore
>> method to use a query that suits my needs i.e retrieving my root library.
>>
>> Now, the resource is loaded in :
>> first time : 30s.
>> second time (editor closed and re-open): 22s.
>> third time (editor closed and re-open) : 30s.
>> fourth time (editor closed and re-open) : 30s
>>
>> All root library children are loaded, I use the default
>> CASCADE_POLICY_ON_CONTAINMENT option. Which are the option values to
>> select to get children loaded when I will expand the related TreeItem
>> only? I'm a bit lost with MERGE, PERSIST, REFRESH, REMOVE values.
>>
>>>>
>>>> 2) On the SessionWrapper provided by the loaded resource, I open a
>>>> transaction.
>>>> I query the db to find my root library (the db contains a root
>>>> library with 10 000 children).
>>>> The query is really fast to retrieve the library : 16ms.
>>> MT>> Yes probably only one record is read
>>
>> SF : I agree with that.
>>
>>>>
>>>> 3) Then, I add this root library to the empty loaded
>>>> HibernateResource content.
>>>> It took 15s more or less. It's long because all library children
>>>> objects are already resolved (checked with the debugger). I haven't
>>>> understood where time is wasted.
>>> MT>> You mean it takes long because the 10000 children are read from
>>> the db?
>> I would be surprised
>>> that the children are read in step 2 (depends on your cascading
>>> options) in
>> the 16ms.
>>> Maybe the collection of children contains proxy objects, which are
>>> resolved?
>>
>> SF: How do I recognize resolved proxy objects against resolved ones ?
>> I would like to get a behavior closed to the EMF one when using proxy
>> containment set to true.
>>
>>>>
>>>> 4) I close the transaction.
>>>> 5)My editor finishes to open with this resource.
>>>>
>>>>
>>>> 2nd scenario (the editor is open with the root library ant its 10 000
>>>> children):
>>>> 1) I select a library either a child of the root one or the root one
>>>> itself.
>>>> 2) I create 10 000 children on the selected library.
>>>> 3) I save the HibernateResource.
>>> MT>> And what performance do you get here?
>>
>> SF: 6 to 7s to create objects directly added to a library contained in
>> the HibernateResource and 43s to save the resource with these 10 000
>> objects added.
>> It is faster to create objects attached to standard resources (XMI, XMl)
>> than the hibernate one.
>>
>> What performance do you expect ?
>>>>
>>>> As performances were not good (for me), I made some tests with
>>>> standard EMF resources to have an idea of standard performances with
>>>> XML files.
>>>>
>>>> In doing theses tests with XML resources, I got better performances
>>>> if I create objects as children of an object not attached to a
>>>> resource. At the end, I add the not attached object to an object
>>>> contained in a resource.
>>>> Ed and I discussed this point in this post.
>>>>
>>>> Stephane.
>>>>
>>>>
>>>> Martin Taal wrote:
>>>>> Hi Stephane,
>>>>> The use case is not completely clear for me yet. I understand this:
>>>>> 1) you create a HibernateResource and load it, it remains empty
>>>>> 2) you use a separate hibernate session to read a library from the
>>>>> db and add it to the hibernate resource?
>>>>>
>>>>> Then you have two scenarios:
>>>>> A- read a library and add it to the resource, then read a second
>>>>> library (with the 10000 children) and add it to the library which is
>>>>> already present in the resource?, or
>>>>> B- read a library (with the 10000 children) from the db and add it
>>>>> to the resource directly
>>>>>
>>>>> or did I miss anything?
>>>>>
>>>>> In which scenario do you see which tree-walking behavior?
>>>>>
>>>>> gr. Martin
>>>>>
>>>>> Stéphane Fournier wrote:
>>>>>> Ed Merks wrote:
>>>>>>
>>>>>>> Stéphane,
>>>>>>
>>>>>>> The assumption being that XMI resources will generally use xmi:id
>>>>>>> which in strict conformance to the XMI specification is required.
>>>>>>> In any case, this is clearly something you can specialize. I.e.,
>>>>>>> you might just use XMLResourceImpl instead...
>>>>>>
>>>>>> I've made the change, now the guard does its job i.e the code in
>>>>>> the guard block is never executed.
>>>>>> I've ran my tests again, it does not change a lot the overall
>>>>>> performances.
>>>>>> Creating a sub tree and adding it at creation end to a library
>>>>>> attached to a resource is always the fastest way.
>>>>>>
>>>>>> Nevertheless, performances with standard resources either XMI or
>>>>>> XML are great to me :)
>>>>>>
>>>>>> My issue is regarding HibernateResource. I do hope that Martin
>>>>>> could give a hint to improve that !
>>>>>>
>>>>>> Stephane.
>>>>>>
>>>>>>> Stéphane Fournier wrote:
>>>>>>>> because in
>>>>>>>> protected boolean isAttachedDetachedHelperRequired()
>>>>>>>> {
>>>>>>>> return useIDs() || super.isAttachedDetachedHelperRequired();
>>>>>>>> }
>>>>>>>>
>>>>>>>> useIDs() always returns true in XmlResourceImpl. In the genmodel,
>>>>>>>> ResourceType is set to NONE.
>>>>>>>>
>>>>>>>> Ed Merks wrote:
>>>>>>>>
>>>>>>>>> Stéphane,
>>>>>>>>
>>>>>>>>> Sorry my point is it's guarded like this:
>>>>>>>>
>>>>>>>>> public void attached(EObject eObject)
>>>>>>>>> {
>>>>>>>>> if (isAttachedDetachedHelperRequired())
>>>>>>>>> {
>>>>>>>>> attachedHelper(eObject);
>>>>>>>>> for (TreeIterator<EObject> tree =
>>>>>>>>> getAllProperContents(eObject); tree.hasNext(); )
>>>>>>>>> {
>>>>>>>>> attachedHelper(tree.next());
>>>>>>>>> }
>>>>>>>>> }
>>>>>>>>> }
>>>>>>>>
>>>>>>>>
>>>>>>>>> protected boolean isAttachedDetachedHelperRequired()
>>>>>>>>> {
>>>>>>>>> return useIDs() || super.isAttachedDetachedHelperRequired();
>>>>>>>>> }
>>>>>>>>
>>>>>>>>> protected boolean useIDs()
>>>>>>>>> {
>>>>>>>>> return eObjectToIDMap != null || idToEObjectMap != null ||
>>>>>>>>> useUUIDs();
>>>>>>>>> }
>>>>>>>>
>>>>>>>>> protected boolean useIDAttributes()
>>>>>>>>> {
>>>>>>>>> return true;
>>>>>>>>> }
>>>>>>>>
>>>>>>>>> protected boolean useUUIDs()
>>>>>>>>> {
>>>>>>>>> return false;
>>>>>>>>> }
>>>>>>>>
>>>>>>>>> protected boolean isAttachedDetachedHelperRequired()
>>>>>>>>> {
>>>>>>>>> return isTrackingModification() ||
>>>>>>>>> getIntrinsicIDToEObjectMap() != null;
>>>>>>>>> }
>>>>>>>>
>>>>>>>>
>>>>>>>>> So generally I would expect the guard to return false by
>>>>>>>>> default. So I'm wondering why it's returning true.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> Stéphane Fournier wrote:
>>>>>>>>>> Ed Merks wrote:
>>>>>>>>>>
>>>>>>>>>>> Stéphane,
>>>>>>>>>>
>>>>>>>>>>> So the tree walk is the answer. Can you tell why it's
>>>>>>>>>>> needing to walk the tree? It's guarded such that the tree
>>>>>>>>>>> walk is avoided in most cases...
>>>>>>>>>>
>>>>>>>>>> Here is my analysis :
>>>>>>>>>> The tree is walked (on the added sub tree) because when I add a
>>>>>>>>>> temp lib with a sub tree containing my 10 000 children, no one
>>>>>>>>>> is attached to a resource.
>>>>>>>>>> When adding the temp lib in a lib attached to a resource, the
>>>>>>>>>> EObjectContainmentWithInverseEList.Resolving#add calls the
>>>>>>>>>> opposite relation i.e setParentBranch(..) that calls
>>>>>>>>>> eBasicSetContainer(..) that calls ResourceImpl.attached(..).
>>>>>>>>>> ResourceImpl.getAllProperContents returns the added sub tree
>>>>>>>>>> objects that are not attached to the parent library resource yet.
>>>>>>>>>>
>>>>>>>>>> Nevertheless, my tests show that's worth calling once
>>>>>>>>>> ResourceImpl.attached() with a tree walk on children rather
>>>>>>>>>> than calling x times this methods without tree walk.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> Stéphane Fournier wrote:
>>>>>>>>>>>> Ed Merks wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Stéphane,
>>>>>>>>>>>>
>>>>>>>>>>>>> Comments below.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>> Stéphane Fournier wrote:
>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Testing Teneo, I've made some performances comparisons
>>>>>>>>>>>>>> against a standard XmiResource.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Scenario : I'm using the ExtendedLibrary meta-model. I have
>>>>>>>>>>>>>> initialized a mysql database (installed on my computer)
>>>>>>>>>>>>>> with a library with 10 000 children : sub-libraries, books,
>>>>>>>>>>>>>> writers, employees...
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> As a library can be contained by another library, Library
>>>>>>>>>>>>>> meta-class is not considered as a top class by Teneo. So,
>>>>>>>>>>>>>> loading my resource gives me an empty resource content. I
>>>>>>>>>>>>>> query the database to retrieve my root library (I tried to
>>>>>>>>>>>>>> load the resource from an uri that specifies the initial
>>>>>>>>>>>>>> query, but the open editor is in read-only).
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Some figures :
>>>>>>>>>>>>>> Empty resource loaded in : 11012ms
>>>>>>>>>>>>>> Root Library retrieved in : 16ms
>>>>>>>>>>>>>> Root library added in the empty resource : 15225ms.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I doesn't understand why it's so long to add the root
>>>>>>>>>>>>>> library in the HibernateResource.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Does anyone have an idea ?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I've made some additional tests against XmiResource.
>>>>>>>>>>>>>> If I add a sub-tree with 10 000 children to a library
>>>>>>>>>>>>>> already contained in a resource, it took 1s.
>>>>>>>>>>>>>> If I add a sub-tree with 10 000 children to a temporary
>>>>>>>>>>>>>> library not attached to a resource yet, it took 200ms +
>>>>>>>>>>>>>> 63ms to attach the temporary library to a library contained
>>>>>>>>>>>>>> in a resource.
>>>>>>>>>>>>>> Does anyone have an idea to explain the performance
>>>>>>>>>>>>>> differences ?
>>>>>>>>>>>>> How much work does ResourceImpl.attached doing? Is it
>>>>>>>>>>>>> walking the whole tree and computing UUIDs or something like
>>>>>>>>>>>>> that?
>>>>>>>>>>>>>>
>>>>>>>>>>>> I'm not using UUIDs, I use the default behavior based on
>>>>>>>>>>>> positions.
>>>>>>>>>>>> When I add 10 000 children to library contained in a
>>>>>>>>>>>> resource, ResourceImpl.attached is called for each object but
>>>>>>>>>>>> that code is never executed :
>>>>>>>>>>>> for (TreeIterator<EObject> tree =
>>>>>>>>>>>> getAllProperContents(eObject); tree.hasNext(); )
>>>>>>>>>>>> {
>>>>>>>>>>>> attachedHelper(tree.next());
>>>>>>>>>>>> }
>>>>>>>>>>>>
>>>>>>>>>>>> When adding a sub tree containing 10 000 objects to a not
>>>>>>>>>>>> attached library, the ResourceImpl.attached is called once
>>>>>>>>>>>> when the temporary lib is added to a resource contained one
>>>>>>>>>>>> but the this time, the TreeIterator is walked to attach every
>>>>>>>>>>>> contained objects.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>>> I export the database content as a xml file.
>>>>>>>>>>>>>> Export resource loaded in : 736ms
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Retrieving the root library in the database is straight
>>>>>>>>>>>>>> forward (16ms), the library children are resolved (checked
>>>>>>>>>>>>>> with the debugger just before adding the root library to
>>>>>>>>>>>>>> the resource content).
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> What is the Teneo option to avoid library children auto
>>>>>>>>>>>>>> resolution when querying for it.
>>>>>>>>>>>>>> I would like to perform the children resolution when
>>>>>>>>>>>>>> expanding the TreeItem only. (I set containment proxy to
>>>>>>>>>>>>>> true in the genmodel). I tested
>>>>>>>>>>>>>> PersistenceOptions.SET_PROXY but I didn't speed up resource
>>>>>>>>>>>>>> loading.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Stephane.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>
>>
>>
>>
Re: HibernateResource vs XmiResourceImpl performances ? [message #425186 is a reply to message #425184] Tue, 18 November 2008 16:26 Go to previous messageGo to next message
Martin Taal is currently offline Martin TaalFriend
Messages: 5468
Registered: July 2009
Senior Member
Hi Stéphane,
The plugin with the hibernate and cglib jar files should have eclipse-buddypolicy set to dependent.
Otherwise cglib can't find the model class files. Do you have that set?

gr. Martin

Stéphane Fournier wrote:
> Hi Martin,
> First of all, thanks for your answers ! :)
> My comments inline.
>
> Stephane.
>
> Martin Taal wrote:
>
>> Hi Stéphane,
>> The hibernate resource will be significantly slower than the xml/xmi
>> store.
> On a standard
>> development system with mysql you can count on inserting 1-2000
>> records per
> second. This without
>> optimizing the db.
>> Hibernate is able to persist complex object graphs. To do this often
> multiple statements are used to
>> persist one object. For example what can happen in hibernate is that
>> first
> an object is inserted in
>> the db with all foreign keys set to null. At that point all id's are
>> known.
> Then update statements
>> are used to update the assocation between two objects.
>
>> The cascade option does not influence loading behavior, but persist/save
> behavior.
>
>> You can set the option PersistenceOptions.SET_PROXY to true. This will
> enable hibernate proxying and
>> make many-to-one relations lazy.
>
> I have tried that and I got ClassNotFoundException. My editor is based
> on two meta-models:
> 1) ExtendedLibrary
> 2) ExtendedExtendedLibrary that only defines one class ExtendedBook ->
> Book defined in ExtendedLibrary.
>
> The data store registers both meta-model root packages i.e one for
> ExtendedBook and another one for ExtendedExtendedBook.
> When opening my editor, I got exceptions due to the ExtendedBook class
> that is not found by a Teneo/Hibernate classloader...
>
> Do you have an idea ? Is it linked to buddy policy ?
>
> I have checked all packages are exported in the plug-in that hosts
> ExtendedBook (library.ext). This plug-in depends on the library one
> which depends and re-exports hibernate & teneo plug-ins. Finally, the
> library.editor plug-in depends on library plug-in and library.ext plug-in.
>
> Here is the exception I got when activating SET_PROXY option :
>
> Caused by: net.sf.cglib.core.CodeGenerationException:
> java.lang.reflect.InvocationTargetException-->null
> at
> net.sf.cglib.core.AbstractClassGenerator.create(AbstractClas sGenerator.java:237)
>
> at net.sf.cglib.proxy.Enhancer.createHelper(Enhancer.java:377)
> at net.sf.cglib.proxy.Enhancer.createClass(Enhancer.java:317)
> at
> org.hibernate.proxy.pojo.cglib.CGLIBLazyInitializer.getProxy Factory(CGLIBLazyInitializer.java:127)
>
> at
> org.hibernate.proxy.pojo.cglib.CGLIBProxyFactory.postInstant iate(CGLIBProxyFactory.java:43)
>
> at
> org.eclipse.emf.teneo.hibernate.tuplizer.EMFTuplizer.buildPr oxyFactory(EMFTuplizer.java:214)
>
> at
> org.hibernate.tuple.entity.AbstractEntityTuplizer.<init>(AbstractEntityTuplizer.java:135)
>
> at
> org.eclipse.emf.teneo.hibernate.tuplizer.EMFTuplizer.<init>(EMFTuplizer.java:76)
>
> ... 119 more
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce ssorImpl.java:39)
>
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe thodAccessorImpl.java:25)
>
> at java.lang.reflect.Method.invoke(Method.java:585)
> at net.sf.cglib.core.ReflectUtils.defineClass(ReflectUtils.java :384)
> at
> net.sf.cglib.core.AbstractClassGenerator.create(AbstractClas sGenerator.java:219)
>
> ... 126 more
> Caused by: java.lang.NoClassDefFoundError:
> com/thalesgroup/mde/tig/ext/extlibrary/extendedlibrary/Exten dedBook
> at java.lang.ClassLoader.defineClass1(Native Method)
> at java.lang.ClassLoader.defineClass(ClassLoader.java:620)
> ... 132 more
>
>
>
>
>> Regarding the load of the resource, it takes far too long. I guess
> everything is loaded because of
>> some traversal through the object graph which loads collections which
>> should
> not be loaded. I think
>> this what you discussed with Ed. I did not completely follow that
> discussion. Can you check this and
>> summarize the behavior you see for me?
>
> Ok, I will check that and I let you know the result of my analysis.
>
>> gr. Martin
>
>> Stéphane Fournier wrote:
>>> Martin Taal wrote:
>>>
>>>> Hi Stéphane,
>>>> See questions inline.
>>>
>>>> gr. Martin
>>>
>>>> Stéphane Fournier wrote:
>>>>> Hi Martin,
>>>>>
>>>>> 1st scenario :
>>>>>
>>>>> 1) I create a HibernateResource that remains empty due to my
>>>>> meta-model (ExtendedLibrary). I have no top class because a Library
>>>>> can contain another one.
>>>>> The elapsed time to load my empty HibernateResource is long from my
>>>>> point of view : 11s more or less.
>>>> MT>> This is long indeed, can you enable hibernate logging (using
>>>> log4j) to
>>> see what hibernate is doing?
>>>
>>> SF : Hibernate is loading data "from Person" query due to
>>> loadUsingTopClasses(SessionWrapper) call. Indeed, Person is the only
>>> one top class in the ExtendedLibrary meta-model. Writer extends
>>> Person thus, writers are loaded, and I think objects involved in
>>> Writer relationships are loaded too. Hence, that loads the entire model.
>>>
>>> I didn't use a file input uri with a query inside, because such an
>>> uri gives an open editor in read-only i.e contextual menu items
>>> provided by Edit are grayed.
>>> I do not want to modify my Editing domain to fix that.
>>> I have overridden the HibernateResource to change the loadFromStore
>>> method to use a query that suits my needs i.e retrieving my root
>>> library.
>>>
>>> Now, the resource is loaded in :
>>> first time : 30s.
>>> second time (editor closed and re-open): 22s.
>>> third time (editor closed and re-open) : 30s.
>>> fourth time (editor closed and re-open) : 30s
>>>
>>> All root library children are loaded, I use the default
>>> CASCADE_POLICY_ON_CONTAINMENT option. Which are the option values to
>>> select to get children loaded when I will expand the related TreeItem
>>> only? I'm a bit lost with MERGE, PERSIST, REFRESH, REMOVE values.
>>>
>>>>>
>>>>> 2) On the SessionWrapper provided by the loaded resource, I open a
>>>>> transaction.
>>>>> I query the db to find my root library (the db contains a root
>>>>> library with 10 000 children).
>>>>> The query is really fast to retrieve the library : 16ms.
>>>> MT>> Yes probably only one record is read
>>>
>>> SF : I agree with that.
>>>
>>>>>
>>>>> 3) Then, I add this root library to the empty loaded
>>>>> HibernateResource content.
>>>>> It took 15s more or less. It's long because all library children
>>>>> objects are already resolved (checked with the debugger). I haven't
>>>>> understood where time is wasted.
>>>> MT>> You mean it takes long because the 10000 children are read from
>>>> the db?
>>> I would be surprised
>>>> that the children are read in step 2 (depends on your cascading
>>>> options) in
>>> the 16ms.
>>>> Maybe the collection of children contains proxy objects, which are
>>>> resolved?
>>>
>>> SF: How do I recognize resolved proxy objects against resolved ones ?
>>> I would like to get a behavior closed to the EMF one when using proxy
>>> containment set to true.
>>>
>>>>>
>>>>> 4) I close the transaction.
>>>>> 5)My editor finishes to open with this resource.
>>>>>
>>>>>
>>>>> 2nd scenario (the editor is open with the root library ant its 10
>>>>> 000 children):
>>>>> 1) I select a library either a child of the root one or the root
>>>>> one itself.
>>>>> 2) I create 10 000 children on the selected library.
>>>>> 3) I save the HibernateResource.
>>>> MT>> And what performance do you get here?
>>>
>>> SF: 6 to 7s to create objects directly added to a library contained
>>> in the HibernateResource and 43s to save the resource with these 10
>>> 000 objects added.
>>> It is faster to create objects attached to standard resources (XMI,
>>> XMl) than the hibernate one.
>>>
>>> What performance do you expect ?
>>>>>
>>>>> As performances were not good (for me), I made some tests with
>>>>> standard EMF resources to have an idea of standard performances
>>>>> with XML files.
>>>>>
>>>>> In doing theses tests with XML resources, I got better performances
>>>>> if I create objects as children of an object not attached to a
>>>>> resource. At the end, I add the not attached object to an object
>>>>> contained in a resource.
>>>>> Ed and I discussed this point in this post.
>>>>>
>>>>> Stephane.
>>>>>
>>>>>
>>>>> Martin Taal wrote:
>>>>>> Hi Stephane,
>>>>>> The use case is not completely clear for me yet. I understand this:
>>>>>> 1) you create a HibernateResource and load it, it remains empty
>>>>>> 2) you use a separate hibernate session to read a library from the
>>>>>> db and add it to the hibernate resource?
>>>>>>
>>>>>> Then you have two scenarios:
>>>>>> A- read a library and add it to the resource, then read a second
>>>>>> library (with the 10000 children) and add it to the library which
>>>>>> is already present in the resource?, or
>>>>>> B- read a library (with the 10000 children) from the db and add it
>>>>>> to the resource directly
>>>>>>
>>>>>> or did I miss anything?
>>>>>>
>>>>>> In which scenario do you see which tree-walking behavior?
>>>>>>
>>>>>> gr. Martin
>>>>>>
>>>>>> Stéphane Fournier wrote:
>>>>>>> Ed Merks wrote:
>>>>>>>
>>>>>>>> Stéphane,
>>>>>>>
>>>>>>>> The assumption being that XMI resources will generally use
>>>>>>>> xmi:id which in strict conformance to the XMI specification is
>>>>>>>> required. In any case, this is clearly something you can
>>>>>>>> specialize. I.e., you might just use XMLResourceImpl instead...
>>>>>>>
>>>>>>> I've made the change, now the guard does its job i.e the code in
>>>>>>> the guard block is never executed.
>>>>>>> I've ran my tests again, it does not change a lot the overall
>>>>>>> performances.
>>>>>>> Creating a sub tree and adding it at creation end to a library
>>>>>>> attached to a resource is always the fastest way.
>>>>>>>
>>>>>>> Nevertheless, performances with standard resources either XMI or
>>>>>>> XML are great to me :)
>>>>>>>
>>>>>>> My issue is regarding HibernateResource. I do hope that Martin
>>>>>>> could give a hint to improve that !
>>>>>>>
>>>>>>> Stephane.
>>>>>>>
>>>>>>>> Stéphane Fournier wrote:
>>>>>>>>> because in
>>>>>>>>> protected boolean isAttachedDetachedHelperRequired()
>>>>>>>>> {
>>>>>>>>> return useIDs() || super.isAttachedDetachedHelperRequired();
>>>>>>>>> }
>>>>>>>>>
>>>>>>>>> useIDs() always returns true in XmlResourceImpl. In the
>>>>>>>>> genmodel, ResourceType is set to NONE.
>>>>>>>>>
>>>>>>>>> Ed Merks wrote:
>>>>>>>>>
>>>>>>>>>> Stéphane,
>>>>>>>>>
>>>>>>>>>> Sorry my point is it's guarded like this:
>>>>>>>>>
>>>>>>>>>> public void attached(EObject eObject)
>>>>>>>>>> {
>>>>>>>>>> if (isAttachedDetachedHelperRequired())
>>>>>>>>>> {
>>>>>>>>>> attachedHelper(eObject);
>>>>>>>>>> for (TreeIterator<EObject> tree =
>>>>>>>>>> getAllProperContents(eObject); tree.hasNext(); )
>>>>>>>>>> {
>>>>>>>>>> attachedHelper(tree.next());
>>>>>>>>>> }
>>>>>>>>>> }
>>>>>>>>>> }
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> protected boolean isAttachedDetachedHelperRequired()
>>>>>>>>>> {
>>>>>>>>>> return useIDs() || super.isAttachedDetachedHelperRequired();
>>>>>>>>>> }
>>>>>>>>>
>>>>>>>>>> protected boolean useIDs()
>>>>>>>>>> {
>>>>>>>>>> return eObjectToIDMap != null || idToEObjectMap != null ||
>>>>>>>>>> useUUIDs();
>>>>>>>>>> }
>>>>>>>>>
>>>>>>>>>> protected boolean useIDAttributes()
>>>>>>>>>> {
>>>>>>>>>> return true;
>>>>>>>>>> }
>>>>>>>>>
>>>>>>>>>> protected boolean useUUIDs()
>>>>>>>>>> {
>>>>>>>>>> return false;
>>>>>>>>>> }
>>>>>>>>>
>>>>>>>>>> protected boolean isAttachedDetachedHelperRequired()
>>>>>>>>>> {
>>>>>>>>>> return isTrackingModification() ||
>>>>>>>>>> getIntrinsicIDToEObjectMap() != null;
>>>>>>>>>> }
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> So generally I would expect the guard to return false by
>>>>>>>>>> default. So I'm wondering why it's returning true.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> Stéphane Fournier wrote:
>>>>>>>>>>> Ed Merks wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Stéphane,
>>>>>>>>>>>
>>>>>>>>>>>> So the tree walk is the answer. Can you tell why it's
>>>>>>>>>>>> needing to walk the tree? It's guarded such that the tree
>>>>>>>>>>>> walk is avoided in most cases...
>>>>>>>>>>>
>>>>>>>>>>> Here is my analysis :
>>>>>>>>>>> The tree is walked (on the added sub tree) because when I add
>>>>>>>>>>> a temp lib with a sub tree containing my 10 000 children, no
>>>>>>>>>>> one is attached to a resource.
>>>>>>>>>>> When adding the temp lib in a lib attached to a resource, the
>>>>>>>>>>> EObjectContainmentWithInverseEList.Resolving#add calls the
>>>>>>>>>>> opposite relation i.e setParentBranch(..) that calls
>>>>>>>>>>> eBasicSetContainer(..) that calls ResourceImpl.attached(..).
>>>>>>>>>>> ResourceImpl.getAllProperContents returns the added sub tree
>>>>>>>>>>> objects that are not attached to the parent library resource
>>>>>>>>>>> yet.
>>>>>>>>>>>
>>>>>>>>>>> Nevertheless, my tests show that's worth calling once
>>>>>>>>>>> ResourceImpl.attached() with a tree walk on children rather
>>>>>>>>>>> than calling x times this methods without tree walk.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> Stéphane Fournier wrote:
>>>>>>>>>>>>> Ed Merks wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Stéphane,
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Comments below.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Stéphane Fournier wrote:
>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Testing Teneo, I've made some performances comparisons
>>>>>>>>>>>>>>> against a standard XmiResource.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Scenario : I'm using the ExtendedLibrary meta-model. I
>>>>>>>>>>>>>>> have initialized a mysql database (installed on my
>>>>>>>>>>>>>>> computer) with a library with 10 000 children :
>>>>>>>>>>>>>>> sub-libraries, books, writers, employees...
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> As a library can be contained by another library, Library
>>>>>>>>>>>>>>> meta-class is not considered as a top class by Teneo. So,
>>>>>>>>>>>>>>> loading my resource gives me an empty resource content. I
>>>>>>>>>>>>>>> query the database to retrieve my root library (I tried
>>>>>>>>>>>>>>> to load the resource from an uri that specifies the
>>>>>>>>>>>>>>> initial query, but the open editor is in read-only).
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Some figures :
>>>>>>>>>>>>>>> Empty resource loaded in : 11012ms
>>>>>>>>>>>>>>> Root Library retrieved in : 16ms
>>>>>>>>>>>>>>> Root library added in the empty resource : 15225ms.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I doesn't understand why it's so long to add the root
>>>>>>>>>>>>>>> library in the HibernateResource.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Does anyone have an idea ?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I've made some additional tests against XmiResource.
>>>>>>>>>>>>>>> If I add a sub-tree with 10 000 children to a library
>>>>>>>>>>>>>>> already contained in a resource, it took 1s.
>>>>>>>>>>>>>>> If I add a sub-tree with 10 000 children to a temporary
>>>>>>>>>>>>>>> library not attached to a resource yet, it took 200ms +
>>>>>>>>>>>>>>> 63ms to attach the temporary library to a library
>>>>>>>>>>>>>>> contained in a resource.
>>>>>>>>>>>>>>> Does anyone have an idea to explain the performance
>>>>>>>>>>>>>>> differences ?
>>>>>>>>>>>>>> How much work does ResourceImpl.attached doing? Is it
>>>>>>>>>>>>>> walking the whole tree and computing UUIDs or something
>>>>>>>>>>>>>> like that?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>> I'm not using UUIDs, I use the default behavior based on
>>>>>>>>>>>>> positions.
>>>>>>>>>>>>> When I add 10 000 children to library contained in a
>>>>>>>>>>>>> resource, ResourceImpl.attached is called for each object
>>>>>>>>>>>>> but that code is never executed :
>>>>>>>>>>>>> for (TreeIterator<EObject> tree =
>>>>>>>>>>>>> getAllProperContents(eObject); tree.hasNext(); )
>>>>>>>>>>>>> {
>>>>>>>>>>>>> attachedHelper(tree.next());
>>>>>>>>>>>>> }
>>>>>>>>>>>>>
>>>>>>>>>>>>> When adding a sub tree containing 10 000 objects to a not
>>>>>>>>>>>>> attached library, the ResourceImpl.attached is called once
>>>>>>>>>>>>> when the temporary lib is added to a resource contained one
>>>>>>>>>>>>> but the this time, the TreeIterator is walked to attach
>>>>>>>>>>>>> every contained objects.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I export the database content as a xml file.
>>>>>>>>>>>>>>> Export resource loaded in : 736ms
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Retrieving the root library in the database is straight
>>>>>>>>>>>>>>> forward (16ms), the library children are resolved
>>>>>>>>>>>>>>> (checked with the debugger just before adding the root
>>>>>>>>>>>>>>> library to the resource content).
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> What is the Teneo option to avoid library children auto
>>>>>>>>>>>>>>> resolution when querying for it.
>>>>>>>>>>>>>>> I would like to perform the children resolution when
>>>>>>>>>>>>>>> expanding the TreeItem only. (I set containment proxy to
>>>>>>>>>>>>>>> true in the genmodel). I tested
>>>>>>>>>>>>>>> PersistenceOptions.SET_PROXY but I didn't speed up
>>>>>>>>>>>>>>> resource loading.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Stephane.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>
>>>
>>>
>>>
>
>
>
>


--

With Regards, Martin Taal

Springsite/Elver.org
Office: Hardwareweg 4, 3821 BV Amersfoort
Postal: Nassaulaan 7, 3941 EC Doorn
The Netherlands
Cell: +31 (0)6 288 48 943
Tel: +31 (0)84 420 2397
Fax: +31 (0)84 225 9307
Mail: mtaal@springsite.com - mtaal@elver.org
Web: www.springsite.com - www.elver.org
Re: HibernateResource vs XmiResourceImpl performances ? [message #425195 is a reply to message #425186] Wed, 19 November 2008 09:59 Go to previous messageGo to next message
Stephane  fournier is currently offline Stephane fournierFriend
Messages: 340
Registered: July 2009
Senior Member
Martin Taal wrote:

> Hi Stéphane,
> The plugin with the hibernate and cglib jar files should have
eclipse-buddypolicy set to dependent.
> Otherwise cglib can't find the model class files. Do you have that set?

I did that and it only works for the library plug-in that directly depends
on the hibernate one. This library plug-in re-exports the hibernate one.
And the library.ext plug-in depends on the library one too. In theory, the
library.ext also depends on the hibernate plug-in. The PDE tooling shows
that with the plug-in dependency feature.
I have removed the library.ext plug-in to continue my tests and now it
works.
Now performances are a bit faster, but I have to figure out why my model
is entirely loaded.

> gr. Martin

> Stéphane Fournier wrote:
>> Hi Martin,
>> First of all, thanks for your answers ! :)
>> My comments inline.
>>
>> Stephane.
>>
>> Martin Taal wrote:
>>
>>> Hi Stéphane,
>>> The hibernate resource will be significantly slower than the xml/xmi
>>> store.
>> On a standard
>>> development system with mysql you can count on inserting 1-2000
>>> records per
>> second. This without
>>> optimizing the db.
>>> Hibernate is able to persist complex object graphs. To do this often
>> multiple statements are used to
>>> persist one object. For example what can happen in hibernate is that
>>> first
>> an object is inserted in
>>> the db with all foreign keys set to null. At that point all id's are
>>> known.
>> Then update statements
>>> are used to update the assocation between two objects.
>>
>>> The cascade option does not influence loading behavior, but persist/save
>> behavior.
>>
>>> You can set the option PersistenceOptions.SET_PROXY to true. This will
>> enable hibernate proxying and
>>> make many-to-one relations lazy.
>>
>> I have tried that and I got ClassNotFoundException. My editor is based
>> on two meta-models:
>> 1) ExtendedLibrary
>> 2) ExtendedExtendedLibrary that only defines one class ExtendedBook ->
>> Book defined in ExtendedLibrary.
>>
>> The data store registers both meta-model root packages i.e one for
>> ExtendedBook and another one for ExtendedExtendedBook.
>> When opening my editor, I got exceptions due to the ExtendedBook class
>> that is not found by a Teneo/Hibernate classloader...
>>
>> Do you have an idea ? Is it linked to buddy policy ?
>>
>> I have checked all packages are exported in the plug-in that hosts
>> ExtendedBook (library.ext). This plug-in depends on the library one
>> which depends and re-exports hibernate & teneo plug-ins. Finally, the
>> library.editor plug-in depends on library plug-in and library.ext plug-in.
>>
>> Here is the exception I got when activating SET_PROXY option :
>>
>> Caused by: net.sf.cglib.core.CodeGenerationException:
>> java.lang.reflect.InvocationTargetException-->null
>> at
>>
net.sf.cglib.core.AbstractClassGenerator.create(AbstractClas sGenerator.java:237)
>>
>> at net.sf.cglib.proxy.Enhancer.createHelper(Enhancer.java:377)
>> at net.sf.cglib.proxy.Enhancer.createClass(Enhancer.java:317)
>> at
>>
org.hibernate.proxy.pojo.cglib.CGLIBLazyInitializer.getProxy Factory(CGLIBLazyInitializer.java:127)
>>
>> at
>>
org.hibernate.proxy.pojo.cglib.CGLIBProxyFactory.postInstant iate(CGLIBProxyFactory.java:43)
>>
>> at
>>
org.eclipse.emf.teneo.hibernate.tuplizer.EMFTuplizer.buildPr oxyFactory(EMFTuplizer.java:214)
>>
>> at
>>
org.hibernate.tuple.entity.AbstractEntityTuplizer.<init>(AbstractEntityTuplizer.java:135)
>>
>> at
>>
org.eclipse.emf.teneo.hibernate.tuplizer.EMFTuplizer.<init>(EMFTuplizer.java:76)
>>
>> ... 119 more
>> Caused by: java.lang.reflect.InvocationTargetException
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at
>>
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce ssorImpl.java:39)
>>
>> at
>>
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe thodAccessorImpl.java:25)
>>
>> at java.lang.reflect.Method.invoke(Method.java:585)
>> at net.sf.cglib.core.ReflectUtils.defineClass(ReflectUtils.java :384)
>> at
>>
net.sf.cglib.core.AbstractClassGenerator.create(AbstractClas sGenerator.java:219)
>>
>> ... 126 more
>> Caused by: java.lang.NoClassDefFoundError:
>> com/thalesgroup/mde/tig/ext/extlibrary/extendedlibrary/Exten dedBook
>> at java.lang.ClassLoader.defineClass1(Native Method)
>> at java.lang.ClassLoader.defineClass(ClassLoader.java:620)
>> ... 132 more
>>
>>
>>
>>
>>> Regarding the load of the resource, it takes far too long. I guess
>> everything is loaded because of
>>> some traversal through the object graph which loads collections which
>>> should
>> not be loaded. I think
>>> this what you discussed with Ed. I did not completely follow that
>> discussion. Can you check this and
>>> summarize the behavior you see for me?
>>
>> Ok, I will check that and I let you know the result of my analysis.
>>
>>> gr. Martin
>>
>>> Stéphane Fournier wrote:
>>>> Martin Taal wrote:
>>>>
>>>>> Hi Stéphane,
>>>>> See questions inline.
>>>>
>>>>> gr. Martin
>>>>
>>>>> Stéphane Fournier wrote:
>>>>>> Hi Martin,
>>>>>>
>>>>>> 1st scenario :
>>>>>>
>>>>>> 1) I create a HibernateResource that remains empty due to my
>>>>>> meta-model (ExtendedLibrary). I have no top class because a Library
>>>>>> can contain another one.
>>>>>> The elapsed time to load my empty HibernateResource is long from my
>>>>>> point of view : 11s more or less.
>>>>> MT>> This is long indeed, can you enable hibernate logging (using
>>>>> log4j) to
>>>> see what hibernate is doing?
>>>>
>>>> SF : Hibernate is loading data "from Person" query due to
>>>> loadUsingTopClasses(SessionWrapper) call. Indeed, Person is the only
>>>> one top class in the ExtendedLibrary meta-model. Writer extends
>>>> Person thus, writers are loaded, and I think objects involved in
>>>> Writer relationships are loaded too. Hence, that loads the entire model.
>>>>
>>>> I didn't use a file input uri with a query inside, because such an
>>>> uri gives an open editor in read-only i.e contextual menu items
>>>> provided by Edit are grayed.
>>>> I do not want to modify my Editing domain to fix that.
>>>> I have overridden the HibernateResource to change the loadFromStore
>>>> method to use a query that suits my needs i.e retrieving my root
>>>> library.
>>>>
>>>> Now, the resource is loaded in :
>>>> first time : 30s.
>>>> second time (editor closed and re-open): 22s.
>>>> third time (editor closed and re-open) : 30s.
>>>> fourth time (editor closed and re-open) : 30s
>>>>
>>>> All root library children are loaded, I use the default
>>>> CASCADE_POLICY_ON_CONTAINMENT option. Which are the option values to
>>>> select to get children loaded when I will expand the related TreeItem
>>>> only? I'm a bit lost with MERGE, PERSIST, REFRESH, REMOVE values.
>>>>
>>>>>>
>>>>>> 2) On the SessionWrapper provided by the loaded resource, I open a
>>>>>> transaction.
>>>>>> I query the db to find my root library (the db contains a root
>>>>>> library with 10 000 children).
>>>>>> The query is really fast to retrieve the library : 16ms.
>>>>> MT>> Yes probably only one record is read
>>>>
>>>> SF : I agree with that.
>>>>
>>>>>>
>>>>>> 3) Then, I add this root library to the empty loaded
>>>>>> HibernateResource content.
>>>>>> It took 15s more or less. It's long because all library children
>>>>>> objects are already resolved (checked with the debugger). I haven't
>>>>>> understood where time is wasted.
>>>>> MT>> You mean it takes long because the 10000 children are read from
>>>>> the db?
>>>> I would be surprised
>>>>> that the children are read in step 2 (depends on your cascading
>>>>> options) in
>>>> the 16ms.
>>>>> Maybe the collection of children contains proxy objects, which are
>>>>> resolved?
>>>>
>>>> SF: How do I recognize resolved proxy objects against resolved ones ?
>>>> I would like to get a behavior closed to the EMF one when using proxy
>>>> containment set to true.
>>>>
>>>>>>
>>>>>> 4) I close the transaction.
>>>>>> 5)My editor finishes to open with this resource.
>>>>>>
>>>>>>
>>>>>> 2nd scenario (the editor is open with the root library ant its 10
>>>>>> 000 children):
>>>>>> 1) I select a library either a child of the root one or the root
>>>>>> one itself.
>>>>>> 2) I create 10 000 children on the selected library.
>>>>>> 3) I save the HibernateResource.
>>>>> MT>> And what performance do you get here?
>>>>
>>>> SF: 6 to 7s to create objects directly added to a library contained
>>>> in the HibernateResource and 43s to save the resource with these 10
>>>> 000 objects added.
>>>> It is faster to create objects attached to standard resources (XMI,
>>>> XMl) than the hibernate one.
>>>>
>>>> What performance do you expect ?
>>>>>>
>>>>>> As performances were not good (for me), I made some tests with
>>>>>> standard EMF resources to have an idea of standard performances
>>>>>> with XML files.
>>>>>>
>>>>>> In doing theses tests with XML resources, I got better performances
>>>>>> if I create objects as children of an object not attached to a
>>>>>> resource. At the end, I add the not attached object to an object
>>>>>> contained in a resource.
>>>>>> Ed and I discussed this point in this post.
>>>>>>
>>>>>> Stephane.
>>>>>>
>>>>>>
>>>>>> Martin Taal wrote:
>>>>>>> Hi Stephane,
>>>>>>> The use case is not completely clear for me yet. I understand this:
>>>>>>> 1) you create a HibernateResource and load it, it remains empty
>>>>>>> 2) you use a separate hibernate session to read a library from the
>>>>>>> db and add it to the hibernate resource?
>>>>>>>
>>>>>>> Then you have two scenarios:
>>>>>>> A- read a library and add it to the resource, then read a second
>>>>>>> library (with the 10000 children) and add it to the library which
>>>>>>> is already present in the resource?, or
>>>>>>> B- read a library (with the 10000 children) from the db and add it
>>>>>>> to the resource directly
>>>>>>>
>>>>>>> or did I miss anything?
>>>>>>>
>>>>>>> In which scenario do you see which tree-walking behavior?
>>>>>>>
>>>>>>> gr. Martin
>>>>>>>
>>>>>>> Stéphane Fournier wrote:
>>>>>>>> Ed Merks wrote:
>>>>>>>>
>>>>>>>>> Stéphane,
>>>>>>>>
>>>>>>>>> The assumption being that XMI resources will generally use
>>>>>>>>> xmi:id which in strict conformance to the XMI specification is
>>>>>>>>> required. In any case, this is clearly something you can
>>>>>>>>> specialize. I.e., you might just use XMLResourceImpl instead...
>>>>>>>>
>>>>>>>> I've made the change, now the guard does its job i.e the code in
>>>>>>>> the guard block is never executed.
>>>>>>>> I've ran my tests again, it does not change a lot the overall
>>>>>>>> performances.
>>>>>>>> Creating a sub tree and adding it at creation end to a library
>>>>>>>> attached to a resource is always the fastest way.
>>>>>>>>
>>>>>>>> Nevertheless, performances with standard resources either XMI or
>>>>>>>> XML are great to me :)
>>>>>>>>
>>>>>>>> My issue is regarding HibernateResource. I do hope that Martin
>>>>>>>> could give a hint to improve that !
>>>>>>>>
>>>>>>>> Stephane.
>>>>>>>>
>>>>>>>>> Stéphane Fournier wrote:
>>>>>>>>>> because in
>>>>>>>>>> protected boolean isAttachedDetachedHelperRequired()
>>>>>>>>>> {
>>>>>>>>>> return useIDs() || super.isAttachedDetachedHelperRequired();
>>>>>>>>>> }
>>>>>>>>>>
>>>>>>>>>> useIDs() always returns true in XmlResourceImpl. In the
>>>>>>>>>> genmodel, ResourceType is set to NONE.
>>>>>>>>>>
>>>>>>>>>> Ed Merks wrote:
>>>>>>>>>>
>>>>>>>>>>> Stéphane,
>>>>>>>>>>
>>>>>>>>>>> Sorry my point is it's guarded like this:
>>>>>>>>>>
>>>>>>>>>>> public void attached(EObject eObject)
>>>>>>>>>>> {
>>>>>>>>>>> if (isAttachedDetachedHelperRequired())
>>>>>>>>>>> {
>>>>>>>>>>> attachedHelper(eObject);
>>>>>>>>>>> for (TreeIterator<EObject> tree =
>>>>>>>>>>> getAllProperContents(eObject); tree.hasNext(); )
>>>>>>>>>>> {
>>>>>>>>>>> attachedHelper(tree.next());
>>>>>>>>>>> }
>>>>>>>>>>> }
>>>>>>>>>>> }
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> protected boolean isAttachedDetachedHelperRequired()
>>>>>>>>>>> {
>>>>>>>>>>> return useIDs() || super.isAttachedDetachedHelperRequired();
>>>>>>>>>>> }
>>>>>>>>>>
>>>>>>>>>>> protected boolean useIDs()
>>>>>>>>>>> {
>>>>>>>>>>> return eObjectToIDMap != null || idToEObjectMap != null ||
>>>>>>>>>>> useUUIDs();
>>>>>>>>>>> }
>>>>>>>>>>
>>>>>>>>>>> protected boolean useIDAttributes()
>>>>>>>>>>> {
>>>>>>>>>>> return true;
>>>>>>>>>>> }
>>>>>>>>>>
>>>>>>>>>>> protected boolean useUUIDs()
>>>>>>>>>>> {
>>>>>>>>>>> return false;
>>>>>>>>>>> }
>>>>>>>>>>
>>>>>>>>>>> protected boolean isAttachedDetachedHelperRequired()
>>>>>>>>>>> {
>>>>>>>>>>> return isTrackingModification() ||
>>>>>>>>>>> getIntrinsicIDToEObjectMap() != null;
>>>>>>>>>>> }
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> So generally I would expect the guard to return false by
>>>>>>>>>>> default. So I'm wondering why it's returning true.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> Stéphane Fournier wrote:
>>>>>>>>>>>> Ed Merks wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Stéphane,
>>>>>>>>>>>>
>>>>>>>>>>>>> So the tree walk is the answer. Can you tell why it's
>>>>>>>>>>>>> needing to walk the tree? It's guarded such that the tree
>>>>>>>>>>>>> walk is avoided in most cases...
>>>>>>>>>>>>
>>>>>>>>>>>> Here is my analysis :
>>>>>>>>>>>> The tree is walked (on the added sub tree) because when I add
>>>>>>>>>>>> a temp lib with a sub tree containing my 10 000 children, no
>>>>>>>>>>>> one is attached to a resource.
>>>>>>>>>>>> When adding the temp lib in a lib attached to a resource, the
>>>>>>>>>>>> EObjectContainmentWithInverseEList.Resolving#add calls the
>>>>>>>>>>>> opposite relation i.e setParentBranch(..) that calls
>>>>>>>>>>>> eBasicSetContainer(..) that calls ResourceImpl.attached(..).
>>>>>>>>>>>> ResourceImpl.getAllProperContents returns the added sub tree
>>>>>>>>>>>> objects that are not attached to the parent library resource
>>>>>>>>>>>> yet.
>>>>>>>>>>>>
>>>>>>>>>>>> Nevertheless, my tests show that's worth calling once
>>>>>>>>>>>> ResourceImpl.attached() with a tree walk on children rather
>>>>>>>>>>>> than calling x times this methods without tree walk.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>> Stéphane Fournier wrote:
>>>>>>>>>>>>>> Ed Merks wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Stéphane,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Comments below.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Stéphane Fournier wrote:
>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Testing Teneo, I've made some performances comparisons
>>>>>>>>>>>>>>>> against a standard XmiResource.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Scenario : I'm using the ExtendedLibrary meta-model. I
>>>>>>>>>>>>>>>> have initialized a mysql database (installed on my
>>>>>>>>>>>>>>>> computer) with a library with 10 000 children :
>>>>>>>>>>>>>>>> sub-libraries, books, writers, employees...
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> As a library can be contained by another library, Library
>>>>>>>>>>>>>>>> meta-class is not considered as a top class by Teneo. So,
>>>>>>>>>>>>>>>> loading my resource gives me an empty resource content. I
>>>>>>>>>>>>>>>> query the database to retrieve my root library (I tried
>>>>>>>>>>>>>>>> to load the resource from an uri that specifies the
>>>>>>>>>>>>>>>> initial query, but the open editor is in read-only).
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Some figures :
>>>>>>>>>>>>>>>> Empty resource loaded in : 11012ms
>>>>>>>>>>>>>>>> Root Library retrieved in : 16ms
>>>>>>>>>>>>>>>> Root library added in the empty resource : 15225ms.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I doesn't understand why it's so long to add the root
>>>>>>>>>>>>>>>> library in the HibernateResource.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Does anyone have an idea ?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I've made some additional tests against XmiResource.
>>>>>>>>>>>>>>>> If I add a sub-tree with 10 000 children to a library
>>>>>>>>>>>>>>>> already contained in a resource, it took 1s.
>>>>>>>>>>>>>>>> If I add a sub-tree with 10 000 children to a temporary
>>>>>>>>>>>>>>>> library not attached to a resource yet, it took 200ms +
>>>>>>>>>>>>>>>> 63ms to attach the temporary library to a library
>>>>>>>>>>>>>>>> contained in a resource.
>>>>>>>>>>>>>>>> Does anyone have an idea to explain the performance
>>>>>>>>>>>>>>>> differences ?
>>>>>>>>>>>>>>> How much work does ResourceImpl.attached doing? Is it
>>>>>>>>>>>>>>> walking the whole tree and computing UUIDs or something
>>>>>>>>>>>>>>> like that?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I'm not using UUIDs, I use the default behavior based on
>>>>>>>>>>>>>> positions.
>>>>>>>>>>>>>> When I add 10 000 children to library contained in a
>>>>>>>>>>>>>> resource, ResourceImpl.attached is called for each object
>>>>>>>>>>>>>> but that code is never executed :
>>>>>>>>>>>>>> for (TreeIterator<EObject> tree =
>>>>>>>>>>>>>> getAllProperContents(eObject); tree.hasNext(); )
>>>>>>>>>>>>>> {
>>>>>>>>>>>>>> attachedHelper(tree.next());
>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> When adding a sub tree containing 10 000 objects to a not
>>>>>>>>>>>>>> attached library, the ResourceImpl.attached is called once
>>>>>>>>>>>>>> when the temporary lib is added to a resource contained one
>>>>>>>>>>>>>> but the this time, the TreeIterator is walked to attach
>>>>>>>>>>>>>> every contained objects.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I export the database content as a xml file.
>>>>>>>>>>>>>>>> Export resource loaded in : 736ms
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Retrieving the root library in the database is straight
>>>>>>>>>>>>>>>> forward (16ms), the library children are resolved
>>>>>>>>>>>>>>>> (checked with the debugger just before adding the root
>>>>>>>>>>>>>>>> library to the resource content).
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> What is the Teneo option to avoid library children auto
>>>>>>>>>>>>>>>> resolution when querying for it.
>>>>>>>>>>>>>>>> I would like to perform the children resolution when
>>>>>>>>>>>>>>>> expanding the TreeItem only. (I set containment proxy to
>>>>>>>>>>>>>>>> true in the genmodel). I tested
>>>>>>>>>>>>>>>> PersistenceOptions.SET_PROXY but I didn't speed up
>>>>>>>>>>>>>>>> resource loading.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Stephane.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>
>>>>
>>>>
>>>>
>>
>>
>>
>>
Re: HibernateResource vs XmiResourceImpl performances ? [message #425219 is a reply to message #425195] Wed, 19 November 2008 23:13 Go to previous message
Martin Taal is currently offline Martin TaalFriend
Messages: 5468
Registered: July 2009
Senior Member
Hi Stéphane,
Apparently buddypolicy dependent means that only the classpath of the directly depending plugin is
searched. In this case the classpath of library as it depends on hibernate. However, the classpath
of library does not contain the libraryext model and can't find it therefore.

Regarding your plugin setup, what I would propose is the following.
- Have a project/plugin which contains the testcases or application code
- let this plugin depend on hibernate, libraryext and library
- remove the dependency of library on hibernate and remove the reexport to libraryext and the
dependence of libraryext on hibernate

gr. Martin

Stéphane Fournier wrote:
> Martin Taal wrote:
>
>> Hi Stéphane,
>> The plugin with the hibernate and cglib jar files should have
> eclipse-buddypolicy set to dependent.
>> Otherwise cglib can't find the model class files. Do you have that set?
>
> I did that and it only works for the library plug-in that directly
> depends on the hibernate one. This library plug-in re-exports the
> hibernate one. And the library.ext plug-in depends on the library one
> too. In theory, the library.ext also depends on the hibernate plug-in.
> The PDE tooling shows that with the plug-in dependency feature. I have
> removed the library.ext plug-in to continue my tests and now it works.
> Now performances are a bit faster, but I have to figure out why my model
> is entirely loaded.
>
>> gr. Martin
>
>> Stéphane Fournier wrote:
>>> Hi Martin,
>>> First of all, thanks for your answers ! :)
>>> My comments inline.
>>>
>>> Stephane.
>>>
>>> Martin Taal wrote:
>>>
>>>> Hi Stéphane,
>>>> The hibernate resource will be significantly slower than the xml/xmi
>>>> store.
>>> On a standard
>>>> development system with mysql you can count on inserting 1-2000
>>>> records per
>>> second. This without
>>>> optimizing the db.
>>>> Hibernate is able to persist complex object graphs. To do this often
>>> multiple statements are used to
>>>> persist one object. For example what can happen in hibernate is that
>>>> first
>>> an object is inserted in
>>>> the db with all foreign keys set to null. At that point all id's are
>>>> known.
>>> Then update statements
>>>> are used to update the assocation between two objects.
>>>
>>>> The cascade option does not influence loading behavior, but
>>>> persist/save
>>> behavior.
>>>
>>>> You can set the option PersistenceOptions.SET_PROXY to true. This will
>>> enable hibernate proxying and
>>>> make many-to-one relations lazy.
>>>
>>> I have tried that and I got ClassNotFoundException. My editor is
>>> based on two meta-models:
>>> 1) ExtendedLibrary
>>> 2) ExtendedExtendedLibrary that only defines one class ExtendedBook
>>> -> Book defined in ExtendedLibrary.
>>>
>>> The data store registers both meta-model root packages i.e one for
>>> ExtendedBook and another one for ExtendedExtendedBook.
>>> When opening my editor, I got exceptions due to the ExtendedBook
>>> class that is not found by a Teneo/Hibernate classloader...
>>>
>>> Do you have an idea ? Is it linked to buddy policy ?
>>>
>>> I have checked all packages are exported in the plug-in that hosts
>>> ExtendedBook (library.ext). This plug-in depends on the library one
>>> which depends and re-exports hibernate & teneo plug-ins. Finally, the
>>> library.editor plug-in depends on library plug-in and library.ext
>>> plug-in.
>>>
>>> Here is the exception I got when activating SET_PROXY option :
>>>
>>> Caused by: net.sf.cglib.core.CodeGenerationException:
>>> java.lang.reflect.InvocationTargetException-->null
>>> at
> net.sf.cglib.core.AbstractClassGenerator.create(AbstractClas sGenerator.java:237)
>
>>>
>>> at net.sf.cglib.proxy.Enhancer.createHelper(Enhancer.java:377)
>>> at net.sf.cglib.proxy.Enhancer.createClass(Enhancer.java:317)
>>> at
> org.hibernate.proxy.pojo.cglib.CGLIBLazyInitializer.getProxy Factory(CGLIBLazyInitializer.java:127)
>
>>>
>>> at
> org.hibernate.proxy.pojo.cglib.CGLIBProxyFactory.postInstant iate(CGLIBProxyFactory.java:43)
>
>>>
>>> at
> org.eclipse.emf.teneo.hibernate.tuplizer.EMFTuplizer.buildPr oxyFactory(EMFTuplizer.java:214)
>
>>>
>>> at
> org.hibernate.tuple.entity.AbstractEntityTuplizer.<init>(AbstractEntityTuplizer.java:135)
>
>>>
>>> at
> org.eclipse.emf.teneo.hibernate.tuplizer.EMFTuplizer.<init>(EMFTuplizer.java:76)
>
>>>
>>> ... 119 more
>>> Caused by: java.lang.reflect.InvocationTargetException
>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce ssorImpl.java:39)
>
>>>
>>> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe thodAccessorImpl.java:25)
>
>>>
>>> at java.lang.reflect.Method.invoke(Method.java:585)
>>> at net.sf.cglib.core.ReflectUtils.defineClass(ReflectUtils.java :384)
>>> at
> net.sf.cglib.core.AbstractClassGenerator.create(AbstractClas sGenerator.java:219)
>
>>>
>>> ... 126 more
>>> Caused by: java.lang.NoClassDefFoundError:
>>> com/thalesgroup/mde/tig/ext/extlibrary/extendedlibrary/Exten dedBook
>>> at java.lang.ClassLoader.defineClass1(Native Method)
>>> at java.lang.ClassLoader.defineClass(ClassLoader.java:620)
>>> ... 132 more
>>>
>>>
>>>
>>>
>>>> Regarding the load of the resource, it takes far too long. I guess
>>> everything is loaded because of
>>>> some traversal through the object graph which loads collections
>>>> which should
>>> not be loaded. I think
>>>> this what you discussed with Ed. I did not completely follow that
>>> discussion. Can you check this and
>>>> summarize the behavior you see for me?
>>>
>>> Ok, I will check that and I let you know the result of my analysis.
>>>
>>>> gr. Martin
>>>
>>>> Stéphane Fournier wrote:
>>>>> Martin Taal wrote:
>>>>>
>>>>>> Hi Stéphane,
>>>>>> See questions inline.
>>>>>
>>>>>> gr. Martin
>>>>>
>>>>>> Stéphane Fournier wrote:
>>>>>>> Hi Martin,
>>>>>>>
>>>>>>> 1st scenario :
>>>>>>>
>>>>>>> 1) I create a HibernateResource that remains empty due to my
>>>>>>> meta-model (ExtendedLibrary). I have no top class because a
>>>>>>> Library can contain another one.
>>>>>>> The elapsed time to load my empty HibernateResource is long from
>>>>>>> my point of view : 11s more or less.
>>>>>> MT>> This is long indeed, can you enable hibernate logging (using
>>>>>> log4j) to
>>>>> see what hibernate is doing?
>>>>>
>>>>> SF : Hibernate is loading data "from Person" query due to
>>>>> loadUsingTopClasses(SessionWrapper) call. Indeed, Person is the
>>>>> only one top class in the ExtendedLibrary meta-model. Writer
>>>>> extends Person thus, writers are loaded, and I think objects
>>>>> involved in Writer relationships are loaded too. Hence, that loads
>>>>> the entire model.
>>>>>
>>>>> I didn't use a file input uri with a query inside, because such an
>>>>> uri gives an open editor in read-only i.e contextual menu items
>>>>> provided by Edit are grayed.
>>>>> I do not want to modify my Editing domain to fix that.
>>>>> I have overridden the HibernateResource to change the loadFromStore
>>>>> method to use a query that suits my needs i.e retrieving my root
>>>>> library.
>>>>>
>>>>> Now, the resource is loaded in :
>>>>> first time : 30s.
>>>>> second time (editor closed and re-open): 22s.
>>>>> third time (editor closed and re-open) : 30s.
>>>>> fourth time (editor closed and re-open) : 30s
>>>>>
>>>>> All root library children are loaded, I use the default
>>>>> CASCADE_POLICY_ON_CONTAINMENT option. Which are the option values
>>>>> to select to get children loaded when I will expand the related
>>>>> TreeItem only? I'm a bit lost with MERGE, PERSIST, REFRESH, REMOVE
>>>>> values.
>>>>>
>>>>>>>
>>>>>>> 2) On the SessionWrapper provided by the loaded resource, I open
>>>>>>> a transaction.
>>>>>>> I query the db to find my root library (the db contains a root
>>>>>>> library with 10 000 children).
>>>>>>> The query is really fast to retrieve the library : 16ms.
>>>>>> MT>> Yes probably only one record is read
>>>>>
>>>>> SF : I agree with that.
>>>>>
>>>>>>>
>>>>>>> 3) Then, I add this root library to the empty loaded
>>>>>>> HibernateResource content.
>>>>>>> It took 15s more or less. It's long because all library children
>>>>>>> objects are already resolved (checked with the debugger). I
>>>>>>> haven't understood where time is wasted.
>>>>>> MT>> You mean it takes long because the 10000 children are read
>>>>>> from the db?
>>>>> I would be surprised
>>>>>> that the children are read in step 2 (depends on your cascading
>>>>>> options) in
>>>>> the 16ms.
>>>>>> Maybe the collection of children contains proxy objects, which are
>>>>>> resolved?
>>>>>
>>>>> SF: How do I recognize resolved proxy objects against resolved ones ?
>>>>> I would like to get a behavior closed to the EMF one when using
>>>>> proxy containment set to true.
>>>>>
>>>>>>>
>>>>>>> 4) I close the transaction.
>>>>>>> 5)My editor finishes to open with this resource.
>>>>>>>
>>>>>>>
>>>>>>> 2nd scenario (the editor is open with the root library ant its 10
>>>>>>> 000 children):
>>>>>>> 1) I select a library either a child of the root one or the root
>>>>>>> one itself.
>>>>>>> 2) I create 10 000 children on the selected library.
>>>>>>> 3) I save the HibernateResource.
>>>>>> MT>> And what performance do you get here?
>>>>>
>>>>> SF: 6 to 7s to create objects directly added to a library contained
>>>>> in the HibernateResource and 43s to save the resource with these 10
>>>>> 000 objects added.
>>>>> It is faster to create objects attached to standard resources (XMI,
>>>>> XMl) than the hibernate one.
>>>>>
>>>>> What performance do you expect ?
>>>>>>>
>>>>>>> As performances were not good (for me), I made some tests with
>>>>>>> standard EMF resources to have an idea of standard performances
>>>>>>> with XML files.
>>>>>>>
>>>>>>> In doing theses tests with XML resources, I got better
>>>>>>> performances if I create objects as children of an object not
>>>>>>> attached to a resource. At the end, I add the not attached object
>>>>>>> to an object contained in a resource.
>>>>>>> Ed and I discussed this point in this post.
>>>>>>>
>>>>>>> Stephane.
>>>>>>>
>>>>>>>
>>>>>>> Martin Taal wrote:
>>>>>>>> Hi Stephane,
>>>>>>>> The use case is not completely clear for me yet. I understand this:
>>>>>>>> 1) you create a HibernateResource and load it, it remains empty
>>>>>>>> 2) you use a separate hibernate session to read a library from
>>>>>>>> the db and add it to the hibernate resource?
>>>>>>>>
>>>>>>>> Then you have two scenarios:
>>>>>>>> A- read a library and add it to the resource, then read a second
>>>>>>>> library (with the 10000 children) and add it to the library
>>>>>>>> which is already present in the resource?, or
>>>>>>>> B- read a library (with the 10000 children) from the db and add
>>>>>>>> it to the resource directly
>>>>>>>>
>>>>>>>> or did I miss anything?
>>>>>>>>
>>>>>>>> In which scenario do you see which tree-walking behavior?
>>>>>>>>
>>>>>>>> gr. Martin
>>>>>>>>
>>>>>>>> Stéphane Fournier wrote:
>>>>>>>>> Ed Merks wrote:
>>>>>>>>>
>>>>>>>>>> Stéphane,
>>>>>>>>>
>>>>>>>>>> The assumption being that XMI resources will generally use
>>>>>>>>>> xmi:id which in strict conformance to the XMI specification is
>>>>>>>>>> required. In any case, this is clearly something you can
>>>>>>>>>> specialize. I.e., you might just use XMLResourceImpl instead...
>>>>>>>>>
>>>>>>>>> I've made the change, now the guard does its job i.e the code
>>>>>>>>> in the guard block is never executed.
>>>>>>>>> I've ran my tests again, it does not change a lot the overall
>>>>>>>>> performances.
>>>>>>>>> Creating a sub tree and adding it at creation end to a library
>>>>>>>>> attached to a resource is always the fastest way.
>>>>>>>>>
>>>>>>>>> Nevertheless, performances with standard resources either XMI
>>>>>>>>> or XML are great to me :)
>>>>>>>>>
>>>>>>>>> My issue is regarding HibernateResource. I do hope that Martin
>>>>>>>>> could give a hint to improve that !
>>>>>>>>>
>>>>>>>>> Stephane.
>>>>>>>>>
>>>>>>>>>> Stéphane Fournier wrote:
>>>>>>>>>>> because in
>>>>>>>>>>> protected boolean isAttachedDetachedHelperRequired()
>>>>>>>>>>> {
>>>>>>>>>>> return useIDs() || super.isAttachedDetachedHelperRequired();
>>>>>>>>>>> }
>>>>>>>>>>>
>>>>>>>>>>> useIDs() always returns true in XmlResourceImpl. In the
>>>>>>>>>>> genmodel, ResourceType is set to NONE.
>>>>>>>>>>>
>>>>>>>>>>> Ed Merks wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Stéphane,
>>>>>>>>>>>
>>>>>>>>>>>> Sorry my point is it's guarded like this:
>>>>>>>>>>>
>>>>>>>>>>>> public void attached(EObject eObject)
>>>>>>>>>>>> {
>>>>>>>>>>>> if (isAttachedDetachedHelperRequired())
>>>>>>>>>>>> {
>>>>>>>>>>>> attachedHelper(eObject);
>>>>>>>>>>>> for (TreeIterator<EObject> tree =
>>>>>>>>>>>> getAllProperContents(eObject); tree.hasNext(); )
>>>>>>>>>>>> {
>>>>>>>>>>>> attachedHelper(tree.next());
>>>>>>>>>>>> }
>>>>>>>>>>>> }
>>>>>>>>>>>> }
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> protected boolean isAttachedDetachedHelperRequired()
>>>>>>>>>>>> {
>>>>>>>>>>>> return useIDs() ||
>>>>>>>>>>>> super.isAttachedDetachedHelperRequired();
>>>>>>>>>>>> }
>>>>>>>>>>>
>>>>>>>>>>>> protected boolean useIDs()
>>>>>>>>>>>> {
>>>>>>>>>>>> return eObjectToIDMap != null || idToEObjectMap != null
>>>>>>>>>>>> || useUUIDs();
>>>>>>>>>>>> }
>>>>>>>>>>>
>>>>>>>>>>>> protected boolean useIDAttributes()
>>>>>>>>>>>> {
>>>>>>>>>>>> return true;
>>>>>>>>>>>> }
>>>>>>>>>>>
>>>>>>>>>>>> protected boolean useUUIDs()
>>>>>>>>>>>> {
>>>>>>>>>>>> return false;
>>>>>>>>>>>> }
>>>>>>>>>>>
>>>>>>>>>>>> protected boolean isAttachedDetachedHelperRequired()
>>>>>>>>>>>> {
>>>>>>>>>>>> return isTrackingModification() ||
>>>>>>>>>>>> getIntrinsicIDToEObjectMap() != null;
>>>>>>>>>>>> }
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> So generally I would expect the guard to return false by
>>>>>>>>>>>> default. So I'm wondering why it's returning true.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> Stéphane Fournier wrote:
>>>>>>>>>>>>> Ed Merks wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Stéphane,
>>>>>>>>>>>>>
>>>>>>>>>>>>>> So the tree walk is the answer. Can you tell why it's
>>>>>>>>>>>>>> needing to walk the tree? It's guarded such that the tree
>>>>>>>>>>>>>> walk is avoided in most cases...
>>>>>>>>>>>>>
>>>>>>>>>>>>> Here is my analysis :
>>>>>>>>>>>>> The tree is walked (on the added sub tree) because when I
>>>>>>>>>>>>> add a temp lib with a sub tree containing my 10 000
>>>>>>>>>>>>> children, no one is attached to a resource.
>>>>>>>>>>>>> When adding the temp lib in a lib attached to a resource,
>>>>>>>>>>>>> the EObjectContainmentWithInverseEList.Resolving#add calls
>>>>>>>>>>>>> the opposite relation i.e setParentBranch(..) that calls
>>>>>>>>>>>>> eBasicSetContainer(..) that calls
>>>>>>>>>>>>> ResourceImpl.attached(..).
>>>>>>>>>>>>> ResourceImpl.getAllProperContents returns the added sub
>>>>>>>>>>>>> tree objects that are not attached to the parent library
>>>>>>>>>>>>> resource yet.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Nevertheless, my tests show that's worth calling once
>>>>>>>>>>>>> ResourceImpl.attached() with a tree walk on children rather
>>>>>>>>>>>>> than calling x times this methods without tree walk.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Stéphane Fournier wrote:
>>>>>>>>>>>>>>> Ed Merks wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Stéphane,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Comments below.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Stéphane Fournier wrote:
>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Testing Teneo, I've made some performances comparisons
>>>>>>>>>>>>>>>>> against a standard XmiResource.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Scenario : I'm using the ExtendedLibrary meta-model. I
>>>>>>>>>>>>>>>>> have initialized a mysql database (installed on my
>>>>>>>>>>>>>>>>> computer) with a library with 10 000 children :
>>>>>>>>>>>>>>>>> sub-libraries, books, writers, employees...
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> As a library can be contained by another library,
>>>>>>>>>>>>>>>>> Library meta-class is not considered as a top class by
>>>>>>>>>>>>>>>>> Teneo. So, loading my resource gives me an empty
>>>>>>>>>>>>>>>>> resource content. I query the database to retrieve my
>>>>>>>>>>>>>>>>> root library (I tried to load the resource from an uri
>>>>>>>>>>>>>>>>> that specifies the initial query, but the open editor
>>>>>>>>>>>>>>>>> is in read-only).
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Some figures :
>>>>>>>>>>>>>>>>> Empty resource loaded in : 11012ms
>>>>>>>>>>>>>>>>> Root Library retrieved in : 16ms
>>>>>>>>>>>>>>>>> Root library added in the empty resource : 15225ms.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I doesn't understand why it's so long to add the root
>>>>>>>>>>>>>>>>> library in the HibernateResource.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Does anyone have an idea ?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I've made some additional tests against XmiResource.
>>>>>>>>>>>>>>>>> If I add a sub-tree with 10 000 children to a library
>>>>>>>>>>>>>>>>> already contained in a resource, it took 1s.
>>>>>>>>>>>>>>>>> If I add a sub-tree with 10 000 children to a temporary
>>>>>>>>>>>>>>>>> library not attached to a resource yet, it took 200ms +
>>>>>>>>>>>>>>>>> 63ms to attach the temporary library to a library
>>>>>>>>>>>>>>>>> contained in a resource.
>>>>>>>>>>>>>>>>> Does anyone have an idea to explain the performance
>>>>>>>>>>>>>>>>> differences ?
>>>>>>>>>>>>>>>> How much work does ResourceImpl.attached doing? Is it
>>>>>>>>>>>>>>>> walking the whole tree and computing UUIDs or something
>>>>>>>>>>>>>>>> like that?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I'm not using UUIDs, I use the default behavior based on
>>>>>>>>>>>>>>> positions.
>>>>>>>>>>>>>>> When I add 10 000 children to library contained in a
>>>>>>>>>>>>>>> resource, ResourceImpl.attached is called for each object
>>>>>>>>>>>>>>> but that code is never executed :
>>>>>>>>>>>>>>> for (TreeIterator<EObject> tree =
>>>>>>>>>>>>>>> getAllProperContents(eObject); tree.hasNext(); )
>>>>>>>>>>>>>>> {
>>>>>>>>>>>>>>> attachedHelper(tree.next());
>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> When adding a sub tree containing 10 000 objects to a not
>>>>>>>>>>>>>>> attached library, the ResourceImpl.attached is called
>>>>>>>>>>>>>>> once when the temporary lib is added to a resource
>>>>>>>>>>>>>>> contained one but the this time, the TreeIterator is
>>>>>>>>>>>>>>> walked to attach every contained objects.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I export the database content as a xml file.
>>>>>>>>>>>>>>>>> Export resource loaded in : 736ms
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Retrieving the root library in the database is straight
>>>>>>>>>>>>>>>>> forward (16ms), the library children are resolved
>>>>>>>>>>>>>>>>> (checked with the debugger just before adding the root
>>>>>>>>>>>>>>>>> library to the resource content).
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> What is the Teneo option to avoid library children auto
>>>>>>>>>>>>>>>>> resolution when querying for it.
>>>>>>>>>>>>>>>>> I would like to perform the children resolution when
>>>>>>>>>>>>>>>>> expanding the TreeItem only. (I set containment proxy
>>>>>>>>>>>>>>>>> to true in the genmodel). I tested
>>>>>>>>>>>>>>>>> PersistenceOptions.SET_PROXY but I didn't speed up
>>>>>>>>>>>>>>>>> resource loading.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Stephane.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>
>>>
>>>
>>>
>
>
>
>


--

With Regards, Martin Taal

Springsite/Elver.org
Office: Hardwareweg 4, 3821 BV Amersfoort
Postal: Nassaulaan 7, 3941 EC Doorn
The Netherlands
Cell: +31 (0)6 288 48 943
Tel: +31 (0)84 420 2397
Fax: +31 (0)84 225 9307
Mail: mtaal@springsite.com - mtaal@elver.org
Web: www.springsite.com - www.elver.org
Previous Topic:Retrieve EditingModel from EObject
Next Topic:[Teneo/JPA] can an @EmbeddedId use type=xsd:IDREF?
Goto Forum:
  


Current Time: Fri Mar 29 14:03:02 GMT 2024

Powered by FUDForum. Page generated in 0.05267 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top