Home » Modeling » EMF "Technology" (Ecore Tools, EMFatic, etc) » Performance problems with udge database
| Performance problems with udge database [message #113370] |
Tue, 26 February 2008 12:36  |
Eclipse User |
|
|
|
Originally posted by: Twister.yahoo.com
Hi,
I have done a RCP with GMF and hibernate and I use TENEO as described on
the elver site.
The root I use is the Eobject "Dataset".
I have created 2 datasets
the first dataset contains more than 8000 EObjects
the second contains only 20 EObjects.
1st problem :
When I open my first dataset, it is very slow, it takes a very long time
even if I open just a partial diagram containing something like 50
EObjects.
2nd problem :
When I open my diagram on my 2nd dataset with 20 EObject, It takes a very
long time as if it watches the all database.
If I create a new Dataset which is empty and I connect to my dataset, it
takes a very long time to open it.
to te contrary, If I have nothing in my database and i create my first
dataset, when i launch it, it is opened very quickly
Is there a problem with teneo or something else?
I'm worried for the RCP I'm working on, because I have very catastrofic
performances when my database grows up
thank's by advance
|
|
|
| Re: Performance problems with udge database [message #113383 is a reply to message #113370] |
Tue, 26 February 2008 13:39  |
| Eclipse User |
|
|
|
Hello,
This may (or may not) be your problem: We have a very large database of
objects that we were trying to load. Even though we selected only a few
of the millions of objects present we'd crash with an out of memory
exception.
It turned out that EMF treats the DB table as a single file and tries to
load the whole file into memory at once! We have 57 million records
in our table so it's not too surprising that we crashed. At 8000
records I'm not sure if you're hitting this problem or if its something
else, but it might be worth considering.
You can check if this is happening to you by turning on the SQL logging.
I have the following three files in my src directory to turn on the logging:
hibernate.properties contains
hibernate.show_sql=true
commons-logging.properties contains
org.apache.commons.logging.Log=org.apache.commons.logging.im pl.SimpleLog
simplelog.properties contains
org.apache.commons.logging.simplelog.defaultlog=ERROR
org.apache.commons.logging.simplelog.showlogname=true
org.apache.commons.logging.simplelog.showShortLogname=false
org.apache.commons.logging.simplelog.showdatetime=false
org.apache.commons.logging.simplelog.log.org.hibernate.type= trace
org.apache.commons.logging.simplelog.log.org.hibernate.SQL=t race
org.apache.commons.logging.simplelog.log.org.eclipse.emf.ten eo.annotations.parser.EAnnotationParserImporter=debug
If you see "select from tablename" you're doing a select * and it's
probably taking you a while to load all those resources. I have an ugly
fix for EMF that takes care of this, but the EMF guys want me to do
make a more elegant solution. If you find this is your problem let me
know and I'll share my work with you.
Good luck with your bug hunt,
Jason Henriksen
Twister wrote:
> Hi,
>
> I have done a RCP with GMF and hibernate and I use TENEO as described on
> the elver site.
>
> The root I use is the Eobject "Dataset".
> I have created 2 datasets
> the first dataset contains more than 8000 EObjects
> the second contains only 20 EObjects.
>
> 1st problem :
> When I open my first dataset, it is very slow, it takes a very long time
> even if I open just a partial diagram containing something like 50
> EObjects.
>
> 2nd problem :
> When I open my diagram on my 2nd dataset with 20 EObject, It takes a
> very long time as if it watches the all database.
> If I create a new Dataset which is empty and I connect to my dataset, it
> takes a very long time to open it.
>
> to te contrary, If I have nothing in my database and i create my first
> dataset, when i launch it, it is opened very quickly
>
> Is there a problem with teneo or something else?
> I'm worried for the RCP I'm working on, because I have very catastrofic
> performances when my database grows up
>
> thank's by advance
>
|
|
| | | | |
| Re: Performance problems with udge database [message #113667 is a reply to message #113655] |
Fri, 29 February 2008 17:18 |
| Eclipse User |
|
|
|
Originally posted by: merks.ca.ibm.com
Martin,
The default behavior for computing the objects that can be added to a
reference is to visit all reachable objects in the resource set. That
could obviously kill all attempts to be lazy. One would need to
specialize this behavior in derived property descriptors to take
advantage of domain knowledge to limit the search...
Martin Taal wrote:
> Hi Twister,
> See my comments below.
>
> gr. Martin
>
> Twister wrote:
>> Hello,
>> I finally managed to get only one Dataset by adding a query on the
>> resource URI. It now looks like :
>> hibernate://?sessionController=testsc&query1=FROM DataSet WHERE title
>> = 'test'
>>
>> But I still have the problem of an expensive browse of the database.
>> My datastore is initialized with SET_PROXY=true.
>>
>> -HQL Queries are ok.
>> -If I load my resource programmatically, it seems that only the
>> information that I need is retreived. But it retreives full
>> collection content in one time, is there any way to load only the nth
>> object in a collection ? Because I have very big collection
> MT>> Hibernate has support for this (extra-lazy loading) but I have
> not had the time yet to implement this. If you enter a bugzilla for
> this then it will be higher on my priority list.
>
>>
>> - The problem is when I want to show properties of an object in the
>> standard propertie View (without diagram editor). I think that it is
>> loading all the object to resolve Ereference.
>> These References are declared as resolveProxies=false and have opposite.
> MT>> I think you need to be more specific, in the end the system needs
> to show something in your properties view (like the identifying string
> of the referenced object). But it seems that it loads more than you
> would like?
>
>>
>> Thanks for your help
>>
>
>
|
|
|
| Re: Performance problems with udge database [message #615571 is a reply to message #113370] |
Tue, 26 February 2008 13:39 |
| Eclipse User |
|
|
|
Hello,
This may (or may not) be your problem: We have a very large database of
objects that we were trying to load. Even though we selected only a few
of the millions of objects present we'd crash with an out of memory
exception.
It turned out that EMF treats the DB table as a single file and tries to
load the whole file into memory at once! We have 57 million records
in our table so it's not too surprising that we crashed. At 8000
records I'm not sure if you're hitting this problem or if its something
else, but it might be worth considering.
You can check if this is happening to you by turning on the SQL logging.
I have the following three files in my src directory to turn on the logging:
hibernate.properties contains
hibernate.show_sql=true
commons-logging.properties contains
org.apache.commons.logging.Log=org.apache.commons.logging.im pl.SimpleLog
simplelog.properties contains
org.apache.commons.logging.simplelog.defaultlog=ERROR
org.apache.commons.logging.simplelog.showlogname=true
org.apache.commons.logging.simplelog.showShortLogname=false
org.apache.commons.logging.simplelog.showdatetime=false
org.apache.commons.logging.simplelog.log.org.hibernate.type= trace
org.apache.commons.logging.simplelog.log.org.hibernate.SQL=t race
org.apache.commons.logging.simplelog.log.org.eclipse.emf.ten eo.annotations.parser.EAnnotationParserImporter=debug
If you see "select from tablename" you're doing a select * and it's
probably taking you a while to load all those resources. I have an ugly
fix for EMF that takes care of this, but the EMF guys want me to do
make a more elegant solution. If you find this is your problem let me
know and I'll share my work with you.
Good luck with your bug hunt,
Jason Henriksen
Twister wrote:
> Hi,
>
> I have done a RCP with GMF and hibernate and I use TENEO as described on
> the elver site.
>
> The root I use is the Eobject "Dataset".
> I have created 2 datasets
> the first dataset contains more than 8000 EObjects
> the second contains only 20 EObjects.
>
> 1st problem :
> When I open my first dataset, it is very slow, it takes a very long time
> even if I open just a partial diagram containing something like 50
> EObjects.
>
> 2nd problem :
> When I open my diagram on my 2nd dataset with 20 EObject, It takes a
> very long time as if it watches the all database.
> If I create a new Dataset which is empty and I connect to my dataset, it
> takes a very long time to open it.
>
> to te contrary, If I have nothing in my database and i create my first
> dataset, when i launch it, it is opened very quickly
>
> Is there a problem with teneo or something else?
> I'm worried for the RCP I'm working on, because I have very catastrofic
> performances when my database grows up
>
> thank's by advance
>
|
|
|
| Re: Performance problems with udge database [message #615576 is a reply to message #113383] |
Wed, 27 February 2008 07:52 |
| Eclipse User |
|
|
|
Originally posted by: frederic.ventura.eurogiciel.fr
thank's a lot for your answer
can you tell me more about your ugly patch for emf?
what have you changed to solve this problem?
I've seen that the root_ID is not present in request whereas it should be
present in all request of my tool. as a result, my RCP seems watch all
database in spite of watching only the root contents
|
|
|
| Re: Performance problems with udge database [message #615577 is a reply to message #113445] |
Wed, 27 February 2008 12:32 |
| Eclipse User |
|
|
|
Hi Twister,
To help you on this I would require some more info, some questions from my side: how do you open the
resources, how does your uri look like? Is the content of the dataset lazily loaded or not?
When you open the resources programmatically (so not in a gmf diagram), what performance do you see
then?
When you do direct hql queries what performance do you see then?
gr. Martin
Twister31 wrote:
> thank's a lot for your answer
> can you tell me more about your ugly patch for emf?
> what have you changed to solve this problem?
> I've seen that the root_ID is not present in request whereas it should
> be present in all request of my tool. as a result, my RCP seems watch
> all database in spite of watching only the root contents
>
--
With Regards, Martin Taal
Springsite/Elver.org
Office: Hardwareweg 4, 3821 BV Amersfoort
Postal: Nassaulaan 7, 3941 EC Doorn
The Netherlands
Tel: +31 (0)84 420 2397
Fax: +31 (0)84 225 9307
Mail: mtaal@springsite.com - mtaal@elver.org
Web: www.springsite.com - www.elver.org
|
|
|
| Re: Performance problems with udge database [message #615587 is a reply to message #113458] |
Fri, 29 February 2008 09:08 |
| Eclipse User |
|
|
|
Originally posted by: sylvain.marcadal.eurogiciel.fr
Hello,
I finally managed to get only one Dataset by adding a query on the
resource URI. It now looks like :
hibernate://?sessionController=testsc&query1=FROM DataSet WHERE title =
'test'
But I still have the problem of an expensive browse of the database.
My datastore is initialized with SET_PROXY=true.
-HQL Queries are ok.
-If I load my resource programmatically, it seems that only the
information that I need is retreived. But it retreives full collection
content in one time, is there any way to load only the nth object in a
collection ? Because I have very big collection
- The problem is when I want to show properties of an object in the
standard propertie View (without diagram editor). I think that it is
loading all the object to resolve Ereference.
These References are declared as resolveProxies=false and have opposite.
Thanks for your help
|
|
|
| Re: Performance problems with udge database [message #615591 is a reply to message #113592] |
Fri, 29 February 2008 17:02 |
| Eclipse User |
|
|
|
Hi Twister,
See my comments below.
gr. Martin
Twister wrote:
> Hello,
> I finally managed to get only one Dataset by adding a query on the
> resource URI. It now looks like :
> hibernate://?sessionController=testsc&query1=FROM DataSet WHERE title =
> 'test'
>
> But I still have the problem of an expensive browse of the database.
> My datastore is initialized with SET_PROXY=true.
>
> -HQL Queries are ok.
> -If I load my resource programmatically, it seems that only the
> information that I need is retreived. But it retreives full collection
> content in one time, is there any way to load only the nth object in a
> collection ? Because I have very big collection
MT>> Hibernate has support for this (extra-lazy loading) but I have not had the time yet to
implement this. If you enter a bugzilla for this then it will be higher on my priority list.
>
> - The problem is when I want to show properties of an object in the
> standard propertie View (without diagram editor). I think that it is
> loading all the object to resolve Ereference.
> These References are declared as resolveProxies=false and have opposite.
MT>> I think you need to be more specific, in the end the system needs to show something in your
properties view (like the identifying string of the referenced object). But it seems that it loads
more than you would like?
>
> Thanks for your help
>
--
With Regards, Martin Taal
Springsite/Elver.org
Office: Hardwareweg 4, 3821 BV Amersfoort
Postal: Nassaulaan 7, 3941 EC Doorn
The Netherlands
Tel: +31 (0)84 420 2397
Fax: +31 (0)84 225 9307
Mail: mtaal@springsite.com - mtaal@elver.org
Web: www.springsite.com - www.elver.org
|
|
|
| Re: Performance problems with udge database [message #615592 is a reply to message #113655] |
Fri, 29 February 2008 17:18 |
| Eclipse User |
|
|
|
Martin,
The default behavior for computing the objects that can be added to a
reference is to visit all reachable objects in the resource set. That
could obviously kill all attempts to be lazy. One would need to
specialize this behavior in derived property descriptors to take
advantage of domain knowledge to limit the search...
Martin Taal wrote:
> Hi Twister,
> See my comments below.
>
> gr. Martin
>
> Twister wrote:
>> Hello,
>> I finally managed to get only one Dataset by adding a query on the
>> resource URI. It now looks like :
>> hibernate://?sessionController=testsc&query1=FROM DataSet WHERE title
>> = 'test'
>>
>> But I still have the problem of an expensive browse of the database.
>> My datastore is initialized with SET_PROXY=true.
>>
>> -HQL Queries are ok.
>> -If I load my resource programmatically, it seems that only the
>> information that I need is retreived. But it retreives full
>> collection content in one time, is there any way to load only the nth
>> object in a collection ? Because I have very big collection
> MT>> Hibernate has support for this (extra-lazy loading) but I have
> not had the time yet to implement this. If you enter a bugzilla for
> this then it will be higher on my priority list.
>
>>
>> - The problem is when I want to show properties of an object in the
>> standard propertie View (without diagram editor). I think that it is
>> loading all the object to resolve Ereference.
>> These References are declared as resolveProxies=false and have opposite.
> MT>> I think you need to be more specific, in the end the system needs
> to show something in your properties view (like the identifying string
> of the referenced object). But it seems that it loads more than you
> would like?
>
>>
>> Thanks for your help
>>
>
>
|
|
|
Goto Forum:
Current Time: Sun May 11 07:48:22 EDT 2025
Powered by FUDForum. Page generated in 0.04138 seconds |
Home