Home » Modeling » EMF "Technology" (Ecore Tools, EMFatic, etc) » Performance problems with udge database
| Performance problems with udge database [message #113370] |
Tue, 26 February 2008 17:36  |
Eclipse User |
|
|
|
Originally posted by: Twister.yahoo.com
Hi,
I have done a RCP with GMF and hibernate and I use TENEO as described on
the elver site.
The root I use is the Eobject "Dataset".
I have created 2 datasets
the first dataset contains more than 8000 EObjects
the second contains only 20 EObjects.
1st problem :
When I open my first dataset, it is very slow, it takes a very long time
even if I open just a partial diagram containing something like 50
EObjects.
2nd problem :
When I open my diagram on my 2nd dataset with 20 EObject, It takes a very
long time as if it watches the all database.
If I create a new Dataset which is empty and I connect to my dataset, it
takes a very long time to open it.
to te contrary, If I have nothing in my database and i create my first
dataset, when i launch it, it is opened very quickly
Is there a problem with teneo or something else?
I'm worried for the RCP I'm working on, because I have very catastrofic
performances when my database grows up
thank's by advance
|
|
|
| Re: Performance problems with udge database [message #113383 is a reply to message #113370] |
Tue, 26 February 2008 18:39  |
 Jason Henriksen
Messages: 231
Registered: July 2009 |
Senior Member |
|
|
Hello,
This may (or may not) be your problem: We have a very large database of
objects that we were trying to load. Even though we selected only a few
of the millions of objects present we'd crash with an out of memory
exception.
It turned out that EMF treats the DB table as a single file and tries to
load the whole file into memory at once! We have 57 million records
in our table so it's not too surprising that we crashed. At 8000
records I'm not sure if you're hitting this problem or if its something
else, but it might be worth considering.
You can check if this is happening to you by turning on the SQL logging.
I have the following three files in my src directory to turn on the logging:
hibernate.properties contains
hibernate.show_sql=true
commons-logging.properties contains
org.apache.commons.logging.Log=org.apache.commons.logging.im pl.SimpleLog
simplelog.properties contains
org.apache.commons.logging.simplelog.defaultlog=ERROR
org.apache.commons.logging.simplelog.showlogname=true
org.apache.commons.logging.simplelog.showShortLogname=false
org.apache.commons.logging.simplelog.showdatetime=false
org.apache.commons.logging.simplelog.log.org.hibernate.type= trace
org.apache.commons.logging.simplelog.log.org.hibernate.SQL=t race
org.apache.commons.logging.simplelog.log.org.eclipse.emf.ten eo.annotations.parser.EAnnotationParserImporter=debug
If you see "select from tablename" you're doing a select * and it's
probably taking you a while to load all those resources. I have an ugly
fix for EMF that takes care of this, but the EMF guys want me to do
make a more elegant solution. If you find this is your problem let me
know and I'll share my work with you.
Good luck with your bug hunt,
Jason Henriksen
Twister wrote:
> Hi,
>
> I have done a RCP with GMF and hibernate and I use TENEO as described on
> the elver site.
>
> The root I use is the Eobject "Dataset".
> I have created 2 datasets
> the first dataset contains more than 8000 EObjects
> the second contains only 20 EObjects.
>
> 1st problem :
> When I open my first dataset, it is very slow, it takes a very long time
> even if I open just a partial diagram containing something like 50
> EObjects.
>
> 2nd problem :
> When I open my diagram on my 2nd dataset with 20 EObject, It takes a
> very long time as if it watches the all database.
> If I create a new Dataset which is empty and I connect to my dataset, it
> takes a very long time to open it.
>
> to te contrary, If I have nothing in my database and i create my first
> dataset, when i launch it, it is opened very quickly
>
> Is there a problem with teneo or something else?
> I'm worried for the RCP I'm working on, because I have very catastrofic
> performances when my database grows up
>
> thank's by advance
>
|
|
|
| Re: Performance problems with udge database [message #113445 is a reply to message #113383] |
Wed, 27 February 2008 12:52 |
| Eclipse User |
|
|
|
Originally posted by: frederic.ventura.eurogiciel.fr
thank's a lot for your answer
can you tell me more about your ugly patch for emf?
what have you changed to solve this problem?
I've seen that the root_ID is not present in request whereas it should be
present in all request of my tool. as a result, my RCP seems watch all
database in spite of watching only the root contents
|
|
| |
| Re: Performance problems with udge database [message #113592 is a reply to message #113458] |
Fri, 29 February 2008 14:08 |
| Eclipse User |
|
|
|
Originally posted by: sylvain.marcadal.eurogiciel.fr
Hello,
I finally managed to get only one Dataset by adding a query on the
resource URI. It now looks like :
hibernate://?sessionController=testsc&query1=FROM DataSet WHERE title =
'test'
But I still have the problem of an expensive browse of the database.
My datastore is initialized with SET_PROXY=true.
-HQL Queries are ok.
-If I load my resource programmatically, it seems that only the
information that I need is retreived. But it retreives full collection
content in one time, is there any way to load only the nth object in a
collection ? Because I have very big collection
- The problem is when I want to show properties of an object in the
standard propertie View (without diagram editor). I think that it is
loading all the object to resolve Ereference.
These References are declared as resolveProxies=false and have opposite.
Thanks for your help
|
|
| |
| Re: Performance problems with udge database [message #113667 is a reply to message #113655] |
Fri, 29 February 2008 22:18 |
| Eclipse User |
|
|
|
Originally posted by: merks.ca.ibm.com
Martin,
The default behavior for computing the objects that can be added to a
reference is to visit all reachable objects in the resource set. That
could obviously kill all attempts to be lazy. One would need to
specialize this behavior in derived property descriptors to take
advantage of domain knowledge to limit the search...
Martin Taal wrote:
> Hi Twister,
> See my comments below.
>
> gr. Martin
>
> Twister wrote:
>> Hello,
>> I finally managed to get only one Dataset by adding a query on the
>> resource URI. It now looks like :
>> hibernate://?sessionController=testsc&query1=FROM DataSet WHERE title
>> = 'test'
>>
>> But I still have the problem of an expensive browse of the database.
>> My datastore is initialized with SET_PROXY=true.
>>
>> -HQL Queries are ok.
>> -If I load my resource programmatically, it seems that only the
>> information that I need is retreived. But it retreives full
>> collection content in one time, is there any way to load only the nth
>> object in a collection ? Because I have very big collection
> MT>> Hibernate has support for this (extra-lazy loading) but I have
> not had the time yet to implement this. If you enter a bugzilla for
> this then it will be higher on my priority list.
>
>>
>> - The problem is when I want to show properties of an object in the
>> standard propertie View (without diagram editor). I think that it is
>> loading all the object to resolve Ereference.
>> These References are declared as resolveProxies=false and have opposite.
> MT>> I think you need to be more specific, in the end the system needs
> to show something in your properties view (like the identifying string
> of the referenced object). But it seems that it loads more than you
> would like?
>
>>
>> Thanks for your help
>>
>
>
|
|
|
| Re: Performance problems with udge database [message #615571 is a reply to message #113370] |
Tue, 26 February 2008 18:39 |
Jason Henriksen
Messages: 231
Registered: July 2009 |
Senior Member |
|
|
Hello,
This may (or may not) be your problem: We have a very large database of
objects that we were trying to load. Even though we selected only a few
of the millions of objects present we'd crash with an out of memory
exception.
It turned out that EMF treats the DB table as a single file and tries to
load the whole file into memory at once! We have 57 million records
in our table so it's not too surprising that we crashed. At 8000
records I'm not sure if you're hitting this problem or if its something
else, but it might be worth considering.
You can check if this is happening to you by turning on the SQL logging.
I have the following three files in my src directory to turn on the logging:
hibernate.properties contains
hibernate.show_sql=true
commons-logging.properties contains
org.apache.commons.logging.Log=org.apache.commons.logging.im pl.SimpleLog
simplelog.properties contains
org.apache.commons.logging.simplelog.defaultlog=ERROR
org.apache.commons.logging.simplelog.showlogname=true
org.apache.commons.logging.simplelog.showShortLogname=false
org.apache.commons.logging.simplelog.showdatetime=false
org.apache.commons.logging.simplelog.log.org.hibernate.type= trace
org.apache.commons.logging.simplelog.log.org.hibernate.SQL=t race
org.apache.commons.logging.simplelog.log.org.eclipse.emf.ten eo.annotations.parser.EAnnotationParserImporter=debug
If you see "select from tablename" you're doing a select * and it's
probably taking you a while to load all those resources. I have an ugly
fix for EMF that takes care of this, but the EMF guys want me to do
make a more elegant solution. If you find this is your problem let me
know and I'll share my work with you.
Good luck with your bug hunt,
Jason Henriksen
Twister wrote:
> Hi,
>
> I have done a RCP with GMF and hibernate and I use TENEO as described on
> the elver site.
>
> The root I use is the Eobject "Dataset".
> I have created 2 datasets
> the first dataset contains more than 8000 EObjects
> the second contains only 20 EObjects.
>
> 1st problem :
> When I open my first dataset, it is very slow, it takes a very long time
> even if I open just a partial diagram containing something like 50
> EObjects.
>
> 2nd problem :
> When I open my diagram on my 2nd dataset with 20 EObject, It takes a
> very long time as if it watches the all database.
> If I create a new Dataset which is empty and I connect to my dataset, it
> takes a very long time to open it.
>
> to te contrary, If I have nothing in my database and i create my first
> dataset, when i launch it, it is opened very quickly
>
> Is there a problem with teneo or something else?
> I'm worried for the RCP I'm working on, because I have very catastrofic
> performances when my database grows up
>
> thank's by advance
>
|
|
|
| Re: Performance problems with udge database [message #615576 is a reply to message #113383] |
Wed, 27 February 2008 12:52 |
| Eclipse User |
|
|
|
Originally posted by: frederic.ventura.eurogiciel.fr
thank's a lot for your answer
can you tell me more about your ugly patch for emf?
what have you changed to solve this problem?
I've seen that the root_ID is not present in request whereas it should be
present in all request of my tool. as a result, my RCP seems watch all
database in spite of watching only the root contents
|
|
| |
| Re: Performance problems with udge database [message #615587 is a reply to message #113458] |
Fri, 29 February 2008 14:08 |
| Eclipse User |
|
|
|
Originally posted by: sylvain.marcadal.eurogiciel.fr
Hello,
I finally managed to get only one Dataset by adding a query on the
resource URI. It now looks like :
hibernate://?sessionController=testsc&query1=FROM DataSet WHERE title =
'test'
But I still have the problem of an expensive browse of the database.
My datastore is initialized with SET_PROXY=true.
-HQL Queries are ok.
-If I load my resource programmatically, it seems that only the
information that I need is retreived. But it retreives full collection
content in one time, is there any way to load only the nth object in a
collection ? Because I have very big collection
- The problem is when I want to show properties of an object in the
standard propertie View (without diagram editor). I think that it is
loading all the object to resolve Ereference.
These References are declared as resolveProxies=false and have opposite.
Thanks for your help
|
|
| |
| Re: Performance problems with udge database [message #615592 is a reply to message #113655] |
Fri, 29 February 2008 22:18 |
Ed Merks
Messages: 32834
Registered: July 2009 |
Senior Member |
|
|
Martin,
The default behavior for computing the objects that can be added to a
reference is to visit all reachable objects in the resource set. That
could obviously kill all attempts to be lazy. One would need to
specialize this behavior in derived property descriptors to take
advantage of domain knowledge to limit the search...
Martin Taal wrote:
> Hi Twister,
> See my comments below.
>
> gr. Martin
>
> Twister wrote:
>> Hello,
>> I finally managed to get only one Dataset by adding a query on the
>> resource URI. It now looks like :
>> hibernate://?sessionController=testsc&query1=FROM DataSet WHERE title
>> = 'test'
>>
>> But I still have the problem of an expensive browse of the database.
>> My datastore is initialized with SET_PROXY=true.
>>
>> -HQL Queries are ok.
>> -If I load my resource programmatically, it seems that only the
>> information that I need is retreived. But it retreives full
>> collection content in one time, is there any way to load only the nth
>> object in a collection ? Because I have very big collection
> MT>> Hibernate has support for this (extra-lazy loading) but I have
> not had the time yet to implement this. If you enter a bugzilla for
> this then it will be higher on my priority list.
>
>>
>> - The problem is when I want to show properties of an object in the
>> standard propertie View (without diagram editor). I think that it is
>> loading all the object to resolve Ereference.
>> These References are declared as resolveProxies=false and have opposite.
> MT>> I think you need to be more specific, in the end the system needs
> to show something in your properties view (like the identifying string
> of the referenced object). But it seems that it loads more than you
> would like?
>
>>
>> Thanks for your help
>>
>
>
Ed Merks
Professional Support: https://www.macromodeling.com/
|
|
|
Goto Forum:
Current Time: Wed Jun 07 06:07:02 GMT 2023
Powered by FUDForum. Page generated in 0.02416 seconds |
] 
Search
Help
Register
Login
Home
