Home » Modeling » EMF "Technology" (Ecore Tools, EMFatic, etc) » Too many join - issue with the HibernateMapping file
|
| Re: Too many join - issue with the HibernateMapping file [message #112799 is a reply to message #112746] |
Wed, 20 February 2008 19:23   |
 Martin Taal
Messages: 5468
Registered: July 2009 |
Senior Member |
|
|
Hi Yigal,
Ok, can you send me the ecore (directly by email)?
gr. Martin
Yigal wrote:
> I have a very complex EMF module (over 150 classes). All classes in the
> model are extend of one of two basic types: Part or Part_Relation.
> Part is an abstract class that identifies a generic part. Part_relation
> associates part to other parts.
> Example: A car engine is a "Part" of the Car assemble. The engine has
> many parts in side it. Those parts are grouped by functionality, like
> moving parts, common parts, and etc. So I have Engine_MovingPart_Rel
> that relates any moving parts to the engine.
>
> Basic relation:
> |----------| |------------| |-------------|
> | | Aggregation | | Association | |
> | Engine |-------------->| Eng_Rel |------------->| Part |
> | |* 1| |1 1| |
> |----------| |------------| |-------------|
>
> This way I can associate different parts to the engine.
> The Engine_MovingPart_Rel extends the Eng_Rel and adds additional
> attributes that makes the assemble.
>
> |----------| |------------| |-------------|
> | | Aggregation | | Association | |
> | Engine |-------------->| Eng_MovePart------------->| Piston |
> | |* 1| Rel |1 1| |
> |----------| |------------| |-------------|
>
> Piston expands "Part".
> |----------| |------------| |-------------|
> | | Aggregation | | Association | |
> | Piston |-------------->| Piston_Rel |------------->| PistonRings |
> | |* 1| |1 1| |
> |----------| |------------| |-------------|
> (and so on and on)
>
> I generated the Hibernate mapping file using the Join option.
> When I create a HibernateResource and load an Engine, and want to LAZY
> load all the different Engine_REL, I get over 69 joins that is an over
> flow in mySql and causes an exception (fail). I investigated the SQL
> commands and found that hibernate wants to load the entire model and
> does not do lazy loading of the relationships. The SQL join all the
> relations with all the parts and relations between those parts and other
> parts.
>
> I tried the following:
> 1. I put the @Proxy attribute on the "Piston" parts, the result was
> combined tables of all abstract classes and database table only on non
> abstract classes (duplication of data). But the issue was still there
> and the join fails with more then 61 joins)
>
> 2. I tries to use the cascade fetch(Lazy) annotation. Did not help.
>
> Currently I use the fist option of generating the the hibernate mapping
> file,but that creates only TWO tables in the Database with all
> attributes in them and the need for over 64 indexes which is not support
> either. The join issue does not exists and I can run my application.
> However a flat database is not acceptable and I have no indexes.
>
> Any solution?
>
> Thanks
> Yigal
--
With Regards, Martin Taal
Springsite/Elver.org
Office: Hardwareweg 4, 3821 BV Amersfoort
Postal: Nassaulaan 7, 3941 EC Doorn
The Netherlands
Tel: +31 (0)84 420 2397
Fax: +31 (0)84 225 9307
Mail: mtaal@springsite.com - mtaal@elver.org
Web: www.springsite.com - www.elver.org
|
|
|
| Re: Too many join - issue with the HibernateMapping file [message #112825 is a reply to message #112799] |
Wed, 20 February 2008 19:56 |
Yigal Spinner
Messages: 127
Registered: July 2009 |
Senior Member |
|
|
Hi Martine,
Sorry but the eCore is an Intellectual Property of my company and can
not be distributed with out an NDA.
The sample below is not the real eCore but a close enough approximation.
I was hopping it would be enough to get some directions to solve my issue.
Yigal
Martin Taal wrote:
> Hi Yigal,
> Ok, can you send me the ecore (directly by email)?
>
> gr. Martin
>
> Yigal wrote:
>> I have a very complex EMF module (over 150 classes). All classes in
>> the model are extend of one of two basic types: Part or Part_Relation.
>> Part is an abstract class that identifies a generic part.
>> Part_relation associates part to other parts.
>> Example: A car engine is a "Part" of the Car assemble. The engine
>> has many parts in side it. Those parts are grouped by functionality,
>> like moving parts, common parts, and etc. So I have
>> Engine_MovingPart_Rel that relates any moving parts to the engine.
>>
>> Basic relation:
>> |----------| |------------| |-------------|
>> | | Aggregation | | Association | |
>> | Engine |-------------->| Eng_Rel |------------->| Part |
>> | |* 1| |1 1| |
>> |----------| |------------| |-------------|
>>
>> This way I can associate different parts to the engine.
>> The Engine_MovingPart_Rel extends the Eng_Rel and adds additional
>> attributes that makes the assemble.
>>
>> |----------| |------------| |-------------|
>> | | Aggregation | | Association | |
>> | Engine |-------------->| Eng_MovePart------------->| Piston |
>> | |* 1| Rel |1 1| |
>> |----------| |------------| |-------------|
>>
>> Piston expands "Part".
>> |----------| |------------| |-------------|
>> | | Aggregation | | Association | |
>> | Piston |-------------->| Piston_Rel |------------->| PistonRings |
>> | |* 1| |1 1| |
>> |----------| |------------| |-------------|
>> (and so on and on)
>>
>> I generated the Hibernate mapping file using the Join option.
>> When I create a HibernateResource and load an Engine, and want to LAZY
>> load all the different Engine_REL, I get over 69 joins that is an over
>> flow in mySql and causes an exception (fail). I investigated the SQL
>> commands and found that hibernate wants to load the entire model and
>> does not do lazy loading of the relationships. The SQL join all the
>> relations with all the parts and relations between those parts and
>> other parts.
>>
>> I tried the following:
>> 1. I put the @Proxy attribute on the "Piston" parts, the result was
>> combined tables of all abstract classes and database table only on non
>> abstract classes (duplication of data). But the issue was still there
>> and the join fails with more then 61 joins)
>>
>> 2. I tries to use the cascade fetch(Lazy) annotation. Did not help.
>>
>> Currently I use the fist option of generating the the hibernate
>> mapping file,but that creates only TWO tables in the Database with all
>> attributes in them and the need for over 64 indexes which is not
>> support either. The join issue does not exists and I can run my
>> application.
>> However a flat database is not acceptable and I have no indexes.
>>
>> Any solution?
>>
>> Thanks
>> Yigal
>
>
|
|
|
| Re: Too many join - issue with the HibernateMapping file [message #112851 is a reply to message #112825] |
Wed, 20 February 2008 21:44 |
Martin Taal
Messages: 5468
Registered: July 2009 |
Senior Member |
|
|
Hi Yigal,
Hmm it is the easiest if I can reproduce the issue (which afaics is mainly a lazy-loading issue)
with a small test model.
What was not completely clear from the model why the eng_rel to part is a 1-1 (1 on both sides), I
would have expected a model like this:
Engine (1)------(*) Eng_Rel (*) -------(1) Part
based on your description. So therefore my confusion.
Does this maybe apply to your situation (minOccurs>0):
http://www.elver.org/hibernate/hibernateresources.html#Valid ation
Lazy loading works in Teneo I have multiple testcases covering this. So it must be something related
to your model (and how Teneo handles it ofcourse). Can you make a small fake subset of your model
which shows the same (eager loading) behavior? Which can be send to me without nda.
gr. Martin
Yigal wrote:
> Hi Martine,
> Sorry but the eCore is an Intellectual Property of my company and can
> not be distributed with out an NDA.
>
> The sample below is not the real eCore but a close enough approximation.
> I was hopping it would be enough to get some directions to solve my issue.
>
> Yigal
>
> Martin Taal wrote:
>> Hi Yigal,
>> Ok, can you send me the ecore (directly by email)?
>>
>> gr. Martin
>>
>> Yigal wrote:
>>> I have a very complex EMF module (over 150 classes). All classes in
>>> the model are extend of one of two basic types: Part or Part_Relation.
>>> Part is an abstract class that identifies a generic part.
>>> Part_relation associates part to other parts.
>>> Example: A car engine is a "Part" of the Car assemble. The engine
>>> has many parts in side it. Those parts are grouped by functionality,
>>> like moving parts, common parts, and etc. So I have
>>> Engine_MovingPart_Rel that relates any moving parts to the engine.
>>>
>>> Basic relation:
>>> |----------| |------------| |-------------|
>>> | | Aggregation | | Association | |
>>> | Engine |-------------->| Eng_Rel |------------->| Part |
>>> | |* 1| |1 1| |
>>> |----------| |------------| |-------------|
>>>
>>> This way I can associate different parts to the engine.
>>> The Engine_MovingPart_Rel extends the Eng_Rel and adds additional
>>> attributes that makes the assemble.
>>>
>>> |----------| |------------| |-------------|
>>> | | Aggregation | | Association | |
>>> | Engine |-------------->| Eng_MovePart------------->| Piston |
>>> | |* 1| Rel |1 1| |
>>> |----------| |------------| |-------------|
>>>
>>> Piston expands "Part".
>>> |----------| |------------| |-------------|
>>> | | Aggregation | | Association | |
>>> | Piston |-------------->| Piston_Rel |------------->| PistonRings |
>>> | |* 1| |1 1| |
>>> |----------| |------------| |-------------|
>>> (and so on and on)
>>>
>>> I generated the Hibernate mapping file using the Join option.
>>> When I create a HibernateResource and load an Engine, and want to
>>> LAZY load all the different Engine_REL, I get over 69 joins that is
>>> an over flow in mySql and causes an exception (fail). I investigated
>>> the SQL commands and found that hibernate wants to load the entire
>>> model and does not do lazy loading of the relationships. The SQL
>>> join all the relations with all the parts and relations between those
>>> parts and other parts.
>>>
>>> I tried the following:
>>> 1. I put the @Proxy attribute on the "Piston" parts, the result was
>>> combined tables of all abstract classes and database table only on
>>> non abstract classes (duplication of data). But the issue was still
>>> there and the join fails with more then 61 joins)
>>>
>>> 2. I tries to use the cascade fetch(Lazy) annotation. Did not help.
>>>
>>> Currently I use the fist option of generating the the hibernate
>>> mapping file,but that creates only TWO tables in the Database with
>>> all attributes in them and the need for over 64 indexes which is not
>>> support either. The join issue does not exists and I can run my
>>> application.
>>> However a flat database is not acceptable and I have no indexes.
>>>
>>> Any solution?
>>>
>>> Thanks
>>> Yigal
>>
>>
--
With Regards, Martin Taal
Springsite/Elver.org
Office: Hardwareweg 4, 3821 BV Amersfoort
Postal: Nassaulaan 7, 3941 EC Doorn
The Netherlands
Tel: +31 (0)84 420 2397
Fax: +31 (0)84 225 9307
Mail: mtaal@springsite.com - mtaal@elver.org
Web: www.springsite.com - www.elver.org
|
|
|
| Re: Too many join - issue with the HibernateMapping file [message #116116 is a reply to message #112851] |
Thu, 27 March 2008 12:27 |
Yigal Spinner
Messages: 127
Registered: July 2009 |
Senior Member |
|
|
Hi Martin,
I figured out the issue. The issue was with hibernate options.
There is an option called MAX_FETCH_DEPTH and the default is set to 3.
In my module a level three at the point of collecting all parts causes
too many join tables. In any case, I want to do lazy loading, so I set
the option to level 0 and now it works fine.
Thanks
Yigal
Martin Taal wrote:
> Hi Yigal,
> Hmm it is the easiest if I can reproduce the issue (which afaics is
> mainly a lazy-loading issue) with a small test model.
>
> What was not completely clear from the model why the eng_rel to part is
> a 1-1 (1 on both sides), I would have expected a model like this:
>
> Engine (1)------(*) Eng_Rel (*) -------(1) Part
>
> based on your description. So therefore my confusion.
>
> Does this maybe apply to your situation (minOccurs>0):
> http://www.elver.org/hibernate/hibernateresources.html#Valid ation
>
> Lazy loading works in Teneo I have multiple testcases covering this. So
> it must be something related to your model (and how Teneo handles it
> ofcourse). Can you make a small fake subset of your model which shows
> the same (eager loading) behavior? Which can be send to me without nda.
>
> gr. Martin
>
> Yigal wrote:
>> Hi Martine,
>> Sorry but the eCore is an Intellectual Property of my company and can
>> not be distributed with out an NDA.
>>
>> The sample below is not the real eCore but a close enough approximation.
>> I was hopping it would be enough to get some directions to solve my
>> issue.
>>
>> Yigal
>>
>> Martin Taal wrote:
>>> Hi Yigal,
>>> Ok, can you send me the ecore (directly by email)?
>>>
>>> gr. Martin
>>>
>>> Yigal wrote:
>>>> I have a very complex EMF module (over 150 classes). All classes in
>>>> the model are extend of one of two basic types: Part or Part_Relation.
>>>> Part is an abstract class that identifies a generic part.
>>>> Part_relation associates part to other parts.
>>>> Example: A car engine is a "Part" of the Car assemble. The engine
>>>> has many parts in side it. Those parts are grouped by functionality,
>>>> like moving parts, common parts, and etc. So I have
>>>> Engine_MovingPart_Rel that relates any moving parts to the engine.
>>>>
>>>> Basic relation:
>>>> |----------| |------------| |-------------|
>>>> | | Aggregation | | Association | |
>>>> | Engine |-------------->| Eng_Rel |------------->| Part |
>>>> | |* 1| |1 1| |
>>>> |----------| |------------| |-------------|
>>>>
>>>> This way I can associate different parts to the engine.
>>>> The Engine_MovingPart_Rel extends the Eng_Rel and adds additional
>>>> attributes that makes the assemble.
>>>>
>>>> |----------| |------------| |-------------|
>>>> | | Aggregation | | Association | |
>>>> | Engine |-------------->| Eng_MovePart------------->| Piston |
>>>> | |* 1| Rel |1 1| |
>>>> |----------| |------------| |-------------|
>>>>
>>>> Piston expands "Part".
>>>> |----------| |------------| |-------------|
>>>> | | Aggregation | | Association | |
>>>> | Piston |-------------->| Piston_Rel |------------->| PistonRings |
>>>> | |* 1| |1 1| |
>>>> |----------| |------------| |-------------|
>>>> (and so on and on)
>>>>
>>>> I generated the Hibernate mapping file using the Join option.
>>>> When I create a HibernateResource and load an Engine, and want to
>>>> LAZY load all the different Engine_REL, I get over 69 joins that is
>>>> an over flow in mySql and causes an exception (fail). I
>>>> investigated the SQL commands and found that hibernate wants to load
>>>> the entire model and does not do lazy loading of the relationships.
>>>> The SQL join all the relations with all the parts and relations
>>>> between those parts and other parts.
>>>>
>>>> I tried the following:
>>>> 1. I put the @Proxy attribute on the "Piston" parts, the result was
>>>> combined tables of all abstract classes and database table only on
>>>> non abstract classes (duplication of data). But the issue was still
>>>> there and the join fails with more then 61 joins)
>>>>
>>>> 2. I tries to use the cascade fetch(Lazy) annotation. Did not help.
>>>>
>>>> Currently I use the fist option of generating the the hibernate
>>>> mapping file,but that creates only TWO tables in the Database with
>>>> all attributes in them and the need for over 64 indexes which is not
>>>> support either. The join issue does not exists and I can run my
>>>> application.
>>>> However a flat database is not acceptable and I have no indexes.
>>>>
>>>> Any solution?
>>>>
>>>> Thanks
>>>> Yigal
>>>
>>>
>
>
|
|
|
| Re: Too many join - issue with the HibernateMapping file [message #116129 is a reply to message #116116] |
Thu, 27 March 2008 12:44 |
Martin Taal
Messages: 5468
Registered: July 2009 |
Senior Member |
|
|
Hi Yigal,
Thanks for the update, I'll add this to the troubleshooting section of the docs.
gr. Martin
Yigal wrote:
> Hi Martin,
> I figured out the issue. The issue was with hibernate options.
> There is an option called MAX_FETCH_DEPTH and the default is set to 3.
> In my module a level three at the point of collecting all parts causes
> too many join tables. In any case, I want to do lazy loading, so I set
> the option to level 0 and now it works fine.
>
> Thanks
> Yigal
>
> Martin Taal wrote:
>> Hi Yigal,
>> Hmm it is the easiest if I can reproduce the issue (which afaics is
>> mainly a lazy-loading issue) with a small test model.
>>
>> What was not completely clear from the model why the eng_rel to part
>> is a 1-1 (1 on both sides), I would have expected a model like this:
>>
>> Engine (1)------(*) Eng_Rel (*) -------(1) Part
>>
>> based on your description. So therefore my confusion.
>>
>> Does this maybe apply to your situation (minOccurs>0):
>> http://www.elver.org/hibernate/hibernateresources.html#Valid ation
>>
>> Lazy loading works in Teneo I have multiple testcases covering this.
>> So it must be something related to your model (and how Teneo handles
>> it ofcourse). Can you make a small fake subset of your model which
>> shows the same (eager loading) behavior? Which can be send to me
>> without nda.
>>
>> gr. Martin
>>
>> Yigal wrote:
>>> Hi Martine,
>>> Sorry but the eCore is an Intellectual Property of my company and can
>>> not be distributed with out an NDA.
>>>
>>> The sample below is not the real eCore but a close enough approximation.
>>> I was hopping it would be enough to get some directions to solve my
>>> issue.
>>>
>>> Yigal
>>>
>>> Martin Taal wrote:
>>>> Hi Yigal,
>>>> Ok, can you send me the ecore (directly by email)?
>>>>
>>>> gr. Martin
>>>>
>>>> Yigal wrote:
>>>>> I have a very complex EMF module (over 150 classes). All classes
>>>>> in the model are extend of one of two basic types: Part or
>>>>> Part_Relation.
>>>>> Part is an abstract class that identifies a generic part.
>>>>> Part_relation associates part to other parts.
>>>>> Example: A car engine is a "Part" of the Car assemble. The engine
>>>>> has many parts in side it. Those parts are grouped by
>>>>> functionality, like moving parts, common parts, and etc. So I have
>>>>> Engine_MovingPart_Rel that relates any moving parts to the engine.
>>>>>
>>>>> Basic relation:
>>>>> |----------| |------------| |-------------|
>>>>> | | Aggregation | | Association | |
>>>>> | Engine |-------------->| Eng_Rel |------------->| Part |
>>>>> | |* 1| |1 1| |
>>>>> |----------| |------------| |-------------|
>>>>>
>>>>> This way I can associate different parts to the engine.
>>>>> The Engine_MovingPart_Rel extends the Eng_Rel and adds additional
>>>>> attributes that makes the assemble.
>>>>>
>>>>> |----------| |------------| |-------------|
>>>>> | | Aggregation | | Association | |
>>>>> | Engine |-------------->| Eng_MovePart------------->| Piston |
>>>>> | |* 1| Rel |1 1| |
>>>>> |----------| |------------| |-------------|
>>>>>
>>>>> Piston expands "Part".
>>>>> |----------| |------------| |-------------|
>>>>> | | Aggregation | | Association | |
>>>>> | Piston |-------------->| Piston_Rel |------------->| PistonRings |
>>>>> | |* 1| |1 1| |
>>>>> |----------| |------------| |-------------|
>>>>> (and so on and on)
>>>>>
>>>>> I generated the Hibernate mapping file using the Join option.
>>>>> When I create a HibernateResource and load an Engine, and want to
>>>>> LAZY load all the different Engine_REL, I get over 69 joins that is
>>>>> an over flow in mySql and causes an exception (fail). I
>>>>> investigated the SQL commands and found that hibernate wants to
>>>>> load the entire model and does not do lazy loading of the
>>>>> relationships. The SQL join all the relations with all the parts
>>>>> and relations between those parts and other parts.
>>>>>
>>>>> I tried the following:
>>>>> 1. I put the @Proxy attribute on the "Piston" parts, the result was
>>>>> combined tables of all abstract classes and database table only on
>>>>> non abstract classes (duplication of data). But the issue was still
>>>>> there and the join fails with more then 61 joins)
>>>>>
>>>>> 2. I tries to use the cascade fetch(Lazy) annotation. Did not help.
>>>>>
>>>>> Currently I use the fist option of generating the the hibernate
>>>>> mapping file,but that creates only TWO tables in the Database with
>>>>> all attributes in them and the need for over 64 indexes which is
>>>>> not support either. The join issue does not exists and I can run
>>>>> my application.
>>>>> However a flat database is not acceptable and I have no indexes.
>>>>>
>>>>> Any solution?
>>>>>
>>>>> Thanks
>>>>> Yigal
>>>>
>>>>
>>
>>
--
With Regards, Martin Taal
Springsite/Elver.org
Office: Hardwareweg 4, 3821 BV Amersfoort
Postal: Nassaulaan 7, 3941 EC Doorn
The Netherlands
Tel: +31 (0)84 420 2397
Fax: +31 (0)84 225 9307
Mail: mtaal@springsite.com - mtaal@elver.org
Web: www.springsite.com - www.elver.org
|
|
|
| Re: Too many join - issue with the HibernateMapping file [message #615527 is a reply to message #112746] |
Wed, 20 February 2008 19:23 |
Martin Taal
Messages: 5468
Registered: July 2009 |
Senior Member |
|
|
Hi Yigal,
Ok, can you send me the ecore (directly by email)?
gr. Martin
Yigal wrote:
> I have a very complex EMF module (over 150 classes). All classes in the
> model are extend of one of two basic types: Part or Part_Relation.
> Part is an abstract class that identifies a generic part. Part_relation
> associates part to other parts.
> Example: A car engine is a "Part" of the Car assemble. The engine has
> many parts in side it. Those parts are grouped by functionality, like
> moving parts, common parts, and etc. So I have Engine_MovingPart_Rel
> that relates any moving parts to the engine.
>
> Basic relation:
> |----------| |------------| |-------------|
> | | Aggregation | | Association | |
> | Engine |-------------->| Eng_Rel |------------->| Part |
> | |* 1| |1 1| |
> |----------| |------------| |-------------|
>
> This way I can associate different parts to the engine.
> The Engine_MovingPart_Rel extends the Eng_Rel and adds additional
> attributes that makes the assemble.
>
> |----------| |------------| |-------------|
> | | Aggregation | | Association | |
> | Engine |-------------->| Eng_MovePart------------->| Piston |
> | |* 1| Rel |1 1| |
> |----------| |------------| |-------------|
>
> Piston expands "Part".
> |----------| |------------| |-------------|
> | | Aggregation | | Association | |
> | Piston |-------------->| Piston_Rel |------------->| PistonRings |
> | |* 1| |1 1| |
> |----------| |------------| |-------------|
> (and so on and on)
>
> I generated the Hibernate mapping file using the Join option.
> When I create a HibernateResource and load an Engine, and want to LAZY
> load all the different Engine_REL, I get over 69 joins that is an over
> flow in mySql and causes an exception (fail). I investigated the SQL
> commands and found that hibernate wants to load the entire model and
> does not do lazy loading of the relationships. The SQL join all the
> relations with all the parts and relations between those parts and other
> parts.
>
> I tried the following:
> 1. I put the @Proxy attribute on the "Piston" parts, the result was
> combined tables of all abstract classes and database table only on non
> abstract classes (duplication of data). But the issue was still there
> and the join fails with more then 61 joins)
>
> 2. I tries to use the cascade fetch(Lazy) annotation. Did not help.
>
> Currently I use the fist option of generating the the hibernate mapping
> file,but that creates only TWO tables in the Database with all
> attributes in them and the need for over 64 indexes which is not support
> either. The join issue does not exists and I can run my application.
> However a flat database is not acceptable and I have no indexes.
>
> Any solution?
>
> Thanks
> Yigal
--
With Regards, Martin Taal
Springsite/Elver.org
Office: Hardwareweg 4, 3821 BV Amersfoort
Postal: Nassaulaan 7, 3941 EC Doorn
The Netherlands
Tel: +31 (0)84 420 2397
Fax: +31 (0)84 225 9307
Mail: mtaal@springsite.com - mtaal@elver.org
Web: www.springsite.com - www.elver.org
|
|
|
| Re: Too many join - issue with the HibernateMapping file [message #615529 is a reply to message #112799] |
Wed, 20 February 2008 19:56 |
Yigal Spinner
Messages: 127
Registered: July 2009 |
Senior Member |
|
|
Hi Martine,
Sorry but the eCore is an Intellectual Property of my company and can
not be distributed with out an NDA.
The sample below is not the real eCore but a close enough approximation.
I was hopping it would be enough to get some directions to solve my issue.
Yigal
Martin Taal wrote:
> Hi Yigal,
> Ok, can you send me the ecore (directly by email)?
>
> gr. Martin
>
> Yigal wrote:
>> I have a very complex EMF module (over 150 classes). All classes in
>> the model are extend of one of two basic types: Part or Part_Relation.
>> Part is an abstract class that identifies a generic part.
>> Part_relation associates part to other parts.
>> Example: A car engine is a "Part" of the Car assemble. The engine
>> has many parts in side it. Those parts are grouped by functionality,
>> like moving parts, common parts, and etc. So I have
>> Engine_MovingPart_Rel that relates any moving parts to the engine.
>>
>> Basic relation:
>> |----------| |------------| |-------------|
>> | | Aggregation | | Association | |
>> | Engine |-------------->| Eng_Rel |------------->| Part |
>> | |* 1| |1 1| |
>> |----------| |------------| |-------------|
>>
>> This way I can associate different parts to the engine.
>> The Engine_MovingPart_Rel extends the Eng_Rel and adds additional
>> attributes that makes the assemble.
>>
>> |----------| |------------| |-------------|
>> | | Aggregation | | Association | |
>> | Engine |-------------->| Eng_MovePart------------->| Piston |
>> | |* 1| Rel |1 1| |
>> |----------| |------------| |-------------|
>>
>> Piston expands "Part".
>> |----------| |------------| |-------------|
>> | | Aggregation | | Association | |
>> | Piston |-------------->| Piston_Rel |------------->| PistonRings |
>> | |* 1| |1 1| |
>> |----------| |------------| |-------------|
>> (and so on and on)
>>
>> I generated the Hibernate mapping file using the Join option.
>> When I create a HibernateResource and load an Engine, and want to LAZY
>> load all the different Engine_REL, I get over 69 joins that is an over
>> flow in mySql and causes an exception (fail). I investigated the SQL
>> commands and found that hibernate wants to load the entire model and
>> does not do lazy loading of the relationships. The SQL join all the
>> relations with all the parts and relations between those parts and
>> other parts.
>>
>> I tried the following:
>> 1. I put the @Proxy attribute on the "Piston" parts, the result was
>> combined tables of all abstract classes and database table only on non
>> abstract classes (duplication of data). But the issue was still there
>> and the join fails with more then 61 joins)
>>
>> 2. I tries to use the cascade fetch(Lazy) annotation. Did not help.
>>
>> Currently I use the fist option of generating the the hibernate
>> mapping file,but that creates only TWO tables in the Database with all
>> attributes in them and the need for over 64 indexes which is not
>> support either. The join issue does not exists and I can run my
>> application.
>> However a flat database is not acceptable and I have no indexes.
>>
>> Any solution?
>>
>> Thanks
>> Yigal
>
>
|
|
|
| Re: Too many join - issue with the HibernateMapping file [message #615531 is a reply to message #112825] |
Wed, 20 February 2008 21:44 |
Martin Taal
Messages: 5468
Registered: July 2009 |
Senior Member |
|
|
Hi Yigal,
Hmm it is the easiest if I can reproduce the issue (which afaics is mainly a lazy-loading issue)
with a small test model.
What was not completely clear from the model why the eng_rel to part is a 1-1 (1 on both sides), I
would have expected a model like this:
Engine (1)------(*) Eng_Rel (*) -------(1) Part
based on your description. So therefore my confusion.
Does this maybe apply to your situation (minOccurs>0):
http://www.elver.org/hibernate/hibernateresources.html#Valid ation
Lazy loading works in Teneo I have multiple testcases covering this. So it must be something related
to your model (and how Teneo handles it ofcourse). Can you make a small fake subset of your model
which shows the same (eager loading) behavior? Which can be send to me without nda.
gr. Martin
Yigal wrote:
> Hi Martine,
> Sorry but the eCore is an Intellectual Property of my company and can
> not be distributed with out an NDA.
>
> The sample below is not the real eCore but a close enough approximation.
> I was hopping it would be enough to get some directions to solve my issue.
>
> Yigal
>
> Martin Taal wrote:
>> Hi Yigal,
>> Ok, can you send me the ecore (directly by email)?
>>
>> gr. Martin
>>
>> Yigal wrote:
>>> I have a very complex EMF module (over 150 classes). All classes in
>>> the model are extend of one of two basic types: Part or Part_Relation.
>>> Part is an abstract class that identifies a generic part.
>>> Part_relation associates part to other parts.
>>> Example: A car engine is a "Part" of the Car assemble. The engine
>>> has many parts in side it. Those parts are grouped by functionality,
>>> like moving parts, common parts, and etc. So I have
>>> Engine_MovingPart_Rel that relates any moving parts to the engine.
>>>
>>> Basic relation:
>>> |----------| |------------| |-------------|
>>> | | Aggregation | | Association | |
>>> | Engine |-------------->| Eng_Rel |------------->| Part |
>>> | |* 1| |1 1| |
>>> |----------| |------------| |-------------|
>>>
>>> This way I can associate different parts to the engine.
>>> The Engine_MovingPart_Rel extends the Eng_Rel and adds additional
>>> attributes that makes the assemble.
>>>
>>> |----------| |------------| |-------------|
>>> | | Aggregation | | Association | |
>>> | Engine |-------------->| Eng_MovePart------------->| Piston |
>>> | |* 1| Rel |1 1| |
>>> |----------| |------------| |-------------|
>>>
>>> Piston expands "Part".
>>> |----------| |------------| |-------------|
>>> | | Aggregation | | Association | |
>>> | Piston |-------------->| Piston_Rel |------------->| PistonRings |
>>> | |* 1| |1 1| |
>>> |----------| |------------| |-------------|
>>> (and so on and on)
>>>
>>> I generated the Hibernate mapping file using the Join option.
>>> When I create a HibernateResource and load an Engine, and want to
>>> LAZY load all the different Engine_REL, I get over 69 joins that is
>>> an over flow in mySql and causes an exception (fail). I investigated
>>> the SQL commands and found that hibernate wants to load the entire
>>> model and does not do lazy loading of the relationships. The SQL
>>> join all the relations with all the parts and relations between those
>>> parts and other parts.
>>>
>>> I tried the following:
>>> 1. I put the @Proxy attribute on the "Piston" parts, the result was
>>> combined tables of all abstract classes and database table only on
>>> non abstract classes (duplication of data). But the issue was still
>>> there and the join fails with more then 61 joins)
>>>
>>> 2. I tries to use the cascade fetch(Lazy) annotation. Did not help.
>>>
>>> Currently I use the fist option of generating the the hibernate
>>> mapping file,but that creates only TWO tables in the Database with
>>> all attributes in them and the need for over 64 indexes which is not
>>> support either. The join issue does not exists and I can run my
>>> application.
>>> However a flat database is not acceptable and I have no indexes.
>>>
>>> Any solution?
>>>
>>> Thanks
>>> Yigal
>>
>>
--
With Regards, Martin Taal
Springsite/Elver.org
Office: Hardwareweg 4, 3821 BV Amersfoort
Postal: Nassaulaan 7, 3941 EC Doorn
The Netherlands
Tel: +31 (0)84 420 2397
Fax: +31 (0)84 225 9307
Mail: mtaal@springsite.com - mtaal@elver.org
Web: www.springsite.com - www.elver.org
|
|
|
| Re: Too many join - issue with the HibernateMapping file [message #616008 is a reply to message #112851] |
Thu, 27 March 2008 12:27 |
Yigal Spinner
Messages: 127
Registered: July 2009 |
Senior Member |
|
|
Hi Martin,
I figured out the issue. The issue was with hibernate options.
There is an option called MAX_FETCH_DEPTH and the default is set to 3.
In my module a level three at the point of collecting all parts causes
too many join tables. In any case, I want to do lazy loading, so I set
the option to level 0 and now it works fine.
Thanks
Yigal
Martin Taal wrote:
> Hi Yigal,
> Hmm it is the easiest if I can reproduce the issue (which afaics is
> mainly a lazy-loading issue) with a small test model.
>
> What was not completely clear from the model why the eng_rel to part is
> a 1-1 (1 on both sides), I would have expected a model like this:
>
> Engine (1)------(*) Eng_Rel (*) -------(1) Part
>
> based on your description. So therefore my confusion.
>
> Does this maybe apply to your situation (minOccurs>0):
> http://www.elver.org/hibernate/hibernateresources.html#Valid ation
>
> Lazy loading works in Teneo I have multiple testcases covering this. So
> it must be something related to your model (and how Teneo handles it
> ofcourse). Can you make a small fake subset of your model which shows
> the same (eager loading) behavior? Which can be send to me without nda.
>
> gr. Martin
>
> Yigal wrote:
>> Hi Martine,
>> Sorry but the eCore is an Intellectual Property of my company and can
>> not be distributed with out an NDA.
>>
>> The sample below is not the real eCore but a close enough approximation.
>> I was hopping it would be enough to get some directions to solve my
>> issue.
>>
>> Yigal
>>
>> Martin Taal wrote:
>>> Hi Yigal,
>>> Ok, can you send me the ecore (directly by email)?
>>>
>>> gr. Martin
>>>
>>> Yigal wrote:
>>>> I have a very complex EMF module (over 150 classes). All classes in
>>>> the model are extend of one of two basic types: Part or Part_Relation.
>>>> Part is an abstract class that identifies a generic part.
>>>> Part_relation associates part to other parts.
>>>> Example: A car engine is a "Part" of the Car assemble. The engine
>>>> has many parts in side it. Those parts are grouped by functionality,
>>>> like moving parts, common parts, and etc. So I have
>>>> Engine_MovingPart_Rel that relates any moving parts to the engine.
>>>>
>>>> Basic relation:
>>>> |----------| |------------| |-------------|
>>>> | | Aggregation | | Association | |
>>>> | Engine |-------------->| Eng_Rel |------------->| Part |
>>>> | |* 1| |1 1| |
>>>> |----------| |------------| |-------------|
>>>>
>>>> This way I can associate different parts to the engine.
>>>> The Engine_MovingPart_Rel extends the Eng_Rel and adds additional
>>>> attributes that makes the assemble.
>>>>
>>>> |----------| |------------| |-------------|
>>>> | | Aggregation | | Association | |
>>>> | Engine |-------------->| Eng_MovePart------------->| Piston |
>>>> | |* 1| Rel |1 1| |
>>>> |----------| |------------| |-------------|
>>>>
>>>> Piston expands "Part".
>>>> |----------| |------------| |-------------|
>>>> | | Aggregation | | Association | |
>>>> | Piston |-------------->| Piston_Rel |------------->| PistonRings |
>>>> | |* 1| |1 1| |
>>>> |----------| |------------| |-------------|
>>>> (and so on and on)
>>>>
>>>> I generated the Hibernate mapping file using the Join option.
>>>> When I create a HibernateResource and load an Engine, and want to
>>>> LAZY load all the different Engine_REL, I get over 69 joins that is
>>>> an over flow in mySql and causes an exception (fail). I
>>>> investigated the SQL commands and found that hibernate wants to load
>>>> the entire model and does not do lazy loading of the relationships.
>>>> The SQL join all the relations with all the parts and relations
>>>> between those parts and other parts.
>>>>
>>>> I tried the following:
>>>> 1. I put the @Proxy attribute on the "Piston" parts, the result was
>>>> combined tables of all abstract classes and database table only on
>>>> non abstract classes (duplication of data). But the issue was still
>>>> there and the join fails with more then 61 joins)
>>>>
>>>> 2. I tries to use the cascade fetch(Lazy) annotation. Did not help.
>>>>
>>>> Currently I use the fist option of generating the the hibernate
>>>> mapping file,but that creates only TWO tables in the Database with
>>>> all attributes in them and the need for over 64 indexes which is not
>>>> support either. The join issue does not exists and I can run my
>>>> application.
>>>> However a flat database is not acceptable and I have no indexes.
>>>>
>>>> Any solution?
>>>>
>>>> Thanks
>>>> Yigal
>>>
>>>
>
>
|
|
|
| Re: Too many join - issue with the HibernateMapping file [message #616009 is a reply to message #116116] |
Thu, 27 March 2008 12:44 |
Martin Taal
Messages: 5468
Registered: July 2009 |
Senior Member |
|
|
Hi Yigal,
Thanks for the update, I'll add this to the troubleshooting section of the docs.
gr. Martin
Yigal wrote:
> Hi Martin,
> I figured out the issue. The issue was with hibernate options.
> There is an option called MAX_FETCH_DEPTH and the default is set to 3.
> In my module a level three at the point of collecting all parts causes
> too many join tables. In any case, I want to do lazy loading, so I set
> the option to level 0 and now it works fine.
>
> Thanks
> Yigal
>
> Martin Taal wrote:
>> Hi Yigal,
>> Hmm it is the easiest if I can reproduce the issue (which afaics is
>> mainly a lazy-loading issue) with a small test model.
>>
>> What was not completely clear from the model why the eng_rel to part
>> is a 1-1 (1 on both sides), I would have expected a model like this:
>>
>> Engine (1)------(*) Eng_Rel (*) -------(1) Part
>>
>> based on your description. So therefore my confusion.
>>
>> Does this maybe apply to your situation (minOccurs>0):
>> http://www.elver.org/hibernate/hibernateresources.html#Valid ation
>>
>> Lazy loading works in Teneo I have multiple testcases covering this.
>> So it must be something related to your model (and how Teneo handles
>> it ofcourse). Can you make a small fake subset of your model which
>> shows the same (eager loading) behavior? Which can be send to me
>> without nda.
>>
>> gr. Martin
>>
>> Yigal wrote:
>>> Hi Martine,
>>> Sorry but the eCore is an Intellectual Property of my company and can
>>> not be distributed with out an NDA.
>>>
>>> The sample below is not the real eCore but a close enough approximation.
>>> I was hopping it would be enough to get some directions to solve my
>>> issue.
>>>
>>> Yigal
>>>
>>> Martin Taal wrote:
>>>> Hi Yigal,
>>>> Ok, can you send me the ecore (directly by email)?
>>>>
>>>> gr. Martin
>>>>
>>>> Yigal wrote:
>>>>> I have a very complex EMF module (over 150 classes). All classes
>>>>> in the model are extend of one of two basic types: Part or
>>>>> Part_Relation.
>>>>> Part is an abstract class that identifies a generic part.
>>>>> Part_relation associates part to other parts.
>>>>> Example: A car engine is a "Part" of the Car assemble. The engine
>>>>> has many parts in side it. Those parts are grouped by
>>>>> functionality, like moving parts, common parts, and etc. So I have
>>>>> Engine_MovingPart_Rel that relates any moving parts to the engine.
>>>>>
>>>>> Basic relation:
>>>>> |----------| |------------| |-------------|
>>>>> | | Aggregation | | Association | |
>>>>> | Engine |-------------->| Eng_Rel |------------->| Part |
>>>>> | |* 1| |1 1| |
>>>>> |----------| |------------| |-------------|
>>>>>
>>>>> This way I can associate different parts to the engine.
>>>>> The Engine_MovingPart_Rel extends the Eng_Rel and adds additional
>>>>> attributes that makes the assemble.
>>>>>
>>>>> |----------| |------------| |-------------|
>>>>> | | Aggregation | | Association | |
>>>>> | Engine |-------------->| Eng_MovePart------------->| Piston |
>>>>> | |* 1| Rel |1 1| |
>>>>> |----------| |------------| |-------------|
>>>>>
>>>>> Piston expands "Part".
>>>>> |----------| |------------| |-------------|
>>>>> | | Aggregation | | Association | |
>>>>> | Piston |-------------->| Piston_Rel |------------->| PistonRings |
>>>>> | |* 1| |1 1| |
>>>>> |----------| |------------| |-------------|
>>>>> (and so on and on)
>>>>>
>>>>> I generated the Hibernate mapping file using the Join option.
>>>>> When I create a HibernateResource and load an Engine, and want to
>>>>> LAZY load all the different Engine_REL, I get over 69 joins that is
>>>>> an over flow in mySql and causes an exception (fail). I
>>>>> investigated the SQL commands and found that hibernate wants to
>>>>> load the entire model and does not do lazy loading of the
>>>>> relationships. The SQL join all the relations with all the parts
>>>>> and relations between those parts and other parts.
>>>>>
>>>>> I tried the following:
>>>>> 1. I put the @Proxy attribute on the "Piston" parts, the result was
>>>>> combined tables of all abstract classes and database table only on
>>>>> non abstract classes (duplication of data). But the issue was still
>>>>> there and the join fails with more then 61 joins)
>>>>>
>>>>> 2. I tries to use the cascade fetch(Lazy) annotation. Did not help.
>>>>>
>>>>> Currently I use the fist option of generating the the hibernate
>>>>> mapping file,but that creates only TWO tables in the Database with
>>>>> all attributes in them and the need for over 64 indexes which is
>>>>> not support either. The join issue does not exists and I can run
>>>>> my application.
>>>>> However a flat database is not acceptable and I have no indexes.
>>>>>
>>>>> Any solution?
>>>>>
>>>>> Thanks
>>>>> Yigal
>>>>
>>>>
>>
>>
--
With Regards, Martin Taal
Springsite/Elver.org
Office: Hardwareweg 4, 3821 BV Amersfoort
Postal: Nassaulaan 7, 3941 EC Doorn
The Netherlands
Tel: +31 (0)84 420 2397
Fax: +31 (0)84 225 9307
Mail: mtaal@springsite.com - mtaal@elver.org
Web: www.springsite.com - www.elver.org
|
|
|
Goto Forum:
Current Time: Fri Apr 26 16:26:00 GMT 2024
Powered by FUDForum. Page generated in 0.03340 seconds |
] 
Search
Help
Register
Login
Home
