Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » EMF » [CDO] Methods to retrieve or cascade not yet loaded referencing objects?
[CDO] Methods to retrieve or cascade not yet loaded referencing objects? [message #508517] Tue, 19 January 2010 09:54 Go to next message
Bjoern Sundin is currently offline Bjoern SundinFriend
Messages: 19
Registered: July 2009
Location: Switzerland
Junior Member
I have an RCP application using EMF and Teneo to store data into a database. As the amount of data has grown bigger the load performance has drastically gone down. Now I am examining the possibilities to use CDO/Net4J to handle the database layer in the application.

The first tests have been very positive! Start up and load behaviour of the application has improved immensely. Also the memory usage seems to perform better.

A problem I haven't been able to solve yet, though, is cascading a delete to objects not yet loaded from the data store. For loaded objects I can use ECrossReferenceAdapters to resolve the cross references. This is obviously not possible for objects not yet loaded from the database. Using a HibernateResource I was able to explicitly query the data store (HbDataStore) for objects referring to a specific object ('datastore.getCrossReferencers'). Is there a similar method one can use within the CDO framework? Or can one e.g. use the CDOView createQuery method to send a specific SQL query to retrieve the referencing objects? Another possibility I see would be to add cascade/update constraints to an EReference in the Ecore model by using annotations. Is there any support for such annotations in CDO?

Thanks
Björn
Re: [CDO] Methods to retrieve or cascade not yet loaded referencing objects? [message #508523 is a reply to message #508517] Tue, 19 January 2010 05:16 Go to previous messageGo to next message
Martin Taal is currently offline Martin TaalFriend
Messages: 5468
Registered: July 2009
Senior Member
Hi Bjoern,
Eike and others can probably also answer some of your questions. To give some other pointers. There is also a Teneo
Hibernate datastore for CDO (you need to use the latest builds from CDO, the 3.0 version). See this thread for some
download links to the relevant plugins:
http://www.eclipse.org/forums/index.php?t=rview&goto=508 014&th=160759

With the CDO Teneo datastore you can use the JPA annotations in your model to control server side cascade (and basically
all other JPA constructs). Also with the teneo datastore you can use HQL queries on the client.

Afaik the CDO API definition of a datastore does not include a cross reference possibility (Eike correct me if I am
wrong). As Teneo standard has this it is very well possible to add this functionality to the CDO Teneo datastore. But
the standard CDO api should support it ofcourse. Maybe Eike can react on this and it can be made into a feature request.

The CDO sql data store allows sql queries on the client (but others can give you more info on that).

I will start today/tomorrow with a (short) wiki article on how to setup a CDO server backed with a Teneo datastore.

gr. Martin

Bjoern Sundin wrote:
> I have an RCP application using EMF and Teneo to store data into a
> database. As the amount of data has grown bigger the load performance
> has drastically gone down. Now I am examining the possibilities to use
> CDO/Net4J to handle the database layer in the application.
> The first tests have been very positive! Start up and load behaviour of
> the application has improved immensely. Also the memory usage seems to
> perform better.
> A problem I haven't been able to solve yet, though, is cascading a
> delete to objects not yet loaded from the data store. For loaded objects
> I can use ECrossReferenceAdapters to resolve the cross references. This
> is obviously not possible for objects not yet loaded from the database.
> Using a HibernateResource I was able to explicitly query the data store
> (HbDataStore) for objects referring to a specific object
> ('datastore.getCrossReferencers'). Is there a similar method one can use
> within the CDO framework? Or can one e.g. use the CDOView createQuery
> method to send a specific SQL query to retrieve the referencing objects?
> Another possibility I see would be to add cascade/update constraints to
> an EReference in the Ecore model by using annotations. Is there any
> support for such annotations in CDO?
>
> Thanks
> Björn


--

With Regards, Martin Taal

Springsite/Elver.org
Office: Hardwareweg 4, 3821 BV Amersfoort
Postal: Nassaulaan 7, 3941 EC Doorn
The Netherlands
Cell: +31 (0)6 288 48 943
Tel: +31 (0)84 420 2397
Fax: +31 (0)84 225 9307
Mail: mtaal@springsite.com - mtaal@elver.org
Web: www.springsite.com - www.elver.org
Re: [CDO] Methods to retrieve or cascade not yet loaded referencing objects? [message #508526 is a reply to message #508523] Tue, 19 January 2010 10:16 Go to previous messageGo to next message
Eike Stepper is currently offline Eike StepperFriend
Messages: 6450
Registered: July 2009
Senior Member
Martin Taal schrieb:
> Hi Bjoern,
> Eike and others can probably also answer some of your questions. To
> give some other pointers. There is also a Teneo Hibernate datastore
> for CDO (you need to use the latest builds from CDO, the 3.0 version).
> See this thread for some download links to the relevant plugins:
> http://www.eclipse.org/forums/index.php?t=rview&goto=508 014&th=160759
>
> With the CDO Teneo datastore you can use the JPA annotations in your
> model to control server side cascade (and basically all other JPA
> constructs). Also with the teneo datastore you can use HQL queries on
> the client.
>
> Afaik the CDO API definition of a datastore does not include a cross
> reference possibility (Eike correct me if I am wrong).
Unfortunately that's correct. We'd appreciate contributions to add this
feature. We'd also like to use that new feature to optionally "handle"
referencing objects at commit time in the server. Bjoern, would you like
to work with us (i.e. with our advice and review) to add these features?

> As Teneo standard has this it is very well possible to add this
> functionality to the CDO Teneo datastore. But the standard CDO api
> should support it ofcourse. Maybe Eike can react on this and it can be
> made into a feature request.
See above ;-)

Cheers
/Eike

----
http://thegordian.blogspot.com
http://twitter.com/eikestepper



>
> The CDO sql data store allows sql queries on the client (but others
> can give you more info on that).
>
> I will start today/tomorrow with a (short) wiki article on how to
> setup a CDO server backed with a Teneo datastore.
>
> gr. Martin
>
> Bjoern Sundin wrote:
>> I have an RCP application using EMF and Teneo to store data into a
>> database. As the amount of data has grown bigger the load performance
>> has drastically gone down. Now I am examining the possibilities to
>> use CDO/Net4J to handle the database layer in the application.
>> The first tests have been very positive! Start up and load behaviour
>> of the application has improved immensely. Also the memory usage
>> seems to perform better. A problem I haven't been able to solve yet,
>> though, is cascading a delete to objects not yet loaded from the data
>> store. For loaded objects I can use ECrossReferenceAdapters to
>> resolve the cross references. This is obviously not possible for
>> objects not yet loaded from the database. Using a HibernateResource I
>> was able to explicitly query the data store (HbDataStore) for objects
>> referring to a specific object ('datastore.getCrossReferencers'). Is
>> there a similar method one can use within the CDO framework? Or can
>> one e.g. use the CDOView createQuery method to send a specific SQL
>> query to retrieve the referencing objects? Another possibility I see
>> would be to add cascade/update constraints to an EReference in the
>> Ecore model by using annotations. Is there any support for such
>> annotations in CDO?
>>
>> Thanks
>> Björn
>
>


Re: [CDO] Methods to retrieve or cascade not yet loaded referencing objects? [message #508630 is a reply to message #508523] Tue, 19 January 2010 16:43 Go to previous messageGo to next message
Bjoern Sundin is currently offline Bjoern SundinFriend
Messages: 19
Registered: July 2009
Location: Switzerland
Junior Member
Hi Martin and thanks for your quick answer.

An up-to-date howto of setting up a CDO server backed up with a Teneo data store would be really great! I must admit that I failed using the current description on elver.org even though I am getting used to set up CDO server/clients by now...

As a matter of fact, setting this up is another path I planned to do in my feasibility tests to optimize the current application. I would probably not have to rebuild too much in such a case, as the application right now is based on EMF, GMF and Teneo anyhow. The cross referencing back to the data store is a very important feature in my case, in order to keep the data consistent without having to load it all into memory.

My only concern in this case is the performance. Using EMF, GMF and Teneo was/is great with smaller models. It became troublesome with huge models, as the framework wanted to load the whole database at start up in an attempt to add adapters to the whole containment tree. This caused not only very long start up times but is also quite memory consuming. I did, however, customize this behaviour to come closer to something like lazy loading, and so achieved almost acceptable performance results.

The tests I've made so far with the same model and (almost) unchanged logic, but based on EMF, GMF and CDO/Net4j was performance wise stunningly good. I would be very content if the tests using EMF, GMF, Teneo and CDO/Net4j would produce similar results. Smile

Where will the Wiki article on how to setup a CDO server backed with a Teneo data store be located?

Cheers
Bjoern


Re: [CDO] Methods to retrieve or cascade not yet loaded referencing objects? [message #508649 is a reply to message #508630] Tue, 19 January 2010 17:31 Go to previous messageGo to next message
Martin Taal is currently offline Martin TaalFriend
Messages: 5468
Registered: July 2009
Senior Member
Hi Bjoern,
The howto/tutorial will be in the wiki.eclipse.org (at least that is my plan). Your remark about the current CDO
tutorial on elver.org is correct, this one will be replaced by the new one (on the eclipse.org wiki).

With the CDO Teneo datastore, Teneo takes care of generating the mapping to the database (so just that part), taking
into account extra JPA annotations in the model. So if you have JPA annotations right now in the model they will be used
also with CDO Teneo.
The actual runtime datastore (querying for objects, updating objects) etc. is completely separate from Teneo and part of
the CDO project (as a separate pluggable datastore). This datastore uses hibernate at runtime. So Teneo is used for
generating the mapping and the CDO Hibernate store takes care of the runtime behavior. From a CDO client perspective the
api is the same for the dbstore and for the hibernatestore (other than that HQL is supported by the Hibernate store and
SQL by the dbstore).

The nice thing with the combination of CDO, Teneo, Hibernate is that you get
- the CDO framework with many great CDO features (lazy loading, efficient memory handling, etc.), great client side
integration
- the mapping flexibility and power provided by Teneo (with possibility to extend/override mappings with basically all
the JPA annotations)
- hibernate at runtime which gives you HQL (an object query language) and support for many relational databases.

There is one larger CDO feature which is not supported by the Teneo datastore, that's auditing. Although I have an idea
on how this can be supported it also has implications for the database schema (foreign key constraints are not supported
in auditing mode).

For the rest practically all testcases pass with the CDO Teneo datastore, the main thing remaining is to write some
docs... which I am doing right now.

gr. Martin

Bjoern Sundin wrote:
> Hi Martin and thanks for your quick answer.
>
> An up-to-date howto of setting up a CDO server backed up with a Teneo
> data store would be really great! I must admit that I failed using the
> current description on elver.org even though I am getting used to set up
> CDO server/clients by now...
> As a matter of fact, setting this up is another path I planned to do in
> my feasibility tests to optimize the current application. I would
> probably not have to rebuild too much in such a case, as the application
> right now is based on EMF, GMF and Teneo anyhow. The cross referencing
> back to the data store is a very important feature in my case, in order
> to keep the data consistent without having to load it all into memory.
>
> My only concern in this case is the performance. Using EMF, GMF and
> Teneo was/is great with smaller models. It became troublesome with huge
> models, as the framework wanted to load the whole database at start up
> in an attempt to add adapters to the whole containment tree. This caused
> not only very long start up times but is also quite memory consuming. I
> did, however, customize this behaviour to come closer to something like
> lazy loading, and so achieved almost acceptable performance results.
> The tests I've made so far with the same model and (almost) unchanged
> logic, but based on EMF, GMF and CDO/Net4j was performance wise
> stunningly good. I would be very content if the tests using EMF, GMF,
> Teneo and CDO/Net4j would produce similar results. :)
>
> Where will the Wiki article on how to setup a CDO server backed with a
> Teneo data store be located?
>
> Cheers
> Bjoern
>
>
>


--

With Regards, Martin Taal

Springsite/Elver.org
Office: Hardwareweg 4, 3821 BV Amersfoort
Postal: Nassaulaan 7, 3941 EC Doorn
The Netherlands
Cell: +31 (0)6 288 48 943
Tel: +31 (0)84 420 2397
Fax: +31 (0)84 225 9307
Mail: mtaal@springsite.com - mtaal@elver.org
Web: www.springsite.com - www.elver.org
Re: [CDO] Methods to retrieve or cascade not yet loaded referencing objects? [message #508753 is a reply to message #508526] Wed, 20 January 2010 08:24 Go to previous messageGo to next message
Bjoern Sundin is currently offline Bjoern SundinFriend
Messages: 19
Registered: July 2009
Location: Switzerland
Junior Member
Hi Eike,

Thanks for offering me the opportunity to contribute to the project. I will have to clear this with my seniors and check how it would fit into the planning before I give an answer, though.

I think there are two sides to be considered:

On the one hand the possibility to control, that the data store correctly handles references (or more correct "referencers") at commit time. That implies that there has to be an optional setting marking the reference for cascade delete (annotation in the ecore file?). This annotation could result in a constraint when creating the database. But this is basically already covered using the Teneo Hibernate Data Store with CDO, as far as I understood. The question is if it is desired to have a "standalone" CDO solution?

On the other hand the client has to be able to control the local model, so that when loading a referencer, to a locally already deleted object, this referencer optionally also would be marked deleted. Another possibility could be to add a CrossReferenceAdapter to loaded objects and at delete time have the client load affected referencers in order to handle them appropriately, possibly according to the defined annotations in the ecore file.

Cheers
Bjoern
Re: [CDO] Methods to retrieve or cascade not yet loaded referencing objects? [message #508761 is a reply to message #508649] Wed, 20 January 2010 08:46 Go to previous messageGo to next message
Bjoern Sundin is currently offline Bjoern SundinFriend
Messages: 19
Registered: July 2009
Location: Switzerland
Junior Member
Hi Martin,

Yes, it looks like that the combination of CDO and the CDO Teneo Datastore is what I will need. I have currently no plans using the audit feature so I'm fine with that for now... Smile

Is the Teneo datastore functionality also reachable from the client? I do have cases where the logic has to check cross references or query the data store within live validation (Java) during user interactions.

I'm looking forward to the wiki cookbook and my results of that part of the feasibility tests!

Cheers
Bjoern
Re: [CDO] Methods to retrieve or cascade not yet loaded referencing objects? [message #508764 is a reply to message #508753] Wed, 20 January 2010 09:15 Go to previous messageGo to next message
Eike Stepper is currently offline Eike StepperFriend
Messages: 6450
Registered: July 2009
Senior Member
Hi Bjoern,

Comments below...


Bjoern Sundin schrieb:
> Hi Eike,
>
> Thanks for offering me the opportunity to contribute to the project. I
> will have to clear this with my seniors and check how it would fit
> into the planning before I give an answer, though.
Excellent!

> I think there are two sides to be considered:
>
> On the one hand the possibility to control, that the data store
> correctly handles references (or more correct "referencers") at commit
> time. That implies that there has to be an optional setting marking
> the reference for cascade delete (annotation in the ecore file?). This
> annotation could result in a constraint when creating the database.
> But this is basically already covered using the Teneo Hibernate Data
> Store with CDO, as far as I understood. The question is if it is
> desired to have a "standalone" CDO solution?
I think this is something that should be handled/provided at repository
level, similar to other optional server features like auditing. The core
framework should provide API for client and SPI for store implementors.

Note that Martin just filed https://bugs.eclipse.org/300149 .

>
> On the other hand the client has to be able to control the local
> model, so that when loading a referencer, to a locally already deleted
> object,
We use to call this a stale reference. See also
org.eclipse.emf.cdo.view.CDOView.Options.setStaleReferenceBe haviour(CDOStaleReferencePolicy)
..

> this referencer optionally also would be marked deleted.
That's a very hard consequence!

> Another possibility could be to add a CrossReferenceAdapter to loaded
> objects
That would obviously not scale well. We'd need exactly the bugzilla
mentioned above to operate on the client independently from the
repository size.

> and at delete time
I'd prefer to handle this at commit time in the server. But your idea is
also an option - when we have this remote cross referencer (see bugzilla
above).

> have the client load affected referencers in order to handle them
> appropriately, possibly according to the defined annotations in the
> ecore file.
If we do it purely on the client I'd prefer a policy-style API. One
possible policy implementation could be to inspect model annotations and
act accordingly.

Cheers
/Eike

----
http://thegordian.blogspot.com
http://twitter.com/eikestepper


Re: [CDO] Methods to retrieve or cascade not yet loaded referencing objects? [message #508765 is a reply to message #508761] Wed, 20 January 2010 04:20 Go to previous messageGo to next message
Eike Stepper is currently offline Eike StepperFriend
Messages: 6450
Registered: July 2009
Senior Member
Bjoern Sundin schrieb:
> Hi Martin,
>
> Yes, it looks like that the combination of CDO and the CDO Teneo
> Datastore is what I will need. I have currently no plans using the
> audit feature so I'm fine with that for now... :)
>
> Is the Teneo datastore functionality also reachable from the client?
If HQL does not already do the trick, the generic query framework could
be used to implement some "system" queries that are handled (see
IQueryHandler) by the HibernateStore.

Cheers
/Eike

----
http://thegordian.blogspot.com
http://twitter.com/eikestepper


Re: [CDO] Methods to retrieve or cascade not yet loaded referencing objects? [message #508768 is a reply to message #508764] Wed, 20 January 2010 09:31 Go to previous message
Bjoern Sundin is currently offline Bjoern SundinFriend
Messages: 19
Registered: July 2009
Location: Switzerland
Junior Member
Yes, I think the enhancement filed by Marting will cover those issues.

Cheers
Bjoern

Previous Topic:[emf-dev] Modify adding the new child/sibling node in EMF
Next Topic:ClassCastException when Loading with dynamic EMF
Goto Forum:
  


Current Time: Wed Nov 13 08:16:24 GMT 2019

Powered by FUDForum. Page generated in 0.02009 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top