Home » Modeling » EMF » [CDO] CDOResources and large datasets
| [CDO] CDOResources and large datasets [message #483501] |
Tue, 01 September 2009 18:55  |
 Mario Winterer
Messages: 136
Registered: July 2009 |
Senior Member |
|
|
Hi!
I wonder about how to best use CDOResources with large datasets.
It seems that every persisted object is associated with a resource, so
there is a one-to-many relation between resource and object, correct?
Lets take an EMF enabled warehouse application with several thousands of
products and customers in the database.
As each product/customer can be edited in a separate editor, it seems
correct to have one resource per product/customer. On the other hand a
search-for-product/-customer function might return a list of
products/customers which would cause hundreds or thousands of resources
to be loaded (I'm assuming that loading an object via query-API will
load the corresponding resource too, won't it?), so it seems better to
have one resource containing all products and one for all customers.
Another approach would be to have one single resource for all domain
objects (products, customers, ...). As CDO will load object state on
demand, scalability should not be affected I think. However, object
handling in editors would be more complex as editors can't work on a per
resource basis any more.
What is your experience concerning resource 'organization'?
Thanks,
Mario
|
|
|
| Re: [CDO] CDOResources and large datasets [message #483521 is a reply to message #483501] |
Tue, 01 September 2009 20:28  |
|
Hi Mario,
Comments below...
Mario Winterer schrieb:
> Hi!
>
> I wonder about how to best use CDOResources with large datasets.
> It seems that every persisted object is associated with a resource, so
> there is a one-to-many relation between resource and object, correct?
Yes, BUT not in the Java reference sense! It's possible to determine the
containing resource for each contained object, but possibly not in one
step. In other words, indirectly contained objects do not store a
reference to their resource.
As a side note, a CDOObject never stores Java references to any other
CDOObject including CDOResources (but that's different from the
statement above, which was about "foreign keys" to the containing resource).
>
> Lets take an EMF enabled warehouse application with several thousands
> of products and customers in the database.
> As each product/customer can be edited in a separate editor, it seems
> correct to have one resource per product/customer.
That only depends on the particular editor, which seems wrong by itself :P
> On the other hand a search-for-product/-customer function might return
> a list of products/customers which would cause hundreds or thousands
> of resources to be loaded (I'm assuming that loading an object via
> query-API will load the corresponding resource too, won't it?), so it
> seems better to have one resource containing all products and one for
> all customers.
It's one of CDO's strengths that it does not really matter how you
distribute your objects over your resources. CDO does not care. I prefer
to see a resource as "an object with a name" or a "well-known entry
point into the persistent object graph". CDO does neither require to
load all contents of a resource, nor to load all the containments of an
object. CDO is completely demand loading based.
Regarding query: CDO has it's own query API, see CDOView.createQuery(),
which executes queries on the server side. Hence it does not affect
demand loading. You can even execute the query asynchronously and
iterate over the result while the server is still working on more
results.Currently the available query languages are determined by the
back-end store (i.e. SQL for DBStore or HQL for HibernateStore).
>
> Another approach would be to have one single resource for all domain
> objects (products, customers, ...). As CDO will load object state on
> demand, scalability should not be affected I think. However, object
> handling in editors would be more complex as editors can't work on a
> per resource basis any more.
>
> What is your experience concerning resource 'organization'?
It depends on the use case. I've seen everything between and including
the two extremes: One huge resource for all objects and many tiny
resources with only one or few objects each. As I said, for CDO it does
not matter.
Making an existing editor infrastructure may be one goal, avoiding
object-based queries another one. Maybe others can jump in with more
best practice hints from their experience. In case you need more in
depth analysis / advice regarding your application, I'm available for
consulting, too ;-)
Cheers
/Eike
----
http://thegordian.blogspot.com
>
> Thanks,
> Mario
Cheers
/Eike
----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper
|
|
|
| Re: [CDO] CDOResources and large datasets [message #483584 is a reply to message #483501] |
Wed, 02 September 2009 08:48 |
Stefan Winkler
Messages: 307
Registered: July 2009
Location: Germany |
Senior Member |
|
|
Mario,
I'd like to add to Eike's comments the following:
Think of a CDO repository as a large object graph of which only the
parts you currently use are materialized in your memory.
Think of a resource in the sense of CDO as a named entry point into this
object graph.
If you store each Customer in one resource this will allow you to
enumerate all customers simply using
getResource(CUTOMER_RESOURCE_URL).getContents()
If you store each Customer in a separate resource enumerating would
involve getting the resource's parent folder and enumerate the resources
and then getting the one content element out of the resource.
So, using one resource seems more straightforward and more readable.
Querying customers in the DBStore would work the same way in both
scenarios by the way, because all customer objects are stored in the
same table.
If you retrieve customers by query it does not matter which of the
approaches you choose (well, maybe the only difference is the amount of
total objects stored in the DB, as a CDOResource -- regarding to the
store -- is nothing more than an ordinary object.
Cheers,
Stefan
Mario Winterer schrieb:
> Hi!
>
> I wonder about how to best use CDOResources with large datasets.
> It seems that every persisted object is associated with a resource, so
> there is a one-to-many relation between resource and object, correct?
>
> Lets take an EMF enabled warehouse application with several thousands
> of products and customers in the database.
> As each product/customer can be edited in a separate editor, it seems
> correct to have one resource per product/customer. On the other hand a
> search-for-product/-customer function might return a list of
> products/customers which would cause hundreds or thousands of
> resources to be loaded (I'm assuming that loading an object via
> query-API will load the corresponding resource too, won't it?), so it
> seems better to have one resource containing all products and one for
> all customers.
>
> Another approach would be to have one single resource for all domain
> objects (products, customers, ...). As CDO will load object state on
> demand, scalability should not be affected I think. However, object
> handling in editors would be more complex as editors can't work on a
> per resource basis any more.
>
> What is your experience concerning resource 'organization'?
>
> Thanks,
> Mario
|
|
|
| Re: [CDO] CDOResources and large datasets [message #483618 is a reply to message #483584] |
Wed, 02 September 2009 11:08 |
Mario Winterer
Messages: 136
Registered: July 2009 |
Senior Member |
|
|
Thanks (to both of you)!
I've got a more clearer picture now of how things could work in my
application.
I think I have to get rid of the idea to have one architecture
(including editors, etc.) that need not be aware of CDO, i.e. an
architecture that uses a transparent persistence mechanisms - either CDO
or any other resource-based technique (e.g. simply saving resources into
files). If I decide to use CDO, editors views etc. will have to be
slightly/completely(?) different.
By the way: Is it somehow easily possible to mix up CDO with some other
persistance mechanism? I'd like to use CDO for one single component of
our application - all other (already existing) components use other ways
to communicate with the server. More specific, is it possible to have a
CDO-enabled model with (cross-document) references to a non-CDO model
objects? I think the main problem is that those referenced objects can
not take part in the CDO's object lifecycle...
Thanks again,
Mario
Stefan Winkler schrieb:
> Mario,
>
> I'd like to add to Eike's comments the following:
>
> Think of a CDO repository as a large object graph of which only the
> parts you currently use are materialized in your memory.
> Think of a resource in the sense of CDO as a named entry point into this
> object graph.
>
> If you store each Customer in one resource this will allow you to
> enumerate all customers simply using
> getResource(CUTOMER_RESOURCE_URL).getContents()
> If you store each Customer in a separate resource enumerating would
> involve getting the resource's parent folder and enumerate the resources
> and then getting the one content element out of the resource.
>
> So, using one resource seems more straightforward and more readable.
>
>
> Querying customers in the DBStore would work the same way in both
> scenarios by the way, because all customer objects are stored in the
> same table.
> If you retrieve customers by query it does not matter which of the
> approaches you choose (well, maybe the only difference is the amount of
> total objects stored in the DB, as a CDOResource -- regarding to the
> store -- is nothing more than an ordinary object.
>
> Cheers,
> Stefan
>
>
>
>
> Mario Winterer schrieb:
>> Hi!
>>
>> I wonder about how to best use CDOResources with large datasets.
>> It seems that every persisted object is associated with a resource, so
>> there is a one-to-many relation between resource and object, correct?
>>
>> Lets take an EMF enabled warehouse application with several thousands
>> of products and customers in the database.
>> As each product/customer can be edited in a separate editor, it seems
>> correct to have one resource per product/customer. On the other hand a
>> search-for-product/-customer function might return a list of
>> products/customers which would cause hundreds or thousands of
>> resources to be loaded (I'm assuming that loading an object via
>> query-API will load the corresponding resource too, won't it?), so it
>> seems better to have one resource containing all products and one for
>> all customers.
>>
>> Another approach would be to have one single resource for all domain
>> objects (products, customers, ...). As CDO will load object state on
>> demand, scalability should not be affected I think. However, object
>> handling in editors would be more complex as editors can't work on a
>> per resource basis any more.
>>
>> What is your experience concerning resource 'organization'?
>>
>> Thanks,
>> Mario
|
|
|
| Re: [CDO] CDOResources and large datasets [message #483670 is a reply to message #483618] |
Wed, 02 September 2009 11:42 |
|
Mario Winterer schrieb:
> Thanks (to both of you)!
You're welcome.
>
> I've got a more clearer picture now of how things could work in my
> application.
> I think I have to get rid of the idea to have one architecture
> (including editors, etc.) that need not be aware of CDO, i.e. an
> architecture that uses a transparent persistence mechanisms - either
> CDO or any other resource-based technique (e.g. simply saving
> resources into files). If I decide to use CDO, editors views etc. will
> have to be slightly/completely(?) different.
Well, CDO is transparent with respect to most of the functionality that
EMF offers through API. But there's a whole lot more about CDO and if
you're going to use this functionality it's somehow hard to talk about
transparency at all :P
On many occasions I found out that t's a nice idea to have an
architecture that is agnostic of specific backend infrastructure. But
later it turned out that the functionality is too reduced then. Maybe
it's a good idea to hide all orthogonal concerns in your own facade? My
eDine application shows some examples on how to do this:
http://thegordian.blogspot.com/2009/04/modeling-goes-enterpr ise.html
>
> By the way: Is it somehow easily possible to mix up CDO with some
> other persistance mechanism? I'd like to use CDO for one single
> component of our application - all other (already existing) components
> use other ways to communicate with the server. More specific, is it
> possible to have a CDO-enabled model with (cross-document) references
> to a non-CDO model objects?
Yes, CDO is capable of maintaining (CDO-) external references. Stefan
will know if the DBStore 2.0 is also capable of storing them.
> I think the main problem is that those referenced objects can not take
> part in the CDO's object lifecycle...
Certainly not ;-)
Cheers
/Eike
----
http://thegordian.blogspot.com
>
> Thanks again,
> Mario
>
> Stefan Winkler schrieb:
>> Mario,
>>
>> I'd like to add to Eike's comments the following:
>>
>> Think of a CDO repository as a large object graph of which only the
>> parts you currently use are materialized in your memory.
>> Think of a resource in the sense of CDO as a named entry point into this
>> object graph.
>>
>> If you store each Customer in one resource this will allow you to
>> enumerate all customers simply using
>> getResource(CUTOMER_RESOURCE_URL).getContents()
>> If you store each Customer in a separate resource enumerating would
>> involve getting the resource's parent folder and enumerate the resources
>> and then getting the one content element out of the resource.
>>
>> So, using one resource seems more straightforward and more readable.
>>
>>
>> Querying customers in the DBStore would work the same way in both
>> scenarios by the way, because all customer objects are stored in the
>> same table.
>> If you retrieve customers by query it does not matter which of the
>> approaches you choose (well, maybe the only difference is the amount of
>> total objects stored in the DB, as a CDOResource -- regarding to the
>> store -- is nothing more than an ordinary object.
>>
>> Cheers,
>> Stefan
>>
>>
>>
>>
>> Mario Winterer schrieb:
>>> Hi!
>>>
>>> I wonder about how to best use CDOResources with large datasets.
>>> It seems that every persisted object is associated with a resource, so
>>> there is a one-to-many relation between resource and object, correct?
>>>
>>> Lets take an EMF enabled warehouse application with several thousands
>>> of products and customers in the database.
>>> As each product/customer can be edited in a separate editor, it seems
>>> correct to have one resource per product/customer. On the other hand a
>>> search-for-product/-customer function might return a list of
>>> products/customers which would cause hundreds or thousands of
>>> resources to be loaded (I'm assuming that loading an object via
>>> query-API will load the corresponding resource too, won't it?), so it
>>> seems better to have one resource containing all products and one for
>>> all customers.
>>>
>>> Another approach would be to have one single resource for all domain
>>> objects (products, customers, ...). As CDO will load object state on
>>> demand, scalability should not be affected I think. However, object
>>> handling in editors would be more complex as editors can't work on a
>>> per resource basis any more.
>>>
>>> What is your experience concerning resource 'organization'?
>>>
>>> Thanks,
>>> Mario
Cheers
/Eike
----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper
|
|
|
| Re: [CDO] CDOResources and large datasets [message #483817 is a reply to message #483670] |
Thu, 03 September 2009 08:00 |
Stefan Winkler
Messages: 307
Registered: July 2009
Location: Germany |
Senior Member |
|
|
More comments below
Eike Stepper schrieb:
>> I've got a more clearer picture now of how things could work in my
>> application.
>> I think I have to get rid of the idea to have one architecture
>> (including editors, etc.) that need not be aware of CDO, i.e. an
>> architecture that uses a transparent persistence mechanisms - either
>> CDO or any other resource-based technique (e.g. simply saving
>> resources into files). If I decide to use CDO, editors views etc.
>> will have to be slightly/completely(?) different.
> Well, CDO is transparent with respect to most of the functionality
> that EMF offers through API. But there's a whole lot more about CDO
> and if you're going to use this functionality it's somehow hard to
> talk about transparency at all :P
I have to contradict Eike in one point. Yes, resource handling seems
transparent in most cases, but just the case you mention would not work
in all situations -- namely saving resources. If you open a transaction
and you open two resources, you could pass both resources to two
different components of your application. (That's what I did at first in
my project. Then a central CDO-aware component could exist which passes
EMF-resources around and every other part of the application would only
use EMF resources.) This does NOT work in that way, however. Because
resource.save() will result in a commit for the transaction and hence,
all resources in the resourceSet will be saved at once.
> On many occasions I found out that t's a nice idea to have an
> architecture that is agnostic of specific backend infrastructure. But
> later it turned out that the functionality is too reduced then. Maybe
> it's a good idea to hide all orthogonal concerns in your own facade?
> My eDine application shows some examples on how to do this:
> http://thegordian.blogspot.com/2009/04/modeling-goes-enterpr ise.html
>>
>> By the way: Is it somehow easily possible to mix up CDO with some
>> other persistance mechanism? I'd like to use CDO for one single
>> component of our application - all other (already existing)
>> components use other ways to communicate with the server. More
>> specific, is it possible to have a CDO-enabled model with
>> (cross-document) references to a non-CDO model objects?
> Yes, CDO is capable of maintaining (CDO-) external references. Stefan
> will know if the DBStore 2.0 is also capable of storing them.
Yes, we should be able to handle them. In this case the DB stores URLs
to the referenced objects which are then resolved at the client side.
So, as long as the client still knows the external resource, external
references should not be a problem.
Cheers,
Stefan
|
|
|
| Re: [CDO] CDOResources and large datasets [message #483943 is a reply to message #483817] |
Thu, 03 September 2009 16:53 |
Mario Winterer
Messages: 136
Registered: July 2009 |
Senior Member |
|
|
See below!
Stefan Winkler schrieb:
> More comments below
>
> Eike Stepper schrieb:
>>> I've got a more clearer picture now of how things could work in my
>>> application.
>>> I think I have to get rid of the idea to have one architecture
>>> (including editors, etc.) that need not be aware of CDO, i.e. an
>>> architecture that uses a transparent persistence mechanisms - either
>>> CDO or any other resource-based technique (e.g. simply saving
>>> resources into files). If I decide to use CDO, editors views etc.
>>> will have to be slightly/completely(?) different.
>> Well, CDO is transparent with respect to most of the functionality
>> that EMF offers through API. But there's a whole lot more about CDO
>> and if you're going to use this functionality it's somehow hard to
>> talk about transparency at all :P
> I have to contradict Eike in one point. Yes, resource handling seems
> transparent in most cases, but just the case you mention would not work
> in all situations -- namely saving resources. If you open a transaction
> and you open two resources, you could pass both resources to two
> different components of your application. (That's what I did at first in
> my project. Then a central CDO-aware component could exist which passes
> EMF-resources around and every other part of the application would only
> use EMF resources.) This does NOT work in that way, however. Because
> resource.save() will result in a commit for the transaction and hence,
> all resources in the resourceSet will be saved at once.
Isn't this a question of transaction scope? If it is required to save
each resource seperately, there could be opened one transaction per
resource, right?
>> On many occasions I found out that t's a nice idea to have an
>> architecture that is agnostic of specific backend infrastructure. But
>> later it turned out that the functionality is too reduced then. Maybe
>> it's a good idea to hide all orthogonal concerns in your own facade?
>> My eDine application shows some examples on how to do this:
>> http://thegordian.blogspot.com/2009/04/modeling-goes-enterpr ise.html
>>> By the way: Is it somehow easily possible to mix up CDO with some
>>> other persistance mechanism? I'd like to use CDO for one single
>>> component of our application - all other (already existing)
>>> components use other ways to communicate with the server. More
>>> specific, is it possible to have a CDO-enabled model with
>>> (cross-document) references to a non-CDO model objects?
>> Yes, CDO is capable of maintaining (CDO-) external references. Stefan
>> will know if the DBStore 2.0 is also capable of storing them.
> Yes, we should be able to handle them. In this case the DB stores URLs
> to the referenced objects which are then resolved at the client side.
> So, as long as the client still knows the external resource, external
> references should not be a problem.
Hm. So for each external reference CDO creates a varchar attribute
containing the URI in the corresponding DB table? My problem is that on
DB side, the model is consistent, i.e. I've got a table ORDER
referencing table CUSTOMER via foreign key customerId. But while ORDER
should be mapped to a CDO-enabled Order-Object, CUSTOMER is still
controlled by another (proprietary) mechanism (using a CustomerResource
that directly accesses the CUSTOMER table on save/load).
So I'd need some way to intercept CDO when it loads an Order and
manipulate the proxy-URI of the referenced Customer object to load the
proprietary CustomerResource on proxy resolution.
(I know that this is a horrible concept, but at the moment there is no
way to replace the existing proprietary solution by CDO - all I can do
is to try to use CDO for all new things that gets added...)
> Cheers,
> Stefan
|
|
|
| Re: [CDO] CDOResources and large datasets [message #483998 is a reply to message #483817] |
Thu, 03 September 2009 20:24 |
|
This is a multi-part message in MIME format.
--------------050602090103010805090006
Content-Type: text/plain; charset=ISO-8859-15; format=flowed
Content-Transfer-Encoding: 7bit
Hi Stefan,
While that seems technically correct I always recommend against exposing
the ResourceSet / Resource infrastructure directly to the business logic
of an application. I prefer to hide the scaffolding part from the
business logic in a single place. Then it's also easier to replace it by
different mechanisms.
Cheers
/Eike
----
http://thegordian.blogspot.com
Stefan Winkler schrieb:
> More comments below
>
> Eike Stepper schrieb:
>
>>> I've got a more clearer picture now of how things could work in my
>>> application.
>>> I think I have to get rid of the idea to have one architecture
>>> (including editors, etc.) that need not be aware of CDO, i.e. an
>>> architecture that uses a transparent persistence mechanisms - either
>>> CDO or any other resource-based technique (e.g. simply saving
>>> resources into files). If I decide to use CDO, editors views etc.
>>> will have to be slightly/completely(?) different.
>>>
>> Well, CDO is transparent with respect to most of the functionality
>> that EMF offers through API. But there's a whole lot more about CDO
>> and if you're going to use this functionality it's somehow hard to
>> talk about transparency at all :P
>>
> I have to contradict Eike in one point. Yes, resource handling seems
> transparent in most cases, but just the case you mention would not work
> in all situations -- namely saving resources. If you open a transaction
> and you open two resources, you could pass both resources to two
> different components of your application. (That's what I did at first in
> my project. Then a central CDO-aware component could exist which passes
> EMF-resources around and every other part of the application would only
> use EMF resources.) This does NOT work in that way, however. Because
> resource.save() will result in a commit for the transaction and hence,
> all resources in the resourceSet will be saved at once.
>
>> On many occasions I found out that t's a nice idea to have an
>> architecture that is agnostic of specific backend infrastructure. But
>> later it turned out that the functionality is too reduced then. Maybe
>> it's a good idea to hide all orthogonal concerns in your own facade?
>> My eDine application shows some examples on how to do this:
>> http://thegordian.blogspot.com/2009/04/modeling-goes-enterpr ise.html
>>
>>> By the way: Is it somehow easily possible to mix up CDO with some
>>> other persistance mechanism? I'd like to use CDO for one single
>>> component of our application - all other (already existing)
>>> components use other ways to communicate with the server. More
>>> specific, is it possible to have a CDO-enabled model with
>>> (cross-document) references to a non-CDO model objects?
>>>
>> Yes, CDO is capable of maintaining (CDO-) external references. Stefan
>> will know if the DBStore 2.0 is also capable of storing them.
>>
> Yes, we should be able to handle them. In this case the DB stores URLs
> to the referenced objects which are then resolved at the client side.
> So, as long as the client still knows the external resource, external
> references should not be a problem.
>
> Cheers,
> Stefan
>
--------------050602090103010805090006
Content-Type: text/html; charset=ISO-8859-15
Content-Transfer-Encoding: 7bit
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=ISO-8859-15"
http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
Hi Stefan,<br>
<br>
While that seems technically correct I always recommend against
exposing the ResourceSet / Resource infrastructure directly to the
business logic of an application. I prefer to hide the scaffolding part
from the business logic in a single place. Then it's also easier to
replace it by different mechanisms.<br>
<br>
Cheers<br>
/Eike<br>
<br>
----<br>
<a class="moz-txt-link-freetext" href="http://thegordian.blogspot.com">http://thegordian.blogspot.com</a><br>
<br>
<br>
<br>
Stefan Winkler schrieb:
<blockquote cite="mid:h7nt38$afc$1@build.eclipse.org" type="cite">
<pre wrap="">More comments below
Eike Stepper schrieb:
</pre>
<blockquote type="cite">
<blockquote type="cite">
<pre wrap="">I've got a more clearer picture now of how things could work in my
application.
I think I have to get rid of the idea to have one architecture
(including editors, etc.) that need not be aware of CDO, i.e. an
architecture that uses a transparent persistence mechanisms - either
CDO or any other resource-based technique (e.g. simply saving
resources into files). If I decide to use CDO, editors views etc.
will have to be slightly/completely(?) different.
</pre>
</blockquote>
<pre wrap="">Well, CDO is transparent with respect to most of the functionality
that EMF offers through API. But there's a whole lot more about CDO
and if you're going to use this functionality it's somehow hard to
talk about transparency at all :P
</pre>
</blockquote>
<pre wrap=""><!---->I have to contradict Eike in one point. Yes, resource handling seems
transparent in most cases, but just the case you mention would not work
in all situations -- namely saving resources. If you open a transaction
and you open two resources, you could pass both resources to two
different components of your application. (That's what I did at first in
my project. Then a central CDO-aware component could exist which passes
EMF-resources around and every other part of the application would only
use EMF resources.) This does NOT work in that way, however. Because
resource.save() will result in a commit for the transaction and hence,
all resources in the resourceSet will be saved at once.
</pre>
<blockquote type="cite">
<pre wrap="">On many occasions I found out that t's a nice idea to have an
architecture that is agnostic of specific backend infrastructure. But
later it turned out that the functionality is too reduced then. Maybe
it's a good idea to hide all orthogonal concerns in your own facade?
My eDine application shows some examples on how to do this:
<a class="moz-txt-link-freetext" href=" http://thegordian.blogspot.com/2009/04/modeling-goes-enterpr ise.html"> http://thegordian.blogspot.com/2009/04/modeling-goes-enterpr ise.html</a>
</pre>
<blockquote type="cite">
<pre wrap="">By the way: Is it somehow easily possible to mix up CDO with some
other persistance mechanism? I'd like to use CDO for one single
component of our application - all other (already existing)
components use other ways to communicate with the server. More
specific, is it possible to have a CDO-enabled model with
(cross-document) references to a non-CDO model objects?
</pre>
</blockquote>
<pre wrap="">Yes, CDO is capable of maintaining (CDO-) external references. Stefan
will know if the DBStore 2.0 is also capable of storing them.
</pre>
</blockquote>
<pre wrap=""><!---->Yes, we should be able to handle them. In this case the DB stores URLs
to the referenced objects which are then resolved at the client side.
So, as long as the client still knows the external resource, external
references should not be a problem.
Cheers,
Stefan
</pre>
</blockquote>
</body>
</html>
--------------050602090103010805090006--
Cheers
/Eike
----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper
|
|
|
| Re: [CDO] CDOResources and large datasets [message #483999 is a reply to message #483943] |
Thu, 03 September 2009 20:33 |
|
Hi Mario,
Comments below...
Mario Winterer schrieb:
> See below!
>
> Stefan Winkler schrieb:
>> More comments below
>>
>> Eike Stepper schrieb:
>>>> I've got a more clearer picture now of how things could work in my
>>>> application.
>>>> I think I have to get rid of the idea to have one architecture
>>>> (including editors, etc.) that need not be aware of CDO, i.e. an
>>>> architecture that uses a transparent persistence mechanisms - either
>>>> CDO or any other resource-based technique (e.g. simply saving
>>>> resources into files). If I decide to use CDO, editors views etc.
>>>> will have to be slightly/completely(?) different.
>>> Well, CDO is transparent with respect to most of the functionality
>>> that EMF offers through API. But there's a whole lot more about CDO
>>> and if you're going to use this functionality it's somehow hard to
>>> talk about transparency at all :P
>> I have to contradict Eike in one point. Yes, resource handling seems
>> transparent in most cases, but just the case you mention would not work
>> in all situations -- namely saving resources. If you open a transaction
>> and you open two resources, you could pass both resources to two
>> different components of your application. (That's what I did at first in
>> my project. Then a central CDO-aware component could exist which passes
>> EMF-resources around and every other part of the application would only
>> use EMF resources.) This does NOT work in that way, however. Because
>> resource.save() will result in a commit for the transaction and hence,
>> all resources in the resourceSet will be saved at once.
>
> Isn't this a question of transaction scope? If it is required to save
> each resource seperately, there could be opened one transaction per
> resource, right?
Right.
>
>>> On many occasions I found out that t's a nice idea to have an
>>> architecture that is agnostic of specific backend infrastructure. But
>>> later it turned out that the functionality is too reduced then. Maybe
>>> it's a good idea to hide all orthogonal concerns in your own facade?
>>> My eDine application shows some examples on how to do this:
>>> http://thegordian.blogspot.com/2009/04/modeling-goes-enterpr ise.html
>>>> By the way: Is it somehow easily possible to mix up CDO with some
>>>> other persistance mechanism? I'd like to use CDO for one single
>>>> component of our application - all other (already existing)
>>>> components use other ways to communicate with the server. More
>>>> specific, is it possible to have a CDO-enabled model with
>>>> (cross-document) references to a non-CDO model objects?
>>> Yes, CDO is capable of maintaining (CDO-) external references. Stefan
>>> will know if the DBStore 2.0 is also capable of storing them.
>> Yes, we should be able to handle them. In this case the DB stores URLs
>> to the referenced objects which are then resolved at the client side.
>> So, as long as the client still knows the external resource, external
>> references should not be a problem.
>
> Hm. So for each external reference CDO creates a varchar attribute
> containing the URI in the corresponding DB table?
I don't think so (Stefan will know it exactly). IMHO the DBStore creates
special CDOIDs for external references that are mapped in a separate
table for all such references because at table creation time it is not
known whether actual reference values will be external, internal, or both.
> My problem is that on DB side, the model is consistent, i.e. I've got
> a table ORDER
You should avoid the table name ORDER since that's a reserved word in SQL.
> referencing table CUSTOMER via foreign key customerId. But while ORDER
> should be mapped to a CDO-enabled Order-Object, CUSTOMER is still
> controlled by another (proprietary) mechanism (using a
> CustomerResource that directly accesses the CUSTOMER table on save/load).
>
> So I'd need some way to intercept CDO when it loads an Order and
> manipulate the proxy-URI of the referenced Customer object to load the
> proprietary CustomerResource on proxy resolution.
CDOIDs and EMF proxies get resolved on the client side. So, whatever
format your "external" object URI has, your resource set must be able to
resolve it. That seems completely unrelated to CDO, right?
>
> (I know that this is a horrible concept, but at the moment there is no
> way to replace the existing proprietary solution by CDO - all I can do
> is to try to use CDO for all new things that gets added...)
While I agree that a consistent approach is probably nicer, EMF should
be able to integrate multiple persistence technologies. All you must do
is configure your resource set properly.
Cheers
/Eike
----
http://thegordian.blogspot.com
>
>
>> Cheers,
>> Stefan
Cheers
/Eike
----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper
|
|
|
| Re: [CDO] CDOResources and large datasets [message #484061 is a reply to message #483943] |
Fri, 04 September 2009 07:00 |
Stefan Winkler
Messages: 307
Registered: July 2009
Location: Germany |
Senior Member |
|
|
Eike has already said most of this. Just to be sure, I'll say it in my
own words ;)
Mario Winterer schrieb:
> Stefan Winkler schrieb:
>> Eike Stepper schrieb:
>>>> I've got a more clearer picture now of how things could work in my
>>>> application.
>>>> I think I have to get rid of the idea to have one architecture
>>>> (including editors, etc.) that need not be aware of CDO, i.e. an
>>>> architecture that uses a transparent persistence mechanisms - either
>>>> CDO or any other resource-based technique (e.g. simply saving
>>>> resources into files). If I decide to use CDO, editors views etc.
>>>> will have to be slightly/completely(?) different.
>>> Well, CDO is transparent with respect to most of the functionality
>>> that EMF offers through API. But there's a whole lot more about CDO
>>> and if you're going to use this functionality it's somehow hard to
>>> talk about transparency at all :P
>> I have to contradict Eike in one point. Yes, resource handling seems
>> transparent in most cases, but just the case you mention would not work
>> in all situations -- namely saving resources. If you open a transaction
>> and you open two resources, you could pass both resources to two
>> different components of your application. (That's what I did at first in
>> my project. Then a central CDO-aware component could exist which passes
>> EMF-resources around and every other part of the application would only
>> use EMF resources.) This does NOT work in that way, however. Because
>> resource.save() will result in a commit for the transaction and hence,
>> all resources in the resourceSet will be saved at once.
>
> Isn't this a question of transaction scope? If it is required to save
> each resource seperately, there could be opened one transaction per
> resource, right?
Absolutely correct. In plain EMF there is no concept of transaction -
this is why you can save resources independently.
In CDO it is as you say: resources a are part of a transaction and as
such are committed together. (and any CDOResource.save() is just
delegated to resource.getTransaction().commit().
>
>>> On many occasions I found out that t's a nice idea to have an
>>> architecture that is agnostic of specific backend infrastructure. But
>>> later it turned out that the functionality is too reduced then. Maybe
>>> it's a good idea to hide all orthogonal concerns in your own facade?
>>> My eDine application shows some examples on how to do this:
>>> http://thegordian.blogspot.com/2009/04/modeling-goes-enterpr ise.html
>>>> By the way: Is it somehow easily possible to mix up CDO with some
>>>> other persistance mechanism? I'd like to use CDO for one single
>>>> component of our application - all other (already existing)
>>>> components use other ways to communicate with the server. More
>>>> specific, is it possible to have a CDO-enabled model with
>>>> (cross-document) references to a non-CDO model objects?
>>> Yes, CDO is capable of maintaining (CDO-) external references. Stefan
>>> will know if the DBStore 2.0 is also capable of storing them.
>> Yes, we should be able to handle them. In this case the DB stores URLs
>> to the referenced objects which are then resolved at the client side.
>> So, as long as the client still knows the external resource, external
>> references should not be a problem.
>
> Hm. So for each external reference CDO creates a varchar attribute
> containing the URI in the corresponding DB table? My problem is that
> on DB side, the model is consistent, i.e. I've got a table ORDER
> referencing table CUSTOMER via foreign key customerId. But while ORDER
> should be mapped to a CDO-enabled Order-Object, CUSTOMER is still
> controlled by another (proprietary) mechanism (using a
> CustomerResource that directly accesses the CUSTOMER table on save/load).
>
> So I'd need some way to intercept CDO when it loads an Order and
> manipulate the proxy-URI of the referenced Customer object to load the
> proprietary CustomerResource on proxy resolution.
Well, what happens in the DBStore is this. If an external EObject is
encountered, the DBStore generates an (negative long) ID as the stand-in
CDOID to be used as foreign key. It also stores in a separate table the
mapping from the negative ID to the URI of the external EObject. The
DBStore implementation works in a way so that the same URI results in
the same ID, so if two CDOObjects refer to the same external object,
they will contain the same foreign key.
If you want to guarantee referential integrity, you would have to write
and install some database code, which is able to resolve the ID to URIs
and then parse the "real" reference out of that URI and check it.
Cheers,
Stefan
|
|
|
Goto Forum:
Current Time: Thu Apr 25 04:03:08 GMT 2024
Powered by FUDForum. Page generated in 0.03407 seconds |
] 
Search
Help
Register
Login
Home
