Eclipse Community Forums: EMF » [CDO] Offline update of CDO objects

Home » Modeling » EMF » [CDO] Offline update of CDO objects

[CDO] Offline update of CDO objects [message #1139013]

Tue, 15 October 2013 09:27

Eclipse User

We are building a crud webservice over a CDO repository. However we intend to make this service stateless. The way updates should work is
1. User reads a CDO object (using a CDOSession that would be closed after the read)
2. User comminicates back the edited attributes.
3. We push the changes back to CDO repository using a new CDOSession.

The question here for step 3 is, can we push changes back to the repository without getting an instance of CDO Object from the repository.

I ask this for efficiency reasons. The fall back obviously would open a transaction , read back the CDO object, apply the changes and commit the transaction.

Can the read back be avoided ? Can we recreate a CDOObject and commit it via a transaction.

Regards,
- Ashish

Re: [CDO] Offline update of CDO objects [message #1139027 is a reply to message #1139013]

Tue, 15 October 2013 09:40

Eclipse User

Hi, Ashish,

What is your plan to avoid dirty updates? It is entirely possible that
between the first session that provided the object to the user and the
second that accepted the user's changes to that object, some other user
has also committed changes to that same object.

The normal usage of CDO repositories is definitely session-oriented and
transactional, to prevent issues such as dirty updates (as well as all
of the benefits of live synchronization etc.).

Cheers,

Christian

On 2013-10-15 13:27:32 +0000, Ashish Shinde said:

> We are building a crud webservice over a CDO repository. However we
> intend to make this service stateless. The way updates should work is
> 1. User reads a CDO object (using a CDOSession that would be closed
> after the read)
> 2. User comminicates back the edited attributes.
> 3. We push the changes back to CDO repository using a new CDOSession.
>
> The question here for step 3 is, can we push changes back to the
> repository without getting an instance of CDO Object from the
> repository.
> I ask this for efficiency reasons. The fall back obviously would open a
> transaction , read back the CDO object, apply the changes and commit
> the transaction.
>
> Can the read back be avoided ? Can we recreate a CDOObject and commit
> it via a transaction.
>
> Regards,
> - Ashish

Re: [CDO] Offline update of CDO objects [message #1139058 is a reply to message #1139027]

Tue, 15 October 2013 10:02

Eclipse User

Hi Christian,

I was hoping that if I can "attach" the changed object to a transaction while preserving the base CDO revision. The hope was that the transaction would fail probably with a conflict exception. If this is a dirty commit then we would have the user resolve the conflict and submit a new CDO object with the latest as the base revision. In our use cases conflicts will be rare and hence a heavy conflict resolution procedure is alright.

Ofcourse if we absolutely have to read object from the repository before commiting the changes, then the we would use object version(?) to detect dirty changes and have the user resolve.

I would like to mention that I am still new to CDO and do not understand all the concepts well. However we benchmarked and liked CDO and are developing a product using CDO repository.

Thanks and regards,
- Ashish

Re: [CDO] Offline update of CDO objects [message #1139194 is a reply to message #1139058]

Tue, 15 October 2013 11:54

Eclipse User

Am 15.10.2013 16:02, schrieb Ashish Shinde:
> Hi Christian,
>
> I was hoping that if I can "attach" the changed object to a transaction while preserving the base CDO revision.
It's still unclear to me what transaction you mean because in your first post you said you're always closing the
CDOSession of your REST service.

It's also unclear to me why you're closing that session repeatedly while it has valuable caches of object revisions
(well, the repository maintains its own cache for those, which could mean that you spare noticable resources by repacing
your client-side cache with CDORevisionCache.NOOP). But I'd suggest to keep a CDOSession and a read-only CDOView open in
the REST service implementation.

BTW. with an open session it's not only cheaper to load objects for the next read request but also to open
CDOTransactions (to apply your changes).

Cheers
/Eike

----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper

> The hope was that the transaction would fail probably with a conflict exception. If this is a dirty commit then we
> would have the user resolve the conflict and submit a new CDO object with the latest as the base revision. In our use
> cases conflicts will be rare and hence a heavy conflict resolution procedure is alright.
>
> Ofcourse if we absolutely have to read object from the repository before commiting the changes, then the we would use
> object version(?) to detect dirty changes and have the user resolve.
>
> I would like to mention that I am still new to CDO and do not understand all the concepts well. However we benchmarked
> and liked CDO and are developing a product using CDO repository.
>
> Thanks and regards,
> - Ashish

Re: [CDO] Offline update of CDO objects [message #1140157 is a reply to message #1139194]

Wed, 16 October 2013 03:01

Eclipse User

Hi Eike,

Sorry about not being clear enough. I meant a CDOTransaction that I would open after opening a new CDOSession for the commit request. Let me try and explain the update flow as I am thinking about it

step 1. rest read request -> open session -> read object -> convert to json
step 2. the user edits the object as json and sends back edited json
step 3. (this is where I need help) recreate a dirty CDO Object from edited json and commit. Ofcourse if the base revision from the read in step 1 is not the same as the head revision we need to flag this as a conflict and the user would resolve it after seeing the latest version and his version.

If I have the session and the original CDO object from sterp 1 in step 3, I could apply the edits from json and commit the object.

However the worry is that this might not scale well. As in we are building a system where 50-100 people will be editing the models and commiting. User sessions could last for a day or a couple of days. The worry was that having a cdosession for each user and also having all the "opened" CDO objects in memory would put heavy RAM requirements on our servers. To top it all there would be programs that would also making bulk edits to these objects.

Again I could be totally wrong with this hypothesis. But this is the reason we wanted to be totally stateless and open a new session for each request.

Hope I made the problem more clearer.

Regards,
- Ashish

> Hi Christian,
>
> I was hoping that if I can "attach" the changed object to a transaction while preserving the base CDO revision.
It's still unclear to me what transaction you mean because in your first post you said you're always closing the
CDOSession of your REST service.

It's also unclear to me why you're closing that session repeatedly while it has valuable caches of object revisions
(well, the repository maintains its own cache for those, which could mean that you spare noticable resources by repacing
your client-side cache with CDORevisionCache.NOOP). But I'd suggest to keep a CDOSession and a read-only CDOView open in
the REST service implementation.

BTW. with an open session it's not only cheaper to load objects for the next read request but also to open
CDOTransactions (to apply your changes).

Cheers
/Eike

----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper

> The hope was that the transaction would fail probably with a conflict exception. If this is a dirty commit then we
> would have the user resolve the conflict and submit a new CDO object with the latest as the base revision. In our use
> cases conflicts will be rare and hence a heavy conflict resolution procedure is alright.
>
> Ofcourse if we absolutely have to read object from the repository before commiting the changes, then the we would use
> object version(?) to detect dirty changes and have the user resolve.
>
> I would like to mention that I am still new to CDO and do not understand all the concepts well. However we benchmarked
> and liked CDO and are developing a product using CDO repository.
>
> Thanks and regards,
> - Ashish

Re: [CDO] Offline update of CDO objects [message #1140475 is a reply to message #1140157]

Wed, 16 October 2013 07:33

Eclipse User

Am 16.10.2013 09:02, schrieb Ashish Shinde:
> Hi Eike,
>
> Sorry about not being clear enough. I meant a CDOTransaction that I would open after opening a new CDOSession for the
> commit request. Let me try and explain the update flow as I am thinking about it
>
> step 1. rest read request -> open session -> read object -> convert to json
So that means:

rest read request -> open session -> *open read-only view* -> read object -> convert to json

Do you keep that session and view open then?

> step 2. the user edits the object as json and sends back edited json
What kind of information does this update request" contain? Deltas? the new values of changed features? The values of
all features of the changed objects? What about additions or deletions?

> step 3. (this is where I need help) recreate a dirty CDO Object from edited json and commit.
Ideally with something like:

CDOTransaction tx = session.openTransaction();
tx.importChanges(inputStream);

Where inputStream delivers the result of a former tx.exportChanges(outputStream).

That seems difficult with your JSON infrastructure, so you'll need to iterate over your change infos and do something like:

for (ChangeInfo changeInfo : changeInfos)
{
CDOObject object = tx.getObject(changeInfo.getID());
if (object.cdoRevision().getVersion() != changeInfo.getVersion())
throw new ConcurrentModificationException();
applyChangeInfo(changeInfo, object);
}
tx.commit();
tx.close();

> Ofcourse if the base revision from the read in step 1 is not the same as the head revision we need to flag this as a
> conflict and the user would resolve it after seeing the latest version and his version.
See my code above.

>
> If I have the session and the original CDO object from sterp 1 in step 3, I could apply the edits from json and commit
> the object.
>
> However the worry is that this might not scale well. As in we are building a system where 50-100 people will be
> editing the models and commiting. User sessions could last for a day or a couple of days. The worry was that having a
> cdosession for each user and also having all the "opened" CDO objects in memory would put heavy RAM requirements on
> our servers. To top it all there would be programs that would also making bulk edits to these objects.
Not sure what that all means but I think my suggestions above addresses it.

>
> Again I could be totally wrong with this hypothesis. But this is the reason we wanted to be totally stateless and open
> a new session for each request.
You should keep the session and a read-only view open to speed up read requests. The session will also speed up opening
new transactions per update request. This could be further optimized to reuse opened transactions from a pool.

Cheers
/Eike

----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper

Re: [CDO] Offline update of CDO objects [message #1143350 is a reply to message #1140475]

Fri, 18 October 2013 02:03

Eclipse User

Hi Eike,

Thank you for the reply. Answers to your questions inline.

Regards,
-Ashish

>Am 16.10.2013 09:02, schrieb Ashish Shinde:
>> Hi Eike,
>>
>> Sorry about not being clear enough. I meant a CDOTransaction that I would open after opening a new CDOSession for the
>> commit request. Let me try and explain the update flow as I am thinking about it
>>
>> step 1. rest read request -> open session -> read object -> convert to json
>So that means:
>
>rest read request -> open session -> *open read-only view* -> read object -> convert to json
>
>Do you keep that session and view open then?

No we plan to close the session and view immediately after reading the object. Unless in line with what you mentioned, session and or view and or transactions can be pooled across users. I guess sessions are tied to a user so this might not work out.

Let me try and explain why the reluctance to keep sessions open.

Our system will support 50-100 users using the rest webservice through a web interface. Each user would be mostly editing model instances. The edit operation invovles doing a bit of reasearch and a single edit could span a day or so. Therefore after a user reads an object he could take a day or so to commit the updates. He might have read a few more referenced object or maybe unrelated objects too.

Also a user web session might expire while he was doing edits. We want to save his partial edits temporarily in the webinterface backend (out of the rest webservice) so that on next web interface login he could start from his partial edits and later on commit.

Of course if we can keep session for a user perpetually open that might work as well, however the question is how much memory is consumed by open sessions. Is it proportional to the number of objects read from a view opened from the session or is it constant. If it is constant then can we keep perpetual session open per user. Will there be memory leaks etc. Again we could periodically close session too to circumvent build up.

I hope I have clarified the situation.

>
>
>> step 2. the user edits the object as json and sends back edited json
>What kind of information does this update request" contain? Deltas? the new values of changed features? The values of
>all features of the changed objects? What about additions or deletions?
>
>> step 3. (this is where I need help) recreate a dirty CDO Object from edited json and commit.
>Ideally with something like:
>
>CDOTransaction tx = session.openTransaction();
>tx.importChanges(inputStream);
>
>Where inputStream delivers the result of a former tx.exportChanges(outputStream).
>
>That seems difficult with your JSON infrastructure, so you'll need to iterate over your change infos and do something like:
>
>for (ChangeInfo changeInfo : changeInfos)
>{
>CDOObject object = tx.getObject(changeInfo.getID());
>if (object.cdoRevision().getVersion() != changeInfo.getVersion())
>throw new ConcurrentModificationException();
>applyChangeInfo(changeInfo, object);
>}
>tx.commit();
>tx.close();
>
>> Ofcourse if the base revision from the read in step 1 is not the same as the head revision we need to flag this as a
>> conflict and the user would resolve it after seeing the latest version and his version.
>See my code above.
>

Re: [CDO] Offline update of CDO objects [message #1143427 is a reply to message #1143350]

Fri, 18 October 2013 03:13

Eclipse User

Am 18.10.2013 08:03, schrieb Ashish Shinde:
> Hi Eike,
>
> Thank you for the reply. Answers to your questions inline.
>
> Regards,
> -Ashish
>
>> Am 16.10.2013 09:02, schrieb Ashish Shinde:
>>> Hi Eike,
>>>
>>> Sorry about not being clear enough. I meant a CDOTransaction that I would open after opening a new CDOSession for
>>> the commit request. Let me try and explain the update flow as I am thinking about it
>>>
>>> step 1. rest read request -> open session -> read object -> convert to json
>> So that means:
>>
>> rest read request -> open session -> *open read-only view* -> read object -> convert to json
>>
>> Do you keep that session and view open then?
>
> No we plan to close the session and view immediately after reading the object. Unless in line with what you mentioned,
> session and or view and or transactions can be pooled across users. I guess sessions are tied to a user so this might
> not work out.
They're only tied to user *names* if you use CDO authentication against the repo. If you don't use any security features
in CDO there's no reason to not reuse a single session across all requests. And then you would certainly want to also
keep the read-only view open, which caches the EObject graph.

Cheers
/Eike

----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper

>
> Let me try and explain why the reluctance to keep sessions open.
> Our system will support 50-100 users using the rest webservice through a web interface. Each user would be mostly
> editing model instances. The edit operation invovles doing a bit of reasearch and a single edit could span a day or
> so. Therefore after a user reads an object he could take a day or so to commit the updates. He might have read a few
> more referenced object or maybe unrelated objects too.
> Also a user web session might expire while he was doing edits. We want to save his partial edits temporarily in the
> webinterface backend (out of the rest webservice) so that on next web interface login he could start from his partial
> edits and later on commit.
> Of course if we can keep session for a user perpetually open that might work as well, however the question is how much
> memory is consumed by open sessions. Is it proportional to the number of objects read from a view opened from the
> session or is it constant. If it is constant then can we keep perpetual session open per user. Will there be memory
> leaks etc. Again we could periodically close session too to circumvent build up.
> I hope I have clarified the situation.
>>
>>
>>> step 2. the user edits the object as json and sends back edited json
>> What kind of information does this update request" contain? Deltas? the new values of changed features? The values of
>> all features of the changed objects? What about additions or deletions?
>>
>>> step 3. (this is where I need help) recreate a dirty CDO Object from edited json and commit.
>> Ideally with something like:
>>
>> CDOTransaction tx = session.openTransaction();
>> tx.importChanges(inputStream);
>>
>> Where inputStream delivers the result of a former tx.exportChanges(outputStream).
>>
>> That seems difficult with your JSON infrastructure, so you'll need to iterate over your change infos and do something
>> like:
>>
>> for (ChangeInfo changeInfo : changeInfos)
>> {
>> CDOObject object = tx.getObject(changeInfo.getID());
>> if (object.cdoRevision().getVersion() != changeInfo.getVersion())
>> throw new ConcurrentModificationException();
>> applyChangeInfo(changeInfo, object);
>> }
>> tx.commit();
>> tx.close();
>>
>>> Ofcourse if the base revision from the read in step 1 is not the same as the head revision we need to flag this as a
>>> conflict and the user would resolve it after seeing the latest version and his version.
>> See my code above.
>>
>

Previous Topic:	Cause an unsettable EInt attribute to be unset by default?
Next Topic:	Cannot add an Inheritance from a new EClass to an EClass imported from another ecore

Goto Forum:

-=] Back to Top [=-

Current Time: Wed Jul 16 04:58:13 EDT 2025

.:: Contact :: Home ::.

Breadcrumbs

Sign up to our Newsletter