Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » EMF » [CDO] disable revisions(delete old revisions automatically)
[CDO] disable revisions [message #691395] Fri, 01 July 2011 07:24 Go to next message
Erdal Karaca is currently offline Erdal Karaca
Messages: 740
Registered: July 2009
Senior Member
I am using CDO with the DB storage and I am using a lot of SQL queries.
SQL queries have all to be made aware of the version of an object, i.e. if I do not want to have old revisions returned, I have to add check the revision by adding a "cdo_version > 0" to the where clause...

How can I "disable" the revision feature in CDO? Is there a setting like disabling audits/branches?
Re: [CDO] disable revisions [message #691426 is a reply to message #691395] Fri, 01 July 2011 08:39 Go to previous messageGo to next message
techteam is currently offline techteam
Messages: 53
Registered: September 2010
Member
Have you looked here
(http://wiki.eclipse.org/CDO/Server_Configuration_Reference) ?

and I think you need you have to configure your CDO server repository like:
<repository name="your_repo_name">
<property name="supportingAudits" value="false"/>
<property name="supportingBranches" value="false"/>
.
.
.
</repository>

Am 01.07.2011 13:24, schrieb Erdal Karaca:
> I am using CDO with the DB storage and I am using a lot of SQL queries.
> SQL queries have all to be made aware of the version of an object, i.e.
> if I do not want to have old revisions returned, I have to add check the
> revision by adding a "cdo_version > 0" to the where clause...
>
> How can I "disable" the revision feature in CDO? Is there a setting like
> disabling audits/branches?
>
Re: [CDO] disable revisions [message #691532 is a reply to message #691426] Fri, 01 July 2011 12:50 Go to previous messageGo to next message
Erdal Karaca is currently offline Erdal Karaca
Messages: 740
Registered: July 2009
Senior Member
I have already disabled audits and branches, but need to disable revisions as well, or at least let cdo automatically delete old revisions of the objects...
Re: [CDO] disable revisions [message #691754 is a reply to message #691532] Sat, 02 July 2011 04:31 Go to previous messageGo to next message
Eike Stepper is currently offline Eike Stepper
Messages: 5500
Registered: July 2009
Senior Member
Am 01.07.2011 18:50, schrieb Erdal Karaca:
> I have already disabled audits
That should lead to automatic deletion/overwrite of "old" revisons" when new ones are being committed.

> and branches, but need to disable revisions as well,
I suspect some sort of misunderstanding here. You can not "disable revisions" per se. CDO will always use them in the communication between the various architectural layers.

> or at least let cdo automatically delete old revisions of the objects...
Do you have evidence that this does not happen with <property name="supportingAudits" value="false"/>?

Cheers
/Eike

----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper
Re: [CDO] disable revisions [message #691755 is a reply to message #691426] Sat, 02 July 2011 04:33 Go to previous messageGo to next message
Eike Stepper is currently offline Eike Stepper
Messages: 5500
Registered: July 2009
Senior Member
Am 01.07.2011 14:39, schrieb Heiko:
> Have you looked here (http://wiki.eclipse.org/CDO/Server_Configuration_Reference) ?
>
> and I think you need you have to configure your CDO server repository like:
> <repository name="your_repo_name">
> <property name="supportingAudits" value="false"/>
Correct. Please also note that this is a "first start initialization property". You must not change it once the repository has been started and initialized for the first time!

Cheers
/Eike

----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper


> <property name="supportingBranches" value="false"/>
> .
> .
> .
> </repository>
>
> Am 01.07.2011 13:24, schrieb Erdal Karaca:
>> I am using CDO with the DB storage and I am using a lot of SQL queries.
>> SQL queries have all to be made aware of the version of an object, i.e.
>> if I do not want to have old revisions returned, I have to add check the
>> revision by adding a "cdo_version > 0" to the where clause...
>>
>> How can I "disable" the revision feature in CDO? Is there a setting like
>> disabling audits/branches?
>>
>
Re: [CDO] disable revisions [message #692294 is a reply to message #691754] Mon, 04 July 2011 03:02 Go to previous messageGo to next message
Erdal Karaca is currently offline Erdal Karaca
Messages: 740
Registered: July 2009
Senior Member
This is an excerpt from my config file:

<repository name="repo1">
<property name="overrideUUID" value="repo1" />
<property name="supportingAudits" value="false" />
<property name="supportingBranches" value="false" />
<property name="verifyingRevisions" value="false" />
<property name="currentLRUCapacity" value="10000" />
<property name="revisedLRUCapacity" value="100" />
<property name="ensureReferentialIntegrity" value="false" />

And an object list (see attachment) with cdo objects that should have been deleted, but instead have negative version numbers...

index.php/fa/3234/0/


Maybe, I am missing something...
Re: [CDO] disable revisions [message #692318 is a reply to message #692294] Mon, 04 July 2011 04:02 Go to previous messageGo to next message
Eike Stepper is currently offline Eike Stepper
Messages: 5500
Registered: July 2009
Senior Member
Am 04.07.2011 09:02, schrieb Erdal Karaca:
> This is an excerpt from my config file:
>
> <repository name="repo1">
> <property name="overrideUUID" value="repo1" />
> <property name="supportingAudits" value="false" />
> <property name="supportingBranches" value="false" />
> <property name="verifyingRevisions" value="false" />
> <property name="currentLRUCapacity" value="10000" />
> <property name="revisedLRUCapacity" value="100" />
The last 3 preoperties are obsolete.

> <property name="ensureReferentialIntegrity" value="false" />
>
> And an object list (see attachment) with cdo objects that should have been deleted, but instead have negative version numbers...
The DBStore uses negative version numbers to indicate the deletion. It's an implementation detail.

Cheers
/Eike

----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper
Re: [CDO] disable revisions [message #692333 is a reply to message #692318] Mon, 04 July 2011 04:49 Go to previous messageGo to next message
Erdal Karaca is currently offline Erdal Karaca
Messages: 740
Registered: July 2009
Senior Member
Thanks for the info.

In some cases the objects are deleted completely. In this case, the version is set to a negative number... Is it possible to instruct CDO to delete them permanently?
I have to check for the version whenever I query the db which is annoying if I have disabled audits/branches...
Re: [CDO] disable revisions [message #692355 is a reply to message #692333] Mon, 04 July 2011 05:05 Go to previous messageGo to next message
Eike Stepper is currently offline Eike Stepper
Messages: 5500
Registered: July 2009
Senior Member
Am 04.07.2011 10:49, schrieb Erdal Karaca:
> Thanks for the info.
>
> In some cases the objects are deleted completely.
Do you see a pattern? I.e. when are they deleted completely?

> In this case, the version is set to a negative number... Is it possible to instruct CDO to delete them permanently?
No, I don't think so, but I can't remember why exactly. Maybe Stefan has more insights?

Cheers
/Eike

----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper
Re: [CDO] disable revisions [message #692356 is a reply to message #692355] Mon, 04 July 2011 05:24 Go to previous messageGo to next message
Eike Stepper is currently offline Eike Stepper
Messages: 5500
Registered: July 2009
Senior Member
Hi Erdal,

Stefan is currently travelling but he asked to submit a bugzilla (please do it) as a reminder that we investigate this further. Maybe there's a chance to change the behaviour for the non-audit case.

Cheers
/Eike

----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper



Am 04.07.2011 11:05, schrieb Eike Stepper:
> Am 04.07.2011 10:49, schrieb Erdal Karaca:
>> Thanks for the info.
>>
>> In some cases the objects are deleted completely.
> Do you see a pattern? I.e. when are they deleted completely?
>
>> In this case, the version is set to a negative number... Is it possible to instruct CDO to delete them permanently?
> No, I don't think so, but I can't remember why exactly. Maybe Stefan has more insights?
>
> Cheers
> /Eike
>
> ----
> http://www.esc-net.de
> http://thegordian.blogspot.com
> http://twitter.com/eikestepper
>
Re: [CDO] disable revisions [message #692357 is a reply to message #692333] Mon, 04 July 2011 05:05 Go to previous messageGo to next message
Eike Stepper is currently offline Eike Stepper
Messages: 5500
Registered: July 2009
Senior Member
Am 04.07.2011 10:49, schrieb Erdal Karaca:
> Thanks for the info.
>
> In some cases the objects are deleted completely.
Do you see a pattern? I.e. when are they deleted completely?

> In this case, the version is set to a negative number... Is it possible to instruct CDO to delete them permanently?
No, I don't think so, but I can't remember why exactly. Maybe Stefan has more insights?

Cheers
/Eike

----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper
Re: [CDO] disable revisions [message #692359 is a reply to message #692355] Mon, 04 July 2011 05:24 Go to previous messageGo to next message
Eike Stepper is currently offline Eike Stepper
Messages: 5500
Registered: July 2009
Senior Member
Hi Erdal,

Stefan is currently travelling but he asked to submit a bugzilla (please do it) as a reminder that we investigate this further. Maybe there's a chance to change the behaviour for the non-audit case.

Cheers
/Eike

----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper



Am 04.07.2011 11:05, schrieb Eike Stepper:
> Am 04.07.2011 10:49, schrieb Erdal Karaca:
>> Thanks for the info.
>>
>> In some cases the objects are deleted completely.
> Do you see a pattern? I.e. when are they deleted completely?
>
>> In this case, the version is set to a negative number... Is it possible to instruct CDO to delete them permanently?
> No, I don't think so, but I can't remember why exactly. Maybe Stefan has more insights?
>
> Cheers
> /Eike
>
> ----
> http://www.esc-net.de
> http://thegordian.blogspot.com
> http://twitter.com/eikestepper
>
Re: [CDO] disable revisions [message #692387 is a reply to message #692356] Mon, 04 July 2011 06:58 Go to previous messageGo to next message
Erdal Karaca is currently offline Erdal Karaca
Messages: 740
Registered: July 2009
Senior Member
Filed bug:
https://bugs.eclipse.org/bugs/show_bug.cgi?id=351068
Re: [CDO] disable revisions [message #988367 is a reply to message #691395] Thu, 29 November 2012 10:59 Go to previous messageGo to next message
Silvestre Martins is currently offline Silvestre Martins
Messages: 41
Registered: July 2009
Member
I have a question related with the support of auditing mode.

Scenario:
Imagine we need to perform a read-only and very time consuming operation. Therefore is can be done in a separate and concurrent procedure. No need to take care about conflicts at the end because is read-only.
This operation must be done in a "snapshot" of the model (such that all reads must look to the same state of a given moment, here the moment when the operation started).

With CDO, this seems to fit very well the feature of Auditing.
The problem is that this requires the database stores all revisions, regardless if we will need to audit it or not, which leads to database growing very fast.

My question is: could we have a mechanism of deleting an old revision?

Or even better, could we, instead of having constantly new revisions being created every time a row is changed, do it only when a specific state was marked to be kept. We could assign a special identifier to these rows, every time the row is changed, and then it would be easy to just remove the old rows.

Is this somehow already possible in CDO?
Re: [CDO] disable revisions [message #988395 is a reply to message #988367] Thu, 29 November 2012 12:35 Go to previous messageGo to next message
Eike Stepper is currently offline Eike Stepper
Messages: 5500
Registered: July 2009
Senior Member
Am 29.11.2012 16:59, schrieb Silvestre Martins:
> I have a question related with the support of auditing mode.
>
> Scenario:
> Imagine we need to perform a read-only and very time consuming operation. Therefore is can be done in a separate and
> concurrent procedure. No need to take care about conflicts at the end because is read-only.
> This operation must be done in a "snapshot" of the model (such that all reads must look to the same state of a given
> moment, here the moment when the operation started).
>
> With CDO, this seems to fit very well the feature of Auditing.
> The problem is that this requires the database stores all revisions, regardless if we will need to audit it or not,
> which leads to database growing very fast.
>
> My question is: could we have a mechanism of deleting an old revision?
>
> Or even better, could we, instead of having constantly new revisions being created every time a row is changed, do it
> only when a specific state was marked to be kept. We could assign a special identifier to these rows, every time the
> row is changed, and then it would be easy to just remove the old rows.
>
> Is this somehow already possible in CDO?
No, not yet. It sounds interesting, though. Are you volunteering to develop such functionality?

BTW. I don't think we should "associate" something (like a " special identifier") with rows explicitely. A collection of
timestamps should do, too. And I wonder if all this must impact the storage layer at all. What about just changing the
server-side revision cache?

Cheers
/Eike

----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper
Re: [CDO] disable revisions [message #988404 is a reply to message #988395] Thu, 29 November 2012 13:53 Go to previous messageGo to next message
Silvestre Martins is currently offline Silvestre Martins
Messages: 41
Registered: July 2009
Member
Eike Stepper wrote on Thu, 29 November 2012 12:35
Am 29.11.2012 16:59, schrieb Silvestre Martins:
> I have a question related with the support of auditing mode.
>
> Scenario:
> Imagine we need to perform a read-only and very time consuming operation. Therefore is can be done in a separate and
> concurrent procedure. No need to take care about conflicts at the end because is read-only.
> This operation must be done in a "snapshot" of the model (such that all reads must look to the same state of a given
> moment, here the moment when the operation started).
>
> With CDO, this seems to fit very well the feature of Auditing.
> The problem is that this requires the database stores all revisions, regardless if we will need to audit it or not,
> which leads to database growing very fast.
>
> My question is: could we have a mechanism of deleting an old revision?
>
> Or even better, could we, instead of having constantly new revisions being created every time a row is changed, do it
> only when a specific state was marked to be kept. We could assign a special identifier to these rows, every time the
> row is changed, and then it would be easy to just remove the old rows.
>
> Is this somehow already possible in CDO?
No, not yet. It sounds interesting, though. Are you volunteering to develop such functionality?

Not really, currently I'm still evaluating if we can adopt CDO in our project. Of course the possibility and how easy is to implement such features (internally or not) will also influence the decision.

Quote:

BTW. I don't think we should "associate" something (like a " special identifier") with rows explicitely. A collection of
timestamps should do, too. And I wonder if all this must impact the storage layer at all. What about just changing the
server-side revision cache?

I don't know the details of implementation of CDO, but I suppose that to support "partial" auditing (i.e., creates different version only when the current version was marked to be kept) would depend on the storage layer.

Anyway, you are right, applying a marker is not efficient, because you would need to update all tables. It would be easier to just create a special table to store "durable views", that would only need an ID and a timestamp.
Then, when deciding if we make an update on the row, or instead insert a new row, we would query this table. If there is an entry in "durable views" with timestamp equal or higher than the timestamp of the existing row being updated, then make an insert and keep existing row.
When removing this "durable view", we just need to remove the entry from this table, plus all rows in all tables where the timestamp is equal or lower than the timestamp of the "durable view" and a newer version of that line exists. As sooner the "durable view" is removed, as sooner we would stop to create new rows instead of update them.
Sounds easy, I just don't if this is more efficient that just enabling the Auditing Smile

[Updated on: Thu, 29 November 2012 13:59]

Report message to a moderator

Re: [CDO] disable revisions [message #988406 is a reply to message #988404] Thu, 29 November 2012 14:02 Go to previous messageGo to next message
Eike Stepper is currently offline Eike Stepper
Messages: 5500
Registered: July 2009
Senior Member
Am 29.11.2012 19:53, schrieb Silvestre Martins:
> Quote:
>> BTW. I don't think we should "associate" something (like a " special identifier") with rows explicitely. A collection
>> of timestamps should do, too. And I wonder if all this must impact the storage layer at all. What about just changing
>> the server-side revision cache?
>
> I don't know the details of implementation of CDO, but I suppose that to support "partial" auditing (i.e., creates
> different version only when the current version was marked to be kept) would depend on the storage layer.
The term "partial auditing" is already reserved for the more horizontal aspect of supporting auditiing only for a subset
of the model classes. It's not supported by CDO, yet.

>
> Anyway, you are right, applying a marker is not efficient, because you would need to update all tables. It would be
> easier to just create a special table to store "durable views", that would only need an ID and a timestamp. Then, when
> deciding if we make an update on the row, or instead insert a new row, we would query this table. If there is an entry
> in "durable views" with timestamp equal or higher than the timestamp of the existing row being updated, then make an
> insert and keep existing row.
> When removing this "durable view", we just need to remove the entry from this table, plus all rows in all tables where
> the timestamp is equal or lower than the timestamp of the "durable view" and a newer version of that line exists. As
> sooner the "durable view" is removed, as sooner we would stop to create new rows instead of update them.
> Sounds easy, I just don't if this is more efficient that just enabling the Auditing :)
Let's discuss the details then when it's clear that someone will actually implement it ;-)

Cheers
/Eike

----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper
Re: [CDO] disable revisions [message #988407 is a reply to message #988406] Thu, 29 November 2012 14:09 Go to previous message
Silvestre Martins is currently offline Silvestre Martins
Messages: 41
Registered: July 2009
Member
Eike Stepper wrote on Thu, 29 November 2012 14:02
Am 29.11.2012 19:53, schrieb Silvestre Martins:
> Quote:
>> BTW. I don't think we should "associate" something (like a " special identifier") with rows explicitely. A collection
>> of timestamps should do, too. And I wonder if all this must impact the storage layer at all. What about just changing
>> the server-side revision cache?
>
> I don't know the details of implementation of CDO, but I suppose that to support "partial" auditing (i.e., creates
> different version only when the current version was marked to be kept) would depend on the storage layer.
The term "partial auditing" is already reserved for the more horizontal aspect of supporting auditiing only for a subset
of the model classes. It's not supported by CDO, yet.

>
> Anyway, you are right, applying a marker is not efficient, because you would need to update all tables. It would be
> easier to just create a special table to store "durable views", that would only need an ID and a timestamp. Then, when
> deciding if we make an update on the row, or instead insert a new row, we would query this table. If there is an entry
> in "durable views" with timestamp equal or higher than the timestamp of the existing row being updated, then make an
> insert and keep existing row.
> When removing this "durable view", we just need to remove the entry from this table, plus all rows in all tables where
> the timestamp is equal or lower than the timestamp of the "durable view" and a newer version of that line exists. As
> sooner the "durable view" is removed, as sooner we would stop to create new rows instead of update them.
> Sounds easy, I just don't if this is more efficient that just enabling the Auditing Smile
Let's discuss the details then when it's clear that someone will actually implement it Wink


Yes, sure. I was just trying to figure out how complex or simple this could be.

Cheers,
Silvestre
Previous Topic:[CDO] CDO Migrate and EMF Model Change Support
Next Topic:[CDO] Problem with concurrent read and write threads
Goto Forum:
  


Current Time: Wed Aug 27 05:03:49 EDT 2014

Powered by FUDForum. Page generated in 0.02625 seconds