Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » EMF » [CDO] Migrate Repository when Ecore changes
[CDO] Migrate Repository when Ecore changes [message #1039705] Fri, 12 April 2013 13:34 Go to next message
Alan McMorran is currently offline Alan McMorranFriend
Messages: 55
Registered: July 2009
Member
We have been using CDO with an H2 database back-end in our software for
the last year and so far it has proved itself invaluable. One problem
we now have is that with clients utilising the database persistence we
cannot make alterations to the underlying Ecore models without breaking
the database.

The situation we are in is that the application's data is defined by an
Ecore model and a CDO-genmodel. We will sometimes get a request from a
client to add extensions to our standard product schema, however this
means that if they install an updated version of the plug-in containing
the Ecore the database schema that had been generated no longer
matches. I realise this is to be expected as it will be looking for
columns that are not present. I am wondering is if there is a facility
to export the entire CDO database (including all revision/branch data -
I can easily extract the current data but I would like to preserve all
the past commits); upgrade the plug-in to the new version of the
schema; then repopulate the database.

I can enforce some restrictions on what will change:
1. There will only ever be additions to the model, no attributes or
references will be removed
2. We can restrict additions to being through multiple-inheritance
(e.g. if we have class A and want to add attribute X we would create a
new class B with the attribtue and have A inherit from B)
3. The generated model code will be from a CDO model (i.e. no legacy support).

However the URI of the base schema cannot change as we want this model
to be a replacement for the base model, not for them to co-exist.

Obviously I am happy to have it run a time-consuming one-time upgrade
process post-update to support this (assuming it's a reasonable time).
I was really wondering if this was something that CDO already supported
(and I'd missed it) or if anyone has done anything similar.

My initial thoughts are that I could export the base data from the
first commit for each resource; extract all the change models up to the
current version; upgrade the plug-in; and then re-import and recreate
the changes from the base+change data. I suspect this may be
time-consuming for large data sets and adding support for branches
would add additional complexity but (in my head at least) I think it
would work.

I am no CDO expert though so any suggestions would be more than welcome!

Alan
Re: [CDO] Migrate Repository when Ecore changes [message #1041692 is a reply to message #1039705] Mon, 15 April 2013 12:22 Go to previous messageGo to next message
Eike Stepper is currently offline Eike StepperFriend
Messages: 6681
Registered: July 2009
Senior Member
Hi Alan,

I'll soon start on an estimate for proper model evolution support. Then I'd like to see if crowd funding is an option
for a bigger feature like this.

In the mean time you could use the CDOServerExporter, transform the resulting XML file to the new model and then use
CDOServerImporter.

Cheers
/Eike

----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper


Am 12.04.2013 15:34, schrieb Alan McMorran:
> We have been using CDO with an H2 database back-end in our software for the last year and so far it has proved itself
> invaluable. One problem we now have is that with clients utilising the database persistence we cannot make alterations
> to the underlying Ecore models without breaking the database.
>
> The situation we are in is that the application's data is defined by an Ecore model and a CDO-genmodel. We will
> sometimes get a request from a client to add extensions to our standard product schema, however this means that if
> they install an updated version of the plug-in containing the Ecore the database schema that had been generated no
> longer matches. I realise this is to be expected as it will be looking for columns that are not present. I am
> wondering is if there is a facility to export the entire CDO database (including all revision/branch data - I can
> easily extract the current data but I would like to preserve all the past commits); upgrade the plug-in to the new
> version of the schema; then repopulate the database.
>
> I can enforce some restrictions on what will change:
> 1. There will only ever be additions to the model, no attributes or references will be removed
> 2. We can restrict additions to being through multiple-inheritance (e.g. if we have class A and want to add attribute
> X we would create a new class B with the attribtue and have A inherit from B)
> 3. The generated model code will be from a CDO model (i.e. no legacy support).
>
> However the URI of the base schema cannot change as we want this model to be a replacement for the base model, not for
> them to co-exist.
>
> Obviously I am happy to have it run a time-consuming one-time upgrade process post-update to support this (assuming
> it's a reasonable time). I was really wondering if this was something that CDO already supported (and I'd missed it)
> or if anyone has done anything similar.
>
> My initial thoughts are that I could export the base data from the first commit for each resource; extract all the
> change models up to the current version; upgrade the plug-in; and then re-import and recreate the changes from the
> base+change data. I suspect this may be time-consuming for large data sets and adding support for branches would add
> additional complexity but (in my head at least) I think it would work.
>
> I am no CDO expert though so any suggestions would be more than welcome!
>
> Alan
>


Re: [CDO] Migrate Repository when Ecore changes [message #1041785 is a reply to message #1041692] Mon, 15 April 2013 15:17 Go to previous messageGo to next message
Alan McMorran is currently offline Alan McMorranFriend
Messages: 55
Registered: July 2009
Member
The CDOServerExporter may actually do what I need! I'll do some experimenting.

Thanks for the pointer

Alan

On 2013-04-15 12:22:24 +0000, Eike Stepper said:

> Hi Alan,
>
> I'll soon start on an estimate for proper model evolution support. Then
> I'd like to see if crowd funding is an option for a bigger feature like
> this.
>
> In the mean time you could use the CDOServerExporter, transform the
> resulting XML file to the new model and then use CDOServerImporter.
>
> Cheers
> /Eike
>
> ----
> http://www.esc-net.de
> http://thegordian.blogspot.com
> http://twitter.com/eikestepper
>
>
> Am 12.04.2013 15:34, schrieb Alan McMorran:
>> We have been using CDO with an H2 database back-end in our software for
>> the last year and so far it has proved itself invaluable. One problem
>> we now have is that with clients utilising the database persistence we
>> cannot make alterations to the underlying Ecore models without breaking
>> the database.
>>
>> The situation we are in is that the application's data is defined by an
>> Ecore model and a CDO-genmodel. We will sometimes get a request from a
>> client to add extensions to our standard product schema, however this
>> means that if they install an updated version of the plug-in containing
>> the Ecore the database schema that had been generated no longer
>> matches. I realise this is to be expected as it will be looking for
>> columns that are not present. I am wondering is if there is a facility
>> to export the entire CDO database (including all revision/branch data -
>> I can easily extract the current data but I would like to preserve all
>> the past commits); upgrade the plug-in to the new version of the
>> schema; then repopulate the database.
>>
>> I can enforce some restrictions on what will change:
>> 1. There will only ever be additions to the model, no attributes or
>> references will be removed
>> 2. We can restrict additions to being through multiple-inheritance
>> (e.g. if we have class A and want to add attribute X we would create a
>> new class B with the attribtue and have A inherit from B)
>> 3. The generated model code will be from a CDO model (i.e. no legacy support).
>>
>> However the URI of the base schema cannot change as we want this model
>> to be a replacement for the base model, not for them to co-exist.
>>
>> Obviously I am happy to have it run a time-consuming one-time upgrade
>> process post-update to support this (assuming it's a reasonable time).
>> I was really wondering if this was something that CDO already supported
>> (and I'd missed it) or if anyone has done anything similar.
>>
>> My initial thoughts are that I could export the base data from the
>> first commit for each resource; extract all the change models up to the
>> current version; upgrade the plug-in; and then re-import and recreate
>> the changes from the base+change data. I suspect this may be
>> time-consuming for large data sets and adding support for branches
>> would add additional complexity but (in my head at least) I think it
>> would work.
>>
>> I am no CDO expert though so any suggestions would be more than welcome!
>>
>> Alan
Re: [CDO] Migrate Repository when Ecore changes [message #1042461 is a reply to message #1041692] Tue, 16 April 2013 13:10 Go to previous messageGo to next message
Christophe Bouhier is currently offline Christophe BouhierFriend
Messages: 937
Registered: July 2009
Senior Member
On 15-04-13 14:22, Eike Stepper wrote:
> Hi Alan,
>
> I'll soon start on an estimate for proper model evolution support. Then
> I'd like to see if crowd funding is an option for a bigger feature like
> this.
That would be really cool.
>
> In the mean time you could use the CDOServerExporter, transform the
> resulting XML file to the new model and then use CDOServerImporter.
>
I have been thinking about how this can be done. Editing the XML is
going to be tough. I foresee the first thing one might want to do is to
update the packageUnit tags, to reflect the changes in the EPackages. I
wonder, if there is a schema for the export format, which could be used
to produce an EMF editor, making editing easy?

As I can't wait for the official feature, we are also exploring another
option. This consists of producing a DB DDL delta, and then applying it
with SQL commands on the DB. I can report later on, how successful this
was.

> Cheers
> /Eike
>
> ----
> http://www.esc-net.de
> http://thegordian.blogspot.com
> http://twitter.com/eikestepper
>
>
> Am 12.04.2013 15:34, schrieb Alan McMorran:
>> We have been using CDO with an H2 database back-end in our software
>> for the last year and so far it has proved itself invaluable. One
>> problem we now have is that with clients utilising the database
>> persistence we cannot make alterations to the underlying Ecore models
>> without breaking the database.
>>
>> The situation we are in is that the application's data is defined by
>> an Ecore model and a CDO-genmodel. We will sometimes get a request
>> from a client to add extensions to our standard product schema,
>> however this means that if they install an updated version of the
>> plug-in containing the Ecore the database schema that had been
>> generated no longer matches. I realise this is to be expected as it
>> will be looking for columns that are not present. I am wondering is if
>> there is a facility to export the entire CDO database (including all
>> revision/branch data - I can easily extract the current data but I
>> would like to preserve all the past commits); upgrade the plug-in to
>> the new version of the schema; then repopulate the database.
>>
>> I can enforce some restrictions on what will change:
>> 1. There will only ever be additions to the model, no attributes or
>> references will be removed
>> 2. We can restrict additions to being through multiple-inheritance
>> (e.g. if we have class A and want to add attribute X we would create a
>> new class B with the attribtue and have A inherit from B)
>> 3. The generated model code will be from a CDO model (i.e. no legacy
>> support).
>>
>> However the URI of the base schema cannot change as we want this model
>> to be a replacement for the base model, not for them to co-exist.
>>
>> Obviously I am happy to have it run a time-consuming one-time upgrade
>> process post-update to support this (assuming it's a reasonable time).
>> I was really wondering if this was something that CDO already
>> supported (and I'd missed it) or if anyone has done anything similar.
>>
>> My initial thoughts are that I could export the base data from the
>> first commit for each resource; extract all the change models up to
>> the current version; upgrade the plug-in; and then re-import and
>> recreate the changes from the base+change data. I suspect this may be
>> time-consuming for large data sets and adding support for branches
>> would add additional complexity but (in my head at least) I think it
>> would work.
>>
>> I am no CDO expert though so any suggestions would be more than welcome!
>>
>> Alan
>>
>
Re: [CDO] Migrate Repository when Ecore changes [message #1042505 is a reply to message #1042461] Tue, 16 April 2013 14:08 Go to previous messageGo to next message
Erdal Karaca is currently offline Erdal KaracaFriend
Messages: 854
Registered: July 2009
Senior Member
Chrisophe, there is a quite interesting framework for migrating sql schemas:

http://liquibase.org/

Unfortunately, the pain point about this solution is how to update the package units.

Christophe Bouhier wrote on Tue, 16 April 2013 15:10
On 15-04-13 14:22, Eike Stepper wrote:
> Hi Alan,
>
> I'll soon start on an estimate for proper model evolution support. Then
> I'd like to see if crowd funding is an option for a bigger feature like
> this.
That would be really cool.
>
> In the mean time you could use the CDOServerExporter, transform the
> resulting XML file to the new model and then use CDOServerImporter.
>
I have been thinking about how this can be done. Editing the XML is
going to be tough. I foresee the first thing one might want to do is to
update the packageUnit tags, to reflect the changes in the EPackages. I
wonder, if there is a schema for the export format, which could be used
to produce an EMF editor, making editing easy?

As I can't wait for the official feature, we are also exploring another
option. This consists of producing a DB DDL delta, and then applying it
with SQL commands on the DB. I can report later on, how successful this
was.

> Cheers
> /Eike
>
> ----
> http://www.esc-net.de
> http://thegordian.blogspot.com
> http://twitter.com/eikestepper
>
>
> Am 12.04.2013 15:34, schrieb Alan McMorran:
>> We have been using CDO with an H2 database back-end in our software
>> for the last year and so far it has proved itself invaluable. One
>> problem we now have is that with clients utilising the database
>> persistence we cannot make alterations to the underlying Ecore models
>> without breaking the database.
>>
>> The situation we are in is that the application's data is defined by
>> an Ecore model and a CDO-genmodel. We will sometimes get a request
>> from a client to add extensions to our standard product schema,
>> however this means that if they install an updated version of the
>> plug-in containing the Ecore the database schema that had been
>> generated no longer matches. I realise this is to be expected as it
>> will be looking for columns that are not present. I am wondering is if
>> there is a facility to export the entire CDO database (including all
>> revision/branch data - I can easily extract the current data but I
>> would like to preserve all the past commits); upgrade the plug-in to
>> the new version of the schema; then repopulate the database.
>>
>> I can enforce some restrictions on what will change:
>> 1. There will only ever be additions to the model, no attributes or
>> references will be removed
>> 2. We can restrict additions to being through multiple-inheritance
>> (e.g. if we have class A and want to add attribute X we would create a
>> new class B with the attribtue and have A inherit from B)
>> 3. The generated model code will be from a CDO model (i.e. no legacy
>> support).
>>
>> However the URI of the base schema cannot change as we want this model
>> to be a replacement for the base model, not for them to co-exist.
>>
>> Obviously I am happy to have it run a time-consuming one-time upgrade
>> process post-update to support this (assuming it's a reasonable time).
>> I was really wondering if this was something that CDO already
>> supported (and I'd missed it) or if anyone has done anything similar.
>>
>> My initial thoughts are that I could export the base data from the
>> first commit for each resource; extract all the change models up to
>> the current version; upgrade the plug-in; and then re-import and
>> recreate the changes from the base+change data. I suspect this may be
>> time-consuming for large data sets and adding support for branches
>> would add additional complexity but (in my head at least) I think it
>> would work.
>>
>> I am no CDO expert though so any suggestions would be more than welcome!
>>
>> Alan
>>
>

Re: [CDO] Migrate Repository when Ecore changes [message #1042580 is a reply to message #1042461] Tue, 16 April 2013 15:52 Go to previous messageGo to next message
Eike Stepper is currently offline Eike StepperFriend
Messages: 6681
Registered: July 2009
Senior Member
Am 16.04.2013 15:10, schrieb Christophe Bouhier:
> On 15-04-13 14:22, Eike Stepper wrote:
>> Hi Alan,
>>
>> I'll soon start on an estimate for proper model evolution support. Then
>> I'd like to see if crowd funding is an option for a bigger feature like
>> this.
> That would be really cool.
>>
>> In the mean time you could use the CDOServerExporter, transform the
>> resulting XML file to the new model and then use CDOServerImporter.
>>
> I have been thinking about how this can be done. Editing the XML is going to be tough. I foresee the first thing one
> might want to do is to update the packageUnit tags, to reflect the changes in the EPackages. I wonder, if there is a
> schema for the export format, which could be used to produce an EMF editor, making editing easy?
No, there's none available because in general these files are just too big to edit them with EMF. Probably you could
derive a schema because the format as such is simple and made to be scalable and readable.

Cheers
/Eike

----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper


>
> As I can't wait for the official feature, we are also exploring another option. This consists of producing a DB DDL
> delta, and then applying it
> with SQL commands on the DB. I can report later on, how successful this was.
>
>> Cheers
>> /Eike
>>
>> ----
>> http://www.esc-net.de
>> http://thegordian.blogspot.com
>> http://twitter.com/eikestepper
>>
>>
>> Am 12.04.2013 15:34, schrieb Alan McMorran:
>>> We have been using CDO with an H2 database back-end in our software
>>> for the last year and so far it has proved itself invaluable. One
>>> problem we now have is that with clients utilising the database
>>> persistence we cannot make alterations to the underlying Ecore models
>>> without breaking the database.
>>>
>>> The situation we are in is that the application's data is defined by
>>> an Ecore model and a CDO-genmodel. We will sometimes get a request
>>> from a client to add extensions to our standard product schema,
>>> however this means that if they install an updated version of the
>>> plug-in containing the Ecore the database schema that had been
>>> generated no longer matches. I realise this is to be expected as it
>>> will be looking for columns that are not present. I am wondering is if
>>> there is a facility to export the entire CDO database (including all
>>> revision/branch data - I can easily extract the current data but I
>>> would like to preserve all the past commits); upgrade the plug-in to
>>> the new version of the schema; then repopulate the database.
>>>
>>> I can enforce some restrictions on what will change:
>>> 1. There will only ever be additions to the model, no attributes or
>>> references will be removed
>>> 2. We can restrict additions to being through multiple-inheritance
>>> (e.g. if we have class A and want to add attribute X we would create a
>>> new class B with the attribtue and have A inherit from B)
>>> 3. The generated model code will be from a CDO model (i.e. no legacy
>>> support).
>>>
>>> However the URI of the base schema cannot change as we want this model
>>> to be a replacement for the base model, not for them to co-exist.
>>>
>>> Obviously I am happy to have it run a time-consuming one-time upgrade
>>> process post-update to support this (assuming it's a reasonable time).
>>> I was really wondering if this was something that CDO already
>>> supported (and I'd missed it) or if anyone has done anything similar.
>>>
>>> My initial thoughts are that I could export the base data from the
>>> first commit for each resource; extract all the change models up to
>>> the current version; upgrade the plug-in; and then re-import and
>>> recreate the changes from the base+change data. I suspect this may be
>>> time-consuming for large data sets and adding support for branches
>>> would add additional complexity but (in my head at least) I think it
>>> would work.
>>>
>>> I am no CDO expert though so any suggestions would be more than welcome!
>>>
>>> Alan
>>>
>>
>


Re: [CDO] Migrate Repository when Ecore changes [message #1042613 is a reply to message #1042580] Tue, 16 April 2013 16:53 Go to previous messageGo to next message
Erdal Karaca is currently offline Erdal KaracaFriend
Messages: 854
Registered: July 2009
Senior Member
If you really want to modify the exported repository data, then I would suggest to use an xml/xquery database (for example, http://basex.org).


Eike Stepper wrote on Tue, 16 April 2013 17:52
Am 16.04.2013 15:10, schrieb Christophe Bouhier:
> On 15-04-13 14:22, Eike Stepper wrote:
>> Hi Alan,
>>
>> I'll soon start on an estimate for proper model evolution support. Then
>> I'd like to see if crowd funding is an option for a bigger feature like
>> this.
> That would be really cool.
>>
>> In the mean time you could use the CDOServerExporter, transform the
>> resulting XML file to the new model and then use CDOServerImporter.
>>
> I have been thinking about how this can be done. Editing the XML is going to be tough. I foresee the first thing one
> might want to do is to update the packageUnit tags, to reflect the changes in the EPackages. I wonder, if there is a
> schema for the export format, which could be used to produce an EMF editor, making editing easy?
No, there's none available because in general these files are just too big to edit them with EMF. Probably you could
derive a schema because the format as such is simple and made to be scalable and readable.

Cheers
/Eike

----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper

Re: [CDO] Migrate Repository when Ecore changes [message #1043050 is a reply to message #1042505] Wed, 17 April 2013 07:52 Go to previous messageGo to next message
Christophe Bouhier is currently offline Christophe BouhierFriend
Messages: 937
Registered: July 2009
Senior Member
On 16-04-13 16:08, Erdal Karaca wrote:
> Chrisophe, there is a quite interesting framework for migrating sql
> schemas:
>
> http://liquibase.org/
>
Thanks Erdal! Do you have experience change a CDO schema with this tool?

> Unfortunately, the pain point about this solution is how to update the
> package units.
>
> Christophe Bouhier wrote on Tue, 16 April 2013 15:10
>> On 15-04-13 14:22, Eike Stepper wrote:
>> > Hi Alan,
>> >
>> > I'll soon start on an estimate for proper model evolution support. Then
>> > I'd like to see if crowd funding is an option for a bigger feature like
>> > this.
>> That would be really cool.
>> >
>> > In the mean time you could use the CDOServerExporter, transform the
>> > resulting XML file to the new model and then use CDOServerImporter.
>> >
>> I have been thinking about how this can be done. Editing the XML is
>> going to be tough. I foresee the first thing one might want to do is
>> to update the packageUnit tags, to reflect the changes in the
>> EPackages. I wonder, if there is a schema for the export format, which
>> could be used to produce an EMF editor, making editing easy?
>>
>> As I can't wait for the official feature, we are also exploring
>> another option. This consists of producing a DB DDL delta, and then
>> applying it
>> with SQL commands on the DB. I can report later on, how successful
>> this was.
>>
>> > Cheers
>> > /Eike
>> >
>> > ----
>> > http://www.esc-net.de
>> > http://thegordian.blogspot.com
>> > http://twitter.com/eikestepper
>> >
>> >
>> > Am 12.04.2013 15:34, schrieb Alan McMorran:
>> >> We have been using CDO with an H2 database back-end in our software
>> >> for the last year and so far it has proved itself invaluable. One
>> >> problem we now have is that with clients utilising the database
>> >> persistence we cannot make alterations to the underlying Ecore models
>> >> without breaking the database.
>> >>
>> >> The situation we are in is that the application's data is defined by
>> >> an Ecore model and a CDO-genmodel. We will sometimes get a request
>> >> from a client to add extensions to our standard product schema,
>> >> however this means that if they install an updated version of the
>> >> plug-in containing the Ecore the database schema that had been
>> >> generated no longer matches. I realise this is to be expected as it
>> >> will be looking for columns that are not present. I am wondering is if
>> >> there is a facility to export the entire CDO database (including all
>> >> revision/branch data - I can easily extract the current data but I
>> >> would like to preserve all the past commits); upgrade the plug-in to
>> >> the new version of the schema; then repopulate the database.
>> >>
>> >> I can enforce some restrictions on what will change:
>> >> 1. There will only ever be additions to the model, no attributes or
>> >> references will be removed
>> >> 2. We can restrict additions to being through multiple-inheritance
>> >> (e.g. if we have class A and want to add attribute X we would create a
>> >> new class B with the attribtue and have A inherit from B)
>> >> 3. The generated model code will be from a CDO model (i.e. no legacy
>> >> support).
>> >>
>> >> However the URI of the base schema cannot change as we want this model
>> >> to be a replacement for the base model, not for them to co-exist.
>> >>
>> >> Obviously I am happy to have it run a time-consuming one-time upgrade
>> >> process post-update to support this (assuming it's a reasonable time).
>> >> I was really wondering if this was something that CDO already
>> >> supported (and I'd missed it) or if anyone has done anything similar.
>> >>
>> >> My initial thoughts are that I could export the base data from the
>> >> first commit for each resource; extract all the change models up to
>> >> the current version; upgrade the plug-in; and then re-import and
>> >> recreate the changes from the base+change data. I suspect this may be
>> >> time-consuming for large data sets and adding support for branches
>> >> would add additional complexity but (in my head at least) I think it
>> >> would work.
>> >>
>> >> I am no CDO expert though so any suggestions would be more than
>> welcome!
>> >>
>> >> Alan
>> >>
>> >
>
>
Re: [CDO] Migrate Repository when Ecore changes [message #1044793 is a reply to message #1043050] Fri, 19 April 2013 10:56 Go to previous messageGo to next message
Erdal Karaca is currently offline Erdal KaracaFriend
Messages: 854
Registered: July 2009
Senior Member
Hi Christophe (sorry for the type in the last post Smile

Yes, we have tried this approach in a project:

Pros:
+ Backend neutral
+ simple workflow
export repo, load it into xml db, "manipulate" data using xquery, save data to new xml file, import xml data

Cons:
- May take long for bigger model data
- you hav to learn xquery
- you have to be able to "transform" your model changes to xquery instructions, though, that may be doable using EMF Compare in a some way


Christophe Bouhier wrote on Wed, 17 April 2013 09:52
On 16-04-13 16:08, Erdal Karaca wrote:
> Chrisophe, there is a quite interesting framework for migrating sql
> schemas:
>
> http://liquibase.org/
>
Thanks Erdal! Do you have experience change a CDO schema with this tool?

> Unfortunately, the pain point about this solution is how to update the
> package units.
>
> Christophe Bouhier wrote on Tue, 16 April 2013 15:10
>> On 15-04-13 14:22, Eike Stepper wrote:
>> > Hi Alan,
>> >
>> > I'll soon start on an estimate for proper model evolution support. Then
>> > I'd like to see if crowd funding is an option for a bigger feature like
>> > this.
>> That would be really cool.
>> >
>> > In the mean time you could use the CDOServerExporter, transform the
>> > resulting XML file to the new model and then use CDOServerImporter.
>> >
>> I have been thinking about how this can be done. Editing the XML is
>> going to be tough. I foresee the first thing one might want to do is
>> to update the packageUnit tags, to reflect the changes in the
>> EPackages. I wonder, if there is a schema for the export format, which
>> could be used to produce an EMF editor, making editing easy?
>>
>> As I can't wait for the official feature, we are also exploring
>> another option. This consists of producing a DB DDL delta, and then
>> applying it
>> with SQL commands on the DB. I can report later on, how successful
>> this was.
>>
>> > Cheers
>> > /Eike
>> >
>> > ----
>> > http://www.esc-net.de
>> > http://thegordian.blogspot.com
>> > http://twitter.com/eikestepper
>> >
>> >
>> > Am 12.04.2013 15:34, schrieb Alan McMorran:
>> >> We have been using CDO with an H2 database back-end in our software
>> >> for the last year and so far it has proved itself invaluable. One
>> >> problem we now have is that with clients utilising the database
>> >> persistence we cannot make alterations to the underlying Ecore models
>> >> without breaking the database.
>> >>
>> >> The situation we are in is that the application's data is defined by
>> >> an Ecore model and a CDO-genmodel. We will sometimes get a request
>> >> from a client to add extensions to our standard product schema,
>> >> however this means that if they install an updated version of the
>> >> plug-in containing the Ecore the database schema that had been
>> >> generated no longer matches. I realise this is to be expected as it
>> >> will be looking for columns that are not present. I am wondering is if
>> >> there is a facility to export the entire CDO database (including all
>> >> revision/branch data - I can easily extract the current data but I
>> >> would like to preserve all the past commits); upgrade the plug-in to
>> >> the new version of the schema; then repopulate the database.
>> >>
>> >> I can enforce some restrictions on what will change:
>> >> 1. There will only ever be additions to the model, no attributes or
>> >> references will be removed
>> >> 2. We can restrict additions to being through multiple-inheritance
>> >> (e.g. if we have class A and want to add attribute X we would create a
>> >> new class B with the attribtue and have A inherit from B)
>> >> 3. The generated model code will be from a CDO model (i.e. no legacy
>> >> support).
>> >>
>> >> However the URI of the base schema cannot change as we want this model
>> >> to be a replacement for the base model, not for them to co-exist.
>> >>
>> >> Obviously I am happy to have it run a time-consuming one-time upgrade
>> >> process post-update to support this (assuming it's a reasonable time).
>> >> I was really wondering if this was something that CDO already
>> >> supported (and I'd missed it) or if anyone has done anything similar.
>> >>
>> >> My initial thoughts are that I could export the base data from the
>> >> first commit for each resource; extract all the change models up to
>> >> the current version; upgrade the plug-in; and then re-import and
>> >> recreate the changes from the base+change data. I suspect this may be
>> >> time-consuming for large data sets and adding support for branches
>> >> would add additional complexity but (in my head at least) I think it
>> >> would work.
>> >>
>> >> I am no CDO expert though so any suggestions would be more than
>> welcome!
>> >>
>> >> Alan
>> >>
>> >
>
>

Re: [CDO] Migrate Repository when Ecore changes [message #1044798 is a reply to message #1044793] Fri, 19 April 2013 11:03 Go to previous message
Erdal Karaca is currently offline Erdal KaracaFriend
Messages: 854
Registered: July 2009
Senior Member
Sorry, Christophe, your question was targeted at liquibase.
Yes, that is the alternative approach that we are using in a project but not yet matured.
It is better suited to the needs of that project.

Pros:
+ in-db model data transformations using SQL DDL/DML
+ fast excecution as all operations run in-db
+ maintains the history of all changes (creates a separate/specific history table)

Cons:
- would only work with the DBStore and/or Hibernate Store (or any other SQL backend)


Erdal Karaca wrote on Fri, 19 April 2013 12:56
Hi Christophe (sorry for the type in the last post Smile

Yes, we have tried this approach in a project:

Pros:
+ Backend neutral
+ simple workflow
export repo, load it into xml db, "manipulate" data using xquery, save data to new xml file, import xml data

Cons:
- May take long for bigger model data
- you hav to learn xquery
- you have to be able to "transform" your model changes to xquery instructions, though, that may be doable using EMF Compare in a some way


Christophe Bouhier wrote on Wed, 17 April 2013 09:52
On 16-04-13 16:08, Erdal Karaca wrote:
> Chrisophe, there is a quite interesting framework for migrating sql
> schemas:
>
> http://liquibase.org/
>
Thanks Erdal! Do you have experience change a CDO schema with this tool?

> Unfortunately, the pain point about this solution is how to update the
> package units.
>
> Christophe Bouhier wrote on Tue, 16 April 2013 15:10
>> On 15-04-13 14:22, Eike Stepper wrote:
>> > Hi Alan,
>> >
>> > I'll soon start on an estimate for proper model evolution support. Then
>> > I'd like to see if crowd funding is an option for a bigger feature like
>> > this.
>> That would be really cool.
>> >
>> > In the mean time you could use the CDOServerExporter, transform the
>> > resulting XML file to the new model and then use CDOServerImporter.
>> >
>> I have been thinking about how this can be done. Editing the XML is
>> going to be tough. I foresee the first thing one might want to do is
>> to update the packageUnit tags, to reflect the changes in the
>> EPackages. I wonder, if there is a schema for the export format, which
>> could be used to produce an EMF editor, making editing easy?
>>
>> As I can't wait for the official feature, we are also exploring
>> another option. This consists of producing a DB DDL delta, and then
>> applying it
>> with SQL commands on the DB. I can report later on, how successful
>> this was.
>>
>> > Cheers
>> > /Eike
>> >
>> > ----
>> > http://www.esc-net.de
>> > http://thegordian.blogspot.com
>> > http://twitter.com/eikestepper
>> >
>> >
>> > Am 12.04.2013 15:34, schrieb Alan McMorran:
>> >> We have been using CDO with an H2 database back-end in our software
>> >> for the last year and so far it has proved itself invaluable. One
>> >> problem we now have is that with clients utilising the database
>> >> persistence we cannot make alterations to the underlying Ecore models
>> >> without breaking the database.
>> >>
>> >> The situation we are in is that the application's data is defined by
>> >> an Ecore model and a CDO-genmodel. We will sometimes get a request
>> >> from a client to add extensions to our standard product schema,
>> >> however this means that if they install an updated version of the
>> >> plug-in containing the Ecore the database schema that had been
>> >> generated no longer matches. I realise this is to be expected as it
>> >> will be looking for columns that are not present. I am wondering is if
>> >> there is a facility to export the entire CDO database (including all
>> >> revision/branch data - I can easily extract the current data but I
>> >> would like to preserve all the past commits); upgrade the plug-in to
>> >> the new version of the schema; then repopulate the database.
>> >>
>> >> I can enforce some restrictions on what will change:
>> >> 1. There will only ever be additions to the model, no attributes or
>> >> references will be removed
>> >> 2. We can restrict additions to being through multiple-inheritance
>> >> (e.g. if we have class A and want to add attribute X we would create a
>> >> new class B with the attribtue and have A inherit from B)
>> >> 3. The generated model code will be from a CDO model (i.e. no legacy
>> >> support).
>> >>
>> >> However the URI of the base schema cannot change as we want this model
>> >> to be a replacement for the base model, not for them to co-exist.
>> >>
>> >> Obviously I am happy to have it run a time-consuming one-time upgrade
>> >> process post-update to support this (assuming it's a reasonable time).
>> >> I was really wondering if this was something that CDO already
>> >> supported (and I'd missed it) or if anyone has done anything similar.
>> >>
>> >> My initial thoughts are that I could export the base data from the
>> >> first commit for each resource; extract all the change models up to
>> >> the current version; upgrade the plug-in; and then re-import and
>> >> recreate the changes from the base+change data. I suspect this may be
>> >> time-consuming for large data sets and adding support for branches
>> >> would add additional complexity but (in my head at least) I think it
>> >> would work.
>> >>
>> >> I am no CDO expert though so any suggestions would be more than
>> welcome!
>> >>
>> >> Alan
>> >>
>> >
>
>


Previous Topic:I need "static" model fields, what is the best way?
Next Topic:Missing XML namespace declaration
Goto Forum:
  


Current Time: Tue Mar 19 05:58:44 GMT 2024

Powered by FUDForum. Page generated in 0.03253 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top