Home » Modeling » EMF » migrate data with Teneo
| migrate data with Teneo [message #424915] |
Mon, 10 November 2008 13:59  |
 Jesper Wermuth
Messages: 6
Registered: July 2009 |
Junior Member |
|
|
Hi.
I have been using Teneo (which is super cool btw) for a while and
currently have several live databases on which I change the schema's
every now and then
Typical updates are
CRUD attribute
* change attribute names
* change attribute types
* remove, delete etc.
* add default data in new attributes
CRUD element
* same as for attribute
Making these updates are both difficult and dangerous, mostly because I
risk loosing data in the process.
Now is the time (for me) to automate this process. I have been
considering 3 possible solutions
1. use xmi to export and then import data
2. wait for emf compare
3. write my own (model based) datamigration language that allows
migration from higher versions to lower and visa versa. I am inspired by
Ruby on Rails migration solution here.
Does anyone know if there are solutions to my problem available ?
Anyways - I like solutions (C) better because its flexible and gives me
control. Heres the outline of the design:
(A) diff(model a v1, model a v2) -> diff model
----------------------------------------------
the diff model will be an EMF model rich enough to express the above
CRUD use cases.
(B) generate_solution_code(model a v1, model a v2, diff model) -> java
code, sql, hql, whatever is needed
----------------------------------------------
A solution could be a bunch of SQL files saying :
update element set e_container_featureid=e_container_featureid-1 where
econtainer_class in(....)
the above sql would be (some of ??) the code needed to add a new
attribute to an already existing table
I might need the original model files in this step to be able to extract
information such as attribute ordering in the xxxPackage java file.
(C) apply solution(credentials + output from B) -> migrated database
----------------------------------------------
log into db - execute sql
execute possibly generated java code
....
Does anyone have comments, help, links etc
Cheers, Jesper
|
|
|
| Re: migrate data with Teneo [message #424928 is a reply to message #424915] |
Mon, 10 November 2008 23:06  |
Martin Taal
Messages: 5468
Registered: July 2009 |
Senior Member |
|
|
Hi Jesper,
I am afraid that I only have experience with models that don't evolve that much or for which only
new elements/attributes/tables are added. This is not that difficult to do (hibernate does it mainly).
I also have done some cases whereby I use a xmi/xml export/import to update the database schema.
So not limited by too much experience/knowledge my preference would also go to a generative approach
using a diff model.
gr. Martin
Jesper Wermuth wrote:
> Hi.
> I have been using Teneo (which is super cool btw) for a while and
> currently have several live databases on which I change the schema's
> every now and then
>
> Typical updates are
> CRUD attribute
> * change attribute names
> * change attribute types
> * remove, delete etc.
> * add default data in new attributes
>
> CRUD element
> * same as for attribute
>
> Making these updates are both difficult and dangerous, mostly because I
> risk loosing data in the process.
>
> Now is the time (for me) to automate this process. I have been
> considering 3 possible solutions
> 1. use xmi to export and then import data
> 2. wait for emf compare
> 3. write my own (model based) datamigration language that allows
> migration from higher versions to lower and visa versa. I am inspired by
> Ruby on Rails migration solution here.
>
> Does anyone know if there are solutions to my problem available ?
>
> Anyways - I like solutions (C) better because its flexible and gives me
> control. Heres the outline of the design:
>
> (A) diff(model a v1, model a v2) -> diff model
> ----------------------------------------------
> the diff model will be an EMF model rich enough to express the above
> CRUD use cases.
>
> (B) generate_solution_code(model a v1, model a v2, diff model) -> java
> code, sql, hql, whatever is needed
> ----------------------------------------------
> A solution could be a bunch of SQL files saying :
> update element set e_container_featureid=e_container_featureid-1 where
> econtainer_class in(....)
> the above sql would be (some of ??) the code needed to add a new
> attribute to an already existing table
>
> I might need the original model files in this step to be able to extract
> information such as attribute ordering in the xxxPackage java file.
>
>
> (C) apply solution(credentials + output from B) -> migrated database
> ----------------------------------------------
> log into db - execute sql
> execute possibly generated java code
> ....
>
> Does anyone have comments, help, links etc
>
> Cheers, Jesper
--
With Regards, Martin Taal
Springsite/Elver.org
Office: Hardwareweg 4, 3821 BV Amersfoort
Postal: Nassaulaan 7, 3941 EC Doorn
The Netherlands
Cell: +31 (0)6 288 48 943
Tel: +31 (0)84 420 2397
Fax: +31 (0)84 225 9307
Mail: mtaal@springsite.com - mtaal@elver.org
Web: www.springsite.com - www.elver.org
|
|
|
| Re: migrate data with Teneo [message #424934 is a reply to message #424928] |
Tue, 11 November 2008 08:31 |
Jesper Wermuth
Messages: 6
Registered: July 2009 |
Junior Member |
|
|
Martin Taal skrev:
> Hi Jesper,
> I am afraid that I only have experience with models that don't evolve
> that much or for which only new elements/attributes/tables are added.
> This is not that difficult to do (hibernate does it mainly).
Thanks Martin. OK. I'll continue on the model approach.
A possible misunderstanding on my behalf caught me in the above lines.
Let me explain:
One of the data migration issues I encounter is when I add a new
attribute to an (abstract) element being referenced to by other elements.
element A <- element B
attr a |--(e_container, featureid)
ref b <----|
In this case I have to move the featureid "ptr" in the referencing
element, or all elements that derive from that element in case of an
abstract element.
Example:
element A <- element B
attr a |--(e_container, featureid-1)
attr c |
ref b <----|
You mention above that hibernate mainly does this. It does not for me.
Am I missing something here ?
Cheers again - Jesper
> I also have done some cases whereby I use a xmi/xml export/import to
> update the database schema.
>
> So not limited by too much experience/knowledge my preference would also
> go to a generative approach using a diff model.
>
> gr. Martin
>
> Jesper Wermuth wrote:
>> Hi.
>> I have been using Teneo (which is super cool btw) for a while and
>> currently have several live databases on which I change the schema's
>> every now and then
>>
>> Typical updates are
>> CRUD attribute
>> * change attribute names
>> * change attribute types
>> * remove, delete etc.
>> * add default data in new attributes
>>
>> CRUD element
>> * same as for attribute
>>
>> Making these updates are both difficult and dangerous, mostly because
>> I risk loosing data in the process.
>>
>> Now is the time (for me) to automate this process. I have been
>> considering 3 possible solutions
>> 1. use xmi to export and then import data
>> 2. wait for emf compare
>> 3. write my own (model based) datamigration language that allows
>> migration from higher versions to lower and visa versa. I am inspired
>> by Ruby on Rails migration solution here.
>>
>> Does anyone know if there are solutions to my problem available ?
>>
>> Anyways - I like solutions (C) better because its flexible and gives
>> me control. Heres the outline of the design:
>>
>> (A) diff(model a v1, model a v2) -> diff model
>> ----------------------------------------------
>> the diff model will be an EMF model rich enough to express the above
>> CRUD use cases.
>>
>> (B) generate_solution_code(model a v1, model a v2, diff model) -> java
>> code, sql, hql, whatever is needed
>> ----------------------------------------------
>> A solution could be a bunch of SQL files saying :
>> update element set e_container_featureid=e_container_featureid-1 where
>> econtainer_class in(....)
>> the above sql would be (some of ??) the code needed to add a new
>> attribute to an already existing table
>>
>> I might need the original model files in this step to be able to
>> extract information such as attribute ordering in the xxxPackage java
>> file.
>>
>>
>> (C) apply solution(credentials + output from B) -> migrated database
>> ----------------------------------------------
>> log into db - execute sql
>> execute possibly generated java code
>> ....
>>
>> Does anyone have comments, help, links etc
>>
>> Cheers, Jesper
>
>
|
|
|
| Re: migrate data with Teneo [message #424935 is a reply to message #424934] |
Tue, 11 November 2008 08:48 |
Martin Taal
Messages: 5468
Registered: July 2009 |
Senior Member |
|
|
Hi Jesper,
I had not thought about this one. Hibernate does not solve this one. In my cases I don't use the
econtainer relationship so then this issue does not occur for me.
gr. Martin
Jesper Wermuth wrote:
> Martin Taal skrev:
>> Hi Jesper,
>> I am afraid that I only have experience with models that don't evolve
>> that much or for which only new elements/attributes/tables are added.
> > This is not that difficult to do (hibernate does it mainly).
>
> Thanks Martin. OK. I'll continue on the model approach.
> A possible misunderstanding on my behalf caught me in the above lines.
> Let me explain:
> One of the data migration issues I encounter is when I add a new
> attribute to an (abstract) element being referenced to by other elements.
>
> element A <- element B
> attr a |--(e_container, featureid)
> ref b <----|
>
> In this case I have to move the featureid "ptr" in the referencing
> element, or all elements that derive from that element in case of an
> abstract element.
> Example:
>
> element A <- element B
> attr a |--(e_container, featureid-1)
> attr c |
> ref b <----|
>
> You mention above that hibernate mainly does this. It does not for me.
> Am I missing something here ?
>
> Cheers again - Jesper
>
>
>> I also have done some cases whereby I use a xmi/xml export/import to
>> update the database schema.
>>
>> So not limited by too much experience/knowledge my preference would
>> also go to a generative approach using a diff model.
>>
>> gr. Martin
>>
>> Jesper Wermuth wrote:
>>> Hi.
>>> I have been using Teneo (which is super cool btw) for a while and
>>> currently have several live databases on which I change the schema's
>>> every now and then
>>>
>>> Typical updates are
>>> CRUD attribute
>>> * change attribute names
>>> * change attribute types
>>> * remove, delete etc.
>>> * add default data in new attributes
>>>
>>> CRUD element
>>> * same as for attribute
>>>
>>> Making these updates are both difficult and dangerous, mostly because
>>> I risk loosing data in the process.
>>>
>>> Now is the time (for me) to automate this process. I have been
>>> considering 3 possible solutions
>>> 1. use xmi to export and then import data
>>> 2. wait for emf compare
>>> 3. write my own (model based) datamigration language that allows
>>> migration from higher versions to lower and visa versa. I am inspired
>>> by Ruby on Rails migration solution here.
>>>
>>> Does anyone know if there are solutions to my problem available ?
>>>
>>> Anyways - I like solutions (C) better because its flexible and gives
>>> me control. Heres the outline of the design:
>>>
>>> (A) diff(model a v1, model a v2) -> diff model
>>> ----------------------------------------------
>>> the diff model will be an EMF model rich enough to express the above
>>> CRUD use cases.
>>>
>>> (B) generate_solution_code(model a v1, model a v2, diff model) -> java
>>> code, sql, hql, whatever is needed
>>> ----------------------------------------------
>>> A solution could be a bunch of SQL files saying :
>>> update element set e_container_featureid=e_container_featureid-1
>>> where econtainer_class in(....)
>>> the above sql would be (some of ??) the code needed to add a new
>>> attribute to an already existing table
>>>
>>> I might need the original model files in this step to be able to
>>> extract information such as attribute ordering in the xxxPackage java
>>> file.
>>>
>>>
>>> (C) apply solution(credentials + output from B) -> migrated database
>>> ----------------------------------------------
>>> log into db - execute sql
>>> execute possibly generated java code
>>> ....
>>>
>>> Does anyone have comments, help, links etc
>>>
>>> Cheers, Jesper
>>
>>
--
With Regards, Martin Taal
Springsite/Elver.org
Office: Hardwareweg 4, 3821 BV Amersfoort
Postal: Nassaulaan 7, 3941 EC Doorn
The Netherlands
Cell: +31 (0)6 288 48 943
Tel: +31 (0)84 420 2397
Fax: +31 (0)84 225 9307
Mail: mtaal@springsite.com - mtaal@elver.org
Web: www.springsite.com - www.elver.org
|
|
|
Goto Forum:
Current Time: Tue Apr 16 09:01:46 GMT 2024
Powered by FUDForum. Page generated in 0.22041 seconds |
] 
Search
Help
Register
Login
Home
