Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » EMF » [CDO] Changing datatypes between model versions
[CDO] Changing datatypes between model versions [message #1690829] Tue, 31 March 2015 14:08 Go to next message
Simon Goodall is currently offline Simon GoodallFriend
Messages: 35
Registered: July 2009
Member
Hi,

I am trying to transform my data model from one version of a metamodel to another one. In this particular case version 1 uses EDate and version 2 uses a custom datatype, but same attribute name. Both metamodels have a different namespace uri.
http://example/model/1: EClass: Example, Attribute date = EDate
http://example/model/2: EClass: Example, Attribute date = Custom EDataType


I currently load the model, manipulate it and then attempt to re-insert the updated data model back into CDO. However I get an exception along the lines of;
org.h2.jdbc.JdbcSQLException: Cannot parse "TIMESTAMP" constant "2010/01"; SQL statement:
INSERT INTO MODEL_EXAMPLE(CDO_ID, CDO_VERSION, CDO_CREATED, CDO_REVISED, CDO_RESOURCE, CDO_CONTAINER, CDO_FEATURE, DATE0, VALUE0) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?) -- (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9) [22007-168]

It appears the despite the different namespace URI, CDO is reusing the original namespace datatypes? Is there a way to resolve this? I have enabled the "qualifiedNames" property in the datastore.

// Load resource from CDO
CDOResource cdoResource = transaction.getResource("...");

// Move data into XMI based data file
{
    ResourceSet rs = new ResourceSetImpl();
    Resource tmpResource = rs.createResource(tmpURI);
    tmpResource.getContents().add(cdoResource.getContents().get(0));
    tmpResource.save(null);
}
// Make sure old reference to datamodel is gone
cdoResource.getContents().clear();

// Externally modify the XMI file
// .....

// Load the update data model instance
{
    ResourceSet rs = new ResourceSetImpl();
    Resource tmpResource = rs.getResource(tmpURI);
    // Move datamodel back into CDO
    cdoResource.getContents().add(tmpResource.getContents().get(0));
}
transaction.commit();



Thanks,
Simon
Re: [CDO] Changing datatypes between model versions [message #1690834 is a reply to message #1690829] Tue, 31 March 2015 14:22 Go to previous messageGo to next message
Eike Stepper is currently offline Eike StepperFriend
Messages: 6682
Registered: July 2009
Senior Member
Hi Simon,

It seems you accept the fact that your import does not preserve the identities of the old objects?

Then it seems to me that you're trying to use both versions of your package in the same repo, right? Different nsURIs
are one precondition for that, but different model names are required, too, because otherwise the "qualifiedNames"
property can't do its job correctly.

Cheers
/Eike

----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper



Am 31.03.2015 um 16:08 schrieb Simon Goodall:
> Hi,
>
> I am trying to transform my data model from one version of a metamodel to another one. In this particular case version
> 1 uses EDate and version 2 uses a custom datatype, but same attribute name. Both metamodels have a different namespace
> uri.
> http://example/model/1: EClass: Example, Attribute date = EDate
> http://example/model/2: EClass: Example, Attribute date = Custom EDataType
>
>
> I currently load the model, manipulate it and then attempt to re-insert the updated data model back into CDO. However
> I get an exception along the lines of;
> org.h2.jdbc.JdbcSQLException: Cannot parse "TIMESTAMP" constant "2010/01"; SQL statement:
> INSERT INTO MODEL_EXAMPLE(CDO_ID, CDO_VERSION, CDO_CREATED, CDO_REVISED, CDO_RESOURCE, CDO_CONTAINER, CDO_FEATURE,
> DATE0, VALUE0) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?) -- (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9) [22007-168]
>
> It appears the despite the different namespace URI, CDO is reusing the original namespace datatypes? Is there a way to
> resolve this? I have enabled the "qualifiedNames" property in the datastore.
>
>
> // Load resource from CDO
> CDOResource cdoResource = transaction.getResource("...");
>
> // Move data into XMI based data file
> {
> ResourceSet rs = new ResourceSetImpl();
> Resource tmpResource = rs.createResource(tmpURI);
> tmpResource.getContents().add(cdoResource.getContents().get(0));
> tmpResource.save(null);
> }
> // Make sure old reference to datamodel is gone
> cdoResource.getContents().clear();
>
> // Externally modify the XMI file
> // .....
>
> // Load the update data model instance
> {
> ResourceSet rs = new ResourceSetImpl();
> Resource tmpResource = rs.getResource(tmpURI);
> // Move datamodel back into CDO
> cdoResource.getContents().add(tmpResource.getContents().get(0));
> }
> transaction.commit();
>
>
>
> Thanks,
> Simon
>


Re: [CDO] Changing datatypes between model versions [message #1690845 is a reply to message #1690834] Tue, 31 March 2015 14:56 Go to previous messageGo to next message
Simon Goodall is currently offline Simon GoodallFriend
Messages: 35
Registered: July 2009
Member
"It seems you accept the fact that your import does not preserve the identities of the old objects?"

Do you mean the internal CDO identifier? Is there a way to preserve this? I am assuming that tracking history across metamodel changes is not feasible. I am interested to know what the wider implications of this are. I am still trying to understand how we can use the wider CDO feature set.

"Then it seems to me that you're trying to use both versions of your package in the same repo, right? Different nsURIs are one precondition for that, but different model names are required, too, because otherwise the "qualifiedNames" property can't do its job correctly."

We have our current meta-model version, and now for CDO, previous meta-model versions registered as dynamic packages. Normally we load these up in a local package registry instance and migrate our data models using the dynamic EObject api. Changing the model name also changes the generated java package. Is it possible to avoid this ? Can we define a custom mapping somewhere?

Thanks,

Simon
Re: [CDO] Changing datatypes between model versions [message #1690847 is a reply to message #1690845] Tue, 31 March 2015 15:12 Go to previous messageGo to next message
Eike Stepper is currently offline Eike StepperFriend
Messages: 6682
Registered: July 2009
Senior Member
Am 31.03.2015 um 16:56 schrieb Simon Goodall:
> "It seems you accept the fact that your import does not preserve the identities of the old objects?"
>
> Do you mean the internal CDO identifier? Is there a way to preserve this? I am assuming that tracking history across
> metamodel changes is not feasible.
Not whenyou use EMF to do the actual instance migration. You may want to search this forum for former replies to the
question of "model evolution" support in CDO. You should find some recipies that make use of the CDOServerExporter and
CDOServerImporter.

> I am interested to know what the wider implications of this are. I am still trying to understand how we can use the
> wider CDO feature set.
>
> "Then it seems to me that you're trying to use both versions of your package in the same repo, right? Different nsURIs
> are one precondition for that, but different model names are required, too, because otherwise the "qualifiedNames"
> property can't do its job correctly."
>
> We have our current meta-model version, and now for CDO, previous meta-model versions registered as dynamic packages.
One problem might be/become that dynamic packages don't handle custom data types very well.

> Normally we load these up in a local package registry instance and migrate our data models using the dynamic EObject
> api. Changing the model name also changes the generated java package. Is it possible to avoid this ? Can we define a
> custom mapping somewhere?
There's a model annotation http://www.eclipse.org/CDO/DBStore (tablename -> xyz) for specific EClasses but I fear we
offer nothing for the EPackage -> tableQualifier mapping so far. If you want to work on that you should have a look at
these members:

org.eclipse.emf.cdo.server.internal.db.mapping.AbstractMappingStrategy.getTableName(EClass, EStructuralFeature)
org.eclipse.emf.cdo.server.internal.db.mapping.AbstractMappingStrategy.getFieldName(EStructuralFeature)
org.eclipse.emf.cdo.server.internal.db.DBAnnotation

Cheers
/Eike

----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper


Re: [CDO] Changing datatypes between model versions [message #1690862 is a reply to message #1690847] Tue, 31 March 2015 16:02 Go to previous messageGo to next message
Simon Goodall is currently offline Simon GoodallFriend
Messages: 35
Registered: July 2009
Member
On 31/03/2015 16:12, Eike Stepper wrote:
> Am 31.03.2015 um 16:56 schrieb Simon Goodall:
>> "It seems you accept the fact that your import does not preserve the
>> identities of the old objects?"
>>
>> Do you mean the internal CDO identifier? Is there a way to preserve
>> this? I am assuming that tracking history across metamodel changes is
>> not feasible.
> Not whenyou use EMF to do the actual instance migration. You may want to
> search this forum for former replies to the question of "model
> evolution" support in CDO. You should find some recipies that make use
> of the CDOServerExporter and CDOServerImporter.
>
I'll take a look, but this works on the repository as a whole rather than parts of it?

>> I am interested to know what the wider implications of this are. I am
>> still trying to understand how we can use the wider CDO feature set.
>>
>> "Then it seems to me that you're trying to use both versions of your
>> package in the same repo, right? Different nsURIs are one precondition
>> for that, but different model names are required, too, because
>> otherwise the "qualifiedNames" property can't do its job correctly."
>>
>> We have our current meta-model version, and now for CDO, previous
>> meta-model versions registered as dynamic packages.
> One problem might be/become that dynamic packages don't handle custom
> data types very well.
>
Ah yes. Is this a problem for CDO - what will it try to do? I override the EFactory methods in my migration code for this.

>> Normally we load these up in a local package registry instance and
>> migrate our data models using the dynamic EObject api. Changing the
>> model name also changes the generated java package. Is it possible to
>> avoid this ? Can we define a custom mapping somewhere?
> There's a model annotation http://www.eclipse.org/CDO/DBStore (tablename
> -> xyz) for specific EClasses but I fear we offer nothing for the
> EPackage -> tableQualifier mapping so far. If you want to work on that
> you should have a look at these members:
>
> org.eclipse.emf.cdo.server.internal.db.mapping.AbstractMappingStrategy.getTableName(EClass,
> EStructuralFeature)
> org.eclipse.emf.cdo.server.internal.db.mapping.AbstractMappingStrategy.getFieldName(EStructuralFeature)
>
> org.eclipse.emf.cdo.server.internal.db.DBAnnotation
>
So here we could add something like TABLE_QUALIFIER to the DBAnnotation and then add the annotation to the EPackage declaration in my ecore file? In
getTableName() add it as a postfix to the generated name? E.g.;

........
public String getTableName(EClass eClass, EStructuralFeature feature)
{
String name = DBAnnotation.TABLE_NAME.getValue(eClass);
if (name == null)
{
name = isQualifiedNames() ? EMFUtil.getQualifiedName(eClass, NAME_SEPARATOR) : eClass.getName();
}

String qualifier = DBAnnotation.TABLE_QUALIFIER.getValue(eClass.getEPackage());
if (qualifier != null)
{
name += NAME_SEPARATOR;
name += qualifier;
}
........


How does AbstractMappingStrategy.getFieldName fit into this?

Simon


> Cheers
> /Eike
>
> ----
> http://www.esc-net.de
> http://thegordian.blogspot.com
> http://twitter.com/eikestepper
>
>
Re: [CDO] Changing datatypes between model versions [message #1690931 is a reply to message #1690862] Wed, 01 April 2015 05:04 Go to previous messageGo to next message
Eike Stepper is currently offline Eike StepperFriend
Messages: 6682
Registered: July 2009
Senior Member
Am 31.03.2015 um 18:02 schrieb Simon Goodall:
> On 31/03/2015 16:12, Eike Stepper wrote:
>> Am 31.03.2015 um 16:56 schrieb Simon Goodall:
>>> "It seems you accept the fact that your import does not preserve the
>>> identities of the old objects?"
>>>
>>> Do you mean the internal CDO identifier? Is there a way to preserve
>>> this? I am assuming that tracking history across metamodel changes is
>>> not feasible.
>> Not whenyou use EMF to do the actual instance migration. You may want to
>> search this forum for former replies to the question of "model
>> evolution" support in CDO. You should find some recipies that make use
>> of the CDOServerExporter and CDOServerImporter.
>>
> I'll take a look, but this works on the repository as a whole rather than parts of it?
Yes.

>
>>> I am interested to know what the wider implications of this are. I am
>>> still trying to understand how we can use the wider CDO feature set.
>>>
>>> "Then it seems to me that you're trying to use both versions of your
>>> package in the same repo, right? Different nsURIs are one precondition
>>> for that, but different model names are required, too, because
>>> otherwise the "qualifiedNames" property can't do its job correctly."
>>>
>>> We have our current meta-model version, and now for CDO, previous
>>> meta-model versions registered as dynamic packages.
>> One problem might be/become that dynamic packages don't handle custom
>> data types very well.
>>
> Ah yes. Is this a problem for CDO - what will it try to do? I override the EFactory methods in my migration code for
> this.
Ok.

>
>>> Normally we load these up in a local package registry instance and
>>> migrate our data models using the dynamic EObject api. Changing the
>>> model name also changes the generated java package. Is it possible to
>>> avoid this ? Can we define a custom mapping somewhere?
>> There's a model annotation http://www.eclipse.org/CDO/DBStore (tablename
>> -> xyz) for specific EClasses but I fear we offer nothing for the
>> EPackage -> tableQualifier mapping so far. If you want to work on that
>> you should have a look at these members:
>>
>> org.eclipse.emf.cdo.server.internal.db.mapping.AbstractMappingStrategy.getTableName(EClass,
>> EStructuralFeature)
>> org.eclipse.emf.cdo.server.internal.db.mapping.AbstractMappingStrategy.getFieldName(EStructuralFeature)
>>
>> org.eclipse.emf.cdo.server.internal.db.DBAnnotation
>>
> So here we could add something like TABLE_QUALIFIER to the DBAnnotation and then add the annotation to the EPackage
> declaration in my ecore file? In getTableName() add it as a postfix to the generated name? E.g.;
>
> .......
> public String getTableName(EClass eClass, EStructuralFeature feature)
> {
> String name = DBAnnotation.TABLE_NAME.getValue(eClass);
> if (name == null)
> {
> name = isQualifiedNames() ? EMFUtil.getQualifiedName(eClass, NAME_SEPARATOR) : eClass.getName();
I would probably replace the call to EMFUtil.getQualifiedName() here and in the other getTableName() method.
> }
>
> String qualifier = DBAnnotation.TABLE_QUALIFIER.getValue(eClass.getEPackage());
> if (qualifier != null)
> {
> name += NAME_SEPARATOR;
> name += qualifier;
> }
> .......
>
>
> How does AbstractMappingStrategy.getFieldName fit into this?
I think that field names are kind of orthogonal here.

Cheers
/Eike

----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper


>
> Simon
>
>
>> Cheers
>> /Eike
>>
>> ----
>> http://www.esc-net.de
>> http://thegordian.blogspot.com
>> http://twitter.com/eikestepper
>>
>>
>


Re: [CDO] Changing datatypes between model versions [message #1691010 is a reply to message #1690931] Wed, 01 April 2015 14:42 Go to previous messageGo to next message
Simon Goodall is currently offline Simon GoodallFriend
Messages: 35
Registered: July 2009
Member
On 01/04/2015 06:04, Eike Stepper wrote:
> Am 31.03.2015 um 18:02 schrieb Simon Goodall:
>> On 31/03/2015 16:12, Eike Stepper wrote:
>>> Am 31.03.2015 um 16:56 schrieb Simon Goodall:
>>>> "It seems you accept the fact that your import does not preserve the
>>>> identities of the old objects?"

It is possible to re-import and manipulate the CDOObject fields to match the ID? (I assume this is not a good idea if it is possible...)

>>>>
>>>> Do you mean the internal CDO identifier? Is there a way to preserve
>>>> this? I am assuming that tracking history across metamodel changes is
>>>> not feasible.
>>> Not whenyou use EMF to do the actual instance migration. You may want to
>>> search this forum for former replies to the question of "model
>>> evolution" support in CDO. You should find some recipies that make use
>>> of the CDOServerExporter and CDOServerImporter.
>>>
>> I'll take a look, but this works on the repository as a whole rather than parts of it?
> Yes.
I can image the size of the XML files can get quite large. Does anyone have experience processing large data dumps this way? I see BaseX is mentioned in
previous posts as a way to transform the XML. Is this still recommended?

I tried running the XML export on my repository and it threw an exception when it hit some serialised objects. It does not seem to handle byte arrays. Due to
some (badly defined) generics in my data model my EIntObject and EDoubleObject objects hit the object serialisation code in the default EFactory code.

>
>>
>>>> I am interested to know what the wider implications of this are. I am
>>>> still trying to understand how we can use the wider CDO feature set.
>>>>
>>>> "Then it seems to me that you're trying to use both versions of your
>>>> package in the same repo, right? Different nsURIs are one precondition
>>>> for that, but different model names are required, too, because
>>>> otherwise the "qualifiedNames" property can't do its job correctly."
>>>>
>>>> We have our current meta-model version, and now for CDO, previous
>>>> meta-model versions registered as dynamic packages.
>>> One problem might be/become that dynamic packages don't handle custom
>>> data types very well.
>>>
>> Ah yes. Is this a problem for CDO - what will it try to do? I override the EFactory methods in my migration code for this.
> Ok.
>
>>
>>>> Normally we load these up in a local package registry instance and
>>>> migrate our data models using the dynamic EObject api. Changing the
>>>> model name also changes the generated java package. Is it possible to
>>>> avoid this ? Can we define a custom mapping somewhere?
>>> There's a model annotation http://www.eclipse.org/CDO/DBStore (tablename
>>> -> xyz) for specific EClasses but I fear we offer nothing for the
>>> EPackage -> tableQualifier mapping so far. If you want to work on that
>>> you should have a look at these members:
>>>
>>> org.eclipse.emf.cdo.server.internal.db.mapping.AbstractMappingStrategy.getTableName(EClass,
>>> EStructuralFeature)
>>> org.eclipse.emf.cdo.server.internal.db.mapping.AbstractMappingStrategy.getFieldName(EStructuralFeature)
>>>
>>> org.eclipse.emf.cdo.server.internal.db.DBAnnotation
>>>
>> So here we could add something like TABLE_QUALIFIER to the DBAnnotation and then add the annotation to the EPackage declaration in my ecore file? In
>> getTableName() add it as a postfix to the generated name? E.g.;
>>
>> .......
>> public String getTableName(EClass eClass, EStructuralFeature feature)
>> {
>> String name = DBAnnotation.TABLE_NAME.getValue(eClass);
>> if (name == null)
>> {
>> name = isQualifiedNames() ? EMFUtil.getQualifiedName(eClass, NAME_SEPARATOR) : eClass.getName();
> I would probably replace the call to EMFUtil.getQualifiedName() here and in the other getTableName() method.
>> }
>>
>> String qualifier = DBAnnotation.TABLE_QUALIFIER.getValue(eClass.getEPackage());
>> if (qualifier != null)
>> {
>> name += NAME_SEPARATOR;
>> name += qualifier;
>> }
>> .......
>>
>>
>> How does AbstractMappingStrategy.getFieldName fit into this?
> I think that field names are kind of orthogonal here.
OK. I'll try this way for now, but looks like the CDOServerExporter is the better way longer term.

Simon
>
> Cheers
> /Eike
>
> ----
> http://www.esc-net.de
> http://thegordian.blogspot.com
> http://twitter.com/eikestepper
>
>
>>
>> Simon
>>
>>
>>> Cheers
>>> /Eike
>>>
>>> ----
>>> http://www.esc-net.de
>>> http://thegordian.blogspot.com
>>> http://twitter.com/eikestepper
>>>
>>>
>>
>
Re: [CDO] Changing datatypes between model versions [message #1691042 is a reply to message #1691010] Wed, 01 April 2015 19:38 Go to previous message
Erdal Karaca is currently offline Erdal KaracaFriend
Messages: 854
Registered: July 2009
Senior Member
If you go for XQuery/BaseX, here are some statistics:

http://docs.basex.org/wiki/Statistics

Simon Goodall wrote on Wed, 01 April 2015 16:42
On 01/04/2015 06:04, Eike Stepper wrote:
I can image the size of the XML files can get quite large. Does anyone have experience processing large data dumps this way? I see BaseX is mentioned in
previous posts as a way to transform the XML. Is this still recommended?

Previous Topic:EOperation is not generated in the sub-class if the super type has one with the same signature
Next Topic:EMF Notification
Goto Forum:
  


Current Time: Fri Mar 29 09:25:45 GMT 2024

Powered by FUDForum. Page generated in 0.03328 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top