Eclipse Community Forums: EMF » ECore model evolution and versioning

Help

Home

Home » Modeling » EMF » ECore model evolution and versioning

Show: Today's Messages :: Show Polls :: Message Navigator

ECore model evolution and versioning [message #1796752]

Thu, 18 October 2018 13:11

Ewoud Werkman

Messages: 28
Registered: January 2018

Junior Member

Hi all,

We are developing a Ecore model that is under heavy development. In the mean time we've developed several tools around this model, such as a Sirius-based editor, and services that e.g. generate instances from raw data. Many of the services are using an XSD that is generated from the model's GenModel.
To keep track of the different versions of the model, we version the namespace URI. With every release we update the version number.

However, now our tooling is extending and the number of instances is growing, updating the namespace URI's in all the files becomes a burden. Especially Sirius' AIRD-files are quite problematic as they contain numerous references to the namespace (compared to just 'updating' the xmlns:ns='xxx' of an model instance).

How can we deal with this issue?
I've opted to get rid of the version number in the namespace, saving us from updating all the files when there are minor changes to the model, but then there is no version information available at all in the files, which I don't like. I know XSD's can be versioned with a version-attribute, but this seems not supported by EMF.

Any ideas? Thanks!

Report message to a moderator

Re: ECore model evolution and versioning [message #1796760 is a reply to message #1796752]

Thu, 18 October 2018 14:34

Ed Willink

Messages: 7655
Registered: July 2009

Senior Member

Hi

While you are in an experimental phase, just change what you need and bear in mind the cost of change. Few model tools support refactoring, but some model tools give good validation of inconsistencies so I recommend developing an exercise script that uses strongly checked tools so that you discover the consequences of your refactorings early. NB ATL and Epsilon give very little type checking. QVTo might be a good choice. OCLinEcore or Complete OCL might be better since it can be activated automatically by a builder.

Once you are in a stable phase you could change namespaces as OMG do, but the ripples are horrible. If you can retain upward compatibility, you can make a case for not changing the nsURI, just as Ecore didn't when generics were introduced.

Bottom line, try very very hard to get the model right before you freeze it.

If you really want a moving version, make it a comment/documentation-only property.

IMHO the correct way to version models is with a tagged GIT commit.

Regards

Ed Willink

Report message to a moderator

Re: ECore model evolution and versioning [message #1796805 is a reply to message #1796760]

Fri, 19 October 2018 06:29

Ed Merks

Messages: 33137
Registered: July 2009

Senior Member

Changing the nsURI is of course a reasonable way to version the model; this way the version is recorded/reflected in the instances. But of course the very fact its recorded in the instances also becomes a burden given you'd like the tools to be able to read older versions as well as the latest version. One approach to support that is to literally keep around the older versions as dynamic models such that older instances can be processed according to the older model version as dynamic instances that can then be automatically transformed into the latest generated version of the model. Depending on how much the structure of the model changes, that can be relatively straight forward. It's still a relatively complex process to manage and of course if you have resources that literally use the model itself (the diagrams, the genmodel), those are things that need to be maintained and versioned like the models themselves...

Indeed an XML Schema has a version attribute and Ecore does not. But Ecore also has annotations so you could easily use an EAnnotation to add a version-like attribute to the EPackage.

Note that an exceedingly useful load/save option in this type of scenario (and evolving model) is XMLResource.OPTION_RECORD_UNKNOWN_FEATURE. If you avoid making "binary incompatible" changes to the model, e.g., don't rename things, don't delete things, it will always be the case that newer tools can read older serializations. With the OPTION_RECORD_UNKNOWN_FEATURE option, it will also be the case that older tools can read newer serializations. The information which isn't recognized (doesn't conform to the model), is simply recorded as "raw" XML in the XMLResource.getEObjectToExtensionMap and that information is written back out again on save, so an older tool doesn't lose the newer information. I mention this because this recorded information can also be post processed directly after loading such that a newer tool can transform information from binary incompatible changes to conform to the newer model. Also, the org.eclipse.emf.ecore.xmi.XMLResource.OPTION_EXTENDED_META_DATA can be used to help automatically transform renamed classes and features.

Ed Merks
Professional Support: https://www.macromodeling.com/

Report message to a moderator

Re: ECore model evolution and versioning [message #1796858 is a reply to message #1796805]

Fri, 19 October 2018 19:18

Ewoud Werkman

Messages: 28
Registered: January 2018

Junior Member

Ed and Ed,

Thanks for your replies and the suggestion to improve our evolution of the model.
Three questions:

We already use Git heavily in the process and tag each 'release', but this information is (if not put in nsURI) not contained in the model itself. So getting a specific version of the model out of Git is straightforward, but the other way around, having a version of the model and then finding out to which git tag it belongs is not straightforward. Any ideas for that (Annotations as Ed mentions?)?
You explain how we could transform older model instances to newer instances, but have trouble understanding how to implement that. I know how to read a model using the old version, but how can I automatically transform that model into the new version (e.g. in the case that we don't change the structure too much)?
Regarding the option to use an EAnnotation is interesting as we have not experimented with EAnnotations yet. How would that information be serialized to an XSD? And this that information than also visible in the resulting instances of the model?

The XMLResource.OPTION_EXTENDED_META_DATA seems quite useful for the Sirius tooling, such that it can read older and newer versions with features it does not know yet. Thanks for mentioning it.

Report message to a moderator

Re: ECore model evolution and versioning [message #1796862 is a reply to message #1796858]

Sat, 20 October 2018 03:07

Ed Merks

Messages: 33137
Registered: July 2009

Senior Member

1. Yes, an EAnnotation can be used to record arbitrary information. E.g., on an EPackage use New Child -> EAnnotation. In the latest release, several choices will be offered here because there is built-in knowledge about some special annotation that are recognized by the framework. Use the Properties view to set the source URI, e.g., model:/version and on the new EAnnotation use New Child -> Details Entry to define an entry, e.g., version="1.0". Of course how you structure the annotation is arbitrary and up to you. Just keep in mind that this information is generated into the Javadoc (of the package interface in this case), and is available in the generated model, so you could use EcoreUtil.getAnnotation(XyzPackage.eINSTANCE, "model/version", "version") to get this information at runtime.
2. You can add load/save hooks using org.eclipse.emf.ecore.xmi.XMLResource.OPTION_RESOURCE_HANDLER so in that handler's postLoad you could process the contents of getEObjectToExtensionMap, which maps each loaded object to the associated arbitrary XML represented by AnyType instances. This article has useful information related to this: https://www.theserverside.com/news/1364302/Binding-XML-to-Java
3. I think each EAnnotation is mapped to XML Schema annotation. At runtime you can use EcoreUtil.getAnnotation( "eObject.eClass.getEPackage(),"model/version", "version) to get the instance's model version. Note that with this approach the serialized instance does not have version information

Ed Merks
Professional Support: https://www.macromodeling.com/

Report message to a moderator

Re: ECore model evolution and versioning [message #1796869 is a reply to message #1796862]

Sat, 20 October 2018 08:25

Ed Willink

Messages: 7655
Registered: July 2009

Senior Member

Hi

Since you are clearly using Ecore as your primary tooling, you might want to review whether the auto-generated XSD does more than satisfy a misguided obsolete requirement.

Regards

Ed Willink

Report message to a moderator

Re: ECore model evolution and versioning [message #1826245 is a reply to message #1796869]

Tue, 21 April 2020 18:52

Marek Jagielski

Messages: 97
Registered: April 2012

Member

Hi,
I was using similar idea with having a version in namespace. I developed an eclipse plugin based on Graphiti. Users could have already diagrams created with previous version when installing a new version of plugin. I didn't set bundles as singletons, so the diagrams could be opened using old bundles that were still in eclipse.

<task xsi:type="constant_1_0_6:Constant" id="My Constant" value="0">
      <outputPort id="Out" parent="/1/@task.0">
        <dataType xsi:type="dt:Int32"/>
      </outputPort>
      <dataType xsi:type="dt:Int32"/>
    </task>

Additionally, plugin was verifying the version of whole Diagram and based on that could see that the diagram was created with older version. Upgrade button got enabled.
When the user clicked on upgrade button I executed a chain of xslt transformations. One xslt transformation for each release that is between diagram and plugin version. So Plugin had xslt recipes for each release upgrade: XSLT[1.0.0 -> 1.0.1], XSLT[1.0.1 -> 1.0.2],...

I am now reevaluating idea so it is why I found this topic. I am asking the same question if having version in namespace is a good pattern.

Report message to a moderator

Re: ECore model evolution and versioning [message #1826246 is a reply to message #1826245]

Tue, 21 April 2020 20:26

Ed Willink

Messages: 7655
Registered: July 2009

Senior Member

Hi

No, unless the change is so big that it has completely different concepts. If the nsURI changes, you potentially have to have separate tooling per version, although you may provide an upgrade facility as the UML project does, but you get into all sorts of horrible complexities.

Much better to have a "version" attribute, so that a generic loader can load the per-version processing can normalise to something compatible and so that neutral tools such as the Ecore Editor don't find themselves using the upgrade facilities.

Much much better to anticipate changes and have flexibility / struggle to adjust changes to be not-incompatible. But obviously that rarely works in the real world.

Regards

Ed Willink

Report message to a moderator

Re: ECore model evolution and versioning [message #1826521 is a reply to message #1826246]

Sun, 26 April 2020 14:34

Marek Jagielski

Messages: 97
Registered: April 2012

Member

Exactly what happened. It was my first approach to EMF. Not knowing where the project will go and being afraid of breaking functionalities I over-engineered the solution. I used quite extensively services and at each stage I had to choose correct version of plugin.
Nevertheless, I try now to simplify the whole solution. Once, that "breaking" changes are really rare if ever, second that I am going to more focus on diagram level version and even not try to load diagram if the version doesn't match but first do needed upgrade.

Thanks,

Marek

Report message to a moderator

Previous Topic:	EMF resource load - SAXParser loading map twice
Next Topic:	CDO: DB Connection leaks caused by InterruptedExceptions

Goto Forum:

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

]

Current Time: Fri Apr 19 07:37:02 GMT 2024

.:: Contact :: Home ::.

Breadcrumbs

Sign up to our Newsletter