Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » TMF (Xtext) » Xtext editor backed by XMI resource
Xtext editor backed by XMI resource [message #1745888] Tue, 18 October 2016 23:01 Go to next message
Luis Solano is currently offline Luis SolanoFriend
Messages: 3
Registered: October 2016
Junior Member
Hi,

I've created a language with Xtext. I'm trying to build some tools on top of it. These tools need to identify parts of the underlying model over time, being resilient to name changes. Effectively, I need a unique identifier for every EObject in my model that never changes.

I was thinking about persisting my Xtext model as an XMI but I'd still like to use the Xtext editor to edit this model: read the xmi as a tree of EObjects with ids, pass that in-memory instance to the XtextEditor. On save, assign ids to the new objects and persist the mode back to xmi.

This doesn't seem possible out of the box and overriding XtextDocumentProvider, XtextDocument and friends seems a big undertaking so I figured I'd ask first.

I'm open to any other options, except exposing those ids to the users of my language Very Happy

Thank you!
Re: Xtext editor backed by XMI resource [message #1745942 is a reply to message #1745888] Wed, 19 October 2016 16:47 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 13637
Registered: July 2009
Senior Member
if the model changes, how do you know the objects itself does not change?
what differs a move rename remove and readd?

is it just the position inside the model?


Need professional support for Xtext, Xpand, EMF?
Go to: https://www.itemis.com/en/it-services/methods-and-tools/xtext
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Re: Xtext editor backed by XMI resource [message #1745943 is a reply to message #1745942] Wed, 19 October 2016 16:59 Go to previous messageGo to next message
Ed Willink is currently offline Ed WillinkFriend
Messages: 6874
Registered: July 2009
Senior Member
Hi

What you say is certainly possible. With Xtext, you have a similar problem as with the UML Model Editor that trashes any pre-existing xmi:ids. You therefore need to ensure that you automatically assign new xmi:ids during Resource.save. For these to be useful they should be deterministic, typically dependent on ancestor class+name/position path. For an arbitrary metamodel this can result in unpleasantly long names. If you have a good understanding on the limitations on your metamodel you can design something much more compact. You can find a variety of semantic xmi:id assigners for UML and OCL in the Eclipse OCL project.

Regards

Ed Willink
Re: Xtext editor backed by XMI resource [message #1745952 is a reply to message #1745942] Wed, 19 October 2016 18:05 Go to previous messageGo to next message
Luis Solano is currently offline Luis SolanoFriend
Messages: 3
Registered: October 2016
Junior Member
Thanks for your responses

Christian Dietrich wrote on Wed, 19 October 2016 16:47
if the model changes, how do you know the objects itself does not change?
what differs a move rename remove and readd?

is it just the position inside the model?


Christian, I'm not sure I understand the first question about the model changing and the objects changing. Could you please elaborate?

Re: differentiating between move-rename and remove-readd, yes. That is a limitation that I'm okay with. Users would have to apply refactors to preserve the identify of things.

Not sure I understand the position question either. I don't identify objects by position and I don't intend to do so. Moving things around, as long as it preserves the assigned id, should have no effect.

Ed Willink wrote on Wed, 19 October 2016 16:59
Hi

What you say is certainly possible. With Xtext, you have a similar problem as with the UML Model Editor that trashes any pre-existing xmi:ids. You therefore need to ensure that you automatically assign new xmi:ids during Resource.save. For these to be useful they should be deterministic, typically dependent on ancestor class+name/position path. For an arbitrary metamodel this can result in unpleasantly long names. If you have a good understanding on the limitations on your metamodel you can design something much more compact. You can find a variety of semantic xmi:id assigners for UML and OCL in the Eclipse OCL project.

Regards

Ed Willink


Ed,

My current approach consists on adding a ID attribute to every metaclass of my .ecore. Upon Resource.save I assign UUIDs to the objects without an ID. Note that I don't rely on xmi:ids because I want to access the ids at the model level. Not sure if it is possible with xms:ids, to be honestly; any ideas are welcome.

When I load the resource, I load my root object from the xmi (with the ids) and pass it to Xtext (right now, by serializing the model and parsing it back, which is a huge hack, but XtextDocumentProvider uses IParserResults)

I believe this option should have deterministic ids, as long as Xtext knows which nodes in the textual AST refer to which EObjects in the model, which I think it does; test pending.

-----

To give some more details, I'm subclassing the following classes:
XtextResource: to override the doSave and doLoad methods. I serialize the model as xmi and write it to the output stream of model.myDsl file. Currently working on the doLoad to read the xmi file and expose it as text to the editor.

XtextDocumentProvider: to override the doSave and 1) add missing IDs and 2) call Resource.save (otherwise it stores the text in the mode.myDsl file directly, no Resource.save involved). I'm also working on the doLoad here, currently the xmi data is being exposed in the editor, not sure why yet.

ITransientValueService: I need to let Xtext know that my ID attribute is transient so that Serialization doesn't fail, even tho it is not marked as transient in the ecore file (we want to store the id attribute in the xmi but not in the textual representation)

In short, I want to implement some poor man's approach to projectional editing with Xtext. I understand that caveats my apply. Is this approach going in the right direction?

That said, if any of this work may be interesting to have as part of Xtext, I'd be happy to contribute back. I'd appreciate feedback about the design regardless.

Re: Xtext editor backed by XMI resource [message #1745953 is a reply to message #1745952] Wed, 19 October 2016 18:41 Go to previous messageGo to next message
Ed Willink is currently offline Ed WillinkFriend
Messages: 6874
Registered: July 2009
Senior Member
Hi

IMHO, UUIDs are only useful for write-once models. Not really your use case.

If by adding an ID, you also incorporate the ID in the grammar then the problem is trivial, and your source text is very ugly.

If the ID does not appear in the grammar, then you might as well use xmi:id. It can be accessed from the model level by a helper function.

Regards

Ed Willink
Re: Xtext editor backed by XMI resource [message #1745955 is a reply to message #1745953] Wed, 19 October 2016 18:57 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 13637
Registered: July 2009
Senior Member
what i meant:

since xtext reparses all the time and does no model merge you cannot distinguish
a rename from a add + remove

=> it will be hard to track this inside the IDs
=> basically the ids have to be type/position based


Need professional support for Xtext, Xpand, EMF?
Go to: https://www.itemis.com/en/it-services/methods-and-tools/xtext
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Re: Xtext editor backed by XMI resource [message #1745959 is a reply to message #1745955] Wed, 19 October 2016 21:18 Go to previous messageGo to next message
Sven Efftinge is currently offline Sven EfftingeFriend
Messages: 83
Registered: January 2016
Location: Kiel
Member

Hi Luis,

This is a really hard problem to solve.
There is no way to keep something like a UUID across sessions if you don't persist them somehow.
The closest you get is to use something like EMF compare to try to identify equal objects based on their state and position.
My recommendation is to reconsider the requirement of being resilient to name changes. Names are very natural identifiers.

Sven
Re: Xtext editor backed by XMI resource [message #1745964 is a reply to message #1745953] Wed, 19 October 2016 23:32 Go to previous messageGo to next message
Luis Solano is currently offline Luis SolanoFriend
Messages: 3
Registered: October 2016
Junior Member
Thanks again for your responses and your clarifications!

Ed Willink wrote on Wed, 19 October 2016 18:41
Hi

IMHO, UUIDs are only useful for write-once models. Not really your use case.

If by adding an ID, you also incorporate the ID in the grammar then the problem is trivial, and your source text is very ugly.

If the ID does not appear in the grammar, then you might as well use xmi:id. It can be accessed from the model level by a helper function.

Regards

Ed Willink


Yes, the IDs won't show in the grammar. I'll look into using xmi:ids.

Not sure why UUIDs won't be useful. If I can make sure that a UUID is assigned once and it stays the same for a given object in my model (the editor is the part that may give me more trouble here).

Christian Dietrich wrote on Wed, 19 October 2016 18:57
what i meant:

since xtext reparses all the time and does no model merge you cannot distinguish
a rename from a add + remove

=> it will be hard to track this inside the IDs
=> basically the ids have to be type/position based


Ah!, the part where Xtext reparses each time is the one I didn't get to experience yet (still working on it). If that is the case, yes, it will be problematic. Maybe it can be workaround?

Sven Efftinge wrote on Wed, 19 October 2016 21:18
Hi Luis,

This is a really hard problem to solve.
There is no way to keep something like a UUID across sessions if you don't persist them somehow.
The closest you get is to use something like EMF compare to try to identify equal objects based on their state and position.
My recommendation is to reconsider the requirement of being resilient to name changes. Names are very natural identifiers.

Sven


Hi Sven,

The ids will be stores across sessions. That's why I want to store the model as xmi, so that I can add the ids before saving.

Funny you mention EMF Compare. The reason I want the stable IDs is so that EMF compare works for my use case.

The requirement of a stable id other than name is legitimate. I want to have a model that acts a a master model. I want to create derived models from the master, which are effectively a clone of the master plus some tweaks. The information I want to keep around is the master model itself and a list of changes to the master so that, when applying them, it yields the child. The master and the child can both rename a given object with distinct names. After that, I want to keep being able to map the object in the master model to the object in the child model. Since names are different in both models, I can't map them.

It is a hard problem to solve, at least in the context of pure textual representations. If the editor reparses the text on every change, creating a new model, then there isn't much to do. On the other hand, if the editor issues changes to the the underlying (and existing model), then it could work. Not 100% sure how it actually works, I guess I'll find out soon enough Smile

[Updated on: Wed, 19 October 2016 23:33]

Report message to a moderator

Re: Xtext editor backed by XMI resource [message #1745975 is a reply to message #1745964] Thu, 20 October 2016 05:32 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 13637
Registered: July 2009
Senior Member
I still do not get it.

If you need in you ID calculator a logic that can distinguish between edit and a remove/add why can't you have it inside emf compares matchers?


Need professional support for Xtext, Xpand, EMF?
Go to: https://www.itemis.com/en/it-services/methods-and-tools/xtext
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Re: Xtext editor backed by XMI resource [message #1745976 is a reply to message #1745975] Thu, 20 October 2016 06:11 Go to previous messageGo to next message
Sven Efftinge is currently offline Sven EfftingeFriend
Messages: 83
Registered: January 2016
Location: Kiel
Member

Maybe you find Stefan Winkler's experiments with EMF Compare, Xtext and CDO helpful.
AFAIR he goes into more detail of how to preserve identity using EMF compare:
https://vimeo.com/98393613
Re: Xtext editor backed by XMI resource [message #1745990 is a reply to message #1745976] Thu, 20 October 2016 09:03 Go to previous messageGo to next message
Ed Willink is currently offline Ed WillinkFriend
Messages: 6874
Registered: July 2009
Senior Member
Hi

UUIDs are fine for write once.

But if you create with a content X, save. Edit, delete X, save. Edit, recreate X. The recreated X gets a new UUID.

Similarly if you merge the identical work of two authors. They have different UUIDs.

If you want IDs to be useful, they must be (semantically) deterministic so that a second 'creation' of the ID recreates that of the first and no other creation collides with a different past or future element.

Using your insight of the hierarchical content, these IDs can be generated very easily and ensure that EMF compare runs fast. If you leave EMF compare to deduce what you know, you can expect to spend a very long time on large models and quite probably get unsatisfactory results after the wait.

Regards

Ed Willink
Re: Xtext editor backed by XMI resource [message #1819421 is a reply to message #1745990] Wed, 15 January 2020 00:12 Go to previous messageGo to next message
Steve Hickman is currently offline Steve HickmanFriend
Messages: 18
Registered: August 2017
Junior Member
I'm interested in preserving object IDs for the same reason. Right now I'm preserving xml:id information in a separate properties file where the key is the fully qualified name (FQN) of the object and the value is the xmi:id. I already use this approach to recreate an XMI file with preserved IDs from an xtext file by loading the xtext file into an XTextResource, then converting that to an XMIResource with useUUID() overridden to return true. Once the resource is converted, I then replace the generated UUIDs with the xmi:ids preserved in the separate properties file.

I would like to extend this capability to support refactor/rename in Eclipse while preserving the xmi:id of each EObject. Tell me if the following makes sense:

modify doLoad to load the hidden properties file associated with each xtext file. After the xtext is parsed and the AST created, update the ids of all the EObjects with the information from the properties file. When a refactor/rename is done, preserve the EObjects and their stored ids but change the name. I then update the properties file to replace the key value pair with the old FQN with a pair using the new FQN. If I can't preserve the EObjects (because XText wants to reparse the entire file and create all new EObjects), I could trap the refactor/rename event, convert the AST back to XMLResource, do the rename there, update the properties file, and then convert back to XTextResource.

This does require cloning the properties file whenever the xtext file is cloned but I consider that a minor inconvenience if the rest of it works.

Does this make sense? Or is there something significant that I'm not thinking of?
Re: Xtext editor backed by XMI resource [message #1819439 is a reply to message #1819421] Wed, 15 January 2020 12:13 Go to previous message
Ed Willink is currently offline Ed WillinkFriend
Messages: 6874
Registered: July 2009
Senior Member
Hi

It can work, but maintaining a magic stable xmi:id properties file is a pain. Just imagine making sure that your user gets the right version checked out from GIT. A guaranteed downstream nightmare.

For OCL I use a different approach; Locally Unique Semantically Sensitive IDentifiers[1]. These are (re-)computed whenever necessary from the hierarchical and local content. Obviously it is essential that the computation is stable and for most metamodels it is easy to come up with a suitable algorithm. NB, xmi:ids only need to be locally unique so there is no need to worry about cross-module clashes. For Ecore you could have readable IDs such as P_packagename, C_packagename_classname etc exploiting the prohibition on ambiguous names in a well-formed Ecore model. Overloaded entities such as Operations require extra care. You of course need coding rules for names that include "_" or other challenging letters. Use of internal content such as names can avoid the refactoring hazards that occur with the standard Ecore xx/@feature-y/@feature-z[44] style of URIs.

Once you have potentially long but readable LUSSIDs, you can then hash them to short xmi:id values. For OCL the hash is to a five Base64 letter representation (six letters if the hash clashes). You can see the usage in [2].

(It would be really nice if EMF allowed the xml namespaces to be used to compress URIs so that e.g. xmlns:gYU="http://www.eclipse.org/ocl/2015/Library.oclas" could facilitate the compression "gYU:#jbMXR" for "http://www.eclipse.org/ocl/2015/Library.oclas#jbMXR".)

Regards

Ed Willink


[1] https://git.eclipse.org/r/plugins/gitiles/ocl/org.eclipse.ocl/+/master/plugins/org.eclipse.ocl.pivot/src/org/eclipse/ocl/pivot/internal/resource/LUSSIDs.java
[2] https://git.eclipse.org/r/plugins/gitiles/ocl/org.eclipse.ocl/+/master/plugins/org.eclipse.ocl.pivot/model-gen/Pivot.oclas
Previous Topic:Whitespace in qualified names
Next Topic:Custom coloring for block of code
Goto Forum:
  


Current Time: Sun Oct 25 19:36:28 GMT 2020

Powered by FUDForum. Page generated in 0.02677 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top