Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » TMF (Xtext) » Context sensitive references?
Context sensitive references? [message #1033966] Thu, 04 April 2013 21:36 Go to next message
Chris Ainsley is currently offline Chris AinsleyFriend
Messages: 78
Registered: March 2010
Location: UK
Member
Hello,

Lets say I have a simple grammar:

Model: items+=KeyValue* ;

KeyValue: name=ID '=' value=STRING;

What I actually want is for some values to be references to keys, and some of them just to be plain strings. For the references, I want to be able to ctrl + click and for that to navigate to the key.

Now, I have metadata associated with these keys and values which will inform me for which keys should the value be a reference, and for which keys, should the values be a string. Everything depends upon XText to be able to dynamically be able to tell what is a reference and what is not a reference, but that information is NOT to be found in the grammar itself.

Is this possible? As a starting point, how should I adjust the above grammar?
Re: Context sensitive references? [message #1034077 is a reply to message #1033966] Fri, 05 April 2013 01:25 Go to previous messageGo to next message
Steve Kallestad is currently offline Steve KallestadFriend
Messages: 62
Registered: March 2013
Member
Why not something like:

Model items+=KeyValue*;

KeyValue: name=ID '=' value=ValueElement;

ValueElement:
 STRING | Keyword;

Keyword: 
 'Some'|'Special'|'Terms'


Alternatively, every cross-reference can be an EObject that you can work with programatically.

Model: items+=KeyValue*

KeyValue name=ID '=' value=[EObject]

But that will require getting into the nitty gritty of a few things.

A little bit easier... you could use cross-references and use a custom linker and validator that would allow for cross references to be invalid in certain situations.
Re: Context sensitive references? [message #1034200 is a reply to message #1034077] Fri, 05 April 2013 06:06 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 13613
Registered: July 2009
Senior Member
Hi,

this is not a reference it is a hyperlink. you may customize IHyperlinkHelper and
What todo there highly depends on your Logik.

Use of The Node Model (NodeModelUtils), EObjectAtOffsetHelper, and the index IResourceDescriptions(Provider) might help.
but as i said: this dependes on your semantics (one file, multiple files, at which places can the hyperlink be)


Need professional support for Xtext, Xpand, EMF?
Go to: https://www.itemis.com/en/it-services/methods-and-tools/xtext
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Advanced Scoping Question 1 [message #1034276 is a reply to message #1034200] Fri, 05 April 2013 08:30 Go to previous messageGo to next message
Chris Ainsley is currently offline Chris AinsleyFriend
Messages: 78
Registered: March 2010
Location: UK
Member
Thanks to both that have answered.

Steve: Your answer looks interesting, and its possible this is what I require. Please check my more specific requirements below and let me know if you still believe that this might be a suitable approach.

Christian: Your answer also looks interesting but there are actually scoping rules that come into play here, as a bad reference should be highlighted.

I hate to be bothersome but I feel that scoping is the one area where I can say that, even after reading the manual, and even after reading the previous replies, I am still very unclear as to the way that XText deals with these things. I have lots of questions, but for now, I am confining myself to asking about once concept at a time, which is why the above example looks strange.

Scoping itself is not too much of problem for me. I've written several compilers in JavaCC, some involving very advanced scoping requirements, and I managed to implement these without too much fuss using plain old JDK APIs. Re-implementing the scoping in XText for the purpose of providing an editor is proving more of a challenge.

My own unfamiliarity with the scoping architecture is the problem, not my knowledge of the scoping rules themselves, and that is where I'm having huge issues. I'm eager to understand how all of this works, but unfortunately the documentation lacks concrete examples beyond the scoping-by-rails examples.

So, I'd like to use the above faux-language, and this thread, as a method of possibly adding a concrete example to a future edition of the XText documentation. My understanding in exchange for sharing that understanding so to speak.

So, here are the requirements:

Advanced Scoping Question 1


  • A DSL that is a list of keys and values (with the '=' as the separator between them).
  • The value field, for sake of simplicity must be an ID type (in reality the value, it will be a custom terminal that consumes tokens up to the end of the line, but not required for the example). IMPORTANT: The implementation cannot use the TERMINAL type to distinguish between a reference an a non-reference. Both a reference value AND a non-reference value are ID type.
  • There is one global scope for key values. The same key cannot be specified twice (I believe this is the default behaviour anyway).
  • If the value field starts with 'ref', then the value field should be a reference to a key via the 4th character onward of the text specified for the value. If the value field does not start with 'ref' then the value field is treated as text only.
  • Holding Ctrl (or the mac equivalent) and clicking a value reference will link to the referenced key, holding ctrl over a non-ref attribute, will do nothing.
  • Broken links will be flagged as errors.
  • The 'ref' prefix rule cannot be part of the .xtext file itself, it must be implemented purely in-code (as the purpose of this example is to document how to manipulate such rules)


Now, this is a completely fake example, but the method to solve this problem is interesting and will reveal how XText works internally to those unfamiliar with EObjects, etc.

Given how limited the requirements are, I'd be happy with a fully working example rather than snippets, and like I said, I'd be willing to document how and why it works.

I'd also like to say, that I have some other more complex concrete scoping examples which upon some answer for this example, I'll be willing to post in a similar fashion. I hope that these examples can be used as a means of documenting the complexities of the scoping architecture of XText. Having a lot of information locked away in these forums is good, but having more single-concept examples in the main documentation is better.

[Updated on: Fri, 05 April 2013 08:36]

Report message to a moderator

Re: Advanced Scoping Question 1 [message #1034302 is a reply to message #1034276] Fri, 05 April 2013 09:07 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 13613
Registered: July 2009
Senior Member
Hi,

if you have references why dont you use references in the grammar?

~Christian


Need professional support for Xtext, Xpand, EMF?
Go to: https://www.itemis.com/en/it-services/methods-and-tools/xtext
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Re: Advanced Scoping Question 1 [message #1034434 is a reply to message #1034302] Fri, 05 April 2013 12:30 Go to previous messageGo to next message
Chris Ainsley is currently offline Chris AinsleyFriend
Messages: 78
Registered: March 2010
Location: UK
Member
Quote:
if you have references why dont you use references in the grammar?


Because, that is the question, how do I implement/define a context sensitive reference in the grammar?
Re: Advanced Scoping Question 1 [message #1034441 is a reply to message #1034434] Fri, 05 April 2013 12:37 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 13613
Registered: July 2009
Senior Member
Hi,

what do you mean with context sensitive? what is the criteria if a string is a reference and if not?


Need professional support for Xtext, Xpand, EMF?
Go to: https://www.itemis.com/en/it-services/methods-and-tools/xtext
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Re: Advanced Scoping Question 1 [message #1034468 is a reply to message #1034434] Fri, 05 April 2013 13:15 Go to previous messageGo to next message
Alexander Nittka is currently offline Alexander NittkaFriend
Messages: 1193
Registered: July 2009
Senior Member
Hi,

if I understand you correctly, what you are trying to do is not supported out of the box. You wrote that a reference in your example case would be recognizable by the prefix ref, so one might try something like
KeyValue: name=ID '=' value=[SomeType|REFID]
REFID: 'ref'(.*) //THIS IS ONLY FOR ILLUSTRATION

with a corresponding value converter trimming the prefix ref. At the same time you wanted "ref" *not* to be fixed in the grammar.

I guess the basic misunderstanding is not about scoping, but what happens before. In the grammar, you must provide a definite way (syntactically) for determining whether something is a reference or not, otherwise the parser will not know what type of object to instantiate (linking and scoping happen much later). It is not possible to write a grammar that expresses "value is a reference in case name meets some semantic condition (like having a prefix that is defined in another file or the preferene page)".

At the moment, the following approach seems most promising to me. You make everything a reference and extending LinkingDiagnosticMessageProvider returning null if something is not intended to be a reference (e.g. because the "link text" does not start with ref).

Alex

P.S.: It is a very common pattern, not to do everything in the grammar (as that is not always possible anyway, but more user friendly even if possible) but in later stages (validation, code completion, scoping).


Need training, onsite consulting or any other kind of help for Xtext?
Go visit http://xtext.itemis.com or send a mail to xtext@itemis.de

[Updated on: Fri, 05 April 2013 13:16]

Report message to a moderator

Re: Advanced Scoping Question 1 [message #1034678 is a reply to message #1034276] Fri, 05 April 2013 19:08 Go to previous message
Steve Kallestad is currently offline Steve KallestadFriend
Messages: 62
Registered: March 2013
Member
Chris Ainsley wrote on Fri, 05 April 2013 04:30
Thanks to both that have answered.

Steve: Your answer looks interesting, and its possible this is what I require. Please check my more specific requirements below and let me know if you still believe that this might be a suitable approach.

Christian: Your answer also looks interesting but there are actually scoping rules that come into play here, as a bad reference should be highlighted.

I hate to be bothersome but I feel that scoping is the one area where I can say that, even after reading the manual, and even after reading the previous replies, I am still very unclear as to the way that XText deals with these things. I have lots of questions, but for now, I am confining myself to asking about once concept at a time, which is why the above example looks strange.

Scoping itself is not too much of problem for me. I've written several compilers in JavaCC, some involving very advanced scoping requirements, and I managed to implement these without too much fuss using plain old JDK APIs. Re-implementing the scoping in XText for the purpose of providing an editor is proving more of a challenge.

My own unfamiliarity with the scoping architecture is the problem, not my knowledge of the scoping rules themselves, and that is where I'm having huge issues. I'm eager to understand how all of this works, but unfortunately the documentation lacks concrete examples beyond the scoping-by-rails examples.

So, I'd like to use the above faux-language, and this thread, as a method of possibly adding a concrete example to a future edition of the XText documentation. My understanding in exchange for sharing that understanding so to speak.

So, here are the requirements:

Advanced Scoping Question 1


  • A DSL that is a list of keys and values (with the '=' as the separator between them).
  • The value field, for sake of simplicity must be an ID type (in reality the value, it will be a custom terminal that consumes tokens up to the end of the line, but not required for the example). IMPORTANT: The implementation cannot use the TERMINAL type to distinguish between a reference an a non-reference. Both a reference value AND a non-reference value are ID type.
  • There is one global scope for key values. The same key cannot be specified twice (I believe this is the default behaviour anyway).
  • If the value field starts with 'ref', then the value field should be a reference to a key via the 4th character onward of the text specified for the value. If the value field does not start with 'ref' then the value field is treated as text only.
  • Holding Ctrl (or the mac equivalent) and clicking a value reference will link to the referenced key, holding ctrl over a non-ref attribute, will do nothing.
  • Broken links will be flagged as errors.
  • The 'ref' prefix rule cannot be part of the .xtext file itself, it must be implemented purely in-code (as the purpose of this example is to document how to manipulate such rules)


Now, this is a completely fake example, but the method to solve this problem is interesting and will reveal how XText works internally to those unfamiliar with EObjects, etc.

Given how limited the requirements are, I'd be happy with a fully working example rather than snippets, and like I said, I'd be willing to document how and why it works.

I'd also like to say, that I have some other more complex concrete scoping examples which upon some answer for this example, I'll be willing to post in a similar fashion. I hope that these examples can be used as a means of documenting the complexities of the scoping architecture of XText. Having a lot of information locked away in these forums is good, but having more single-concept examples in the main documentation is better.


I'm reading your post and kind of smiling at my own confusion. I complained about the documentation a week or so ago because there are times when it has me pulling my hair. My real problem isn't that things aren't clearly documented, they just aren't documented in a way that *I* easily understand, and getting a strong grasp on some of basic concepts sometimes takes significant effort. I rarely run into this on other projects, but at the same time I'm not just coming at XText as a neophyte, but eclipse plugins, osgi, and EMF. All the technical background in the world isn't going to make this stuff easy.

All that is to say that thinking about things abstractly actually makes understanding XText more difficult. People have a hard time interpreting your questions and choosing a best answer because like anything else there are 10 different ways to accomplish a given task and each has drawbacks and positives.

So let me take a step back here and see if I can't give some general advice from my vastly historical (1 month) of experience here Smile.

Right off the bat, the first thing I would recommend is to try not to define everything in the grammar. It should provide structure, but it shouldn't be overly definitive. That way minor tweaks down the road such as adding a keyword will be easier.

Here's a blog entry on using libraries to define things. This one is more complex but lays out a lot of detail.

The reason I bring this up was this quote:
Quote:

The 'ref' prefix rule cannot be part of the .xtext file itself


I had some similar issues conceptually early on - "I don't want to have to define everything in the grammar because that's an unacceptable restriction on a separate effort".

Now as far as alternative cross-references goes - this is an easy to understand getting started point.

Cross-references aren't hyperlinks, granted, but I haven't delved into hyperlinks just yet. I do know that hitting F3 on an element with a cross-reference it will take you to the element that it references. It's a one-way trip - at least the way I have implemented things.

In my DSL, I have function calls and function definitions. Hitting F3 on a function call will take you to the function definition.

Out of the box, this worked with a simple cross-reference declaration, but when I wanted to implement different signatures - linking to a different function declaration based on parameter types and not just the name - I had to start learning about scoping, linking, and defining my grammar a little more generically. My starting point was the links above. My productivity happened when I started coding instead of reading. Trial and error, debugging, and lots of caffeine.

Outside of simple cross-references, you have an "either this or that" thing going on. That's typically handled by a simple abstraction.

I have this declared:
ParameterSet:
	param+=Parameter (',' param+=Parameter)*
;
Parameter:
	DataParameter | StringParameter | BareWordParameter | Integer
;

Each of these different parameters is defined slightly differently and the BareWordParameter is the only one that will have a cross-reference.

So Strings are valid, numbers are valid, but if it's a Bareword in order for it to be valid there has to exist something that it references.

-----
I hope that some of this or all of this is relevant/helpful, but I don't know that I fully understand what your getting at.

I honestly think that simply posting an example of the actual language or a simplified version makes it easier for everyone.

something like:
x = y
z = "some string"
w = ref x

with a question like "when I control click ref x I want my cursor to jump to the "x = y" line.

The easier the question is to digest, the better the answers. Just my opinion. I don't follow my own advice, but I wish I would. Smile



Previous Topic:How to imply a value for a field via the grammar
Next Topic:Dynamic Template Proposal Hover Display
Goto Forum:
  


Current Time: Mon Sep 28 23:24:56 GMT 2020

Powered by FUDForum. Page generated in 0.02209 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top