| [Xtext 2.4.3] What to do if 'keyword' can occur as an ID as well? [message #1219328] |
Tue, 03 December 2013 06:07  |
Eclipse User |
|
|
|
Hi,
I am currently devloping a grammar for a distributed file format. Each of these files has its own grammar, therefore my model looks like this:
...
terminal PLUS: '+';
terminal MINUS: '-';
terminal DOT: '.';
terminal ID : ('^'|'$'|DOT)? ('a'..'z'|'A'..'Z'|'_') (('a'..'z'|'A'..'Z'|'_'|'0'..'9'|PLUS|MINUS)|DOT)*;
...
Model:
AFile | BFile;
AFile:
'Entity' '{'
...
'name' '=' name=ID
... ;
BFile: ABCRecord | DEFRecord | TOPRecord;
ABCRecord: ... ;
DEFRecord: ... ;
TOPRecord: 'TOP' valA=INT valB=INT valC=INT ';' ;
Everything worked well until I found out that the following construct is legal:
file A:
Entity {
...
name=TOP
...
}
file B:
ABC 111 222 333; 33 484
DEF 333 444 555 xyz; 333
TOP 894 23 979;
As you can see I need 'TOP' in file A to be recognized as an ID, whereas in file B TOP denotes the beginning of a data line.
If I understand aright, the problem is that the lexer consumes 'TOP' before it applies the 'ID' rule.
Is there a nice way to deal with such a situation, e.g. telling the lexer that it should delay the recognition of 'TOP' tokens until the ID rule was applied to the input?
Thank you in advance!
|
|
|
|
| Re: [Xtext 2.4.3] What to do if 'keyword' can occur as an ID as well? [message #1219333 is a reply to message #1219330] |
Tue, 03 December 2013 06:43  |
| Eclipse User |
|
|
|
Quote:
Because the "Entity" denotes that you are inside the AFile rules which do not recognise a "TOP" keyword.
So IIRC there is no conflict at all.
Hm, that's what I thought at first, too. But if I open file A with my Runtime Eclipse, then all occurences of 'TOP' are recognized as a keyword, causing an error ("Mismatched input 'TOP', expecting RULE_ID").
My standalone test tool reports the same syntax error.
Quote:
In the standard XTEXT terminals, ther is a build-in escape for IDs implemented:
a "^" in front of the ID which separetes the ID from a keyword is removed automatically.
So entering name=^TOP will be turned into a simple TOP
Yes, I am aware of that. Unfortunately I cannot modify the source files. The whole file format is a lot more complex, so I really need to avoid messing around with file contents manually.
--- EDIT ---
I changed the 'TOP' rule to 'TOP ', and that did the job, as there must be a whitespace anyway.
This works for now, and it does not smell as badly as manually crawling the input files. But if anybody has a even better solution I would be very happy 
--- /EDIT ---
[Updated on: Tue, 03 December 2013 07:05] by Moderator |
|
|
|
| Re: [Xtext 2.4.3] What to do if 'keyword' can occur as an ID as well? [message #1219353 is a reply to message #1219333] |
Tue, 03 December 2013 08:48 |
| Eclipse User |
|
|
|
Hi Wayne,
for that you should declare a non-terminal rule like this:
MyIdWithKeywords :
ID | 'keyword1' | 'keyword2';
And then use that rule at the appropriate places.
Sven
Am 12/3/13 12:43 PM, schrieb Wayne Campbell:
> Quote:
>> Because the "Entity" denotes that you are inside the AFile rules which
>> do not recognise a "TOP" keyword.
>> So IIRC there is no conflict at all.
>
> Hm, that's what I thought at first, too. But if I open file A with my
> Runtime Eclipse, then all occurences of 'TOP' are recognized as a
> keyword, causing an error ("Mismatched input 'TOP', expecting RULE_ID").
> My standalone test tool reports the same syntax error.
>
>
> In the standard XTEXT terminals, ther is a build-in escape for IDs
> implemented:
> a "^" in front of the ID which separetes the ID from a keyword is
> removed automatically.
> So entering name=^TOP will be turned into a simple TOP
>
> Yes, I am aware of that. Unfortunately I cannot modify the source files.
> The whole file format is a lot more complex, so I really need to avoid
> messing around with file contents manually.
--
Need professional support for Xtext or other Eclipse Modeling technologies?
Go to: http://xtext.itemis.com
Twitter : @svenefftinge
Blog : http://blog.efftinge.de
|
|
|
|
|
|
| Re: [Xtext 2.4.3] What to do if 'keyword' can occur as an ID as well? [message #1219531 is a reply to message #1219518] |
Wed, 04 December 2013 07:29 |
| Eclipse User |
|
|
|
Hi Ed,
thanks for pointing out a context where this will cause trouble.
I had not encountered this situation in the legacy areas I inspected,
just a simple KEYWORD myname KEYWORD1= abc, def, xyz
where the second keyword is "KEYWORD1="
So as long as a situation like the one you mentioned can be excluded,
it seems to be an acceptable way to specify the first KEYWORD as "KEYWORD ".
This way, an existing DSL will still be useable if it reads:
KEYWORD KEYWORD KEYWORD1= KEYWORD, KEYWORD1
[Updated on: Wed, 04 December 2013 10:21] by Moderator |
|
|
Powered by
FUDForum. Page generated in 0.30076 seconds
Home