| Terminals with whitespace [message #1062987] |
Tue, 11 June 2013 11:47  |
Eclipse User |
|
|
|
I am trying to create an Xtext project for a language whose general form is a set of comma separated lists. For example:
ObjectName,
Field 1, !- Comment describing field 1
Field 2, !- Comment describing field 2
Field N; !- Comment describing filed N
Here is my problem: The fields can be any alphanumeric string and may contain spaces. So I originally created the terminal this way:
terminal ALPHANUMERIC :
(
('0'..'9' | 'A'..'Z' | 'a'..'z') |
('0'..'9' | 'A'..'Z' | 'a'..'z') ('0'..'9' | 'A'..'Z' | 'a'..'z' | ' ')* ('0'..'9' | 'A'..'Z' | 'a'..'z')
)
;
Whitespace is allowed on either side of the field (the actual string will trim off whitespace between the fields and commas).
The problem is that when I use this, and have a space between the field and the comma, e.g.:
The parser interprets the space as part of the alphanumeric string and complains because it thinks the field is not complete (i.e. it does not end in a non-space character).
Anyone have a suggestion of how to get around this problem?
|
|
|
| Re: Terminals with whitespace [message #1063002 is a reply to message #1062987] |
Tue, 11 June 2013 13:22  |
| Eclipse User |
|
|
|
If you are using the default terminals, you will end up with
overlapping terminals, and lots of problems.
Why not simply use a data rule - e.g. something like:
AlphaNumeric : (ID | Number | NonCommaPunctuation)+ ;
NonCommaPunctuation : ':' | ';' | ... ;
....
Regards
- henrik
On 2013-11-06 17:47, Neal Kruis wrote:
> I am trying to create an Xtext project for a language whose general form
> is a set of comma separated lists. For example:
>
>
> ObjectName,
> Field 1, !- Comment describing field 1
> Field 2, !- Comment describing field 2
> Field N; !- Comment describing filed N
>
>
> Here is my problem: The fields can be any alphanumeric string and may
> contain spaces. So I originally created the terminal this way:
>
>
> terminal ALPHANUMERIC :
> ( ('0'..'9' | 'A'..'Z' | 'a'..'z') |
> ('0'..'9' | 'A'..'Z' | 'a'..'z') ('0'..'9' | 'A'..'Z' | 'a'..'z' |
> ' ')* ('0'..'9' | 'A'..'Z' | 'a'..'z')
> )
> ;
>
>
> Whitespace is allowed on either side of the field (the actual string
> will trim off whitespace between the fields and commas).
>
> The problem is that when I use this, and have a space between the field
> and the comma, e.g.:
>
>
> Field X ,
>
>
> The parser interprets the space as part of the alphanumeric string and
> complains because it thinks the field is not complete (i.e. it does not
> end in a non-space character).
>
> Anyone have a suggestion of how to get around this problem?
|
|
|
|
|
|
|
|
|
|
| Re: Terminals with whitespace [message #1063233 is a reply to message #1063226] |
Wed, 12 June 2013 14:33 |
| Eclipse User |
|
|
|
Ok. Thanks for all of your patience with me as I'm still climbing the Xtext learning curve.
I'm still having the same issue that I originally had where, because whitespace (specifically tabs and spaces) is allowed in the alphanumeric filed, the parser thinks any whitespace between the Field token and the following comma is going to be a continuation of the Field token and I get an error because it doesn't detect the ending alphanumeric character.
Here is what I have:
InputDefinition hidden(SL_COMMENT, WS):
objects += Object*
;
Object:
fields+=Field','
(fields+=Field',')*
fields+=Field';'
;
Field:
ALPHANUMERIC
;
terminal ALPHANUMERIC :
('0'..'9' | 'A'..'Z' | 'a'..'z')* (('\t' | ' ')+ ('0'..'9' | 'A'..'Z' | 'a'..'z')+)*
;
terminal SL_COMMENT : '!' !('\n'|'\r')* ('\r'? '\n')?;
terminal WS : (' '|'\t'|'\r'|'\n')+;
terminal OTHER : .;
So in this example:
Object,
Field 1 ,
Field 2,
Filed N;
I get an error because of the space after field 1 which the parser is interpreting as an incomplete addition to the field token instead of hidden whitespace before the comma.
I hope that makes sense. Please let me know if I can clarify anything in my example.
[Updated on: Wed, 12 June 2013 14:33] by Moderator |
|
|
| Re: Terminals with whitespace [message #1063237 is a reply to message #1063233] |
Wed, 12 June 2013 15:05 |
| Eclipse User |
|
|
|
Hi,
still dont know why you care about ws at all the flowing seem to work for me
grammar org.xtext.example.mydsl1.MyDsl hidden(SL_COMMENT, WS)
import "http://www.eclipse.org/emf/2002/Ecore"
generate myDsl "http://www.xtext.org/example/mydsl1/MyDsl"
InputDefinition:
objects += Object*
;
Object:
fields+=Field','
(fields+=Field',')*
fields+=Field';'
;
Field:
ALPHANUMERIC+
;
terminal ALPHANUMERIC :
('0'..'9' | 'A'..'Z' | 'a'..'z')+
;
terminal SL_COMMENT : '!' !('\n'|'\r')* ('\r'? '\n')?;
terminal WS : (' '|'\t'|'\r'|'\n')+;
terminal OTHER : .; |
|
|
Powered by
FUDForum. Page generated in 0.05699 seconds
Home