|
| Re: Terminals with whitespace [message #1063002 is a reply to message #1062987] |
Tue, 11 June 2013 17:22   |
 Henrik Lindberg
Messages: 2509
Registered: July 2009 |
Senior Member |
|
|
If you are using the default terminals, you will end up with
overlapping terminals, and lots of problems.
Why not simply use a data rule - e.g. something like:
AlphaNumeric : (ID | Number | NonCommaPunctuation)+ ;
NonCommaPunctuation : ':' | ';' | ... ;
....
Regards
- henrik
On 2013-11-06 17:47, Neal Kruis wrote:
> I am trying to create an Xtext project for a language whose general form
> is a set of comma separated lists. For example:
>
>
> ObjectName,
> Field 1, !- Comment describing field 1
> Field 2, !- Comment describing field 2
> Field N; !- Comment describing filed N
>
>
> Here is my problem: The fields can be any alphanumeric string and may
> contain spaces. So I originally created the terminal this way:
>
>
> terminal ALPHANUMERIC :
> ( ('0'..'9' | 'A'..'Z' | 'a'..'z') |
> ('0'..'9' | 'A'..'Z' | 'a'..'z') ('0'..'9' | 'A'..'Z' | 'a'..'z' |
> ' ')* ('0'..'9' | 'A'..'Z' | 'a'..'z')
> )
> ;
>
>
> Whitespace is allowed on either side of the field (the actual string
> will trim off whitespace between the fields and commas).
>
> The problem is that when I use this, and have a space between the field
> and the comma, e.g.:
>
>
> Field X ,
>
>
> The parser interprets the space as part of the alphanumeric string and
> complains because it thinks the field is not complete (i.e. it does not
> end in a non-space character).
>
> Anyone have a suggestion of how to get around this problem?
|
|
|
|
|
| Re: Terminals with whitespace [message #1063044 is a reply to message #1063027] |
Tue, 11 June 2013 21:20 |
Neal Kruis
Messages: 15
Registered: October 2012 |
Junior Member |
|
|
The problem is that I don't want spaces hidden from the tokens.
I want the parser to be able to recognize everything between the first non-whitespace character and the last non-whitespace character (including spaces) as a field. All the other whitespace between the commas (or semicolon if it's the last field) should be hidden.
[Updated on: Tue, 11 June 2013 21:27] Report message to a moderator |
|
|
|
|
|
|
| Re: Terminals with whitespace [message #1063233 is a reply to message #1063226] |
Wed, 12 June 2013 18:33 |
Neal Kruis
Messages: 15
Registered: October 2012 |
Junior Member |
|
|
Ok. Thanks for all of your patience with me as I'm still climbing the Xtext learning curve.
I'm still having the same issue that I originally had where, because whitespace (specifically tabs and spaces) is allowed in the alphanumeric filed, the parser thinks any whitespace between the Field token and the following comma is going to be a continuation of the Field token and I get an error because it doesn't detect the ending alphanumeric character.
Here is what I have:
InputDefinition hidden(SL_COMMENT, WS):
objects += Object*
;
Object:
fields+=Field','
(fields+=Field',')*
fields+=Field';'
;
Field:
ALPHANUMERIC
;
terminal ALPHANUMERIC :
('0'..'9' | 'A'..'Z' | 'a'..'z')* (('\t' | ' ')+ ('0'..'9' | 'A'..'Z' | 'a'..'z')+)*
;
terminal SL_COMMENT : '!' !('\n'|'\r')* ('\r'? '\n')?;
terminal WS : (' '|'\t'|'\r'|'\n')+;
terminal OTHER : .;
So in this example:
Object,
Field 1 ,
Field 2,
Filed N;
I get an error because of the space after field 1 which the parser is interpreting as an incomplete addition to the field token instead of hidden whitespace before the comma.
I hope that makes sense. Please let me know if I can clarify anything in my example.
[Updated on: Wed, 12 June 2013 18:33] Report message to a moderator |
|
|
| Re: Terminals with whitespace [message #1063237 is a reply to message #1063233] |
Wed, 12 June 2013 19:05 |
|
Hi,
still dont know why you care about ws at all the flowing seem to work for me
grammar org.xtext.example.mydsl1.MyDsl hidden(SL_COMMENT, WS)
import "http://www.eclipse.org/emf/2002/Ecore"
generate myDsl "http://www.xtext.org/example/mydsl1/MyDsl"
InputDefinition:
objects += Object*
;
Object:
fields+=Field','
(fields+=Field',')*
fields+=Field';'
;
Field:
ALPHANUMERIC+
;
terminal ALPHANUMERIC :
('0'..'9' | 'A'..'Z' | 'a'..'z')+
;
terminal SL_COMMENT : '!' !('\n'|'\r')* ('\r'? '\n')?;
terminal WS : (' '|'\t'|'\r'|'\n')+;
terminal OTHER : .;
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
|
|
|
Powered by
FUDForum. Page generated in 0.04941 seconds
] 
Search
Help
Register
Login
Home
