Eclipse Community Forums: TMF (Xtext) » Backslashes as line ending characters

Home » Modeling » TMF (Xtext) » Backslashes as line ending characters

Backslashes as line ending characters [message #1045133]

Fri, 19 April 2013 16:44

Eclipse User

Greetings Xtext world,

Is it possible to implement a language in Xtext that supports optional backslash line endings? Our legacy language needs to be able to put a backslash anywhere in the file to signal to the external compiler tools that it should continue parsing on the next line. I would eventually like to implement a formatter that adds these backslash line endings to help organize long lines of similar data into a column.

I tried to include a backslash in the whitespace rule, which has the obvious consequence that Xtext does not care if anything comes after the backslash - it's hidden after all. I then thought I could implement a value converter for the whitespace rule and check that only a newline character exists after the backslash, but the corresponding toValue(String string, INode node) method is never called. This is probably because the whitespace rule is hidden.

I need to be able to detect the backslash character and then flag an error or warning in the editor if anything besides a newline occurs after the backslash. Does anyone know if this is possible in Xtext or could offer some advice?

Thanks in advance!
Brad

Re: Backslashes as line ending characters [message #1045599 is a reply to message #1045133]

Sat, 20 April 2013 10:30

Eclipse User

Did you try having only ('\\' '\n') | ('\n') as newline? I.e. only allow
newline or backslash newline as "a newline"?

If you want the newlines to be separate tokens (i.e. backslash newline
is two tokens), or if you accept whites pace between the backslash and
the newline it gets a bit more complicated and you may need to use an
external lexer where you can perform both look ahead and have syntactic
predicates.

Regards
- henrik

On 2013-19-04 22:44, Brad Riching wrote:
> Greetings Xtext world,
>
> Is it possible to implement a language in Xtext that supports optional
> backslash line endings? Our legacy language needs to be able to put a
> backslash anywhere in the file to signal to the external compiler tools
> that it should continue parsing on the next line. I would eventually
> like to implement a formatter that adds these backslash line endings to
> help organize long lines of similar data into a column.
>
> I tried to include a backslash in the whitespace rule, which has the
> obvious consequence that Xtext does not care if anything comes after the
> backslash - it's hidden after all. I then thought I could implement a
> value converter for the whitespace rule and check that only a newline
> character exists after the backslash, but the corresponding
> toValue(String string, INode node) method is never called. This is
> probably because the whitespace rule is hidden.
>
> I need to be able to detect the backslash character and then flag an
> error or warning in the editor if anything besides a newline occurs
> after the backslash. Does anyone know if this is possible in Xtext or
> could offer some advice?
>
> Thanks in advance!
> Brad
>
>

Re: Backslashes as line ending characters [message #1046939 is a reply to message #1045599]

Mon, 22 April 2013 11:19

Eclipse User

Thanks Henrik!

With your help I came up with the following whitespace rule (for windows machines only):

terminal WS: 
	(' '|'\t'|'\r'|'\n'| ('\\' (' '|'\t')* '\r' '\n'))+;

Now in my formatter, I would like to be able to add a backslash along with the native line ending characters after some elements of my model. In my case, I have a description of a wiring harness connector, and a list of all of its pins -- all of which are on one line. The list contains comma-separated names and numbers that I would like to appear indented and properly formatted with a backslash line ending sequence after each pin. Essentially I would like to take the entire line and transform it into a column of pins in the connector. Do you know if this is possible with the formatter API that ships with Xtext or if will I need to write my own?

Re: Backslashes as line ending characters [message #1047256 is a reply to message #1046939]

Mon, 22 April 2013 21:19

Eclipse User

On 2013-22-04 17:19, Brad Riching wrote:
> Thanks Henrik!
>
>
> With your help I came up with the following whitespace rule (for windows
> machines only):
>
>
> terminal WS: (' '|'\t'|'\r'|'\n'| ('\\' (' '|'\t')* '\r' '\n'))+;
>
>
> Now in my formatter, I would like to be able to add a backslash along
> with the native line ending characters after some elements of my model.
> In my case, I have a description of a wiring harness connector, and a
> list of all of its pins -- all of which are on one line. The list
> contains comma-separated names and numbers that I would like to appear
> indented and properly formatted with a backslash line ending sequence
> after each pin. Essentially I would like to take the entire line and
> transform it into a column of pins in the connector. Do you know if
> this is possible with the formatter API that ships with Xtext or if will
> I need to write my own?

afaik, you can't use the default formatter to insert anything but
whitespace and linebreaks, so it would need to go into the serializer.
You may be able to combine the serializer with the formatter so that you
only have to deal with this special aspect of serialization and
formatting. One problem is naturally to both have whitespace preserving
rules (you want to keep the inserted whitspace) and formatting at the
same time.

Unless you have a complex language and complicated formatting, the best
option is probably to have the formatting done as part of the serializer
(i.e. always format it the way you want it).

Regards
- henrik

Re: Backslashes as line ending characters [message #1047421 is a reply to message #1047256]

Tue, 23 April 2013 03:21

Eclipse User

Henrik Lindberg wrote on Tue, 23 April 2013 03:19

afaik, you can't use the default formatter to insert anything but
whitespace and linebreaks, so it would need to go into the serializer.

c.setSpace("\\\n").after(...)

Or am I missing something here?

Re: Backslashes as line ending characters [message #1047636 is a reply to message #1047421]

Tue, 23 April 2013 09:13

Eclipse User

On 2013-23-04 9:21, Claudio Heeg wrote:
> Henrik Lindberg wrote on Tue, 23 April 2013 03:19
>> afaik, you can't use the default formatter to insert anything but
>> whitespace and linebreaks, so it would need to go into the serializer.
>
> c.setSpace("\\\n").after(...)
> Or am I missing something here?

Maybe I did :)

- henrik

Re: Backslashes as line ending characters [message #1049423 is a reply to message #1047636]

Thu, 25 April 2013 17:05

Eclipse User

Once again I'm simply amazed with Xtext. Thanks for the help!

Re: Backslashes as line ending characters [message #1745508 is a reply to message #1047421]

Tue, 11 October 2016 15:02

Eclipse User

I know this was three years ago that this thread was being discussed, but I am also interested in being able to correctly process with Xtext "\" at the end of a line like the C pre-processor does. I am novice to Xtext and I don't understand what:

c.setSpace("\\\n").after(...)

does or where it should be placed. The serializer is mentioned, but when I look at the files in my Xtext language serializer folder (2.10), I don't see any code similar to this.

Could someone explain to me the neophyte what this setSpace does and where it could be placed?

Thanks,
Larry

Re: Backslashes as line ending characters [message #1745509 is a reply to message #1745508]

Tue, 11 October 2016 15:27

Eclipse User

Can you elaborate on what you want to actually want to do
The code mentioned Obote goes to the (old) formatter (1) code
To get stuff for the serializer generated you need to configure the workflow
Sth like (serializer={generatextendstub=true})

So still the question is this about parsing or about serialist ion/formatting

Previous Topic:	Thank you!
Next Topic:	Strange Serialization

Goto Forum:

-=] Back to Top [=-

Current Time: Sat Jul 26 12:56:17 EDT 2025

.:: Contact :: Home ::.

Breadcrumbs

Sign up to our Newsletter