Eclipse Community Forums: TMF (Xtext) » Question regarding lexer and IDValueConverter

Help

Home

Home » Modeling » TMF (Xtext) » Question regarding lexer and IDValueConverter

Show: Today's Messages :: Show Polls :: Message Navigator

Question regarding lexer and IDValueConverter [message #638149]

Wed, 10 November 2010 08:16

Eclipse User

Originally posted by: koester.matthias.gmx.de

Hi,

I have some problems when using keywords of my language as identifiers
in my grammar. Is it possible to use the
IDValueConverter/AbstractLexerBasedConverter class to cope with this
problem? Sonce '^' is already a keyword in my language I think I then
have to change the default impl.
After looking at the source code I haven't seen if this conversion is
performed before the lexer scans the input stream, but perhaps I didn't
get it. Would be nice if someone could give me a hint ;-)

Best regards,
Matthias

Report message to a moderator

Re: Question regarding lexer and IDValueConverter [message #638159 is a reply to message #638149]

Wed, 10 November 2010 09:02

Alexander Nittka

Messages: 1193
Registered: July 2009

Senior Member

Hi,

you could redefine the ID terminal rule using a different escape character and adapt the value converter accordingly.
Have a look at IDValueConverter, AbstractIDValueConverter and DefaultTerminalConverters. I guess you'd bind a class extending IDValueConverter overriding toEscapedString and toValue using your own escape character (if that's the only change of your new ID rule).

Another possibility (not recommended) for using keywords as IDs (directly) is to introduce a datatype rule and use that rule wherever you would use ID.

IdIncludingKeywords: ID|'keyword1'|'keyword2'|...;

The problem is that a keyword will be highlighted as such wherever it appears (even if it is used as id). You would have to change that by customising semantic highlighting.

Alex

[Updated on: Wed, 10 November 2010 09:08]

Report message to a moderator

Re: Question regarding lexer and IDValueConverter [message #638647 is a reply to message #638149]

Thu, 11 November 2010 23:26

Henrik Lindberg

Messages: 2509
Registered: July 2009

Senior Member

The ^ is removed by the default value converter for the ID terminal. If
you redefine ID to not include the ^ you also need to provide a
different value converter.

An alternative is to use a construct such as
ID | "keyword1" | "keyword2" | ...
in your grammar for keywords that are acceptable as ID in some context.
When doing so you also need to add syntax coloring since the kw will get
kw terminal styling.
Regards
- henrik

Matthias Köster <koester.matthias@gmx.de> wrote:
> Hi,
>
> I have some problems when using keywords of my language as identifiers
> in my grammar. Is it possible to use the
> IDValueConverter/AbstractLexerBasedConverter class to cope with this
> problem? Sonce '^' is already a keyword in my language I think I then
> have to change the default impl.
> After looking at the source code I haven't seen if this conversion is
> performed before the lexer scans the input stream, but perhaps I
> didn't get it. Would be nice if someone could give me a hint ;-)
>
> Best regards,
> Matthias

--
- henrik

Report message to a moderator

Previous Topic:	XtextEditor + editor input which is not a file
Next Topic:	How to set xtextEditor readonly.

Goto Forum:

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

]

Current Time: Fri Apr 26 04:31:00 GMT 2024

.:: Contact :: Home ::.

Breadcrumbs

Sign up to our Newsletter