Eclipse Community Forums: TMF (Xtext) » Sharing how to override the lexer

Help

Home

Home » Modeling » TMF (Xtext) » Sharing how to override the lexer

Show: Today's Messages :: Show Polls :: Message Navigator

Sharing how to override the lexer [message #1776411]

Thu, 16 November 2017 09:38

Markus Holzem

Messages: 7
Registered: November 2017

Junior Member

Hi,

I have searched forums, blogs, and a lot of other places how to enhance Xtext with a precompiler. The only post I found was https://www.eclipse.org/forums/index.php/t/366839/ pointing me a bit in the right direction.

After a longer time of trial and error I found a way to override the lexer that is generated by Xtext using some internal knowledge how Xtext seems to work and think. I found

a way to trace the tokens that are generated by the lexer. This makes it far easier to understand what goes wrong in a DSL definition.
a way to insert, split, and redefine tokens in the token stream, giving them additional synthetic terminal types.
a way to insert tokens with zero length making it possible to keep the inserted tokens in sync with the editor.

What is the best way to share the information? After searching the web I think there is some demand to find some answers to these questions and I would preferably like to add something to the Xtext documentation instead of putting it somewhere on the web in a separate blog.

So - what is the best way to share?

Markus

Report message to a moderator

Re: Sharing how to override the lexer [message #1776413 is a reply to message #1776411]

Thu, 16 November 2017 09:59

Karsten Thoms

Messages: 762
Registered: July 2009
Location: Dortmund, Germany

Senior Member

Hi Markus! The Xtext manual is written in a manner to describe general concepts and common extension machanisms. You are solving a very specific use case. We usually don't add such things to the manual.

To share your solution approch you might consider creating a minimal sample project on GitHub and write the challenge and the specific solution to the project's README.MD. You may write a blog post about that, too, and pointing to your example project.

Would be great if you would do this!

~Karsten

Report message to a moderator

Previous Topic:	Custom validation of constraints from existing metamodel
Next Topic:	Loading file-based ecore files from xText DSML

Goto Forum:

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

]

Current Time: Fri Apr 26 21:29:44 GMT 2024

.:: Contact :: Home ::.

Breadcrumbs

Sign up to our Newsletter