Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » TMF (Xtext) » Sharing how to override the lexer
Sharing how to override the lexer [message #1776411] Thu, 16 November 2017 09:38 Go to next message
Markus Holzem is currently offline Markus HolzemFriend
Messages: 7
Registered: November 2017
Junior Member
Hi,

I have searched forums, blogs, and a lot of other places how to enhance Xtext with a precompiler. The only post I found was https://www.eclipse.org/forums/index.php/t/366839/ pointing me a bit in the right direction.

After a longer time of trial and error I found a way to override the lexer that is generated by Xtext using some internal knowledge how Xtext seems to work and think. I found

  • a way to trace the tokens that are generated by the lexer. This makes it far easier to understand what goes wrong in a DSL definition.
  • a way to insert, split, and redefine tokens in the token stream, giving them additional synthetic terminal types.
  • a way to insert tokens with zero length making it possible to keep the inserted tokens in sync with the editor.

What is the best way to share the information? After searching the web I think there is some demand to find some answers to these questions and I would preferably like to add something to the Xtext documentation instead of putting it somewhere on the web in a separate blog.

So - what is the best way to share?

Markus
Re: Sharing how to override the lexer [message #1776413 is a reply to message #1776411] Thu, 16 November 2017 09:59 Go to previous message
Karsten Thoms is currently offline Karsten ThomsFriend
Messages: 618
Registered: July 2009
Location: Dortmund, Germany
Senior Member

Hi Markus! The Xtext manual is written in a manner to describe general concepts and common extension machanisms. You are solving a very specific use case. We usually don't add such things to the manual.

To share your solution approch you might consider creating a minimal sample project on GitHub and write the challenge and the specific solution to the project's README.MD. You may write a blog post about that, too, and pointing to your example project.

Would be great if you would do this!

~Karsten
Previous Topic:Custom validation of constraints from existing metamodel
Next Topic:Loading file-based ecore files from xText DSML
Goto Forum:
  


Current Time: Wed Sep 26 08:22:53 GMT 2018

Powered by FUDForum. Page generated in 0.01531 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top