|Python-like indentation and editor problems [message #1015898]
||Mon, 04 March 2013 08:38
| Shavarsh Odajavan
Registered: March 2013
Hi, i'm writing a DSL which handles blocks in a (modified) Python-like way. A block starts with a newline followed by a number of tabs, and each tab added is one more nested level.|
I'm using the AbstractSplittingTokenSource class (IndentTokenSource.java file) to handle indents/dedents and everything works but sometimes i'm having trouble with the editor.
In particular, the artificial indent/dedent tokens have to be set in their attributes (text, type, line, char position in line, start index, stop index) to create a consistent node model but they are "invisible" to the editor. This causes the editor to behave in a strange way (fake error highlighting even if no error is reported by the console, shifted highliting for errors, outline shifted highliting etc.).
My opinion is these problems come out because artificial tokens are overlapped with other tokens.
I would like to ask if someone has figured out how to get this working normally.
Project can be found in the attachment
Thanks in advance
|Re: Python-like indentation and editor problems [message #1052910 is a reply to message #1015898]
||Wed, 01 May 2013 09:24
| Mark Butler
Registered: May 2013
I am also interested in writing an indentation based language like Shavarsh. Thanks Sharvash for posting such a complete example. Unfortunately I can't get it the example, or a number of others, to work. |
I've been looking back through the Eclipse email lists, and also seen this project
http : // code.google.com/a/eclipselabs.org/p/todotext/
(sorry not to post full URLs, but this forum won't let me)
Ralf Ebert's code for Todotext mentioned above is no longer available from Github but
I discovered it is in the Eclipse test repository.
http : // eclipse.googlesource.com/tmf/org.eclipse.xtext/+/master/tests/org.eclipse.xtext.tests/src/org/eclipse/xtext/testlanguages/indent/
I also found the original thread about the work
and the Eclipse issue raised about it
I also found an alternative implementation here
http : // eclipse-snippets.googlecode.com/svn/trunk/xtext-indenting/org.xtext.example.mydsl/src/org/xtext/example/
I have tried all three versions but I cannot get them to work. In the issue Ralf raised, he said there was a problem due to "partial parsing" because the indent sensitive language needs a stateful lexer. He then proposed a series of patches, but looking at the XText code. When I run the lexer in the debugger, it does seem to be doing some kind of partial parsing, and this leads to the fake errors that Sharvash describes.
I could not find the files that Ralf patches to solve this problem, so I wonder if those changes seem long gone? Can anybody confirm is this the case?
There is interest in parsing indentation based grammars as the question has come up on Stackoverflow -
http : // stackoverflow.com/questions/7167834/xtext-grammar-for-language-with-significant-semantic-whitespace
Other tools like Parboiled do support this "out-the-box"
https : // github.com/sirthias/parboiled/wiki/Indentation-Based-Grammar
so it would be great if XText could do the same.
Powered by FUDForum
. Page generated in 0.02896 seconds