| Using ANTLR or Xtext for YAML editor [message #464172] |
Sun, 02 August 2009 14:28  |
Eclipse User |
|
|
|
Hi,
I am implementing an YAML (www.yaml.org) editor in Eclipse called YEdit
(http://code.google.com/p/yedit/).
The editor currently uses two parsers. One for the syntax high-lighting
and one to generate and outline view and to check the syntax.
The parser that is used for syntax highlighting is a rule based scanner
that works ok for its purpose, but it is not powerful enough to correctly
parse all YAML. It doesn't seem to be practical to extend the parser so
that it can parse all YAML either.
The parser used for the outline view and the syntax checking correctly
parses all YAML, but is meant to be used for static files. This has been
ok so far, but I have some ideas for new features that requires that the
editor is aware of the structure of the file continuously. Requiring a
complete re-parse of the entire file for each change seems impractical.
For these reason I have plans to either create a new YAML parser in ANTLR
or rewrite the editor to use Xtext. The problem is that it is hard for me
to evaluate which approach will be most sensible and I hope that someone
here could provide some insights. Some questions that are relevant.
1. Can Xtext correctly parse a language where whitespaces are significant?
Like YAML and Python.
2. How often does Xtext update the underlying model for a file?
3. Do ANTLR provide direct support for creating an incremental parser or
is this something that requires a bit of hacking.
4. Do Xtext perform well for large files (10000 lines or more) and complex
grammars?
5. Can error messages for syntax errors be tailored in Xtext to provide
the user with more help?
6. Any recommondations or other things that you can think of?
- Øystein Torget
|
|
|
| Re: Using ANTLR or Xtext for YAML editor [message #464678 is a reply to message #464172] |
Sun, 02 August 2009 16:03  |
| Eclipse User |
|
|
|
In my opinion, Xtext would be just the perfect choice for YAML! You could
avoid the duplication of parsers since Xtext infers everything from your
grammar definition, and has all the features (and more) that currently
YEdit has. Unfortunately AFAIK there is no EBNF specification for YAML,
but since you managed to write the ANTLR parser, it is not impossible, so
good luck!
Regarding the first question (whitespace significance), this is of course
possible, read Chapter 2 of
http://www.eclipse.org/Xtext/documentation/latest/xtext.html - especially
the section Terminal rules. I don't know the other answers, but I would be
interested in them, too. :)
thSoft
|
|
|
| Re: Using ANTLR or Xtext for YAML editor [message #467846 is a reply to message #464172] |
Mon, 03 August 2009 03:08 |
| Eclipse User |
|
|
|
Hi Øystein,
please find my answer below.
Am 02.08.2009 20:28 Uhr, schrieb Øystein Torget:
> Hi,
>
> I am implementing an YAML (www.yaml.org) editor in Eclipse called YEdit
> (http://code.google.com/p/yedit/).
> The editor currently uses two parsers. One for the syntax high-lighting
> and one to generate and outline view and to check the syntax.
> The parser that is used for syntax highlighting is a rule based scanner
> that works ok for its purpose, but it is not powerful enough to
> correctly parse all YAML. It doesn't seem to be practical to extend the
> parser so that it can parse all YAML either.
> The parser used for the outline view and the syntax checking correctly
> parses all YAML, but is meant to be used for static files. This has been
> ok so far, but I have some ideas for new features that requires that the
> editor is aware of the structure of the file continuously. Requiring a
> complete re-parse of the entire file for each change seems impractical.
>
> For these reason I have plans to either create a new YAML parser in
> ANTLR or rewrite the editor to use Xtext. The problem is that it is hard
> for me to evaluate which approach will be most sensible and I hope that
> someone here could provide some insights. Some questions that are relevant.
> 1. Can Xtext correctly parse a language where whitespaces are
> significant? Like YAML and Python.
That is possible, but I guess it is quite a tough task. As far as I know
the ANTLR book contains an example on how to do this with ANTLR. Well I
think that it is not possibly to apply the trick with Lexer#emit on
Xtext generated grammars too but it should be possible to find a way to
describe indent and dedent.
> 2. How often does Xtext update the underlying model for a file?
That depends on how often you modify your input document. We track the
changes in the document and if the delay between typing and waiting is
long enough, Xtext parses the affected part of the document and merges
to previous semantic model with the newly created objects.
> 3. Do ANTLR provide direct support for creating an incremental parser or
> is this something that requires a bit of hacking.
You can call any ANTLR rule directly from the parser, but it is helpful
to be sure about the required lookahead and the state of the decision
tree. The answer is: is requires a bit of hacking.
> 4. Do Xtext perform well for large files (10000 lines or more) and
> complex grammars?
Xtext's performance does not depend on the complexity of your grammar
(althoug it is definitly helpful to provide a grammar that does not
require backtracking ...). Depending on your linking and validation
algorithms Xtext can handle this size more or less gracefully. We had
tests with 80.000 lines of code and it was possible to edit this files
altough there was some kind of latency.
> 5. Can error messages for syntax errors be tailored in Xtext to provide
> the user with more help?
That is not possible at the moment for error messages from the parser
and the linker. Validation messages and under your control.
> 6. Any recommondations or other things that you can think of?
>
I don't know YAML that well so currently there are no other things that
I can mention except from: Good luck :)
If you have any further questions, feel free to ask them.
> - Øystein Torget
>
Hope that helps,
Sebastian
--
Need professional support for Eclipse Modeling?
Go visit: http://xtext.itemis.com
|
|
|
|
|
| Re: Using ANTLR or Xtext for YAML editor [message #1389893 is a reply to message #479121] |
Sat, 28 June 2014 10:31 |
| Eclipse User |
|
|
|
Congratulations on the success of yedit.
I see that you did indeed use antlr. Wondering if you tried using xtext. If so, what issues did you encounter? I'm guessing that lexing was a challenge.
The motivation for these questions is that I'd like to leverage xtext for an EMF resource that reads and writes ECore models as YAML. Any insights on using xtext would be helpful.
[Updated on: Mon, 30 June 2014 16:29] by Moderator |
|
|
Powered by
FUDForum. Page generated in 0.07551 seconds
Home