Home » Modeling » TMF (Xtext) » [Xtext] Formatter/Pretty Printer needed
| [Xtext] Formatter/Pretty Printer needed [message #47545] |
Wed, 03 June 2009 13:27  |
Eclipse User |
|
|
|
Hello,
I'm changing the Xtext document content via the following piece of (speudo)
code
->snip
....<some model manipulations of the resource>...
ByteArrayOutputStream out = new ByteArrayOutputStream();
XTextResource.save(out, null);
XtextDocument.set(out.toString());
<- snip
and get the whole content reformatted.
Everything is put into the first line of the editor! Also, parentheses and
semicolons are added where I don't expect it and some elements are reordered
in a different way.
I'm using the Eclipse Modeling IDE RC2 from last Monday. This behaviour is
even worse than e.g. in MS7 that I had before where at least the rough
formatting was kept.
What can I do in order to prevent the addition of parentheses and semicolons
and the reordering of elements (note that the document is valid also without
these parentheses and semicolons)?
Is there a pretty printer or a formatter available?
Ciao, Michael
|
|
| |
| Re: [Xtext] Formatter/Pretty Printer needed [message #47846 is a reply to message #47635] |
Thu, 04 June 2009 06:04  |
| Eclipse User |
|
|
|
Hello Sebastian,
I have an initial, nicely formatted model. An instance mmtest of MyObject
contains an instance mmtestSubObject of MySubObject and this one contains 2
other instances: mmtestSubSubObject1 of MySubSubObject1 &
mmtestSubSubObject2 of MySubSubObject2. Everything syntactically correct:
MyObject mmtest {
MySubObject mmtestSubObject {
MySubSubObject1 mmtestSubSubObject1 ;
MySubSubObject2 mmtestSubSubObject2 {
}
}
}
Then I programmatically add a new instance newSubSubObject1 of
MySubSubObject1 under mmtestSubObject and under mmtestSubSubObject2 via the
code below (....<some model manipulations of the resource>...) and then the
editor content looks like this:
MyObject mmtest { MySubObject mmtestSubObject { MySubSubObject2
mmtestSubSubObject2 { MySubSubObject1 newSubSubObject1 ; } ; MySubSubObject1
mmtestSubSubObject1 { } ; MySubSubObject1 newSubSubObject1 { } ; } ; } ;
Everything is in the first line, additionally I get several new parenthesis
and semicolons, also the elements are reordered. This can be seen if one
manually reformats this line to:
MyObject mmtest {
MySubObject mmtestSubObject {
MySubSubObject2 mmtestSubSubObject2 {
MySubSubObject1 newSubSubObject1 ;
} ;
MySubSubObject1 mmtestSubSubObject1 { } ;
MySubSubObject1 newSubSubObject1 { } ; <- 2 new parentheses "{ }" &
position changed, now below mmtestSubSubObject2
} ;
} ;
You can see that after newSubSubObject1 I get 2 parentheses "{ }" with a
space between, one sees some new semicolons and the element mentioned is now
below the element mmtestSubSubObject2.
Unfortunately I can not post the grammar here, but nevertheless, it should
be reproducable for you, at least the semocolons and the parentheses in a
sample code.
Debugging shows that the statement XtextResource.save below already creates
that single line in the buffer "out" and the XtextDocument simply shows it
in the editor. It seams that the resource save call throws away the
formatting.
As I could just see now, also comments are completely thrown away :( Test it
out!
Thanks in advance and ciao, Michael
"Sebastian Zarnekow" <Sebastian.Zarnekow@itemis.de> wrote in message
news:h06gd2$7ib$1@build.eclipse.org...
> Hi Michael,
>
> we are not aware of any concrete problems with the current (recently
> refactored and improved) implementation of the serializer. Can you be more
> specific here and provide a short sample that illustrates the described
> problem.
>
> What do you mean by "addition of parentheses and semicolons"? Is the
> serialized model not parseable? Does it contain any errors?
>
> As far as I know, whitespace handling will be improved in RC4.
>
> Sorry for inconvenience,
> Sebastian
>
> Am 03.06.2009 19:27 Uhr, schrieb Michael M
|
|
|
| Re: [Xtext] Formatter/Pretty Printer needed [message #47876 is a reply to message #47846] |
Thu, 04 June 2009 06:11 |
| Eclipse User |
|
|
|
Hi Michael,
thanks for your detailed description.
Reordering of elements is clearly a bug, as it may change the semantics
of your model.
Formatting and pretty printing is not yet supported by
XtextResource.save(..).
I'm confident that some improvements will make it into the next RC4,
which will be released on Tuesday.
Regards,
Sebastian
Am 04.06.2009 12:04 Uhr, schrieb Michael Mühlberg:
> Hello Sebastian,
>
> I have an initial, nicely formatted model. An instance mmtest of MyObject
> contains an instance mmtestSubObject of MySubObject and this one contains 2
> other instances: mmtestSubSubObject1 of MySubSubObject1&
> mmtestSubSubObject2 of MySubSubObject2. Everything syntactically correct:
>
> MyObject mmtest {
> MySubObject mmtestSubObject {
> MySubSubObject1 mmtestSubSubObject1 ;
> MySubSubObject2 mmtestSubSubObject2 {
> }
> }
> }
>
> Then I programmatically add a new instance newSubSubObject1 of
> MySubSubObject1 under mmtestSubObject and under mmtestSubSubObject2 via the
> code below (....<some model manipulations of the resource>...) and then the
> editor content looks like this:
>
> MyObject mmtest { MySubObject mmtestSubObject { MySubSubObject2
> mmtestSubSubObject2 { MySubSubObject1 newSubSubObject1 ; } ; MySubSubObject1
> mmtestSubSubObject1 { } ; MySubSubObject1 newSubSubObject1 { } ; } ; } ;
>
> Everything is in the first line, additionally I get several new parenthesis
> and semicolons, also the elements are reordered. This can be seen if one
> manually reformats this line to:
>
> MyObject mmtest {
> MySubObject mmtestSubObject {
> MySubSubObject2 mmtestSubSubObject2 {
> MySubSubObject1 newSubSubObject1 ;
> } ;
> MySubSubObject1 mmtestSubSubObject1 { } ;
> MySubSubObject1 newSubSubObject1 { } ;<- 2 new parentheses "{ }"&
> position changed, now below mmtestSubSubObject2
> } ;
> } ;
>
> You can see that after newSubSubObject1 I get 2 parentheses "{ }" with a
> space between, one sees some new semicolons and the element mentioned is now
> below the element mmtestSubSubObject2.
>
> Unfortunately I can not post the grammar here, but nevertheless, it should
> be reproducable for you, at least the semocolons and the parentheses in a
> sample code.
>
> Debugging shows that the statement XtextResource.save below already creates
> that single line in the buffer "out" and the XtextDocument simply shows it
> in the editor. It seams that the resource save call throws away the
> formatting.
>
> As I could just see now, also comments are completely thrown away :( Test it
> out!
>
> Thanks in advance and ciao, Michael
>
>
> "Sebastian Zarnekow"<Sebastian.Zarnekow@itemis.de> wrote in message
> news:h06gd2$7ib$1@build.eclipse.org...
>> Hi Michael,
>>
>> we are not aware of any concrete problems with the current (recently
>> refactored and improved) implementation of the serializer. Can you be more
>> specific here and provide a short sample that illustrates the described
>> problem.
>>
>> What do you mean by "addition of parentheses and semicolons"? Is the
>> serialized model not parseable? Does it contain any errors?
>>
>> As far as I know, whitespace handling will be improved in RC4.
>>
>> Sorry for inconvenience,
>> Sebastian
>>
>> Am 03.06.2009 19:27 Uhr, schrieb Michael Mühlberg:
>>> Hello,
>>>
>>> I'm changing the Xtext document content via the following piece of
>>> (speudo)
>>> code
>>>
>>> ->snip
>>> ....<some model manipulations of the resource>...
>>> ByteArrayOutputStream out = new ByteArrayOutputStream();
>>> XTextResource.save(out, null);
>>> XtextDocument.set(out.toString());
>>> <- snip
>>>
>>> and get the whole content reformatted.
>>>
>>> Everything is put into the first line of the editor! Also, parentheses
>>> and
>>> semicolons are added where I don't expect it and some elements are
>>> reordered
>>> in a different way.
>>>
>>> I'm using the Eclipse Modeling IDE RC2 from last Monday. This behaviour
>>> is
>>> even worse than e.g. in MS7 that I had before where at least the rough
>>> formatting was kept.
>>>
>>> What can I do in order to prevent the addition of parentheses and
>>> semicolons
>>> and the reordering of elements (note that the document is valid also
>>> without
>>> these parentheses and semicolons)?
>>>
>>> Is there a pretty printer or a formatter available?
>>>
>>> Ciao, Michael
>>>
>>>
>
>
|
|
|
| Re: [Xtext] Formatter/Pretty Printer needed [message #48313 is a reply to message #47545] |
Fri, 05 June 2009 04:23 |
| Eclipse User |
|
|
|
In a project we are about to do something similar. However, because the
model itself does not contain comments, line breaks, spacing etc. we plan
to insert the new elements (as "text" LeafNodes) in the parse tree
directly (rather than the model), but then we have to make sure that the
new text is actually parsed again, in order to correctly instantiate the
"new" model. That way the original layout information is not lost. Of
course it would be great if the standard serialiser would automatically
use the parse tree for serialisation (if there is one attached to the
model). (A hint, in case you did not know: Using the NodeUtil-class you
can navigate from model element to parse-tree node and back).
There would still be problems, I guess. I have not looked very deeply into
the matter yet, so I may be wrong: The parse tree node attached to a model
element contains all comments and white spaces that follow the elements
belonging to the previous model element (so assuming your Grammar rule
ends with ';' everything "hidden" - comments, white spaces - following
that semicolon will be stored in the composite parse node attached to the
following model element).
Simply speaking, comments are attached to the following model element.
That is, deleting a model element (hence deleting the corresponding
composite node in the parse tree) also deletes the comment (unless you
first extract it or have a more sophisticated deletion strategy). Adding a
model element will add it before the comment which is not alway wanted
(e.g. in //Here we begin with the main methods).
In other words, serialising a model change not damaging the layout is
everything but trivial (insert before or after comments, which white
spaces actually belong to the next model element (in order to retain
indentation)...). And even if the standard serialisation will be improved,
you may have to do a lot of additional work in order to get exactly the
result you want.
|
|
|
| Re: [Xtext] Formatter/Pretty Printer needed [message #48372 is a reply to message #47846] |
Fri, 05 June 2009 05:53 |
| Eclipse User |
|
|
|
Hi Michael,
I've done some major improvements which have arrived in CVS yesterday
and will be available with RC4.
Now, the AbstractFormattingTokenSerializer has been replaced with the
AbstractDeclarativeFormatter and it is possible to apply formatting
*while* keeping comments.
The XtextResource has an option named OPTION_FORMAT of type Boolean
which can be specified for the resource.save(outputstream, options) call
and which specifies whether the existing whitespaces should be kept or
be replaced by the suggestions of the formatter. Comments are kept in
both scenarios.
Furthermore, I've enabled formatting for the "CTRL+SHIFT+F"-Action in
the dditor per default. If that is your use case, you won't need to
manipulate your document manually anymore. For details, see
ContentFormatterFactory.java
With regards to the problems you've mentioned:
1. Everything in one line: make sure you have configured your language
to use the AbstractDeclarativeFormatter (as of RC4) or
AbstractFormattingTokenSerializer (RC3 or earlier) and that you've
specified rules where to insert linebreaks.
2. Inserted curly braces: This is a known problem:
https://bugs.eclipse.org/bugs/show_bug.cgi?id=275855 The issue is that
the serializer needs to serialize the longest path with the intention to
serialize all model elements. the needed improvement is to detect
whether there is a shorter path which would still consume all model
elements. Currently, you can work around this by tweaking your grammar:
Instead of
MyClass: "class" name=ID ("{" myattrs=Attr* "}")?
use
MyClass: "class" name=ID ("{" myattrs=Attr+ "}")?
this way you ensure that the curly brackets are only allowed (and
serialized if there is at least one atribute.
3. The reordering of elements. This sounds like a bug, I'll have to try
to reproduce that the next days
regards,
Moritz
> Hello Sebastian,
>
> I have an initial, nicely formatted model. An instance mmtest of MyObject
> contains an instance mmtestSubObject of MySubObject and this one contains 2
> other instances: mmtestSubSubObject1 of MySubSubObject1 &
> mmtestSubSubObject2 of MySubSubObject2. Everything syntactically correct:
>
> MyObject mmtest {
> MySubObject mmtestSubObject {
> MySubSubObject1 mmtestSubSubObject1 ;
> MySubSubObject2 mmtestSubSubObject2 {
> }
> }
> }
>
> Then I programmatically add a new instance newSubSubObject1 of
> MySubSubObject1 under mmtestSubObject and under mmtestSubSubObject2 via the
> code below (....<some model manipulations of the resource>...) and then the
> editor content looks like this:
>
> MyObject mmtest { MySubObject mmtestSubObject { MySubSubObject2
> mmtestSubSubObject2 { MySubSubObject1 newSubSubObject1 ; } ; MySubSubObject1
> mmtestSubSubObject1 { } ; MySubSubObject1 newSubSubObject1 { } ; } ; } ;
>
> Everything is in the first line, additionally I get several new parenthesis
> and semicolons, also the elements are reordered. This can be seen if one
> manually reformats this line to:
>
> MyObject mmtest {
> MySubObject mmtestSubObject {
> MySubSubObject2 mmtestSubSubObject2 {
> MySubSubObject1 newSubSubObject1 ;
> } ;
> MySubSubObject1 mmtestSubSubObject1 { } ;
> MySubSubObject1 newSubSubObject1 { } ; <- 2 new parentheses "{ }" &
> position changed, now below mmtestSubSubObject2
> } ;
> } ;
>
> You can see that after newSubSubObject1 I get 2 parentheses "{ }" with a
> space between, one sees some new semicolons and the element mentioned is now
> below the element mmtestSubSubObject2.
>
> Unfortunately I can not post the grammar here, but nevertheless, it should
> be reproducable for you, at least the semocolons and the parentheses in a
> sample code.
>
> Debugging shows that the statement XtextResource.save below already creates
> that single line in the buffer "out" and the XtextDocument simply shows it
> in the editor. It seams that the resource save call throws away the
> formatting.
>
> As I could just see now, also comments are completely thrown away :( Test it
> out!
>
> Thanks in advance and ciao, Michael
>
>
> "Sebastian Zarnekow" <Sebastian.Zarnekow@itemis.de> wrote in message
> news:h06gd2$7ib$1@build.eclipse.org...
>> Hi Michael,
>>
>> we are not aware of any concrete problems with the current (recently
>> refactored and improved) implementation of the serializer. Can you be more
>> specific here and provide a short sample that illustrates the described
>> problem.
>>
>> What do you mean by "addition of parentheses and semicolons"? Is the
>> serialized model not parseable? Does it contain any errors?
>>
>> As far as I know, whitespace handling will be improved in RC4.
>>
>> Sorry for inconvenience,
>> Sebastian
>>
>> Am 03.06.2009 19:27 Uhr, schrieb Michael Mühlberg:
>>> Hello,
>>>
>>> I'm changing the Xtext document content via the following piece of
>>> (speudo)
>>> code
>>>
>>> ->snip
>>> ....<some model manipulations of the resource>...
>>> ByteArrayOutputStream out = new ByteArrayOutputStream();
>>> XTextResource.save(out, null);
>>> XtextDocument.set(out.toString());
>>> <- snip
>>>
>>> and get the whole content reformatted.
>>>
>>> Everything is put into the first line of the editor! Also, parentheses
>>> and
>>> semicolons are added where I don't expect it and some elements are
>>> reordered
>>> in a different way.
>>>
>>> I'm using the Eclipse Modeling IDE RC2 from last Monday. This behaviour
>>> is
>>> even worse than e.g. in MS7 that I had before where at least the rough
>>> formatting was kept.
>>>
>>> What can I do in order to prevent the addition of parentheses and
>>> semicolons
>>> and the reordering of elements (note that the document is valid also
>>> without
>>> these parentheses and semicolons)?
>>>
>>> Is there a pretty printer or a formatter available?
>>>
>>> Ciao, Michael
>>>
>>>
>
>
|
|
|
| Re: [Xtext] Formatter/Pretty Printer needed [message #48402 is a reply to message #48313] |
Fri, 05 June 2009 06:09 |
| Eclipse User |
|
|
|
Hi Alexander,
what do you actually intend to do? It is generally not a good idea to
manipulate the NodeModel manually. If you want to insert whitespaces, it
is a better idea to implement a formatter. If you want to insert
comments, hm... it might be an option to include comments in your
semantic model. If you want to insert semantic tokens - do it in the
semantic model and re-run the ParseTreeCOnstructor. I might be wrong,
but I don't think CompositeNodes/LeafNodes are actually removed when
removing elements from the semantic model. after all, the node model is
a stand-alone containment tree.
hth,
Moritz
> In a project we are about to do something similar. However, because the
> model itself does not contain comments, line breaks, spacing etc. we
> plan to insert the new elements (as "text" LeafNodes) in the parse tree
> directly (rather than the model), but then we have to make sure that the
> new text is actually parsed again, in order to correctly instantiate the
> "new" model. That way the original layout information is not lost. Of
> course it would be great if the standard serialiser would automatically
> use the parse tree for serialisation (if there is one attached to the
> model). (A hint, in case you did not know: Using the NodeUtil-class you
> can navigate from model element to parse-tree node and back).
>
> There would still be problems, I guess. I have not looked very deeply
> into the matter yet, so I may be wrong: The parse tree node attached to
> a model element contains all comments and white spaces that follow the
> elements belonging to the previous model element (so assuming your
> Grammar rule ends with ';' everything "hidden" - comments, white spaces
> - following that semicolon will be stored in the composite parse node
> attached to the following model element).
> Simply speaking, comments are attached to the following model element.
> That is, deleting a model element (hence deleting the corresponding
> composite node in the parse tree) also deletes the comment (unless you
> first extract it or have a more sophisticated deletion strategy). Adding
> a model element will add it before the comment which is not alway wanted
> (e.g. in //Here we begin with the main methods).
>
> In other words, serialising a model change not damaging the layout is
> everything but trivial (insert before or after comments, which white
> spaces actually belong to the next model element (in order to retain
> indentation)...). And even if the standard serialisation will be
> improved, you may have to do a lot of additional work in order to get
> exactly the result you want.
>
|
|
| | | | |
Goto Forum:
Current Time: Mon Nov 03 23:06:48 EST 2025
Powered by FUDForum. Page generated in 0.05500 seconds |
Home