Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » TMF (Xtext) » How to generate language with comments
How to generate language with comments [message #775781] Fri, 06 January 2012 17:03 Go to next message
Thomas Williams is currently offline Thomas WilliamsFriend
Messages: 15
Registered: January 2012
Junior Member
Dear Xtexters,

(Apologies if I'm not using the correct terms below, please feel free to correct me).

I've worked out how to insert a new type-object into my ECore model and pretty-print when serializing that model, but I'm struggling in my requirement to easily insert comments attached to these type-objects.

From another post I've seen how to do very basic comment insertion by extending HiddenTokenSequencer, but it's pretty unpleasant and I haven't worked out how to handle pretty-printing so that they integrate with the indenting of the serialized type-objects.

Is there an API for a better way?

Related to this, how can I add another feature to the generated EObject type, e.g. a "newComment" field that I can populate when I programatically create a type-object and then query from the HiddenTokenSequencer when I want to insert that comment into the serialized text?

Sincerely,

Tom
Re: How to generate language with comments [message #776242 is a reply to message #775781] Sat, 07 January 2012 23:37 Go to previous messageGo to next message
Henrik Lindberg is currently offline Henrik LindbergFriend
Messages: 2509
Registered: July 2009
Senior Member
On 2012-06-01 18:03, Thomas Williams wrote:
> Dear Xtexters,
>
> (Apologies if I'm not using the correct terms below, please feel free to
> correct me).
>
> I've worked out how to insert a new type-object into my ECore model and
> pretty-print when serializing that model, but I'm struggling in my
> requirement to easily insert comments attached to these type-objects.
>
> From another post I've seen how to do very basic comment insertion by
> extending HiddenTokenSequencer, but it's pretty unpleasant and I haven't
> worked out how to handle pretty-printing so that they integrate with the
> indenting of the serialized type-objects.
>
> Is there an API for a better way?
>
Well, I don't know about better, but perhaps more straight forward; you
could always do it in two passes - first serialize the model, and then
modify it textually (since you have the nodes at this point you know the
offsets and lengths of everything). (Hint it is easier to do it "in
reverse" - i.e. iterate over text from last char to first and insert
comments (also in reverse). This is easier as you then do not have to
recompute all the offsets for every inserted comment. Finally, reverse
the resulting string).

Then parse the result again (now the node model knows about the
comments) and then serialize it with formatting turned on. Then it is at
least as pretty as what can be done with the formatter.

> Related to this, how can I add another feature to the generated EObject
> type, e.g. a "newComment" field that I can populate when I
> programatically create a type-object and then query from the
> HiddenTokenSequencer when I want to insert that comment into the
> serialized text?
>
The easiest is to use an adapter. This way you do not have to alter the
model to be able to associate a comments with something in the model.

If you combine an adapter with the above suggestion, as you iterate over
the node model, you can also get the semantic object. If that semantic
object has an adapter with a comment, you output comments (according to
some convention (before or after the element, etc.)).

Alternatively, if you want to add things to the model you need to switch
from a generated to an external (manually maintained model). You can
start with a copy of the generated one. (Generally, this is what
typically what you want at some point in the life cycle of your language
anyway).

Hope that helps.
- henrik
Re: How to generate language with comments [message #776849 is a reply to message #776242] Mon, 09 January 2012 11:32 Go to previous messageGo to next message
Lasse  indg is currently offline Lasse indgFriend
Messages: 12
Registered: July 2009
Junior Member
Did you look at this:
http://www.eclipse.org/forums/index.php/m/769264/#msg_769264
Re: How to generate language with comments [message #776981 is a reply to message #776849] Mon, 09 January 2012 15:57 Go to previous messageGo to next message
Thomas Williams is currently offline Thomas WilliamsFriend
Messages: 15
Registered: January 2012
Junior Member
Lasse indg wrote on Mon, 09 January 2012 06:32
Did you look at this ...


Briefly, but found it required rather more knowledge than I currently have about Xtext and Eclipse. At the moment I'm being lazy, I need to evaluate whether Xtext is the correct technology for a project (does it tick all the boxes included not being tied into the eclipse IDE) without having the time to learn many of what are probably the pre-requisite (Eclipse) technologies. This ability to generate code with comments is one of those boxes.
Re: How to generate language with comments [message #776986 is a reply to message #776981] Mon, 09 January 2012 16:07 Go to previous messageGo to next message
Thomas Williams is currently offline Thomas WilliamsFriend
Messages: 15
Registered: January 2012
Junior Member
Henrik,

Thanks for your reply; I'm going to ask some newbie follow up about adapters.

Quote:
The easiest is to use an adapter. This way you do not have to alter the
model to be able to associate a comments with something in the model.


By applying educated guessing I seem to have achieved what you suggest with:

package psenterprise.xtext;

import org.eclipse.emf.common.notify.impl.AdapterImpl;

public class NewCommentAdapter extends AdapterImpl {

    private final String m_CommentText;
	
    public NewCommentAdapter(String commentText) {
        m_CommentText = commentText;
    }

    @Override
    public boolean isAdapterForType(java.lang.Object type) {
        return type == NewCommentAdapter.class;
    }

    public String getCommentText() {
        return m_CommentText;
    }
}


Setting the adapter with:

    EList<Adapter> adapters = myEObject.eAdapters();
    adapters.add(new NewCommentAdapter(comment));


And getting the adapter with:

    NewCommentAdapter adapter = (NewCommentAdapter) EcoreUtil.getRegisteredAdapter(myEObject, NewCommentAdapter.class);
    if (adapter != null) {
        String comment = adapter.getCommentText();
    }


But I suspect someone is going to tell me that I'm being naive and the proper way to do it is ... ?

Quote:

Alternatively, if you want to add things to the model you need to switch
from a generated to an external (manually maintained model). You can
start with a copy of the generated one. (Generally, this is what
typically what you want at some point in the life cycle of your language
anyway).


Using a manually maintained model is something I'm hoping to avoid (I'm evaluating Xtext as a replacement for a system that currently uses antlr to generate a custom semantic model). I would be interested if you could provide me with a suggestion of (or reference to) some reasons why even with Xtext a manually maintained model may ultimately be required.

Sincerely,

Tom
Re: How to generate language with comments [message #777030 is a reply to message #776986] Mon, 09 January 2012 17:25 Go to previous messageGo to next message
Henrik Lindberg is currently offline Henrik LindbergFriend
Messages: 2509
Registered: July 2009
Senior Member
Hi,

On 2012-09-01 17:07, Thomas Williams wrote:
> Henrik,
>
> Thanks for your reply; I'm going to ask some newbie follow up about
> adapters.
>
> Quote:
>> The easiest is to use an adapter. This way you do not have to alter
>> the model to be able to associate a comments with something in the model.
>
>
> By applying educated guessing I seem to have achieved what you suggest
> with:
>
>
> package psenterprise.xtext;
>
> import org.eclipse.emf.common.notify.impl.AdapterImpl;
>
> public class NewCommentAdapter extends AdapterImpl {
>
> private final String m_CommentText;
>
> public NewCommentAdapter(String commentText) {
> m_CommentText = commentText;
> }
>
> @Override
> public boolean isAdapterForType(java.lang.Object type) {
> return type == NewCommentAdapter.class;
> }
>
> public String getCommentText() {
> return m_CommentText;
> }
> }
>
>
> Setting the adapter with:
>
>
> EList<Adapter> adapters = myEObject.eAdapters();
> adapters.add(new NewCommentAdapter(comment));
>
>
> And getting the adapter with:
>
>
> NewCommentAdapter adapter = (NewCommentAdapter)
> EcoreUtil.getRegisteredAdapter(myEObject, NewCommentAdapter.class);
> if (adapter != null) {
> String comment = adapter.getCommentText();
> }
>
>
> But I suspect someone is going to tell me that I'm being naive and the
> proper way to do it is ... ?
>
It is basically as simple as that...

If ypu want to, you can take a look at the class
org.cloudsmith.geppetto.pp.dsl.adapters.DocumentationAdapter
(Found at github cloudsmith/geppetto) and the related classes in the
same package.

The DocumentationAdapter contains documentation comments similar to
JavaDoc (but is somewhat complicated in the puppet language since the
documentation consists of multiple single line comments, but that is all
done outside of the adapters).

> Quote:
>> Alternatively, if you want to add things to the model you need to
>> switch from a generated to an external (manually maintained model).
>> You can start with a copy of the generated one. (Generally, this is
>> what typically what you want at some point in the life cycle of your
>> language anyway).
>
>
> Using a manually maintained model is something I'm hoping to avoid (I'm
> evaluating Xtext as a replacement for a system that currently uses antlr
> to generate a custom semantic model). I would be interested if you could
> provide me with a suggestion of (or reference to) some reasons why even
> with Xtext a manually maintained model may ultimately be required.
>
The reason you want to maintain it manually (as I see it) is mainly to
avoid unexpected changes in the model as a side effect of changing the
grammar (some constructs in a grammar can cause drastic changes in the
model). This is bad on its own, but esp. bad if the model is also
considered API.

For some types of models you are actually required to maintain the model
manually as the generator simply does not support everything you may
want to do, and in some cases produces models that are suboptimal.

BTW, I find maintaining the model manually to be very easy and not time
consuming at all. But it is very valuable to have it generated at the
start of a project when exploring language design. In fact, although it
is possible to affect the model by using certain constructs in the
grammar, I found it to be much easier to change this in an external
model thus leaving the grammar free of this concern.

Regards
- henrik
Re: How to generate language with comments [message #778198 is a reply to message #777030] Wed, 11 January 2012 16:02 Go to previous messageGo to next message
Thomas Williams is currently offline Thomas WilliamsFriend
Messages: 15
Registered: January 2012
Junior Member
Thanks Henrik,

Quote:

Well, I don't know about better, but perhaps more straight forward;


Straight forward is always good Wink

Quote:
you
could always do it in two passes - first serialize the model, and then
modify it textually (since you have the nodes at this point you know the
offsets and lengths of everything).


Do you have the nodes (for the new stuff)? I thought you just had the text, that's what ISerializer#serialize() returns?

Quote:
(Hint it is easier to do it "in
reverse" - i.e. iterate over text from last char to first and insert
comments (also in reverse). This is easier as you then do not have to
recompute all the offsets for every inserted comment. Finally, reverse
the resulting string).
Then parse the result again (now the node model knows about the
comments) and then serialize it with formatting turned on. Then it is at
least as pretty as what can be done with the formatter.


I thought I'd seen the formatter only working on 'stuff' that I had inserted programatically into the semantic model - and thus did not already have a representation (and formatting) recorded in the node model ?

Regards,

Tom
Re: How to generate language with comments [message #778355 is a reply to message #778198] Thu, 12 January 2012 14:58 Go to previous messageGo to next message
Thomas Williams is currently offline Thomas WilliamsFriend
Messages: 15
Registered: January 2012
Junior Member
Dear Henrik,

After overcoming my laziness and re-reading the documentation I believe I have managed to answer my own questions and do something inspired by your suggestion, but I believe I'm still not quite understanding what you said and thus may be doing something sub-optimal.

If I can summarize my understanding of your suggestion:
 1. Programatically add new element to semantic model.
 2. Serialize the semantic model.
 3. Parse the serialized model created at (2).
    - You didn't say this but I think it's necessary because we need a node model that has the offsets including the "new element".
 4. Somehow identify the "new element" in the new semantic model created at (3)
 5. Use NodeModelUtils to locate the INode for the "new element"
 6. Use String manipulation to insert the comment before/after the "new element" using offsets from the node model.


You then said:
Quote:
Then parse the result again (now the node model knows about the
comments) and then serialize it with formatting turned on. Then it is at
least as pretty as what can be done with the formatter.


I don't understand. Are you suggesting using Formatting=true in the save options? Wouldn't that re-format the entire text, not just the newly inserted comments?

Finally you ended with:
Quote:
If you combine an adapter with the above suggestion, as you iterate over
the node model, you can also get the semantic object. If that semantic
object has an adapter with a comment, you output comments (according to
some convention (before or after the element, etc.)).


I don't understand. Any such adapter would be attached to the new element / EObject at step (1), by the time we reach step (6) don't we have a completely different EObject (in the new semantic model created at step (3)) representing that element?

Sincerely,

Tom
Re: How to generate language with comments [message #778424 is a reply to message #778198] Thu, 12 January 2012 00:55 Go to previous messageGo to next message
Henrik Lindberg is currently offline Henrik LindbergFriend
Messages: 2509
Registered: July 2009
Senior Member
On 2012-11-01 17:02, Thomas Williams wrote:
> Thanks Henrik,
>
> Quote:
>> Well, I don't know about better, but perhaps more straight forward;
>
>
> Straight forward is always good ;)
>
> Quote:
>> you could always do it in two passes - first serialize the model, and
>> then modify it textually (since you have the nodes at this point you
>> know the offsets and lengths of everything).
>
>
> Do you have the nodes (for the new stuff)? I thought you just had the
> text, that's what ISerializer#serialize() returns?
>
You have a model. You serialize it to text. At that point, I am not sure
if a node model is constructed, or if it is easy to reconciliate it with
the model instance you have. If you again parse the generated text, you
should get an equivalent model but probably not the same instances.

Maybe there is a smarter way of affecting the serializer to insert
comments found in adapters.

> Quote:
>> (Hint it is easier to do it "in reverse" - i.e. iterate over text from
>> last char to first and insert comments (also in reverse). This is
>> easier as you then do not have to recompute all the offsets for every
>> inserted comment. Finally, reverse the resulting string).
>> Then parse the result again (now the node model knows about the
>> comments) and then serialize it with formatting turned on. Then it is
>> at least as pretty as what can be done with the formatter.
>
>
> I thought I'd seen the formatter only working on 'stuff' that I had
> inserted programatically into the semantic model - and thus did not
> already have a representation (and formatting) recorded in the node model ?
>
You can ask for serialization with formatting (when you have a model,
and a node model). When you only have a model, it will always format
(there is no formatting to preserve).

Regards
- henrik
Re: How to generate language with comments [message #780835 is a reply to message #778424] Thu, 19 January 2012 14:53 Go to previous message
Thomas Williams is currently offline Thomas WilliamsFriend
Messages: 15
Registered: January 2012
Junior Member
Thanks for all your help Henrik,

I didn't come to a satisfactory solution and for the moment I am allowing it to lapse as my evaluation effort on Xtext is over.

Sincerely,

Tom
Previous Topic:Serializer fails with InvalidConcreteSyntaxException
Next Topic:how to extract STRING content?
Goto Forum:
  


Current Time: Fri Apr 19 21:54:38 GMT 2024

Powered by FUDForum. Page generated in 0.03471 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top