Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » EMF » Code Generator broken in EMF 2.8.x and Eclipse 3.8 / 4.2?(Workaround: Enable 'Process Javadoc comments' in workspace settings)
Code Generator broken in EMF 2.8.x and Eclipse 3.8 / 4.2? [message #997451] Mon, 07 January 2013 15:17 Go to next message
Yves Alter is currently offline Yves AlterFriend
Messages: 11
Registered: January 2013
Junior Member
Hello all

We have a problem with the EMF Tooling Code Generator that we first noticed in Eclipse 3.8/4.2. The code generation process does not reflect all changes in the model/genmodel anymore. In particular, we are able to reproduce the problem by executing the following steps:

Prerequisites: An EMF.Edit generated source code of an EMF model, runtime version 2.7.


  1. In the generator model, we change the 'Notify' property of a reference from 'false' to 'true'. The container class of the reference uses the 'Singleton' provider type.
  2. We re-generate the model and edit code.



  • Expected result: The *ItemProvider#notifyChanged() method will be extended with code that dispatches the notification for the feature.
  • Actual result: The notifyChanged() method is untouched. Only some import are added to the *ItemProvider class (because we organize imports manually after code generation).


This happens with Eclipse 3.8.0, 3.8.1, 4.2.0, 4.2.1 (ie. with EMF 2.8.0 and 2.8.1). When we generate the Edit code of the very same model/genmodel with Eclipse 3.7.2 (EMF 2.7), the *ItemProvider class is generated as expected. Of course the @generated annotation is present. We did not modify the *ItemProvider classes besides the organization of imports.

Does somebody know the cause of this problem? Is it 'just' a bug in the generator tooling?

[Updated on: Wed, 09 January 2013 16:39]

Report message to a moderator

Re: EMF Code Generator broken in 2.8.x? [message #997656 is a reply to message #997451] Mon, 07 January 2013 16:18 Go to previous messageGo to next message
Ed Merks is currently offline Ed MerksFriend
Messages: 26289
Registered: July 2009
Senior Member
Yves,

Comments below.


On 07/01/2013 5:02 PM, Yves Alter wrote:
> Hello all
>
> We have a problem with the EMF Tooling Code Generator that we first
> noticed in Eclipse 3.8/4.2. The code generation process does not
> reflect all changes in the model/genmodel anymore. In particular, we
> are able to reproduce the problem by executing the following steps:
>
> Prerequisites: An EMF.Edit generated source code of an EMF model,
> runtime version 2.7.
>
>
> In the generator model, we change the 'Notify' property of a reference
> from 'false' to 'true'. The container class of the reference uses the
> 'Singleton' provider type.
> We re-generate the model and edit code.
>
>
>
> Expected result: The *ItemProvider#notifyChanged() method will be
> extended with code that dispatches the notification for the feature.
> Actual result: The notifyChanged() method is untouched. Only some
> import are added to the *ItemProvider class (because we organize
> imports manually after code generation).
So it's definitely regenerating and it works for me locally... If you
delete the entire *.java file, does it produce the expected results?
You don't have an @generated NOT on the class or on that method?
>
>
> This happens with Eclipse 3.8.0, 3.8.1, 4.2.0, 4.2.1 (ie. with EMF
> 2.8.0 and 2.8.1). When we generate the Edit code of the very same
> model/genmodel with Eclipse 3.7.2 (EMF 2.7), the *ItemProvider class
> is generated as expected. Of course the @generated annotation is
> present. We did not modify the *ItemProvider classes besides the
> organization of imports.
>
> Does somebody know the cause of this problem? Is it 'just' a bug in
> the generator tooling?
Try deleting the whole file to see if it's being produced at all. You'd
be the first to complain about such an issue since the release, so
there's most likely something specific about your situation that you've
note noticed nor told me about...
Re: EMF Code Generator broken in 2.8.x? [message #997731 is a reply to message #997656] Tue, 08 January 2013 16:07 Go to previous messageGo to next message
Yves Alter is currently offline Yves AlterFriend
Messages: 11
Registered: January 2013
Junior Member
Thank your for your answer, Ed.

If we delete the .java file, the file gets regenerated with the expected result. Only the merge does not work. The class and the method(s) do have @generated annotations and no @generated NOT at all. And as I already mentioned, opening the very same workspace with Eclipse 3.7.2 and regenerating the code leads to a successful merge.

We verified the wrong behavior on two different systems with Eclipse 4.2.1, 3.8.1 and 3.8.0 (on the same EMF projects checked out from SCM, though). Is there any information I could provide you to help us determining the cause?
Re: EMF Code Generator broken in 2.8.x? [message #997737 is a reply to message #997731] Tue, 08 January 2013 16:25 Go to previous messageGo to next message
Ed Merks is currently offline Ed MerksFriend
Messages: 26289
Registered: July 2009
Senior Member
Yves,

It all sounds very odd. The fact that imports are added indicates that
the file is generated and the JMerge is running to some extent. You've
also confirmed that the file is also generated with the correct contents
when no merging is involved. So that implies there is some problem
with merging the methods themselves (or maybe just this method in
particular). If in the properly regenerated result you make a change in
that method body (add a line comment) and then generate again, that
comment should be removed. Is that the case, or is merging failing to
overwrite the body?


On 08/01/2013 5:08 PM, Yves Alter wrote:
> Thank your for your answer, Ed.
>
> If we delete the .java file, the file gets regenerated with the
> expected result. Only the merge does not work. The class and the
> method(s) do have @generated annotations and no @generated NOT at all.
You can use the Eclipse history or the SCM history to see how this file
differs from the one you deleted. Do those differences tell you
anything interesting or useful?
> And as I already mentioned, opening the very same workspace with
> Eclipse 3.7.2 and regenerating the code leads to a successful merge.
>
> We verified the wrong behavior on two different systems with Eclipse
> 4.2.1, 3.8.1 and 3.8.0 (on the same EMF projects checked out from SCM,
> though). Is there any information I could provide you to help us
> determining the cause?
I still think it's something odd about that file's contents. Maybe the
answers to these latest questions will help narrow things down.
Re: EMF Code Generator broken in 2.8.x? [message #998020 is a reply to message #997737] Wed, 09 January 2013 07:49 Go to previous messageGo to next message
Yves Alter is currently offline Yves AlterFriend
Messages: 11
Registered: January 2013
Junior Member

The change (comment) stays. I probably have to add, that generating the edit code executes very fast, the progress bar just flicks through. In eclipse <3.8 it takes noticeably longer.

Also after systematic testing with the *ItemProvider of EMF Edit generated code, it seems that imports are also untouched. My previous statement that import were regenerated was probably about some other class in the model. Sorry about that.

So here are the actual results of the test: I did a structural modification of the model (added a reference to a class). I then generated the model (not Edit) with eclipse 3.8.2 that led to changes to some files:

- The interface of the modelled class, got the getter/setter pair
- The model implementation class got a new field and getter/setter/basicGetter for the reference.
- The package interface got a new constant with the feature id of the new feature, a new meta object for the reference, and the meta object literal.
- The package implementation class got the implementation of the getter for meta object reference.

I then re-generated the same model (without deleting the previously generated source) in eclipse 3.7.2 that created the following delta:

- In the interface of the modelled class that got the new reference, the class-comment was updated (like "<li>{@link testsuitePM.TestExecutionResult#getFoobar <em>Foobar</em>}</li>"). Very odd: The actual getter/setter (including method-comment) was already generated by EMF 2.8!
- This also applies for the implementation class of the modelled class. Additionally, the implementation class got an updated 'reflective' eGet()/eSet()/eUnset()/eIsSet() (to include the new feature)
- The model *Package interface got an updated the feature count constant (from 2 to 3, where as 3 is correct).
- The package implementation createPackageContents() got a new createEReference() and the initializePackageContents() a new initEReference() call.

So in short: Additions were successfully applied to existing code in EMF 2.8, but changes were not. And it happens for both model and edit code.

Re: EMF Code Generator broken in 2.8.x? [message #998027 is a reply to message #998020] Wed, 09 January 2013 08:10 Go to previous messageGo to next message
Ed Willink is currently offline Ed WillinkFriend
Messages: 4189
Registered: July 2009
Senior Member
Hi

On 09/01/2013 07:49, Yves Alter wrote:
> The change (comment) stays. I probably have to add, that generating
> the edit code executes very fast, the progress bar just flicks
> through. In eclipse <3.8 it takes noticeably longer.
That is a sure sign that something threw an Exception and terminated the
generation prematurely.

Have you checked the Error Log? If you're lucky there may be an
Exception trace.

Have you Validated all your models? The basic GenModel doesn't always
diagnose every possible ill-formed model. GenModel extensions can be
much more temperamental.

If you start your Eclipse with:

-Xrunjdwp:server=y,transport=dt_socket,address=14143,suspend=n

you may use the Remote Java Application, Debug launch configuration in
another Eclipse to connect to port 14143 and intercept the exceptions
while the first Eclipse genmodels too quickly.

Regards

Ed Willink
Re: EMF Code Generator broken in 2.8.x? [message #998032 is a reply to message #998020] Wed, 09 January 2013 08:23 Go to previous messageGo to next message
Ed Merks is currently offline Ed MerksFriend
Messages: 26289
Registered: July 2009
Senior Member
Yves,

Comments below.

On 09/01/2013 8:49 AM, Yves Alter wrote:
>
> The change (comment) stays. I probably have to add, that generating
> the edit code executes very fast, the progress bar just flicks
> through. In eclipse <3.8 it takes noticeably longer.
There's nothing in your Error log that might give clues?
>
> Also after systematic testing with the *ItemProvider of EMF Edit
> generated code, it seems that imports are also untouched. My previous
> statement that import were regenerated was probably about some other
> class in the model. Sorry about that.
I see.
>
> So here are the actual results of the test: I did a structural
> modification of the model (added a reference to a class). I then
> generated the model (not Edit) with eclipse 3.8.2 that led to changes
> to some files:
>
> - The interface of the modelled class, got the getter/setter pair
> - The model implementation class got a new field and
> getter/setter/basicGetter for the reference.
> - The package interface got a new constant with the feature id of the
> new feature, a new meta object for the reference, and the meta object
> literal.
> - The package implementation class got the implementation of the
> getter for meta object reference.
So new things are definitely pushed in... The package initialization
code changed to actually create it EReference?
>
> I then re-generated the same model (without deleting the previously
> generated source) in eclipse 3.7.2 that created the following delta:
>
> - In the interface of the modelled class that got the new reference,
> the class-comment was updated (like "<li>{@link
> testsuitePM.TestExecutionResult#getFoobar <em>Foobar</em>}</li>").
> Very odd: The actual getter/setter (including method-comment) was
> already generated by EMF 2.8!
So it sounds like you have a general problem with the 2.8 generation
that nothing is pulled from the new source into the existing target
source, only new things are pushed in...
> - This also applies for the implementation class of the modelled
> class. Additionally, the implementation class got an updated
> 'reflective' eGet()/eSet()/eUnset()/eIsSet() (to include the new feature)
So again, with 2.8 generation, these didn't get pulled...
> - The model *Package interface got an updated the feature count
> constant (from 2 to 3, where as 3 is correct).
More problems with pull...
> - The package implementation createPackageContents() got a new
> createEReference() and the initializePackageContents() a new
> initEReference() call.
That's what I was asking about earlier.
>
> So in short: Additions were successfully applied to existing code in
> EMF 2.8, but changes were not. And it happens for both model and edit
> code.
The fact that push worked, implies that (as you confirmed previously)
the right things are generated and that the matching between new source
and existing target is working properly (otherwise JMerge would not know
which things should get pushed). So for some reason, pull is not working
at all. Of course uncounted users would have noticed if that was just
systematically broken and our tools tests would fail, so there must be
something environmental involved...

If I could reproduce the problem, I'd set a breakpoint in
org.eclipse.emf.codegen.merge.java.JMerger.applyPullRules(JNode, JNode)
to see why it does nothing.

Can you reproduce this problem with some small test case model?
>
>
Re: EMF Code Generator broken in 2.8.x? [message #998082 is a reply to message #998032] Wed, 09 January 2013 10:43 Go to previous messageGo to next message
Yves Alter is currently offline Yves AlterFriend
Messages: 11
Registered: January 2013
Junior Member
Thanks for your help! I now setup debugging for code generation. At first sight, it looks like the applyPullRules never makes the branch into the first (huge) condition, even though I do see the the correctly generated code within the sourceNode object and the existing code in the targetNode object. So there clearly is a difference. I'm going to investigate which part of the condition causes this behavior for the affected method.

Edit: There is no error in the error log.

[Updated on: Wed, 09 January 2013 10:43]

Report message to a moderator

Re: EMF Code Generator broken in 2.8.x? [message #998173 is a reply to message #998082] Wed, 09 January 2013 13:57 Go to previous messageGo to next message
Yves Alter is currently offline Yves AlterFriend
Messages: 11
Registered: January 2013
Junior Member
Ok, I traced the applyPullRules method up to the point were the PullRule for a feature method with name "getBody" and feature class "JMethod" is evaluated. I assume this is the one that gets and merges the body of the method? If not, I guess you can stop reading now.

This pull rule is skipped, because the last condition
targetPatternDictionary.isMarkedUp(pullRule.getTargetMarkup(), pullRule.getTargetParentMarkup(), targetNode)

evaluates to false.

pullRule.getTargetMarkup() returns the pattern "^gen$"
pullRule.getTargetParentMarkup() returns null

The methodMap in the targetPatternDictionary contains an entry for the affected method (notifyChanged) whose value is the ASTJMethod with the correct content.

Looking at the implementation of JPatternDictionary#isMarkedUp() reveils that either markupPattern must be null (it isn't) or the targetNode must be marked up with the pattern. This is not the case, since the markupMap is empty. I quickly checked, and no overload of JPatternDictionary.markupNode() is ever called. So before I dig any deeper, do you know if I'm on the right track at all?
Re: EMF Code Generator broken in 2.8.x? [message #998186 is a reply to message #998173] Wed, 09 January 2013 14:17 Go to previous messageGo to next message
Ed Merks is currently offline Ed MerksFriend
Messages: 26289
Registered: July 2009
Senior Member
Yves,

Yes, it sounds like you're on the right track! It's really behaving as
if those methods don't have @generated on them. This dictionary
pattern should be finding that.

<merge:dictionaryPattern
name="generatedUnmodifiableMembers"
select="Member/getComment"
match="@\s*(gen)erated\s*(This
field/method[^(?:\n\r?|\r\n?)]*)*(?:\n\r?|\r\n?)"/>

The pattern matching is done in
org.eclipse.emf.codegen.merge.java.JPatternDictionary.markupNode(String,
DictionaryPattern, JNode). It will be good to see what "selection"
strings come in. All the standard patterns look at the comment, so
this should be the full comment string. This should cause pretty much
everything to end up with "gen" (or "model") in the call to
org.eclipse.emf.codegen.merge.java.JPatternDictionary.markupNode(String,
JNode). Something must be going wrong astray in this stage...

On 09/01/2013 2:57 PM, Yves Alter wrote:
> Ok, I traced the applyPullRules method up to the point were the
> PullRule for a feature method with name "getBody" and feature class
> "JMethod" is evaluated. I assume this is the one that gets and merges
> the body of the method? If not, I guess you can stop reading now.
>
> This pull rule is skipped, because the last condition
> targetPatternDictionary.isMarkedUp(pullRule.getTargetMarkup(),
> pullRule.getTargetParentMarkup(), targetNode)
> evaluates to false.
>
> pullRule.getTargetMarkup() returns the pattern "^gen$"
> pullRule.getTargetParentMarkup() returns null
>
> The methodMap in the targetPatternDictionary contains an entry for the
> affected method (notifyChanged) whose value is the ASTJMethod with the
> correct content.
>
> Looking at the implementation of JPatternDictionary#isMarkedUp()
> reveils that either markupPattern must be null (it isn't) or the
> targetNode must be marked up with the pattern. This is not the case,
> since the markupMap is empty. I quickly checked, and no overload of
> JPatternDictionary.markupNode() is ever called. So before I dig any
> deeper, do you know if I'm on the right track at all?
>
Re: EMF Code Generator broken in 2.8.x? [message #998194 is a reply to message #998186] Wed, 09 January 2013 14:37 Go to previous messageGo to next message
Yves Alter is currently offline Yves AlterFriend
Messages: 11
Registered: January 2013
Junior Member
I was already debugging JPatternDictionary#beforeVisit() and selection is null, so markupNode is never called.

Edit: As you assumed, the ASTJMethod does not contain any signs of javadoc (optionalDocComment is null), but if I look at the scanner of the AST, the JavaDoc is present. Somehow smells like a JDT issue?

[Updated on: Wed, 09 January 2013 14:52]

Report message to a moderator

Re: EMF Code Generator broken in 2.8.x? [message #998206 is a reply to message #998194] Wed, 09 January 2013 15:05 Go to previous messageGo to next message
Ed Merks is currently offline Ed MerksFriend
Messages: 26289
Registered: July 2009
Senior Member
Yves,

Yes, that's what I suspected. After all, if there's a string value,
the pattern is known to work.

So that starts to point to some issue in
org.eclipse.emf.codegen.merge.java.facade.ast.ASTJMember.getComment()
returning null. That suggests
org.eclipse.emf.codegen.merge.java.facade.ast.ASTFacadeHelper.toString(ASTNode)
is not returning a good result. Have a look at what's going on there...

On 09/01/2013 3:37 PM, Yves Alter wrote:
> I was already debugging JPatternDictionary#beforeVisit() and selection
> is null, so markupNode is never called.
Re: EMF Code Generator broken in 2.8.x? [message #998211 is a reply to message #998206] Wed, 09 January 2013 15:18 Go to previous messageGo to next message
Yves Alter is currently offline Yves AlterFriend
Messages: 11
Registered: January 2013
Junior Member
Yes, I'd already edited my previous post... the reason is that MethodDeclaration#optionalDocComment is null - and that is returned by getASTNode().getJavadoc(), which is used in the implementation of ASTJMember#getComment(). I think I'll start looking at the JDT source. Tumbling down the rabbit hole...
Re: EMF Code Generator broken in 2.8.x? [message #998216 is a reply to message #998211] Wed, 09 January 2013 15:34 Go to previous messageGo to next message
Ed Merks is currently offline Ed MerksFriend
Messages: 26289
Registered: July 2009
Senior Member
Yves,

Is getASTNode().getJavadoc() returning null? I get the sense that
there's some kind of bad option floating around in your environment
that's telling JDT not to process comments because it's not possible for
JDT to have a bug in this widely used code... Maybe for
Help->Preferences->Java->Compiler->Javadoc you have "Process Javadoc
comments" set to false. I'll test that now...


On 09/01/2013 4:18 PM, Yves Alter wrote:
> Yes, I'd already edited my previous post... the reason is that
> MethodDeclaration#optionalDocComment is null - and that is returned by
> getASTNode().getJavadoc(), which is used in the implementation of
> ASTJMember#getComment(). I think I'll start looking at the JDT source.
> Tumbling down the rabbit hole...
Re: EMF Code Generator broken in 2.8.x? [message #998218 is a reply to message #998216] Wed, 09 January 2013 15:40 Go to previous messageGo to next message
Ed Merks is currently offline Ed MerksFriend
Messages: 26289
Registered: July 2009
Senior Member
Yves,

Yes, I believe that's exactly the problem! Please confirm and open a
bugzilla. Of course the workaround is obvious, but we should be sure we
set this option to true, regardless of the workspace preference.


On 09/01/2013 4:34 PM, Ed Merks wrote:
> Yves,
>
> Is getASTNode().getJavadoc() returning null? I get the sense that
> there's some kind of bad option floating around in your environment
> that's telling JDT not to process comments because it's not possible
> for JDT to have a bug in this widely used code... Maybe for
> Help->Preferences->Java->Compiler->Javadoc you have "Process Javadoc
> comments" set to false. I'll test that now...
>
>
> On 09/01/2013 4:18 PM, Yves Alter wrote:
>> Yes, I'd already edited my previous post... the reason is that
>> MethodDeclaration#optionalDocComment is null - and that is returned
>> by getASTNode().getJavadoc(), which is used in the implementation of
>> ASTJMember#getComment(). I think I'll start looking at the JDT
>> source. Tumbling down the rabbit hole...
>
Re: EMF Code Generator broken in 2.8.x? [message #998222 is a reply to message #998218] Wed, 09 January 2013 15:48 Go to previous messageGo to next message
Yves Alter is currently offline Yves AlterFriend
Messages: 11
Registered: January 2013
Junior Member

Unbelievable! That option was actually disabled and enabling it fixed code generation. It is very interesting, that the problem did not occur in eclipse 3.7.2, as I used the same workspace (and in 3.7.2 this option was also deselected, I don't know why though).
Re: EMF Code Generator broken in 2.8.x? [message #998245 is a reply to message #998218] Wed, 09 January 2013 16:37 Go to previous messageGo to next message
Yves Alter is currently offline Yves AlterFriend
Messages: 11
Registered: January 2013
Junior Member
Ed, thank you very much for you help! I filed the bug under https://bugs.eclipse.org/bugs/show_bug.cgi?id=397776
Re: EMF Code Generator broken in 2.8.x? [message #998519 is a reply to message #998245] Thu, 10 January 2013 06:51 Go to previous message
Ed Merks is currently offline Ed MerksFriend
Messages: 26289
Registered: July 2009
Senior Member
Yves,

In the end I helped you help yourself (most people won't take this kind
of initiative). I'm happy we tracked it down so that I have an
opportunity to ensure no one else falls into this trap in the future.


On 09/01/2013 5:37 PM, Yves Alter wrote:
> Ed, thank you very much for you help! I filed the bug under
> https://bugs.eclipse.org/bugs/show_bug.cgi?id=397776
>
Previous Topic:[CDO] CDO+db4o run from RCP application
Next Topic:Loading resource from jar file
Goto Forum:
  


Current Time: Mon Dec 22 23:34:00 GMT 2014

Powered by FUDForum. Page generated in 0.07054 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software