Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » TMF (Xtext) » Chaining of code generators
Chaining of code generators [message #1690092] Wed, 25 March 2015 08:46 Go to next message
Nils Ruhr is currently offline Nils RuhrFriend
Messages: 9
Registered: January 2015
Junior Member
Hi,

I have the following use case with two DSLs:

- Grammar Foo (inherits from Xbase)
-- file extension: *.foo
-- FooGenerator generates a *.java file in src-gen folder

- Grammar Bar (inherits from Xbase)
-- Bar is a high level abstraction language for Foo
-- file extension: *.bar
-- BarGenerator generates a *.foo file in src-gen folder

The intention is that for every *.bar file there is a *.foo file and a *.java file.

Unfortunately saving a *.bar file does not trigger the XtextBuilder for the generated /src-gen/*.foo file and the *.java file is therefore never generated.

Is there any way to configure the builder to be triggered for the *.foo file?

I understand I can inject the FooGenerator into the BarBenerator. However I'm not sure whether this approach is appropriate usage of Xtext code generation.

Thanks,
Nils
Re: Chaining of code generators [message #1690526 is a reply to message #1690092] Sat, 28 March 2015 01:02 Go to previous messageGo to next message
Patrice Brendamour is currently offline Patrice BrendamourFriend
Messages: 24
Registered: March 2015
Junior Member
Yeah.

Same issue here. Although we actually have references between our DSLs, the child-DSL files never really change (thus don't show up in Eclipse's deltas) and thus aren't build.
Re: Chaining of code generators [message #1690531 is a reply to message #1690526] Sat, 28 March 2015 05:43 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 13884
Registered: July 2009
Senior Member
Hi,

this is something different.

(case a) generator of lang a generates lang b files and generator of lang b is not triggert

(case b) change to file of lang a does not trigger generator of dependent files of lang b.

for case b: i don't know how your references look like but you may digg into
org.eclipse.xtext.resource.IResourceDescription.Manager.isAffected(Delta, IResourceDescription)
org.eclipse.xtext.resource.IResourceDescription.Manager.isAffected(Collection<Delta>, IResourceDescription, IResourceDescriptions)

https://www.eclipse.org/forums/index.php/m/1269072/?srch=isaffected#msg_1269072


Need professional support for Xtext, Xpand, EMF?
Go to: https://www.itemis.com/en/it-services/methods-and-tools/xtext
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Re: Chaining of code generators [message #1690540 is a reply to message #1690531] Sat, 28 March 2015 13:57 Go to previous messageGo to next message
Patrice Brendamour is currently offline Patrice BrendamourFriend
Messages: 24
Registered: March 2015
Junior Member
Thanks Christian for the pointers.
However, I haven't been able to figure out how to do this.

Lets say I have two DSLs A and B. I define an Object in A and reference it in B (there's also an import statement in B that points to A).
If I change that Object in A, the IResourceDescription.Manager for B is never even called.
Re: Chaining of code generators [message #1690541 is a reply to message #1690540] Sat, 28 March 2015 14:06 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 13884
Registered: July 2009
Senior Member
No but the one for a and it should find out that b is (i dont know the details of the impl. You might use he debugger to find out whst happens).

Need professional support for Xtext, Xpand, EMF?
Go to: https://www.itemis.com/en/it-services/methods-and-tools/xtext
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Re: Chaining of code generators [message #1690542 is a reply to message #1690541] Sat, 28 March 2015 14:16 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 13884
Registered: July 2009
Senior Member
Btw with a simple example this works fine

Need professional support for Xtext, Xpand, EMF?
Go to: https://www.itemis.com/en/it-services/methods-and-tools/xtext
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Re: Chaining of code generators [message #1690543 is a reply to message #1690542] Sat, 28 March 2015 14:31 Go to previous messageGo to next message
Patrice Brendamour is currently offline Patrice BrendamourFriend
Messages: 24
Registered: March 2015
Junior Member
Well no.
As far as I understand it (doing some digging in the code), the Manager for A is called but that one can only decide whether A is affected or not. That's it.
So far, there doesn't seem to be a way for the Manager for A to tell Xtext/Eclipse to also trigger a build for B
Re: Chaining of code generators [message #1690544 is a reply to message #1690543] Sat, 28 March 2015 14:41 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 13884
Registered: July 2009
Senior Member
Hi in my exaple i altered file a (really altered not only touched since there is a change detection. Now the methodof themanager for b is called and returns true which triggers a build for b

Need professional support for Xtext, Xpand, EMF?
Go to: https://www.itemis.com/en/it-services/methods-and-tools/xtext
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Re: Chaining of code generators [message #1690545 is a reply to message #1690544] Sat, 28 March 2015 14:48 Go to previous messageGo to next message
Patrice Brendamour is currently offline Patrice BrendamourFriend
Messages: 24
Registered: March 2015
Junior Member
hmm.. Interesting.
Doesn't do it for me.

It works the other way around (changing B) but not if I only change A
Re: Chaining of code generators [message #1690546 is a reply to message #1690545] Sat, 28 March 2015 14:53 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 13884
Registered: July 2009
Senior Member
Can you share the code of the dsls and two example projects. Maybe there is something wrong with your customizations. Btw which xtext version do you use

Need professional support for Xtext, Xpand, EMF?
Go to: https://www.itemis.com/en/it-services/methods-and-tools/xtext
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Re: Chaining of code generators [message #1690547 is a reply to message #1690546] Sat, 28 March 2015 15:00 Go to previous messageGo to next message
Patrice Brendamour is currently offline Patrice BrendamourFriend
Messages: 24
Registered: March 2015
Junior Member
2.8.0

I don't think I can share the code. It's proprietary so I'd need to ask for permission to publish/share it first.
Maybe I can share the relevant parts without having to share the entire project?

Re: Chaining of code generators [message #1690550 is a reply to message #1690547] Sat, 28 March 2015 15:09 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 13884
Registered: July 2009
Senior Member
Creating a minimal example that reproduces the problem will help as well. Maybe it even helps you to find out what goes wrong. I used 281.

Need professional support for Xtext, Xpand, EMF?
Go to: https://www.itemis.com/en/it-services/methods-and-tools/xtext
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Re: Chaining of code generators [message #1690591 is a reply to message #1690550] Sun, 29 March 2015 17:15 Go to previous messageGo to next message
Patrice Brendamour is currently offline Patrice BrendamourFriend
Messages: 24
Registered: March 2015
Junior Member
Ok. I dug a bit deeper and figured out what's going on:

What's really happening is, that the Manager for B can't determine it is affected because the definition in A that changed is not directly referenced in B. It's a little complicated to explain, but let's use this as an abstract example to illustrate what I mean:

DSL A:
SomeClass {
 String variable
}

method someMethod {
 params: String aString
 returns SomeClass
}


DSL B:
someMethod: /api/{aString} 


So, "aString" in DSL B clearly references the param in DSL A. But if I now change the return statement of someMethod, DSL B doesn't get build.
(In reality it's a little bit more complicated than that, but I hope you still get the point.)

There's a reference between DSL B and DSL A, so all I need to do is to figure out, that DSL A was changed.... And that's where I'm stuck now, because there doesn't seem to be any reference to DSL A in the deltas I get in the Manager.
Re: Chaining of code generators [message #1690592 is a reply to message #1690591] Sun, 29 March 2015 17:37 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 13884
Registered: July 2009
Senior Member
hmmm, maybe you can detect the change in your case? or make the test a bit more generic and return true if anything from file a is used in file b

Need professional support for Xtext, Xpand, EMF?
Go to: https://www.itemis.com/en/it-services/methods-and-tools/xtext
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Re: Chaining of code generators [message #1690595 is a reply to message #1690592] Sun, 29 March 2015 18:07 Go to previous messageGo to next message
Patrice Brendamour is currently offline Patrice BrendamourFriend
Messages: 24
Registered: March 2015
Junior Member
That's what I thought.. but that would require that I know which A-file changed so I can check whether the B-file candidate is using something from that file (I don't want to build every B-file when ONE A-file changes).

Or, for that matter, to even know that an A-file changed. Neither one seems to be possible (or I just don't see it...)

[Updated on: Sun, 29 March 2015 18:08]

Report message to a moderator

Re: Chaining of code generators [message #1690596 is a reply to message #1690595] Sun, 29 March 2015 18:12 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 13884
Registered: July 2009
Senior Member
the information is there. (in the deltas old/new resoucedescriptions uri)

Need professional support for Xtext, Xpand, EMF?
Go to: https://www.itemis.com/en/it-services/methods-and-tools/xtext
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de

[Updated on: Sun, 29 March 2015 18:18]

Report message to a moderator

Re: Chaining of code generators [message #1690597 is a reply to message #1690596] Sun, 29 March 2015 18:23 Go to previous messageGo to next message
Patrice Brendamour is currently offline Patrice BrendamourFriend
Messages: 24
Registered: March 2015
Junior Member
nope..
The deltas contain references to generated Java files, e.g. java:/Objects/com.bitzeche.looptest.CreateLoopInContextUserIsAuthenticatedResponseEvent
Not very useful

[Updated on: Sun, 29 March 2015 18:23]

Report message to a moderator

Re: Chaining of code generators [message #1690598 is a reply to message #1690597] Sun, 29 March 2015 18:51 Go to previous messageGo to next message
Patrice Brendamour is currently offline Patrice BrendamourFriend
Messages: 24
Registered: March 2015
Junior Member
One step closer to even understanding what's going on. That build where I get the deltas with Java class references happens AFTER the A-file was built. So, it first builds the A-file, which changes a few Java classes, which in turn triggers another build.
And the isAffected method for the b-files is only triggered for that second run.

Which brings me back to the initial question. How do I trigger the build for the b-file when the a-file is built?
I noticed that the deltas are empty for that first run, which is why the ClusteringBuilderState stops looking for affected Resources (in queueAffectedResources)
Re: Chaining of code generators [message #1690599 is a reply to message #1690598] Sun, 29 March 2015 18:55 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 13884
Registered: July 2009
Senior Member
Hi,

i still do not get your problem: if you do a build on file a the isaffected for file b will be called during that build. and if isaffected returns true, the file b will be build as well
here is the stacktrace from my example

MyDslBRDM.isAffected(Collection<Delta>, IResourceDescription, IResourceDescriptions) line: 16
ClusteringBuilderState.queueAffectedResources(Set<URI>, IResourceDescriptions, CurrentDescriptions, Collection<Delta>, Collection<Delta>, BuildData, IProgressMonitor) line: 534
ClusteringBuilderState.doUpdate(BuildData, ResourceDescriptionsData, IProgressMonitor) line: 306
ClusteringBuilderState(AbstractBuilderState).update(BuildData, IProgressMonitor) line: 115
XtextBuilder.doBuild(ToBeBuilt, IProgressMonitor, IXtextBuilderParticipant$BuildType) line: 242
XtextBuilder.incrementalBuild(IResourceDelta, IProgressMonitor) line: 222
XtextBuilder.build(int, Map, IProgressMonitor) line: 117

since i have no idea whats going on in your case why this is not happening i cannot give you futher hints


Need professional support for Xtext, Xpand, EMF?
Go to: https://www.itemis.com/en/it-services/methods-and-tools/xtext
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Re: Chaining of code generators [message #1690600 is a reply to message #1690599] Sun, 29 March 2015 19:14 Go to previous messageGo to next message
Patrice Brendamour is currently offline Patrice BrendamourFriend
Messages: 24
Registered: March 2015
Junior Member
Yeah.. that doesn't happen for me.

ClusteringBuilderState.queueAffectedResources doesn't even check for affected resources, because there are no deltas (line 510)
Re: Chaining of code generators [message #1690601 is a reply to message #1690600] Sun, 29 March 2015 19:20 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 13884
Registered: July 2009
Senior Member
so you may need to change the indexing of dsl a to change that?

Need professional support for Xtext, Xpand, EMF?
Go to: https://www.itemis.com/en/it-services/methods-and-tools/xtext
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Re: Chaining of code generators [message #1690602 is a reply to message #1690601] Sun, 29 March 2015 19:27 Go to previous messageGo to next message
Patrice Brendamour is currently offline Patrice BrendamourFriend
Messages: 24
Registered: March 2015
Junior Member
How would I do that?
Re: Chaining of code generators [message #1690603 is a reply to message #1690602] Sun, 29 March 2015 19:29 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 13884
Registered: July 2009
Senior Member
by changing the defaultresoucedescriptionstrategy in a manner that all relenant information is stored in the index. eg the return type of the method

Need professional support for Xtext, Xpand, EMF?
Go to: https://www.itemis.com/en/it-services/methods-and-tools/xtext
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Re: Chaining of code generators [message #1690604 is a reply to message #1690603] Sun, 29 March 2015 19:46 Go to previous messageGo to next message
Patrice Brendamour is currently offline Patrice BrendamourFriend
Messages: 24
Registered: March 2015
Junior Member
hmm... I might give that a try.

meanwhile, another nugget of information:
The reason why there aren't any deltas for the initial build is, that the DeltaConverter discards any Deltas related to the DSL file. No idea whether this is how it's supposed to be, but that's what I'm seeing
Not sure how changing the index would change that?
Re: Chaining of code generators [message #1690605 is a reply to message #1690604] Sun, 29 March 2015 19:54 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 13884
Registered: July 2009
Senior Member
Hi,

please try it with this simple dsls and compare the observations with yours:

grammar org.xtext.example.mydsla.MyDslA with org.eclipse.xtext.common.Terminals

generate myDslA "http://www.xtext.org/example/mydsla/MyDslA"

Model:
	Packages+=Package*
;
Package:
	"package" name=ID
	definitions+=Definition*
;	
	
Definition:
	'define' name=ID;
grammar org.xtext.example.mydslb.MyDslB with org.eclipse.xtext.common.Terminals

generate myDslB "http://www.xtext.org/example/mydslb/MyDslB"

import "http://www.xtext.org/example/mydsla/MyDslA" as a

Model:
	imports+=Import*
	usages+=Usage*;
	
Import:
	"import" importedNamespace=FQNWC
;	

FQNWC:
	ID ("." ID)* (".""*")?
;
	
Usage:
	'use' def=[a::Definition] '!';


Need professional support for Xtext, Xpand, EMF?
Go to: https://www.itemis.com/en/it-services/methods-and-tools/xtext
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Re: Chaining of code generators [message #1690606 is a reply to message #1690605] Sun, 29 March 2015 20:09 Go to previous messageGo to next message
Patrice Brendamour is currently offline Patrice BrendamourFriend
Messages: 24
Registered: March 2015
Junior Member
ok. I'll do that. Might take me a while though.

I just discovered, that Eclipse thinks, nothing has changed. When it calls the haveEObjectDescriptionsChanged() method on the ResourceDescriptionDelta for the DSL file I changed, that method returns false indicating there were no changes.
Re: Chaining of code generators [message #1690607 is a reply to message #1690606] Sun, 29 March 2015 20:13 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 13884
Registered: July 2009
Senior Member
yes this is why you should add the whole signature to the EObjectDescription of the method, e.g. by storing params and return types to the user data map or within the qualified names

Need professional support for Xtext, Xpand, EMF?
Go to: https://www.itemis.com/en/it-services/methods-and-tools/xtext
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Re: Chaining of code generators [message #1690612 is a reply to message #1690607] Mon, 30 March 2015 00:27 Go to previous message
Patrice Brendamour is currently offline Patrice BrendamourFriend
Messages: 24
Registered: March 2015
Junior Member
Awesome. That did it.

I wasn't aware this was required (or even necessary).

Thanks for your patience!
Previous Topic:Using a custom Xbase-compiler
Next Topic:plugin Dependencies 2.8
Goto Forum:
  


Current Time: Sat Apr 17 09:24:51 GMT 2021

Powered by FUDForum. Page generated in 0.03533 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top