Eclipse Community Forums: TMF (Xtext) » Chaining of code generators

Home » Modeling » TMF (Xtext) » Chaining of code generators

Chaining of code generators [message #1690092]

Wed, 25 March 2015 04:46

Eclipse User

Hi,

I have the following use case with two DSLs:

- Grammar Foo (inherits from Xbase)
-- file extension: *.foo
-- FooGenerator generates a *.java file in src-gen folder

- Grammar Bar (inherits from Xbase)
-- Bar is a high level abstraction language for Foo
-- file extension: *.bar
-- BarGenerator generates a *.foo file in src-gen folder

The intention is that for every *.bar file there is a *.foo file and a *.java file.

Unfortunately saving a *.bar file does not trigger the XtextBuilder for the generated /src-gen/*.foo file and the *.java file is therefore never generated.

Is there any way to configure the builder to be triggered for the *.foo file?

I understand I can inject the FooGenerator into the BarBenerator. However I'm not sure whether this approach is appropriate usage of Xtext code generation.

Thanks,
Nils

Re: Chaining of code generators [message #1690526 is a reply to message #1690092]

Fri, 27 March 2015 21:02

Eclipse User

Yeah.

Same issue here. Although we actually have references between our DSLs, the child-DSL files never really change (thus don't show up in Eclipse's deltas) and thus aren't build.

Re: Chaining of code generators [message #1690531 is a reply to message #1690526]

Sat, 28 March 2015 01:43

Eclipse User

Hi,

this is something different.

(case a) generator of lang a generates lang b files and generator of lang b is not triggert

(case b) change to file of lang a does not trigger generator of dependent files of lang b.

for case b: i don't know how your references look like but you may digg into
org.eclipse.xtext.resource.IResourceDescription.Manager.isAffected(Delta, IResourceDescription)
org.eclipse.xtext.resource.IResourceDescription.Manager.isAffected(Collection<Delta>, IResourceDescription, IResourceDescriptions)

https://www.eclipse.org/forums/index.php/m/1269072/?srch=isaffected#msg_1269072

Re: Chaining of code generators [message #1690540 is a reply to message #1690531]

Sat, 28 March 2015 09:57

Eclipse User

Thanks Christian for the pointers.
However, I haven't been able to figure out how to do this.

Lets say I have two DSLs A and B. I define an Object in A and reference it in B (there's also an import statement in B that points to A).
If I change that Object in A, the IResourceDescription.Manager for B is never even called.

Re: Chaining of code generators [message #1690541 is a reply to message #1690540]

Sat, 28 March 2015 10:06

Eclipse User

No but the one for a and it should find out that b is (i dont know the details of the impl. You might use he debugger to find out whst happens).

Re: Chaining of code generators [message #1690542 is a reply to message #1690541]

Sat, 28 March 2015 10:16

Eclipse User

Btw with a simple example this works fine

Re: Chaining of code generators [message #1690543 is a reply to message #1690542]

Sat, 28 March 2015 10:31

Eclipse User

Well no.
As far as I understand it (doing some digging in the code), the Manager for A is called but that one can only decide whether A is affected or not. That's it.
So far, there doesn't seem to be a way for the Manager for A to tell Xtext/Eclipse to also trigger a build for B

Re: Chaining of code generators [message #1690544 is a reply to message #1690543]

Sat, 28 March 2015 10:41

Eclipse User

Hi in my exaple i altered file a (really altered not only touched since there is a change detection. Now the methodof themanager for b is called and returns true which triggers a build for b

Re: Chaining of code generators [message #1690545 is a reply to message #1690544]

Sat, 28 March 2015 10:48

Eclipse User

hmm.. Interesting.
Doesn't do it for me.

It works the other way around (changing B) but not if I only change A

Re: Chaining of code generators [message #1690546 is a reply to message #1690545]

Sat, 28 March 2015 10:53

Eclipse User

Can you share the code of the dsls and two example projects. Maybe there is something wrong with your customizations. Btw which xtext version do you use

Re: Chaining of code generators [message #1690547 is a reply to message #1690546]

Sat, 28 March 2015 11:00

Eclipse User

2.8.0

I don't think I can share the code. It's proprietary so I'd need to ask for permission to publish/share it first.
Maybe I can share the relevant parts without having to share the entire project?

Re: Chaining of code generators [message #1690550 is a reply to message #1690547]

Sat, 28 March 2015 11:09

Eclipse User

Creating a minimal example that reproduces the problem will help as well. Maybe it even helps you to find out what goes wrong. I used 281.

Re: Chaining of code generators [message #1690591 is a reply to message #1690550]

Sun, 29 March 2015 13:15

Eclipse User

Ok. I dug a bit deeper and figured out what's going on:

What's really happening is, that the Manager for B can't determine it is affected because the definition in A that changed is not directly referenced in B. It's a little complicated to explain, but let's use this as an abstract example to illustrate what I mean:

DSL A:

SomeClass {
 String variable
}

method someMethod {
 params: String aString
 returns SomeClass
}

DSL B:

someMethod: /api/{aString}

So, "aString" in DSL B clearly references the param in DSL A. But if I now change the return statement of someMethod, DSL B doesn't get build.
(In reality it's a little bit more complicated than that, but I hope you still get the point.)

There's a reference between DSL B and DSL A, so all I need to do is to figure out, that DSL A was changed.... And that's where I'm stuck now, because there doesn't seem to be any reference to DSL A in the deltas I get in the Manager.

Re: Chaining of code generators [message #1690592 is a reply to message #1690591]

Sun, 29 March 2015 13:37

Eclipse User

hmmm, maybe you can detect the change in your case? or make the test a bit more generic and return true if anything from file a is used in file b

Re: Chaining of code generators [message #1690595 is a reply to message #1690592]

Sun, 29 March 2015 14:07

Eclipse User

That's what I thought.. but that would require that I know which A-file changed so I can check whether the B-file candidate is using something from that file (I don't want to build every B-file when ONE A-file changes).

Or, for that matter, to even know that an A-file changed. Neither one seems to be possible (or I just don't see it...)

[Updated on: Sun, 29 March 2015 14:08] by Moderator

Re: Chaining of code generators [message #1690596 is a reply to message #1690595]

Sun, 29 March 2015 14:12

Eclipse User

the information is there. (in the deltas old/new resoucedescriptions uri)

[Updated on: Sun, 29 March 2015 14:18] by Moderator

Re: Chaining of code generators [message #1690597 is a reply to message #1690596]

Sun, 29 March 2015 14:23

Eclipse User

nope..
The deltas contain references to generated Java files, e.g. java:/Objects/com.bitzeche.looptest.CreateLoopInContextUserIsAuthenticatedResponseEvent
Not very useful

[Updated on: Sun, 29 March 2015 14:23] by Moderator

Re: Chaining of code generators [message #1690598 is a reply to message #1690597]

Sun, 29 March 2015 14:51

Eclipse User

One step closer to even understanding what's going on. That build where I get the deltas with Java class references happens AFTER the A-file was built. So, it first builds the A-file, which changes a few Java classes, which in turn triggers another build.
And the isAffected method for the b-files is only triggered for that second run.

Which brings me back to the initial question. How do I trigger the build for the b-file when the a-file is built?
I noticed that the deltas are empty for that first run, which is why the ClusteringBuilderState stops looking for affected Resources (in queueAffectedResources)

Re: Chaining of code generators [message #1690599 is a reply to message #1690598]

Sun, 29 March 2015 14:55

Eclipse User

Hi,

i still do not get your problem: if you do a build on file a the isaffected for file b will be called during that build. and if isaffected returns true, the file b will be build as well
here is the stacktrace from my example

MyDslBRDM.isAffected(Collection<Delta>, IResourceDescription, IResourceDescriptions) line: 16
ClusteringBuilderState.queueAffectedResources(Set<URI>, IResourceDescriptions, CurrentDescriptions, Collection<Delta>, Collection<Delta>, BuildData, IProgressMonitor) line: 534
ClusteringBuilderState.doUpdate(BuildData, ResourceDescriptionsData, IProgressMonitor) line: 306
ClusteringBuilderState(AbstractBuilderState).update(BuildData, IProgressMonitor) line: 115
XtextBuilder.doBuild(ToBeBuilt, IProgressMonitor, IXtextBuilderParticipant$BuildType) line: 242
XtextBuilder.incrementalBuild(IResourceDelta, IProgressMonitor) line: 222
XtextBuilder.build(int, Map, IProgressMonitor) line: 117

since i have no idea whats going on in your case why this is not happening i cannot give you futher hints

Re: Chaining of code generators [message #1690600 is a reply to message #1690599]

Sun, 29 March 2015 15:14

Eclipse User

Yeah.. that doesn't happen for me.

ClusteringBuilderState.queueAffectedResources doesn't even check for affected resources, because there are no deltas (line 510)

Re: Chaining of code generators [message #1690601 is a reply to message #1690600]

Sun, 29 March 2015 15:20

Eclipse User

so you may need to change the indexing of dsl a to change that?

Re: Chaining of code generators [message #1690602 is a reply to message #1690601]

Sun, 29 March 2015 15:27

Eclipse User

How would I do that?

Re: Chaining of code generators [message #1690603 is a reply to message #1690602]

Sun, 29 March 2015 15:29

Eclipse User

by changing the defaultresoucedescriptionstrategy in a manner that all relenant information is stored in the index. eg the return type of the method

Re: Chaining of code generators [message #1690604 is a reply to message #1690603]

Sun, 29 March 2015 15:46

Eclipse User

hmm... I might give that a try.

meanwhile, another nugget of information:
The reason why there aren't any deltas for the initial build is, that the DeltaConverter discards any Deltas related to the DSL file. No idea whether this is how it's supposed to be, but that's what I'm seeing
Not sure how changing the index would change that?

Re: Chaining of code generators [message #1690605 is a reply to message #1690604]

Sun, 29 March 2015 15:54

Eclipse User

Hi,

please try it with this simple dsls and compare the observations with yours:

grammar org.xtext.example.mydsla.MyDslA with org.eclipse.xtext.common.Terminals

generate myDslA "http://www.xtext.org/example/mydsla/MyDslA"

Model:
	Packages+=Package*
;
Package:
	"package" name=ID
	definitions+=Definition*
;	
	
Definition:
	'define' name=ID;

grammar org.xtext.example.mydslb.MyDslB with org.eclipse.xtext.common.Terminals

generate myDslB "http://www.xtext.org/example/mydslb/MyDslB"

import "http://www.xtext.org/example/mydsla/MyDslA" as a

Model:
	imports+=Import*
	usages+=Usage*;
	
Import:
	"import" importedNamespace=FQNWC
;	

FQNWC:
	ID ("." ID)* (".""*")?
;
	
Usage:
	'use' def=[a::Definition] '!';

Re: Chaining of code generators [message #1690606 is a reply to message #1690605]

Sun, 29 March 2015 16:09

Eclipse User

ok. I'll do that. Might take me a while though.

I just discovered, that Eclipse thinks, nothing has changed. When it calls the haveEObjectDescriptionsChanged() method on the ResourceDescriptionDelta for the DSL file I changed, that method returns false indicating there were no changes.

Re: Chaining of code generators [message #1690607 is a reply to message #1690606]

Sun, 29 March 2015 16:13

Eclipse User

yes this is why you should add the whole signature to the EObjectDescription of the method, e.g. by storing params and return types to the user data map or within the qualified names

Re: Chaining of code generators [message #1690612 is a reply to message #1690607]

Sun, 29 March 2015 20:27

Eclipse User

Awesome. That did it.

I wasn't aware this was required (or even necessary).

Thanks for your patience!

Previous Topic:	Using a custom Xbase-compiler
Next Topic:	plugin Dependencies 2.8

Goto Forum:

-=] Back to Top [=-

Current Time: Tue Jul 15 09:37:52 EDT 2025

.:: Contact :: Home ::.

Breadcrumbs

Sign up to our Newsletter