Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » TMF (Xtext) » Building an xtext language using gradle, in the main thread
Building an xtext language using gradle, in the main thread [message #1852468] Tue, 17 May 2022 11:57 Go to next message
Andrzej Wasowski is currently offline Andrzej WasowskiFriend
Messages: 15
Registered: June 2015
Junior Member

Gradle upgrades have invalidated my clunky xtext build scripts, and I am giving them a new shot (somehow the "compile" dependency depends differently than "implementation" but let's not discuss it now). I am trying to use the opportunity to also make the xtext language build faster, by not spawning another JVM instance, which my old JavaExec task did. I came up with the following, which looks pretty close, but not quite there yet:

plugins {
  id "org.xtext.xtend"
  id "java-library"
}
dependencies {
  implementation project(':dsldesign.expr')
  implementation "org.eclipse.emf:org.eclipse.emf.mwe2.launch:${mweVersion}"
  implementation "org.eclipse.xtend:org.eclipse.xtend.lib:${xtendVersion}"
  implementation "org.eclipse.xtext:org.eclipse.xtext.common.types:${xtextVersion}"
  implementation "org.eclipse.xtext:org.eclipse.xtext.generator:${xtextVersion}"
  implementation "org.eclipse.xtext:org.eclipse.xtext.xbase:${xtextVersion}"
  implementation "org.eclipse.xtext:org.eclipse.xtext.xtext.generator:${xtextVersion}"
  implementation "org.eclipse.xtext:org.eclipse.xtext:${xtextVersion}"
  implementation "org.eclipse.xtext:xtext-antlr-generator:[2.1.1, 3)"
}

def root = rootProject.projectDir.toURI ()
tasks.register ('exprXtextCode') {
    doLast {
      def classloader = new URLClassLoader (
        [root.toURL ()] as URL[], 
        gradle.class.classLoader
      )
      configurations
        .compileClasspath
        .each { 
          logger.info ("Adding to classpath: ${it.toURI ()}")
          classloader.addURL (it.toURI ().toURL ()) 
        }
      def mwe2 = classloader
        .loadClass('org.eclipse.emf.mwe2.launch.runtime.Mwe2Launcher')
        .newInstance ()
      logger.lifecycle ("MWE2 launcher class loaded.")
      logger.lifecycle ("Starting MWE2 workflow in ${root.path}")
      mwe2.run([
          "${root.toURL ()}/dsldesign.expr.xtext/src/main/java/dsldesign/expr/xtext/GenerateExpr.mwe2",
          "-p", 
          "rootPath=${root.path}"] as String[]
      )
      logger.lifecycle ("MWE2 workflow completed.")
    }
}
compileJava.dependsOn exprXtextCode


I believe that I have all the jars in dependencies as I should (at least in the old set up they were enough). The first dependency is on the EMF project with the model/genmodel. The task instantiates a class loader, then puts all the dependencies on the current classpath. Then we load Mwe2Launcher and use the run functions to start it, with the root set to the root of the project (this is a nested-project, so root is the parent). The MWE2 script is unsurprising, as created by the wizard some years ago.

Unfortunately attempting to invoke the task produces this error message:

...
Adding to classpath: file:/home/wasowski/.gradle/caches/modules-2/files-2.1/antlr/antlr/2.7.7/83cd2cd674a217ade95a4bb83a8a14f351f48bd0/antlr-2.7.7.jar                                                                                                               
MWE2 launcher class loaded.                                                                                                                                                                                                                                    
Starting MWE2 workflow in /home/wasowski/work/dsldesign/                                                                                                                                                                                                       
Initializing Xtext generator
Adding generated EPackage 'org.eclipse.xtext.common.types.TypesPackage'
No project file found for /home/wasowski/work/dsldesign/dsldesign.expr/build/classes/java/main. The folder was neither an Eclipse 'bin' folder, nor a Maven 'target/classes' folder, nor a Gradle bin/main folder
Registering project dsldesign.expr.xtext at 'file:/home/wasowski/work/dsldesign/dsldesign.expr.xtext/'
Registering project dsldesign.expr.xtext at 'file:/home/wasowski/work/dsldesign/dsldesign.expr.xtext/'
Using resourceSet registry. The registered Packages will not be registered in the global EPackage.Registry.INSTANCE!
Registered GenModel 'http://www.eclipse.org/Xtext/Xbase/XAnnotations' from 'platform:/resource/org.eclipse.xtext.xbase/model/Xbase.genmodel'
Registered GenModel 'http://www.eclipse.org/xtext/xbase/Xtype' from 'platform:/resource/org.eclipse.xtext.xbase/model/Xbase.genmodel'
Registered GenModel 'http://www.eclipse.org/xtext/xbase/Xbase' from 'platform:/resource/org.eclipse.xtext.xbase/model/Xbase.genmodel'
Registered GenModel 'http://www.eclipse.org/xtext/common/JavaVMTypes' from 'platform:/resource/org.eclipse.xtext.common.types/model/JavaVMTypes.genmodel'
Registered GenModel 'http://www.dsl.design/dsldesign.expr' from 'platform:/resource/dsldesign.expr/model/expr.genmodel'
[XtextLinkingDiagnostic: null:1 Couldn't resolve reference to Grammar 'org.eclipse.xtext.common.Terminals'., XtextLinkingDiagnostic: null:19 STRING cannot be resolved to a rule, XtextLinkingDiagnostic: null:19 ID cannot be resolved to a rule, TransformationDiagnostic: null:18 Datatype rules may only use other datatype rules, lexer rules and keywords. (ErrorCode: InvalidDatatypeRule)]


There is disturbing attempt to find something in the build tree and there strange double registration for my project (the current project), after this the Xtext generator seem to proceed as it should, until it fails to find the Terminals grammar. This one should be in the org.eclipse.xtext:org.eclipse.xtext:${xtextVersion} jar, and I have checked that the jar is in the classpath.

I am starting to get desperate, and considering switching to a JavaExec task again, attempting to fix my old ways (although the above is nicely fast). Any suggestions of what could be wrong? Any leads are highly appreciated...

[Updated on: Tue, 17 May 2022 12:00]

Report message to a moderator

Re: Building an xtext language using gradle, in the main thread [message #1852469 is a reply to message #1852468] Tue, 17 May 2022 12:03 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 14349
Registered: July 2009
Senior Member
i propose you remote debug.
is there a reason you dont use "the default thing" what the wizard creates


Need professional support for Xtext, Xpand, EMF?
Go to: https://www.itemis.com/en/it-services/methods-and-tools/xtext
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Re: Building an xtext language using gradle, in the main thread [message #1852470 is a reply to message #1852469] Tue, 17 May 2022 12:07 Go to previous messageGo to next message
Andrzej Wasowski is currently offline Andrzej WasowskiFriend
Messages: 15
Registered: June 2015
Junior Member

Thanks for an unbelievably fast answer.

This piece of code is a part of the large build thing involving many programming languages and tools - I couldn't figure out how to marry it with the wizard thing. But I can try to scaffold a trial project and try to learn from there ... you're probably right. Does the default thing use the plugin, or is it juts plan gradle? Does it run MWE2 from gradle or in a separate JVM?
Re: Building an xtext language using gradle, in the main thread [message #1852471 is a reply to message #1852470] Tue, 17 May 2022 12:09 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 14349
Registered: July 2009
Senior Member
we just call mwe as java main with default gradle means with proper classpath

Need professional support for Xtext, Xpand, EMF?
Go to: https://www.itemis.com/en/it-services/methods-and-tools/xtext
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de

[Updated on: Tue, 17 May 2022 12:10]

Report message to a moderator

Re: Building an xtext language using gradle, in the main thread [message #1852474 is a reply to message #1852471] Tue, 17 May 2022 13:12 Go to previous messageGo to next message
Andrzej Wasowski is currently offline Andrzej WasowskiFriend
Messages: 15
Registered: June 2015
Junior Member

ah, then it is what I used to do - probably got it from there. I will have a look how you have upgraded this for newer gradle. Thanks.
Re: Building an xtext language using gradle, in the main thread [message #1852476 is a reply to message #1852474] Tue, 17 May 2022 19:05 Go to previous messageGo to next message
Andrzej Wasowski is currently offline Andrzej WasowskiFriend
Messages: 15
Registered: June 2015
Junior Member

So following you suggestion I have dropped the idea of using Mwe2Launcher.run, generated an example project using the eclipse wizard, and extracted what I could from it, using Mwe2Launcher.main. I am succesfully running the xtext generators using this build script for a subproject:

plugins {
  id "org.xtext.xtend"
  id "java-library"
}

sourceCompatibility = javaVersion
targetCompatibility = javaVersion

dependencies {

  implementation project(path: ':dsldesign.expr', configuration:'archives')

  implementation "org.eclipse.emf:org.eclipse.emf.mwe2.launch:${mweVersion}"
  implementation "org.eclipse.xtext:org.eclipse.xtext.generator:${xtextVersion}"
  implementation "org.eclipse.xtext:org.eclipse.xtext.xbase:${xtextVersion}"
  implementation "org.eclipse.xtext:org.eclipse.xtext:${xtextVersion}"

  testImplementation "org.eclipse.xtext:org.eclipse.xtext.testing:${xtextVersion}"
  testImplementation "org.eclipse.xtext:org.eclipse.xtext.xbase.testing:${xtextVersion}"
  testImplementation "org.junit.jupiter:junit-jupiter-api:${junitVersion}"
  testRuntimeOnly "org.junit.jupiter:junit-jupiter-engine:${junitVersion}"
}


sourceSets {
  main {
    java.srcDir 'src/main/xtext-gen'
    resources.srcDir 'src/main/xtext-gen'
    xtendOutputDir = 'src/main/xtend-gen'
  }
  test {
    java.srcDir 'src/test/xtext-gen'
    resources.srcDir 'src/test/xtext-gen'
    xtendOutputDir = 'src/test/xtend-gen'
  }
}

jar {
  from('model') {
    into('model')
  }
  from(sourceSets.main.allSource) {
    include '**/*.xtext'
  }
  manifest {
    attributes 'Bundle-SymbolicName': project.name
  }
}

task generateXtextLanguage (type: JavaExec) {
  mainClass = 'org.eclipse.emf.mwe2.launch.runtime.Mwe2Launcher'
  classpath = configurations.compileClasspath
  inputs.file "src/main/java/dsldesign/expr/xtext/GenerateExpr.mwe2"
  inputs.file "src/main/java/dsldesign/expr/xtext/Expr.xtext"
  outputs.dir "src/main/xtext-gen"
  args += "src/main/java/dsldesign/expr/xtext/GenerateExpr.mwe2"
  args += "-p"
  args += "rootPath=/${rootProject.projectDir}"
}

processResources.dependsOn generateXtextLanguage
generateXtext.dependsOn generateXtextLanguage
compileJava.dependsOn generateXtextLanguage
clean.dependsOn (cleanGenerateXtextLanguage)


It seems that including "configuration:'archives'" in the ecore project dependency was essential - otherwise I was only getting the java files from there, not the full jar with the model (perhaps the script of that ecore project could be improved to include the genmodel in the default configuration, instead of this complication here).

The task generateXtextLanguage works now, nicely populating xtext-gen/. But the task 'generateXtext' fails with a null pointer exception and no diagnostics. Any idea what could be going wrong?

Caused by: java.lang.NullPointerException                                                                                                                                                                                                                      
        at org.eclipse.xtext.xbase.compiler.JvmModelGenerator.reassignSuperType(JvmModelGenerator.java:1076)                                                                                                                                                   
        at org.eclipse.xtext.xbase.compiler.JvmModelGenerator.assignThisAndSuper(JvmModelGenerator.java:1049)                                                                                                                                                  
        at org.eclipse.xtext.xbase.compiler.JvmModelGenerator.generateType(JvmModelGenerator.java:214)                       
        at org.eclipse.xtext.xbase.compiler.JvmModelGenerator._internalDoGenerate(JvmModelGenerator.java:202)                                                                                                                                                  
        at org.eclipse.xtext.xbase.compiler.JvmModelGenerator.internalDoGenerate(JvmModelGenerator.java:1597)                                                                                                                                                  
        at org.eclipse.xtext.xbase.compiler.JvmModelGenerator.doGenerate(JvmModelGenerator.java:185)         
        at org.eclipse.xtend.core.compiler.XtendGenerator.doGenerate(XtendGenerator.java:96)                                                                                                                                                                   
        at org.eclipse.xtend.core.compiler.XtendGenerator.doGenerate(XtendGenerator.java:112)         
        at org.eclipse.xtext.generator.GeneratorDelegate.doGenerate(GeneratorDelegate.java:44)                                                                                                                                                                 
        at org.eclipse.xtext.generator.GeneratorDelegate.generate(GeneratorDelegate.java:35)
        at org.eclipse.xtext.build.IncrementalBuilder$InternalStatefulIncrementalBuilder.generate(IncrementalBuilder.java:349)
        at org.eclipse.xtext.build.IncrementalBuilder$InternalStatefulIncrementalBuilder.lambda$launch$2(IncrementalBuilder.java:277)
        at com.google.common.collect.Iterators$6.transform(Iterators.java:783)
        at com.google.common.collect.TransformedIterator.next(TransformedIterator.java:47)
        at com.google.common.collect.FluentIterable.copyInto(FluentIterable.java:791)
        at org.eclipse.xtext.build.ClusteringStorageAwareResourceLoader.executeClustered(ClusteringStorageAwareResourceLoader.java:69)
        at org.eclipse.xtext.build.BuildContext.executeClustered(BuildContext.java:55)
        at org.eclipse.xtext.build.IncrementalBuilder$InternalStatefulIncrementalBuilder.launch(IncrementalBuilder.java:259)
        at org.eclipse.xtext.build.IncrementalBuilder.build(IncrementalBuilder.java:412)
        at org.eclipse.xtext.build.IncrementalBuilder.build(IncrementalBuilder.java:394)
        at org.xtext.gradle.builder.XtextGradleBuilder.doBuild(XtextGradleBuilder.java:322)
        at org.xtext.gradle.builder.XtextGradleBuilder.build(XtextGradleBuilder.java:194)
        at org.xtext.gradle.tasks.XtextGenerate.generate(XtextGenerate.java:195)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at org.gradle.internal.reflect.JavaMethod.invoke(JavaMethod.java:104)
        at org.gradle.api.internal.project.taskfactory.IncrementalTaskInputsTaskAction.doExecute(IncrementalTaskInputsTaskAction.java:47)
...


The exception seems to be thrown somewhere deeply in the compiler (above), but I checked applying this procedure to another Xtext language (same setup), and landed with exactly the same exception. So I am inclined to think that it is a setup problem.

[Updated on: Tue, 17 May 2022 20:13]

Report message to a moderator

Re: Building an xtext language using gradle, in the main thread [message #1852479 is a reply to message #1852476] Tue, 17 May 2022 21:44 Go to previous messageGo to next message
Andrzej Wasowski is currently offline Andrzej WasowskiFriend
Messages: 15
Registered: June 2015
Junior Member

I got this to run, by downgrading my jvm from java-13-openjdk-amd64/bin/java to java-11-openjdk-amd64/bin/java (more precisely to openjdk version "11.0.15" 2022-04-19). It might have also helped that I switched the javaVersion for language input and output from Java 8 to Java 11 in the gradle settings file (but I doubt it - most surely it was the JVM).
Re: Building an xtext language using gradle, in the main thread [message #1852483 is a reply to message #1852479] Wed, 18 May 2022 05:26 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 14349
Registered: July 2009
Senior Member
which xtext and mwe and asm versions do you use? you dont seem to use the xtext bom?

Need professional support for Xtext, Xpand, EMF?
Go to: https://www.itemis.com/en/it-services/methods-and-tools/xtext
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de

[Updated on: Wed, 18 May 2022 05:26]

Report message to a moderator

Re: Building an xtext language using gradle, in the main thread [message #1852484 is a reply to message #1852483] Wed, 18 May 2022 05:37 Go to previous messageGo to next message
Andrzej Wasowski is currently offline Andrzej WasowskiFriend
Messages: 15
Registered: June 2015
Junior Member

I have no idea what is xtext bom, I tried to just include it in the classpath like the wizard does, but it did not seem to help (in retrospect: perhaps it was shadowed due to ordering issues?). This is my version properties file:
  ext {
    emfVersion = '2.15.+'
    xmiVersion = '2.15.+'
    flexmarkVersion = '0.35.10'
    guiceVersion = '4.2.3'
    javaVersion = '11'
    junitVersion = '5.6.0'
    kiamaVersion = '2.5.0'
    kotlinVersion = '1.3.71'
    mweVersion = '2.11.0'
    oslibVersion = '0.7.1'
    paigesVersion = '0.4.2'
    parboiledVersion = '2.3.0'
    scalaVersion = '2.13.5'
    scalaVersionSuffix = '2.13'
    scalacheckVersion = '1-15'
    scalatestVersion = '3.2.7'
    xtendVersion = '2.26.0'
    xtextVersion = '2.26.0'
  }


I do not even know what asm is :), but it sounds like I should know.

[Updated on: Wed, 18 May 2022 05:59]

Report message to a moderator

Re: Building an xtext language using gradle, in the main thread [message #1852486 is a reply to message #1852484] Wed, 18 May 2022 08:08 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 14349
Registered: July 2009
Senior Member
at least guice seems old. mwe2 too

Need professional support for Xtext, Xpand, EMF?
Go to: https://www.itemis.com/en/it-services/methods-and-tools/xtext
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Fixed subject: Building an xtext language using gradle, in a separate main thread, with a BOM [message #1852523 is a reply to message #1852486] Fri, 20 May 2022 07:53 Go to previous messageGo to next message
Andrzej Wasowski is currently offline Andrzej WasowskiFriend
Messages: 15
Registered: June 2015
Junior Member

Thanks for nudging me. I have learnt about BOMs and used the Xtext BOM. The following seems to work and is a bit more maintainable than above (pasting mostly for the googlers out there, but feedback is of course welcomed):

plugins {
  id "java-library"
  id "org.xtext.xtend"
}

sourceCompatibility = javaVersion
targetCompatibility = javaVersion

dependencies {
  implementation project(path: ':dsldesign.expr', configuration:'archives')

  api "org.eclipse.xtext:org.eclipse.xtext:${xtextVersion}"

  implementation platform("org.eclipse.xtext:xtext-dev-bom:${xtextVersion}")

  implementation "org.eclipse.emf:org.eclipse.emf.mwe2.launch"
  implementation "org.eclipse.xtext:org.eclipse.xtext.common.types:${xtextVersion}"
  implementation "org.eclipse.xtext:org.eclipse.xtext.generator:${xtextVersion}"
  implementation "org.eclipse.xtext:org.eclipse.xtext.testing:${xtextVersion}"
  implementation "org.eclipse.xtext:org.eclipse.xtext.xbase:${xtextVersion}"
  implementation "org.eclipse.xtext:org.eclipse.xtext.xtext.generator:${xtextVersion}"
  implementation "org.eclipse.xtext:xtext-antlr-generator"

  testImplementation "org.eclipse.xtext:org.eclipse.xtext.testing"
  testImplementation "org.eclipse.xtext:org.eclipse.xtext.xbase.testing:${xtextVersion}"
  testImplementation "org.junit.jupiter:junit-jupiter-api"
  testRuntimeOnly "org.junit.jupiter:junit-jupiter-engine"
}

sourceSets {
  main {
    java.srcDir 'src/main/xtext-gen'
    xtendOutputDir = 'src/main/xtend-gen'
    resources.srcDir 'src/test/xtext-gen'
  }
  test {
    java.srcDir 'src/test/xtext-gen'
    resources.srcDir 'src/test/xtext-gen'
    xtendOutputDir = 'src/test/xtend-gen'
  }
}

jar {
  from('model') {
    into('model')
  }
  from(sourceSets.main.allSource) {
    include '**/*.xtext'
  }
  manifest {
    attributes 'Bundle-SymbolicName': project.name
  }
}

task generateXtextLanguage (type: JavaExec) {
  mainClass = 'org.eclipse.emf.mwe2.launch.runtime.Mwe2Launcher'
  classpath = configurations.compileClasspath
  inputs.file "src/main/java/dsldesign/expr/xtext/GenerateExpr.mwe2"
  inputs.file "src/main/java/dsldesign/expr/xtext/Expr.xtext"
  outputs.dir "src/main/xtext-gen"
  args += "src/main/java/dsldesign/expr/xtext/GenerateExpr.mwe2"
  args += "-p"
  args += "rootPath=/${rootProject.projectDir}"
}

processResources.dependsOn (generateXtextLanguage)
generateXtext.dependsOn (generateXtextLanguage)
compileJava.dependsOn (generateXtextLanguage)
clean.dependsOn (cleanGenerateXtextLanguage)

[Updated on: Fri, 20 May 2022 07:55]

Report message to a moderator

Re: Fixed subject: Building an xtext language using gradle, in a separate main thread, with a BOM [message #1852787 is a reply to message #1852523] Fri, 03 June 2022 17:07 Go to previous message
Andrzej Wasowski is currently offline Andrzej WasowskiFriend
Messages: 15
Registered: June 2015
Junior Member

removed.

[Updated on: Fri, 03 June 2022 17:40]

Report message to a moderator

Previous Topic:How to add a documentation in code-completion?
Next Topic:Generate templates.xml?
Goto Forum:
  


Current Time: Wed Sep 28 04:28:02 GMT 2022

Powered by FUDForum. Page generated in 0.02587 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top