| Building an xtext language using gradle, in the main thread [message #1852468] |
Tue, 17 May 2022 11:57  |
|
Gradle upgrades have invalidated my clunky xtext build scripts, and I am giving them a new shot (somehow the "compile" dependency depends differently than "implementation" but let's not discuss it now). I am trying to use the opportunity to also make the xtext language build faster, by not spawning another JVM instance, which my old JavaExec task did. I came up with the following, which looks pretty close, but not quite there yet:
plugins {
id "org.xtext.xtend"
id "java-library"
}
dependencies {
implementation project(':dsldesign.expr')
implementation "org.eclipse.emf:org.eclipse.emf.mwe2.launch:${mweVersion}"
implementation "org.eclipse.xtend:org.eclipse.xtend.lib:${xtendVersion}"
implementation "org.eclipse.xtext:org.eclipse.xtext.common.types:${xtextVersion}"
implementation "org.eclipse.xtext:org.eclipse.xtext.generator:${xtextVersion}"
implementation "org.eclipse.xtext:org.eclipse.xtext.xbase:${xtextVersion}"
implementation "org.eclipse.xtext:org.eclipse.xtext.xtext.generator:${xtextVersion}"
implementation "org.eclipse.xtext:org.eclipse.xtext:${xtextVersion}"
implementation "org.eclipse.xtext:xtext-antlr-generator:[2.1.1, 3)"
}
def root = rootProject.projectDir.toURI ()
tasks.register ('exprXtextCode') {
doLast {
def classloader = new URLClassLoader (
[root.toURL ()] as URL[],
gradle.class.classLoader
)
configurations
.compileClasspath
.each {
logger.info ("Adding to classpath: ${it.toURI ()}")
classloader.addURL (it.toURI ().toURL ())
}
def mwe2 = classloader
.loadClass('org.eclipse.emf.mwe2.launch.runtime.Mwe2Launcher')
.newInstance ()
logger.lifecycle ("MWE2 launcher class loaded.")
logger.lifecycle ("Starting MWE2 workflow in ${root.path}")
mwe2.run([
"${root.toURL ()}/dsldesign.expr.xtext/src/main/java/dsldesign/expr/xtext/GenerateExpr.mwe2",
"-p",
"rootPath=${root.path}"] as String[]
)
logger.lifecycle ("MWE2 workflow completed.")
}
}
compileJava.dependsOn exprXtextCode
I believe that I have all the jars in dependencies as I should (at least in the old set up they were enough). The first dependency is on the EMF project with the model/genmodel. The task instantiates a class loader, then puts all the dependencies on the current classpath. Then we load Mwe2Launcher and use the run functions to start it, with the root set to the root of the project (this is a nested-project, so root is the parent). The MWE2 script is unsurprising, as created by the wizard some years ago.
Unfortunately attempting to invoke the task produces this error message:
...
Adding to classpath: file:/home/wasowski/.gradle/caches/modules-2/files-2.1/antlr/antlr/2.7.7/83cd2cd674a217ade95a4bb83a8a14f351f48bd0/antlr-2.7.7.jar
MWE2 launcher class loaded.
Starting MWE2 workflow in /home/wasowski/work/dsldesign/
Initializing Xtext generator
Adding generated EPackage 'org.eclipse.xtext.common.types.TypesPackage'
No project file found for /home/wasowski/work/dsldesign/dsldesign.expr/build/classes/java/main. The folder was neither an Eclipse 'bin' folder, nor a Maven 'target/classes' folder, nor a Gradle bin/main folder
Registering project dsldesign.expr.xtext at 'file:/home/wasowski/work/dsldesign/dsldesign.expr.xtext/'
Registering project dsldesign.expr.xtext at 'file:/home/wasowski/work/dsldesign/dsldesign.expr.xtext/'
Using resourceSet registry. The registered Packages will not be registered in the global EPackage.Registry.INSTANCE!
Registered GenModel 'http://www.eclipse.org/Xtext/Xbase/XAnnotations' from 'platform:/resource/org.eclipse.xtext.xbase/model/Xbase.genmodel'
Registered GenModel 'http://www.eclipse.org/xtext/xbase/Xtype' from 'platform:/resource/org.eclipse.xtext.xbase/model/Xbase.genmodel'
Registered GenModel 'http://www.eclipse.org/xtext/xbase/Xbase' from 'platform:/resource/org.eclipse.xtext.xbase/model/Xbase.genmodel'
Registered GenModel 'http://www.eclipse.org/xtext/common/JavaVMTypes' from 'platform:/resource/org.eclipse.xtext.common.types/model/JavaVMTypes.genmodel'
Registered GenModel 'http://www.dsl.design/dsldesign.expr' from 'platform:/resource/dsldesign.expr/model/expr.genmodel'
[XtextLinkingDiagnostic: null:1 Couldn't resolve reference to Grammar 'org.eclipse.xtext.common.Terminals'., XtextLinkingDiagnostic: null:19 STRING cannot be resolved to a rule, XtextLinkingDiagnostic: null:19 ID cannot be resolved to a rule, TransformationDiagnostic: null:18 Datatype rules may only use other datatype rules, lexer rules and keywords. (ErrorCode: InvalidDatatypeRule)]
There is disturbing attempt to find something in the build tree and there strange double registration for my project (the current project), after this the Xtext generator seem to proceed as it should, until it fails to find the Terminals grammar. This one should be in the org.eclipse.xtext:org.eclipse.xtext:${xtextVersion} jar, and I have checked that the jar is in the classpath.
I am starting to get desperate, and considering switching to a JavaExec task again, attempting to fix my old ways (although the above is nicely fast). Any suggestions of what could be wrong? Any leads are highly appreciated...
[Updated on: Tue, 17 May 2022 12:00] Report message to a moderator |
|
|
|
| Re: Building an xtext language using gradle, in the main thread [message #1852470 is a reply to message #1852469] |
Tue, 17 May 2022 12:07  |
|
Thanks for an unbelievably fast answer.
This piece of code is a part of the large build thing involving many programming languages and tools - I couldn't figure out how to marry it with the wizard thing. But I can try to scaffold a trial project and try to learn from there ... you're probably right. Does the default thing use the plugin, or is it juts plan gradle? Does it run MWE2 from gradle or in a separate JVM?
|
|
|
|
|
| Re: Building an xtext language using gradle, in the main thread [message #1852476 is a reply to message #1852474] |
Tue, 17 May 2022 19:05 |
|
So following you suggestion I have dropped the idea of using Mwe2Launcher.run, generated an example project using the eclipse wizard, and extracted what I could from it, using Mwe2Launcher.main. I am succesfully running the xtext generators using this build script for a subproject:
plugins {
id "org.xtext.xtend"
id "java-library"
}
sourceCompatibility = javaVersion
targetCompatibility = javaVersion
dependencies {
implementation project(path: ':dsldesign.expr', configuration:'archives')
implementation "org.eclipse.emf:org.eclipse.emf.mwe2.launch:${mweVersion}"
implementation "org.eclipse.xtext:org.eclipse.xtext.generator:${xtextVersion}"
implementation "org.eclipse.xtext:org.eclipse.xtext.xbase:${xtextVersion}"
implementation "org.eclipse.xtext:org.eclipse.xtext:${xtextVersion}"
testImplementation "org.eclipse.xtext:org.eclipse.xtext.testing:${xtextVersion}"
testImplementation "org.eclipse.xtext:org.eclipse.xtext.xbase.testing:${xtextVersion}"
testImplementation "org.junit.jupiter:junit-jupiter-api:${junitVersion}"
testRuntimeOnly "org.junit.jupiter:junit-jupiter-engine:${junitVersion}"
}
sourceSets {
main {
java.srcDir 'src/main/xtext-gen'
resources.srcDir 'src/main/xtext-gen'
xtendOutputDir = 'src/main/xtend-gen'
}
test {
java.srcDir 'src/test/xtext-gen'
resources.srcDir 'src/test/xtext-gen'
xtendOutputDir = 'src/test/xtend-gen'
}
}
jar {
from('model') {
into('model')
}
from(sourceSets.main.allSource) {
include '**/*.xtext'
}
manifest {
attributes 'Bundle-SymbolicName': project.name
}
}
task generateXtextLanguage (type: JavaExec) {
mainClass = 'org.eclipse.emf.mwe2.launch.runtime.Mwe2Launcher'
classpath = configurations.compileClasspath
inputs.file "src/main/java/dsldesign/expr/xtext/GenerateExpr.mwe2"
inputs.file "src/main/java/dsldesign/expr/xtext/Expr.xtext"
outputs.dir "src/main/xtext-gen"
args += "src/main/java/dsldesign/expr/xtext/GenerateExpr.mwe2"
args += "-p"
args += "rootPath=/${rootProject.projectDir}"
}
processResources.dependsOn generateXtextLanguage
generateXtext.dependsOn generateXtextLanguage
compileJava.dependsOn generateXtextLanguage
clean.dependsOn (cleanGenerateXtextLanguage)
It seems that including "configuration:'archives'" in the ecore project dependency was essential - otherwise I was only getting the java files from there, not the full jar with the model (perhaps the script of that ecore project could be improved to include the genmodel in the default configuration, instead of this complication here).
The task generateXtextLanguage works now, nicely populating xtext-gen/. But the task 'generateXtext' fails with a null pointer exception and no diagnostics. Any idea what could be going wrong?
Caused by: java.lang.NullPointerException
at org.eclipse.xtext.xbase.compiler.JvmModelGenerator.reassignSuperType(JvmModelGenerator.java:1076)
at org.eclipse.xtext.xbase.compiler.JvmModelGenerator.assignThisAndSuper(JvmModelGenerator.java:1049)
at org.eclipse.xtext.xbase.compiler.JvmModelGenerator.generateType(JvmModelGenerator.java:214)
at org.eclipse.xtext.xbase.compiler.JvmModelGenerator._internalDoGenerate(JvmModelGenerator.java:202)
at org.eclipse.xtext.xbase.compiler.JvmModelGenerator.internalDoGenerate(JvmModelGenerator.java:1597)
at org.eclipse.xtext.xbase.compiler.JvmModelGenerator.doGenerate(JvmModelGenerator.java:185)
at org.eclipse.xtend.core.compiler.XtendGenerator.doGenerate(XtendGenerator.java:96)
at org.eclipse.xtend.core.compiler.XtendGenerator.doGenerate(XtendGenerator.java:112)
at org.eclipse.xtext.generator.GeneratorDelegate.doGenerate(GeneratorDelegate.java:44)
at org.eclipse.xtext.generator.GeneratorDelegate.generate(GeneratorDelegate.java:35)
at org.eclipse.xtext.build.IncrementalBuilder$InternalStatefulIncrementalBuilder.generate(IncrementalBuilder.java:349)
at org.eclipse.xtext.build.IncrementalBuilder$InternalStatefulIncrementalBuilder.lambda$launch$2(IncrementalBuilder.java:277)
at com.google.common.collect.Iterators$6.transform(Iterators.java:783)
at com.google.common.collect.TransformedIterator.next(TransformedIterator.java:47)
at com.google.common.collect.FluentIterable.copyInto(FluentIterable.java:791)
at org.eclipse.xtext.build.ClusteringStorageAwareResourceLoader.executeClustered(ClusteringStorageAwareResourceLoader.java:69)
at org.eclipse.xtext.build.BuildContext.executeClustered(BuildContext.java:55)
at org.eclipse.xtext.build.IncrementalBuilder$InternalStatefulIncrementalBuilder.launch(IncrementalBuilder.java:259)
at org.eclipse.xtext.build.IncrementalBuilder.build(IncrementalBuilder.java:412)
at org.eclipse.xtext.build.IncrementalBuilder.build(IncrementalBuilder.java:394)
at org.xtext.gradle.builder.XtextGradleBuilder.doBuild(XtextGradleBuilder.java:322)
at org.xtext.gradle.builder.XtextGradleBuilder.build(XtextGradleBuilder.java:194)
at org.xtext.gradle.tasks.XtextGenerate.generate(XtextGenerate.java:195)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at org.gradle.internal.reflect.JavaMethod.invoke(JavaMethod.java:104)
at org.gradle.api.internal.project.taskfactory.IncrementalTaskInputsTaskAction.doExecute(IncrementalTaskInputsTaskAction.java:47)
...
The exception seems to be thrown somewhere deeply in the compiler (above), but I checked applying this procedure to another Xtext language (same setup), and landed with exactly the same exception. So I am inclined to think that it is a setup problem.
[Updated on: Tue, 17 May 2022 20:13] Report message to a moderator |
|
|
| Re: Building an xtext language using gradle, in the main thread [message #1852479 is a reply to message #1852476] |
Tue, 17 May 2022 21:44 |
|
|
I got this to run, by downgrading my jvm from java-13-openjdk-amd64/bin/java to java-11-openjdk-amd64/bin/java (more precisely to openjdk version "11.0.15" 2022-04-19). It might have also helped that I switched the javaVersion for language input and output from Java 8 to Java 11 in the gradle settings file (but I doubt it - most surely it was the JVM).
|
|
|
|
| Re: Building an xtext language using gradle, in the main thread [message #1852484 is a reply to message #1852483] |
Wed, 18 May 2022 05:37 |
|
I have no idea what is xtext bom, I tried to just include it in the classpath like the wizard does, but it did not seem to help (in retrospect: perhaps it was shadowed due to ordering issues?). This is my version properties file:
ext {
emfVersion = '2.15.+'
xmiVersion = '2.15.+'
flexmarkVersion = '0.35.10'
guiceVersion = '4.2.3'
javaVersion = '11'
junitVersion = '5.6.0'
kiamaVersion = '2.5.0'
kotlinVersion = '1.3.71'
mweVersion = '2.11.0'
oslibVersion = '0.7.1'
paigesVersion = '0.4.2'
parboiledVersion = '2.3.0'
scalaVersion = '2.13.5'
scalaVersionSuffix = '2.13'
scalacheckVersion = '1-15'
scalatestVersion = '3.2.7'
xtendVersion = '2.26.0'
xtextVersion = '2.26.0'
}
I do not even know what asm is :), but it sounds like I should know.
[Updated on: Wed, 18 May 2022 05:59] Report message to a moderator |
|
|
|
| Fixed subject: Building an xtext language using gradle, in a separate main thread, with a BOM [message #1852523 is a reply to message #1852486] |
Fri, 20 May 2022 07:53 |
|
Thanks for nudging me. I have learnt about BOMs and used the Xtext BOM. The following seems to work and is a bit more maintainable than above (pasting mostly for the googlers out there, but feedback is of course welcomed):
plugins {
id "java-library"
id "org.xtext.xtend"
}
sourceCompatibility = javaVersion
targetCompatibility = javaVersion
dependencies {
implementation project(path: ':dsldesign.expr', configuration:'archives')
api "org.eclipse.xtext:org.eclipse.xtext:${xtextVersion}"
implementation platform("org.eclipse.xtext:xtext-dev-bom:${xtextVersion}")
implementation "org.eclipse.emf:org.eclipse.emf.mwe2.launch"
implementation "org.eclipse.xtext:org.eclipse.xtext.common.types:${xtextVersion}"
implementation "org.eclipse.xtext:org.eclipse.xtext.generator:${xtextVersion}"
implementation "org.eclipse.xtext:org.eclipse.xtext.testing:${xtextVersion}"
implementation "org.eclipse.xtext:org.eclipse.xtext.xbase:${xtextVersion}"
implementation "org.eclipse.xtext:org.eclipse.xtext.xtext.generator:${xtextVersion}"
implementation "org.eclipse.xtext:xtext-antlr-generator"
testImplementation "org.eclipse.xtext:org.eclipse.xtext.testing"
testImplementation "org.eclipse.xtext:org.eclipse.xtext.xbase.testing:${xtextVersion}"
testImplementation "org.junit.jupiter:junit-jupiter-api"
testRuntimeOnly "org.junit.jupiter:junit-jupiter-engine"
}
sourceSets {
main {
java.srcDir 'src/main/xtext-gen'
xtendOutputDir = 'src/main/xtend-gen'
resources.srcDir 'src/test/xtext-gen'
}
test {
java.srcDir 'src/test/xtext-gen'
resources.srcDir 'src/test/xtext-gen'
xtendOutputDir = 'src/test/xtend-gen'
}
}
jar {
from('model') {
into('model')
}
from(sourceSets.main.allSource) {
include '**/*.xtext'
}
manifest {
attributes 'Bundle-SymbolicName': project.name
}
}
task generateXtextLanguage (type: JavaExec) {
mainClass = 'org.eclipse.emf.mwe2.launch.runtime.Mwe2Launcher'
classpath = configurations.compileClasspath
inputs.file "src/main/java/dsldesign/expr/xtext/GenerateExpr.mwe2"
inputs.file "src/main/java/dsldesign/expr/xtext/Expr.xtext"
outputs.dir "src/main/xtext-gen"
args += "src/main/java/dsldesign/expr/xtext/GenerateExpr.mwe2"
args += "-p"
args += "rootPath=/${rootProject.projectDir}"
}
processResources.dependsOn (generateXtextLanguage)
generateXtext.dependsOn (generateXtextLanguage)
compileJava.dependsOn (generateXtextLanguage)
clean.dependsOn (cleanGenerateXtextLanguage)
[Updated on: Fri, 20 May 2022 07:55] Report message to a moderator |
|
|
|
Powered by
FUDForum. Page generated in 0.05011 seconds
] 
Search
Help
Register
Login
Home
