Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » EMF » Xtext Generator making random mistakes while generating java classes(Xext Generator making random mistakes while generating java classes)
Xtext Generator making random mistakes while generating java classes [message #1796037] Thu, 04 October 2018 11:52 Go to next message
Mehmetcan Sinir is currently offline Mehmetcan SinirFriend
Messages: 54
Registered: September 2018
Member
I have a quite large grammar.

When I generate the Xtext Artifacts, many many java classes are generated. Generators run successfully. However, almost always some of the classes which are generated have random mistakes in them, which makes them unable to compile.

It is usually only one or two .java classes among almost a hundred which have mistakes, and it is not the same classes, the mistakes seem to come randomly.

Bit it is usually inside the the package implementation class.

Here is an example mistake:

  public EClass getActionDeclaration()
  {
    if (actionDeclarationEClass == null)
    {
      actionDeclarationEClass = (EClass)EPackage.Registry.INSTANCE.getEPackage(HmiPackage.eNS_URI).getEClassifiers().g(96);
    }
    return compositeActionEClass;
  }


this code should actually look like this:

  public EClass getActionDeclaration()
  {
    if (actionDeclarationEClass == null)
    {
      actionDeclarationEClass = (EClass)EPackage.Registry.INSTANCE.getEPackage(HmiPackage.eNS_URI).getEClassifiers().get(92);
    }
    return actionDeclarationEClass;
  }


so the instead of "get" there is "g"

Here is my Workflow, if that helps:

module de.bmw.rhmi.GenerateHmi

import org.eclipse.xtext.xtext.generator.*
import org.eclipse.xtext.xtext.generator.model.project.*
import org.eclipse.emf.mwe.utils.*

var rootPath = ".."
var projectName = "de.bmw.rhmi"
var runtimeProject = "../${projectName}"

Workflow {

	bean = StandaloneSetup {
		scanClassPath = true
		platformUri = "${rootPath}"
		registerGenModelFile = "platform:/resource/de.bmw.rhmi.resource/model/generated/Strings.genmodel"
		registerGenModelFile = "platform:/resource/de.bmw.rhmi.resource/model/Image.genmodel"
		registerGenModelFile = "platform:/resource/de.bmw.rhmi/model/generated/App.genmodel"
		registerGenModelFile = "platform:/resource/de.bmw.rhmi/model/generated/Hmi.genmodel"
		registerGeneratedEPackage = "de.bmw.rhmi.resource.image.ImagePackage"
		registerGeneratedEPackage = "de.bmw.rhmi.resource.strings.StringsPackage"
	}

	component = XtextGenerator {
		configuration = {
			project = StandardProjectConfig {
				baseName = "de.bmw.rhmi"
				rootPath = rootPath
				runtimeTest = {
					enabled = true
				}
				genericIde = {
					enabled = true
				}
				mavenLayout = true
			}
			code = {
				encoding = "UTF-8"
				lineDelimiter = "\n"
				fileHeader = "/*\n * generated by Xtext \${version}\n */"
			}
		}

		language = XtextGeneratorLanguage {
			name = "de.bmw.rhmi.App"
			fileExtensions = "rapp"
			referencedResource = "platform:/resource/org.eclipse.xtext.xbase/model/Xbase.genmodel"
			fragment = grammarAccess.GrammarAccessFragment2 {}
			fragment = ecore.EMFGeneratorFragment2 {}
			fragment = serializer.SerializerFragment2 {}
			fragment = resourceFactory.ResourceFactoryFragment2 {}
			fragment = parser.antlr.XtextAntlrGeneratorFragment2 {}
			fragment = validation.ValidatorFragment2 {}
			fragment = scoping.ImportNamespacesScopingFragment2 {}
			fragment = exporting.QualifiedNamesFragment2 {}
			fragment = builder.BuilderIntegrationFragment2 {}
			fragment = formatting.Formatter2Fragment2 {}
			fragment = ui.labeling.LabelProviderFragment2 {}
			fragment = ui.outline.QuickOutlineFragment2 {}
			fragment = ui.outline.OutlineTreeProviderFragment2 {}
			fragment = ui.quickfix.QuickfixProviderFragment2 {}
			fragment = ui.contentAssist.ContentAssistFragment2 {}
			fragment = junit.JUnitFragment {}
			fragment = ui.refactoring.RefactorElementNameFragment2 {}
			fragment = ui.templates.CodetemplatesGeneratorFragment2 {}
			fragment = ui.compare.CompareFragment2 {}
			fragment = idea.parser.antlr.XtextAntlrIDEAGeneratorFragment {}
			fragment = ui.projectWizard.TemplateProjectWizardFragment {}
			fragment = ui.fileWizard.TemplateFileWizardFragment {}
		}

		language = XtextGeneratorLanguage {
			referencedResource = "platform:/resource/de.bmw.rhmi/src/main/java/de/bmw/rhmi/App.xtext"
			name = "de.bmw.rhmi.Hmi"
			fileExtensions = "hmi"
			referencedResource = "platform:/resource/org.eclipse.xtext.xbase/model/Xbase.genmodel"
			fragment = grammarAccess.GrammarAccessFragment2 {}
			fragment = ecore.EMFGeneratorFragment2 {}
			fragment = serializer.SerializerFragment2 {}
			fragment = resourceFactory.ResourceFactoryFragment2 {}
			fragment = parser.antlr.XtextAntlrGeneratorFragment2 {}
			fragment = validation.ValidatorFragment2 {}
			fragment = scoping.ImportNamespacesScopingFragment2 {}
			fragment = exporting.QualifiedNamesFragment2 {}
			fragment = builder.BuilderIntegrationFragment2 {}
			fragment = formatting.Formatter2Fragment2 {}
			fragment = ui.labeling.LabelProviderFragment2 {}
			fragment = ui.outline.QuickOutlineFragment2 {}
			fragment = ui.outline.OutlineTreeProviderFragment2 {}
			fragment = ui.quickfix.QuickfixProviderFragment2 {}
			fragment = ui.contentAssist.ContentAssistFragment2 {}
			fragment = junit.JUnitFragment {}
			fragment = ui.refactoring.RefactorElementNameFragment2 {}
			fragment = ui.templates.CodetemplatesGeneratorFragment2 {}
			fragment = ui.compare.CompareFragment2 {}
			fragment = idea.parser.antlr.XtextAntlrIDEAGeneratorFragment {}
			fragment = ui.projectWizard.TemplateProjectWizardFragment {}
			fragment = ui.fileWizard.TemplateFileWizardFragment {}
		}
	}
}


Why is this happening?

My Idea was to increase the timeout variable for the generator, but I am not sure if that has to do with it or how to do it.
Re: Xtext Generator making random mistakes while generating java classes [message #1796039 is a reply to message #1796037] Thu, 04 October 2018 11:59 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 12934
Registered: July 2009
Senior Member
@ed do you have any idea what could cause this?

Need professional support for Xtext, Xpand, EMF?
Go to: https://www.itemis.com/en/xtext/
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Re: Xtext Generator making random mistakes while generating java classes [message #1796041 is a reply to message #1796039] Thu, 04 October 2018 12:13 Go to previous messageGo to next message
Ed Willink is currently offline Ed WillinkFriend
Messages: 6393
Registered: July 2009
Senior Member
Hi

It is of course 'impossible'.

My guess is that two 'generators' are running concurrently and so the bogus text occurs where one overtakes another when writing the file.

Regards

Ed Willink
Re: Xtext Generator making random mistakes while generating java classes [message #1796043 is a reply to message #1796041] Thu, 04 October 2018 12:22 Go to previous messageGo to next message
Mehmetcan Sinir is currently offline Mehmetcan SinirFriend
Messages: 54
Registered: September 2018
Member
but why are two generators running simultaneously? This wasn't happening before and it suddenly started happening.
Re: Xtext Generator making random mistakes while generating java classes [message #1796044 is a reply to message #1796043] Thu, 04 October 2018 12:24 Go to previous messageGo to next message
Mehmetcan Sinir is currently offline Mehmetcan SinirFriend
Messages: 54
Registered: September 2018
Member
also here is for reference:

Build.gradle of the parent project

buildscript {
	repositories {
		jcenter()
		maven {
			url "https://plugins.gradle.org/m2/"
		}
	}
	dependencies {
		classpath 'org.xtext:xtext-gradle-plugin:1.0.21'
		classpath 'com.moowork.gradle:gradle-node-plugin:1.2.0'
		classpath 'net.researchgate:gradle-release:2.6.0'
	}
}

apply plugin: 'net.researchgate.release'
apply plugin: 'com.moowork.node'
node {
	version = '8.11.1'
	npmVersion = '5.6.0'
	download = true
}

configure(subprojects.findAll { it.name.startsWith('de.bmw') }) {
	ext.xtextVersion = '2.14.0'
	repositories {
		maven {
			url 'https://oss.sonatype.org/content/repositories/snapshots'
		}
		jcenter()
	}
	
	apply plugin: 'java'
	apply plugin: 'org.xtext.xtend'
	apply from: "${rootDir}/gradle/source-layout.gradle"
	apply from: "${rootDir}/gradle/maven-deployment.gradle"
	apply plugin: 'eclipse'
	apply plugin: 'idea'
	
	group = 'de.bmw.rhmi'
	version = '1.0.0-SNAPSHOT'
	
	sourceCompatibility = '1.8'
	targetCompatibility = '1.8'
	
	configurations.all {
		exclude group: 'asm'
	}
}

task npmInstallVsce(type: NpmTask, dependsOn: npmSetup) {
	ext.destPath = "$rootProject.projectDir/node_modules/vsce" 
	outputs.dir(destPath)
	group 'Node'
	description 'Installs the NodeJS package "Visual Studio Code Extension Manager"'
	args = [ 'install', 'vsce' ]
}

// Configuration for vscode projects
configure(subprojects.findAll { it.name.startsWith('vscode') }) {

	apply plugin: 'com.moowork.node'
	node {
	    version = '6.11.3'
	    npmVersion = '3.10.10'
	    download = true
	}

	def inputFiles = fileTree(
		dir: projectDir, 
		excludes: [ 'out/**', '.gitignore', '.gradle/**', 'build/**', '*.gradle' ]
	)

	npmInstall {
		inputs.files(inputFiles)
		outputs.dir('out')
	}

	//this task creates the vscode-extension-self-contained-0.0.14.visx file
	task vscodeExtension(dependsOn: [npmInstall, npmInstallVsce], type: NodeTask) {
		//set the destination directory
		ext.destDir = new File(buildDir, 'vscode')
		//set the archive name
		ext.archiveName = "$project.name-${project.version}.vsix"
		//set the destination path
		ext.destPath = "$destDir/$archiveName"
		inputs.with {
			files inputFiles
			dir npmInstallVsce.destPath
		}
		outputs.dir destDir
		doFirst {
			destDir.mkdirs()
		}
		script = file("$npmInstallVsce.destPath/out/vsce")
		args = [ 'package', '--out', destPath ]
		execOverrides {
			workingDir = projectDir
		}
	}

	task clean {
		doLast {
			delete vscodeExtension.destDir
			delete 'out' // output of npmInstall - don't want to delete node_modules
		}
		
	}
}

// Workaround for issue https://github.com/researchgate/gradle-release/issues/144
task build {
	dependsOn subprojects.findResults { it.tasks.findByName('build') }
}



build.gradle of the sub project where the grammer is defined

dependencies {
	testCompile "junit:junit:4.12"
	testCompile "org.eclipse.xtext:org.eclipse.xtext.testing:${xtextVersion}"
	testCompile "org.eclipse.xtext:org.eclipse.xtext.xbase.testing:${xtextVersion}"
	compile "org.eclipse.xtext:org.eclipse.xtext:${xtextVersion}"
	compile "org.eclipse.xtext:org.eclipse.xtext.xbase:${xtextVersion}"
	compile group: 'org.eclipse.emf', name: 'org.eclipse.emf.mwe2.launch', version: '2.9.1.201705291010'
	compile group: 'org.eclipse.core', name: 'org.eclipse.core.resources', version: '3.7.100'
	compile group: 'xom', name: 'xom', version: '1.2.5'
	compile group: 'javax.xml.bind', name: 'jaxb-api', version: '2.2.5'
	compile group: 'com.sun.xml.bind', name: 'jaxb-impl', version: '2.2.5'
}
configurations {
	mwe2 {
		extendsFrom compile
	}
}

dependencies {
	compile project(':de.bmw.rhmi.resource')
	mwe2 "org.eclipse.emf:org.eclipse.emf.mwe2.launch:2.9.1.201705291010"
	mwe2 "org.eclipse.xtext:org.eclipse.xtext.common.types:${xtextVersion}"
	mwe2 "org.eclipse.xtext:org.eclipse.xtext.xtext.generator:${xtextVersion}"
	mwe2 "org.eclipse.xtext:xtext-antlr-generator:[2.1.1, 3)"
}

task generateXtextLanguage(type: JavaExec) {
	main = 'org.eclipse.emf.mwe2.launch.runtime.Mwe2Launcher'
	classpath = configurations.mwe2
	inputs.file "src/main/java/de/bmw/rhmi/GenerateHmi.mwe2"
	inputs.file "src/main/java/de/bmw/rhmi/Hmi.xtext"
	outputs.dir "src/main/xtext-gen"
	args += "src/main/java/de/bmw/rhmi/GenerateHmi.mwe2"
	args += "-p"
	args += "rootPath=/${projectDir}/.."
}

generateXtext.dependsOn(generateXtextLanguage)
clean.dependsOn(cleanGenerateXtextLanguage)
eclipse.classpath.plusConfigurations += [configurations.mwe2]



I am trying to build a VSCode extension
Re: Xtext Generator making random mistakes while generating java classes [message #1796045 is a reply to message #1796044] Thu, 04 October 2018 12:28 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 12934
Registered: July 2009
Senior Member
Are you sure you don't run a second build e.g. in eclipse in parallel

Need professional support for Xtext, Xpand, EMF?
Go to: https://www.itemis.com/en/xtext/
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Re: Xtext Generator making random mistakes while generating java classes [message #1796047 is a reply to message #1796045] Thu, 04 October 2018 12:34 Go to previous messageGo to next message
Mehmetcan Sinir is currently offline Mehmetcan SinirFriend
Messages: 54
Registered: September 2018
Member
Christian Dietrich wrote on Thu, 04 October 2018 12:28
Are you sure you don't run a second build e.g. in eclipse in parallel



How can I check this?=) Build automatically was selected I just deselected it now. Let's see if that makes a difference

[Updated on: Thu, 04 October 2018 12:36]

Report message to a moderator

Re: Xtext Generator making random mistakes while generating java classes [message #1796048 is a reply to message #1796047] Thu, 04 October 2018 12:37 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 12934
Registered: July 2009
Senior Member
Simply make sure the first generator run is done before you start the second eg use progress view and console ( should print done) to monitor

Need professional support for Xtext, Xpand, EMF?
Go to: https://www.itemis.com/en/xtext/
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Re: Xtext Generator making random mistakes while generating java classes [message #1796055 is a reply to message #1796048] Thu, 04 October 2018 13:46 Go to previous messageGo to next message
Ed Willink is currently offline Ed WillinkFriend
Messages: 6393
Registered: July 2009
Senior Member
Hi

Your original report suggested a regular problem, so it is unlikely to be associated with a manual activity. I suspect that for some reason multiple builds run from a single trigger. Possibly due to multiple registered (platform) builders/natures, possibly due to multiple EF Generators. Perhaps some EMF-based project that I am not familiar with auto-builds from genmodels.

In your position I would instrument XMIResourceImpl.save(*) to get some insight into the multiples.

Regards

Ed Willink
Re: Xtext Generator making random mistakes while generating java classes [message #1796060 is a reply to message #1796055] Thu, 04 October 2018 15:19 Go to previous messageGo to next message
Mehmetcan Sinir is currently offline Mehmetcan SinirFriend
Messages: 54
Registered: September 2018
Member
Ed Willink wrote on Thu, 04 October 2018 13:46
Hi

Your original report suggested a regular problem, so it is unlikely to be associated with a manual activity. I suspect that for some reason multiple builds run from a single trigger. Possibly due to multiple registered (platform) builders/natures, possibly due to multiple EF Generators. Perhaps some EMF-based project that I am not familiar with auto-builds from genmodels.

In your position I would instrument XMIResourceImpl.save(*) to get some insight into the multiples.

Regards

Ed Willink

@Ed Willink

How do I do that exactly?

Here is the interesting parts of the console output when I run the builder.


1207 [main] INFO  text.xtext.generator.XtextGenerator  - Initializing Xtext generator [b][/b]


1894 [main] INFO  lipse.emf.mwe.utils.StandaloneSetup  - Using resourceSet registry. The registered Packages will not be registered in the global EPackage.Registry.INSTANCE!
5283 [main] INFO  text.xtext.generator.XtextGenerator  - Generating de.bmw.rhmi.App
8578 [main] INFO  nerator.ecore.EMFGeneratorFragment2  - Generating EMF model code
8635 [main] INFO  clipse.emf.mwe.utils.GenModelHelper  - Registered GenModel 'http://www.bmw.de/rhmi/app' from 'platform:/resource/de.bmw.rhmi/model/generated/App.genmodel'
14550 [main] INFO  text.xtext.generator.XtextGenerator  - Generating de.bmw.rhmi.Hmi
32123 [main] INFO  nerator.ecore.EMFGeneratorFragment2  - Generating EMF model code
32203 [main] INFO  clipse.emf.mwe.utils.GenModelHelper  - Registered GenModel 'http://www.bmw.de/rhmi/hmi' from 'platform:/resource/de.bmw.rhmi/model/generated/Hmi.genmodel'
103360 [main] INFO  text.xtext.generator.XtextGenerator  - Generating common infrastructure
103362 [main] INFO  .emf.mwe2.runtime.workflow.Workflow  - Done.



Can I run the mwe2 workflow through the console and with a debug option so that I know what is going on exactly?

Something is definitely wrong.

For example here is part of the InternalHmiParser.java class which is also generated and which is a complete code mish mash:

      try {
            // InternalHmi.g:24073:2: ( ( ( (lv_value_0_0= ruleIntLiteral ) ) (otherlv_1= ',' ( (lv_value_2_0= ruleIntLiteral ) ) ) ) )
            // InternalHmi.g:24074:2: ( ( (lv_value_0_0= ruleIntLiteral ) ) (otherlv_1= ',' ( (lv_value_2_0= ruleIntLiteral ) ) ) )
            {
            // InternalHmi.g:24074:2: ( ( (lv_value_0_0= ruleIntLiteral ) ) (otherlv_1= ',' ( (lv_value_2_0= ruleIntLiteral ) ) ) )
            // InternalHmi.g:24075:3: ( (lv_value_0_0= ruleIntLiteral ) ) (otherlv_1= ',' ( (lv_value_2_0= ruleIntLiteral ) ) )
            {
            // InternalHmi.g:24075:3: ( (lv_value_0_0= ruleIntLiteral ) )
            // InternalHmi.g:24076:4: (lv_value_0_0= ruleIntLiteral )
            {
            // InternalHmi.g:24076:4: (lv_value_0_0= ruleIntLiteral )
            // InternalHmi.g:24077:5: lv_value_0_0= ruleIntLiteral
            {

            					newCompositeNode(grammarAccess.getTableLayoutValueAccess().getValueIntLiteralParserRuleCall_0_0());
            				
            pushFollow(FollowSets000.FOLLOW_123);
            lv_value_0_0=ruleIntLiteral();

            state._fsp--;


            					if (current==null) {
            						current = createModelElementForParent(grammarAccess.getTableLayoutValueRule());
            					}
            					add(
            						current,
            						"value",
            						lv_value_0_0,
            						"de.bmw.rhmi.Hmi.IntLiteral");
            					afterParserOrEnumRuleCall();
            				

            }


            }

            // InternalHmi.g:24094:3: (otherlv_1= ',' ( (lv_value_2_0= ruleIntLiteral ) ) )
            // InternalHmi.g:24095:4: otherlv_1= ',' ( (lv_value_2_0= ruleIntLiteral ) )
            {
            otherlv_1=(Token)match(input,71,FollowSets000.FOLLOW_25); 

            				newLeafNode(otherlv_1, grammarAccess.getTableLayoutValueAccess().getCommaKeyword_1_0());
            			
            // InternalHmi.g:24099:4: ( (lv_value_2_0= ruleIntLiteral ) )
            // InternalHmi.g:24100:5: (lv_value_2_0= ruleIntLiteral )
            {
            // InternalHmi.g:24100:5: (lv_value_2_0= ruleIntLiteral )
            // InternalHmi.g:24101:6: lv_value_2_0= ruleIntLiteral
            {

            						newCompositeNode(grammarAccess.getTableLayoutValueAccess().getValueIntLiteralParserRuleCall_1_1_0());
            					
            pushFollow(FollowSets000.FOLLOW_2);
            lv_value_2_0=ruleIntLiteral();

            state._fsp--;


            						if (current==null) {
            							current = createModelElementForParent(grammarAccess.getTableLayoutValueRule());
            						}
            						add(
            							current,
            							"value",
            							lv_value_2_0,
            							"de.bmw.rhmi.Hmi.IntLiteral");
            						afterParserOrEnumRuleCall();
            					

            }


            }


            }


            }


            }


            	leaveRule();

        }

            catch (RecognitionException re) {
                recover(input,re);
                appendSkippedTokens();
Re: Xtext Generator making random mistakes while generating java classes [message #1796075 is a reply to message #1796060] Thu, 04 October 2018 17:19 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 12934
Registered: July 2009
Senior Member
Did you try to play with antlr timeouts

https://www.eclipse.org/forums/index.php/m/1769932/?srch=Xconversiontimeout#msg_1769932

Otherwise having something reproducible would hell


Need professional support for Xtext, Xpand, EMF?
Go to: https://www.itemis.com/en/xtext/
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Re: Xtext Generator making random mistakes while generating java classes [message #1796080 is a reply to message #1796075] Thu, 04 October 2018 18:17 Go to previous message
Ed Willink is currently offline Ed WillinkFriend
Messages: 6393
Registered: July 2009
Senior Member
Hi

I would consider it unlikely that the console will give you a complete and true record of the problem.

If you have a known suspect component you might manage to guess your way to a solution.

But I think you have to debug. I often check out EMF modules into my workspace so that I can add println's to them. Then I run in a nested Eclipse and the log guides me to where to set more relevant breakpoints.

My suspicion of colliding writes turning "get()" into "g()" might be a wrong. In which case you will instead need to debug how a "g()" got emitted and what got corrupted to cause it ....

Regards

Ed Willink
Previous Topic:UnsupportedOperationException when setting a model attribute
Next Topic:load EMF instance model and then create a object in it programmatically
Goto Forum:
  


Current Time: Mon Jun 24 11:46:32 GMT 2019

Powered by FUDForum. Page generated in 0.01999 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top