Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » VIATRA » VIATRA - Query Generic API
VIATRA - Query Generic API [message #1760293] Tue, 25 April 2017 15:11 Go to next message
Lars Fritsche is currently offline Lars FritscheFriend
Messages: 6
Registered: April 2017
Junior Member
Hi,

I just tried to use the generic API to use my patterns without generating code.
The reason why I want to do this is because the code generation takes too long even for our smaller projects.

I found this link that describes the usage of the generic API: https://wiki.eclipse.org/VIATRA/Query/UserDocumentation/API/Advanced

My code differs in the call to EMFPatternLanguageStandaloneSetup which seems to have changed to a static invocation.
Apart from that I do not want to load an .eiq but a .vql file (.eiq is incquery that has been merged into Viatra?), so I changed that, too.

Below is my code and the error that appears at the line that should be orange.

Somehow, he doesn't seem to recognize the internal structure of the .vql file but as far as I can tell, it is the same as in the old .eiq files.

So my questions are:


  1. Did I use the correct API?
  2. Maybe there is another register function for .vql files that has to be used?
  3. Are there any disadvantages of using the generic API?


Regards,
Lars


val patternLocation = "model/patterns/";
	val fileNameList = new ArrayList<String>

	/* Transformation-related extensions */
	extension EventDrivenTransformation transformation

	/* Transformation rule-related extensions */
	extension EventDrivenTransformationRuleFactory = new EventDrivenTransformationRuleFactory

	val builder = new SpecificationBuilder();

	protected ViatraQueryEngine engine
	protected Resource resource

	val patternMap = new HashMap<String, Pattern>

	private TGGRuntimeUtil tggRuntimeUtil

	// protected EventDrivenTransformationRule<?,?> exampleRule
	new(ResourceSet set, TGGRuntimeUtil tggRuntimeUtil) {
		this.resource = resource
		this.tggRuntimeUtil = tggRuntimeUtil
     	
     	        EMFPatternLanguageStandaloneSetup.doSetup
     	
		registerFileNames()

		registerPattern(set)

		tggRuntimeUtil.getCSPProvider().registerFactory(new CDToDocAttrCondDefLibrary());

		// Create EMF scope and EMF IncQuery engine based on the resource
		val scope = new EMFScope(set)
		engine = ViatraQueryEngine.on(scope);

	}

	private def registerFileNames() {
		fileNameList.add("Axiom.vql");
	}

	private def registerPattern(ResourceSet set) {
		ResourceSet resourceSet = new ResourceSetImpl
		for (s : fileNameList) {
			[color=orangeval] val Resource patternResource = resourceSet.getResource(URI.createFileURI(patternLocation + s), true);[/color]
			// navigate to the pattern definition that we want
			if (patternResource != null) {
				if (patternResource.getErrors().size() == 0 && patternResource.getContents().size() >= 1) {
					val EObject	topElement = patternResource.getContents().get(0);
					if (topElement instanceof PatternModel) {
						for (p : (topElement as PatternModel).getPatterns()) {
							patternMap.put(p.name, p)
						}
					}
				}
			}
		}
	}


Exception in thread "main" java.lang.NoSuchFieldError: EOF_TOKEN
	at org.eclipse.xtext.parser.antlr.Lexer.nextToken(Lexer.java:59)
	at org.antlr.runtime.BufferedTokenStream.fetch(BufferedTokenStream.java:143)
	at org.antlr.runtime.BufferedTokenStream.sync(BufferedTokenStream.java:137)
	at org.antlr.runtime.BufferedTokenStream.fill(BufferedTokenStream.java:286)
	at org.antlr.runtime.BufferedTokenStream.toString(BufferedTokenStream.java:251)
	at org.eclipse.xtext.parser.antlr.AbstractInternalAntlrParser.parse(AbstractInternalAntlrParser.java:541)
	at org.eclipse.xtext.parser.antlr.AbstractAntlrParser.doParse(AbstractAntlrParser.java:102)
	at org.eclipse.xtext.parser.antlr.AbstractAntlrParser.parse(AbstractAntlrParser.java:84)
	at org.eclipse.xtext.parser.antlr.AbstractAntlrParser.doParse(AbstractAntlrParser.java:62)
	at org.eclipse.xtext.parser.AbstractParser.parse(AbstractParser.java:33)
	at org.eclipse.xtext.resource.XtextResource.doLoad(XtextResource.java:170)
	at org.eclipse.xtext.linking.lazy.LazyLinkingResource.doLoad(LazyLinkingResource.java:100)
	at org.eclipse.emf.ecore.resource.impl.ResourceImpl.load(ResourceImpl.java:1518)
	at org.eclipse.emf.ecore.resource.impl.ResourceImpl.load(ResourceImpl.java:1297)
	at org.eclipse.xtext.resource.persistence.StorageAwareResource.load(StorageAwareResource.java:81)
	at org.eclipse.emf.ecore.resource.impl.ResourceSetImpl.demandLoad(ResourceSetImpl.java:259)
	at org.eclipse.emf.ecore.resource.impl.ResourceSetImpl.demandLoadHelper(ResourceSetImpl.java:274)
	at org.eclipse.emf.ecore.resource.impl.ResourceSetImpl.getResource(ResourceSetImpl.java:406)
	at org.emoflon.ibex.tgg.run.CDToDocTransformation_NoCodeGen.registerPattern(CDToDocTransformation_NoCodeGen.java:117)

[Updated on: Tue, 25 April 2017 15:12]

Report message to a moderator

Re: VIATRA - Query Generic API [message #1760314 is a reply to message #1760293] Tue, 25 April 2017 18:34 Go to previous messageGo to next message
Zoltan Ujhelyi is currently offline Zoltan UjhelyiFriend
Messages: 335
Registered: July 2015
Senior Member
Hi Lars,

thank you for the detailed feedback, it seems we were indeed having incorrect documentation/snippets about the generic api.

0. Indeed, eiq was the old file extension for EMF-IncQuery, the use of vql is indeed required. The doSetup call was always static, the use of the constructor is and was unnecessary. I have updated the code snippet to represent the current version.
1. Yes, this is the generic API that can be used to parse the model.
2. The vql registration seems to be executed correctly, as seen in the exception trace, as the Antlr-based parser has started.
3. Yes, there are disadvantages, more specifically (a) you have a dependency on Xtext during runtime, (b) parsing the patterns has some overhead during initialization, (c) the execution speed of check and eval expressions is very noticeably slower in generic queries and (d) the generated code includes a type-safe wrapper for the matcher, etc. classes, making it easier to integrate to your application.

About the runtime exception, I can only guess what might have gone wrong; my first guess would be a missing parenthesis, semicolon or brace; or somehow the file URI is not resolved correctly. However, without seeing the vql file itself this is hard to decide. Or you could have a look at the executable version of our headless example [1]. You might have to change the version ranges of the VIATRA dependencies, but the code itself still works with up-to-date VIATRA versions.

I also would like to respond to your comment about the compiler being too slow. We have already received feedback on that and we are planning to address it, most likely this summer. Can you please give us some details about your environment, more specifically

a) How big/complex is your metamodel?
b) How is your metamodel available in your environment? Is it installed in your host, is it available in your workspace or only available in the target platform?
c) How big/many queries do you have?
d) Do you have one big query file or multiple, smaller files?

Thank you very much,
Zoltán

[1] http://git.eclipse.org/c/viatra/org.eclipse.viatra.examples.git/tree/query/headless
Re: VIATRA - Query Generic API [message #1760401 is a reply to message #1760314] Wed, 26 April 2017 16:20 Go to previous messageGo to next message
Lars Fritsche is currently offline Lars FritscheFriend
Messages: 6
Registered: April 2017
Junior Member
Hi Zoltán,

thank you for the fast reply.

In my opinion the headless example looks exactly like what I did.
However, I pruned my .vql file to only contain one pattern but the error still consists.
I also checked if the URI is wrong but if I change it only slightly, I'll get a FileNotFound exception.

If you don't see any obvious errors in my pattern, I'll try out the headless example but what concerns me about this error is that
I actually get generated code that is executable.
When you generate code, then I guess you have to load the .vql files analogously which obviously is working but when I try to load it, it is not.

package org.emoflon.ibex.tgg.common

import "platform:/plugin/org.emoflon.ibex.tgg.core.runtime/model/Runtime.ecore" as dep_ibex
import "http://www.eclipse.org/emf/2002/Ecore" as dep_ecore


pattern marked(p:TGGRuleApplication, o:EObject){
	TGGRuleApplication.createdSrc(p, o);
}


Regarding the code generation performance:
a) In the current project we are using mainly 3 metamodels.
The first represents a clazzdiagram with packages and classes (so only two EClasses)
Second is a file system with folders and files like DocFiles and BinaryFiles (in sum 4 EClasses)
The third metamodel connects both and consists of three ECLasses for which each connects one EClass of the first metamodel with one of the second.

Apart from that we use Ecore but only EObject is referenced.

In sum, we maybe reference about 8 EClasses within our patterns

b)
Apart from Ecore, our metamodels lie within our workspace.

c-d)
We are generating 8 .vql files with in average 25 queries.
Those queries with patterns are matching mostly up to 3-4 elements. (10 patterns in total per file)
All others contain invocations to other patterns (also external ones from other .vql files).

For this project we already have a build time of 3 minutes or more.

What I'd like to do would be to disable code generation and when the development has been finished, I can turn it on again to benefit from the advantages.

Regards,
Lars
Re: VIATRA - Query Generic API [message #1760405 is a reply to message #1760401] Wed, 26 April 2017 16:50 Go to previous messageGo to next message
Zoltan Ujhelyi is currently offline Zoltan UjhelyiFriend
Messages: 335
Registered: July 2015
Senior Member
Hi Lars,

I have taken another look at the exception trace, and I would like you to check what version of Antlr is being used in your target platform, as it seems to me that the EOF token from Antlr is not found as a corresponding field. Xtext only works with Antlr 3.2, and I have seen it not work with Antlr 3.4 or newer. If this is not the issue, I would like to ask you to provide some environment I can reproduce it, as your code seems appropriate in this scenario and I am very interested in what might have gone wrong.

About the builder performance: this seems like the case we have already found. For the short term, I can offer you one of two workarounds:

* Install the remaining EPackages to your host Eclipse as well. From there resolution is much faster.
* You could try out an experimental feature available since VIATRA 1.5.2: there is a Preference setting under VIATRA/Query Tooling called 'Disable automatic update of target platform metamodels'. By default it is turned off, as in case of your metamodel being updated this will cause VIATRA not to detect these updates, but we have found it sometimes helps with performance. We plan this feature is only temporary, as we hope to find a better solution that does not require such side effects as this one; thus this setting might go away in future VIATRA versions without any specific warning.

Hope this helps,
Zoltán
Re: VIATRA - Query Generic API [message #1760510 is a reply to message #1760405] Thu, 27 April 2017 19:42 Go to previous messageGo to next message
Lars Fritsche is currently offline Lars FritscheFriend
Messages: 6
Registered: April 2017
Junior Member
Hi Zoltán,

that was the problem!
I had a dependency to ANTLR 3.5.2 from some other plugin which I threw out and now it is working as expected.
Thank you!

I'll try out the performance tips in the near future but I still got a little problem with the API.
I'm now getting this exception here:

Exception in thread "main" java.lang.IllegalStateException: Query org.emoflon.ibex.tgg.axiom.Axiom_FWD contains errors.
	at com.google.common.base.Preconditions.checkState(Preconditions.java:177)
	at org.eclipse.viatra.query.patternlanguage.emf.specification.GenericEMFPatternPQuery.getDisjunctBodies(GenericEMFPatternPQuery.java:165)
	at org.eclipse.viatra.query.runtime.matchers.psystem.rewriters.PDisjunctionRewriter.rewrite(PDisjunctionRewriter.java:26)
	at org.eclipse.viatra.query.runtime.rete.construction.plancompiler.ReteRecipeCompiler.compileProduction(ReteRecipeCompiler.java:258)
	at org.eclipse.viatra.query.runtime.rete.construction.plancompiler.ReteRecipeCompiler.getCompiledForm(ReteRecipeCompiler.java:201)
	at org.eclipse.viatra.query.runtime.rete.boundary.ReteBoundary.accessProductionTrace(ReteBoundary.java:349)
	at org.eclipse.viatra.query.runtime.rete.matcher.ReteEngine$1.call(ReteEngine.java:192)
	at org.eclipse.viatra.query.runtime.rete.matcher.ReteEngine$1.call(ReteEngine.java:1)
	at org.eclipse.viatra.query.runtime.base.core.NavigationHelperImpl.coalesceTraversals(NavigationHelperImpl.java:1233)
	at org.eclipse.viatra.query.runtime.emf.EMFQueryRuntimeContext.coalesceTraversals(EMFQueryRuntimeContext.java:114)
	at org.eclipse.viatra.query.runtime.rete.matcher.ReteEngine.constructionWrapper(ReteEngine.java:241)
	at org.eclipse.viatra.query.runtime.rete.matcher.ReteEngine.accessMatcher(ReteEngine.java:188)
	at org.eclipse.viatra.query.runtime.rete.matcher.ReteEngine.getResultProvider(ReteEngine.java:513)
	at org.eclipse.viatra.query.runtime.internal.apiimpl.ViatraQueryEngineImpl.getResultProviderInternal(ViatraQueryEngineImpl.java:557)
	at org.eclipse.viatra.query.runtime.internal.apiimpl.ViatraQueryEngineImpl.getResultProviderInternal(ViatraQueryEngineImpl.java:545)
	at org.eclipse.viatra.query.runtime.internal.apiimpl.ViatraQueryEngineImpl.getResultProvider(ViatraQueryEngineImpl.java:535)
	at org.eclipse.viatra.query.runtime.internal.apiimpl.ViatraQueryEngineImpl.getMatcher(ViatraQueryEngineImpl.java:198)
	at org.eclipse.viatra.query.runtime.internal.apiimpl.ViatraQueryEngineImpl.getMatcher(ViatraQueryEngineImpl.java:175)
	at org.eclipse.viatra.query.patternlanguage.emf.specification.GenericPatternMatcher.on(GenericPatternMatcher.java:106)
	at org.eclipse.viatra.query.patternlanguage.emf.specification.GenericQuerySpecification.instantiate(GenericQuerySpecification.java:56)
	at org.eclipse.viatra.query.patternlanguage.emf.specification.GenericQuerySpecification.instantiate(GenericQuerySpecification.java:1)
	at org.eclipse.viatra.query.runtime.api.impl.BaseQuerySpecification.getMatcher(BaseQuerySpecification.java:71)
	at org.eclipse.viatra.transformation.evm.specific.event.ViatraQueryEventSourceSpecification.getMatcher(ViatraQueryEventSourceSpecification.java:52)
	at org.eclipse.viatra.transformation.evm.specific.event.ViatraQueryEventSource.<init>(ViatraQueryEventSource.java:35)
	at org.eclipse.viatra.transformation.evm.specific.event.ViatraQueryEventRealm.createSource(ViatraQueryEventRealm.java:40)
	at org.eclipse.viatra.transformation.evm.specific.event.ViatraQueryRuleInstanceBuilder.prepareRuleInstance(ViatraQueryRuleInstanceBuilder.java:30)
	at org.eclipse.viatra.transformation.evm.api.RuleSpecification.instantiateRule(RuleSpecification.java:99)
	at org.eclipse.viatra.transformation.evm.api.RuleBase.instantiateRule(RuleBase.java:78)
	at org.eclipse.viatra.transformation.evm.api.RuleEngine.addRule(RuleEngine.java:112)
	at org.eclipse.viatra.transformation.evm.api.RuleEngine.addRule(RuleEngine.java:94)
	at org.eclipse.viatra.transformation.runtime.emf.transformation.eventdriven.EventDrivenTransformation$EventDrivenTransformationBuilder.build(EventDrivenTransformation.java:129)
	at org.emoflon.ibex.tgg.run.CDToDocTransformation_NoCodeGen.createTransformation(CDToDocTransformation_NoCodeGen.java:153)


However, I have debugged a lot today and while creating a minimal example for you, I realized that this problem does not occur when the imports in my .vql file are only referencing:

import "http://www.eclipse.org/emf/2002/Ecore" as dep_1


The problem only occurs when I replace the dependency with my own metamodel, just like:

import "platform:/resource/CDToDoc2/model/CD.ecore" as dep_1


Do you know what the problem might be?
When I generate code from my patterns it does work.

I created a minimal example of this problem and I hope that I deleted all unnecessary dependencies.
Could you maybe have a look at this?

https://www.dropbox.com/s/e4yf1t6c5fd0ym6/Viatra_CodegenBug.zip?dl=0

Regards and thanks,
Lars


PS: In your documentation to the generic API, you only have a code snippet on how to generate generic matchers but not on how to insert the GenericQuerySpecifications into the EventDrivenTransformationRuleBuilder which is what I did.
Maybe this would also help others if there is an example for this in the documentation.
Re: VIATRA - Query Generic API [message #1760547 is a reply to message #1760510] Fri, 28 April 2017 08:26 Go to previous message
Zoltan Ujhelyi is currently offline Zoltan UjhelyiFriend
Messages: 335
Registered: July 2015
Senior Member
Hi Lars,

thanks for the reproduction, now I see what is the problem here: if you are using VIATRA outside Eclipse environments, you also have to set up a few things manually:

* First, you have to register your EMF metamodels into the EPackage registry.
* If you have generated code from your EPackage, you can simply call YourPackage.eINSTANCE that will load your EPackage, and register it and its dependencies to the registry.
* If you want to use your ecore files directly, you have to open it as an EMF resource, and call EPackage.Registry.INSTANCE.put() with the appropriate nsURI/EPackage calls. Furthermore, make sure you use the same nsURI that you are importing in your VQL file. This is used for metamodel resolution.
* Remark: if you don't have the classes generated from EPackages available in the classpath, you cannot use the generated code from the patterns. Still, our recommendation is try to use the generated code if possible, as it results in better
performance and less mistakes because the type-safe API.
* The other important aspect is calling out standalone setup; but that part was already done by you. This also ensured that the "http://www.eclipse.org/emf/2002/Ecore" metamodel was already available in the EPackage registry.

Remark: These steps are not required for the IDE, as it tries to be clever and figure out where your EPackages are; on the other hand, that is one of the main reasons that the editor/builder is so slow.

Some further remarks: platform:/resource and platform:/plugin urls do not work in standalone applications by default. platform:/resource urls can work if a corresponding entry is added to URIMappingRegistryImpl.INSTANCE; I don't know of any way to make platform:/plugin uris work in standalone environments.

About the documentation: you are right, some further example on how to use GenericQuerySpecification with either EVM or the transformation API should be added there. Our only excuse is that we did not use generic queries in quite some time, so its documentation became somewhat stale.

Some further information about detecting such issues manually (to avoid the exceptions): after you create a query specification, you could query its status and list of errors as well; this means at least you have a preliminary idea about what might have gone wrong. Or you could execute EMF Validation on the resource with the pattern definition and check whether there are any validation rules available.

Furthermore, I have opened bug https://bugs.eclipse.org/bugs/show_bug.cgi?id=515934 to find such errors earlier, and provide a better error message for erroneous patterns here.

Best regards,
Zoltán
Previous Topic:Persistence of internal state
Next Topic:ViatraQuery - Synchronisation of Threads
Goto Forum:
  


Current Time: Thu Nov 23 14:50:52 GMT 2017

Powered by FUDForum. Page generated in 0.01783 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software