Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » TMF (Xtext) » Resolving imports in standalone generator
Resolving imports in standalone generator [message #775090] Thu, 05 January 2012 09:59 Go to next message
Lasse  indg is currently offline Lasse indgFriend
Messages: 12
Registered: July 2009
Junior Member
Hi,

I realize that this has been asked before, but I just can't find the solution.
So bear with me as I try to ask this question in a simple way instead of posting heaps of code that doesn't work, I'll just post some simpler more naive code that doesn't work either ;-)

I am trying to write a standalone wrapper for my code generator, and I am having trouble with the imports.

I have this simple domainmodel derived model:
* I is not based om XBase
* Uses namespace imports
grammar com.mydsl.SchemaDSL with org.eclipse.xtext.common.Terminals

generate schemaDSL "http://com.mydsl.SchemaDSL"

SchemaDSL:
	PackageDeclaration
	imports+=Import* 
	elements+=Entity*;

PackageDeclaration:
	'namespace' name=QualifiedName;

Import:
	'import' importedNamespace=QualifiedName;

Entity: Element;

Element:
	'element' name=ValidID ('extends' superType=[Entity])? '{'
		properties+=Property*
	'}';

Property:
	name=ValidID ':' type=[Entity];

QualifiedName:
	ID ('.' ID)*;

ValidID:
	ID
;

The above model is slightly simplified. I can post the real one if needed.

Using this model I can create the following simple files:
main.model:
namespace com.mydsl.main

import com.mydsl.ref.ReferencedEntity

element MainEntity {
	x : LocalEntity
	y : ReferencedEntity
} 

element LocalEntity {
	name : String
}


ref.model:
namespace com.mydsl.ref

element ReferencedEntity {
	number : Integer
}


I can launch the generated eclipse UI, and it generates files beautifully.

This standalone generator is inspired by the one from the state machine example.
	public static void main(String[] args) {
		Injector injector = new SchemaDSLStandaloneSetupGenerated().createInjectorAndDoEMFRegistration();
		Main2 main = injector.getInstance(Main2.class);
		// load the resource
		ResourceSet set = main.resourceSetProvider.get();
		Resource resource = set.getResource(URI.createURI(args[0]), true);
		
		// validate the resource
		List<Issue> list = main.validator.validate(resource, CheckMode.ALL, CancelIndicator.NullImpl);
		if (!list.isEmpty()) {
			for (Issue issue : list) {
				System.err.println(issue);
			}
			System.exit(-1);
		}
		
		// configure and start the generator
		main.fileAccess.setOutputPath("src-gen/");
		main.generator.doGenerate(resource, main.fileAccess);
		
		System.out.println("Code generation finished.");
	}


After adding all dependencies to my classpath I can use that to compile ref.model without errors.
When I compile main.model, it fails in the validator:
ERROR:Couldn't resolve reference to Entity 'ReferencedEntity'. (src\main.model line : 7)


I have put ref.model on my classpath, but that didn't make any difference.

http://www.eclipse.org/Xtext/documentation/1_0_1/xtext.html#processing_Xtext_models
Quote:
If one Resource references another Resource, EMF will automatically load that other Resource into the same ResourceSet as soon as the cross-reference is resolved. Resolution of cross-references is done lazy, i.e. on first access.


Somehow I need to get my standalone EMF setup to resolve the resources, but I just can't find out how to do it.

The closest I got was manually scanning my file system and adding all files to the resourceset like this:
for (String modelFile : getSourceFiles(srcFolder) {
    Resource modelResource = set.getResource(URI.createURI(modelFile), true);
    resource.getContents().addAll(modelResource.getContents());
}


I could make this post a lot longer by trying to explain all the things I have tried and failed with. But I think it is already too long.

This must have been solved many times by the community, since a standalone generator for any DSL must be the first requirement for deployment.

I hope that you can help me in the right direction.

Thanks in advance.
Re: Resolving imports in standalone generator [message #775126 is a reply to message #775090] Thu, 05 January 2012 11:36 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 13357
Registered: July 2009
Senior Member
Hi,

you have the solution in the hand so i dont see what is the problem

Resource resource = set.getResource(URI.createURI(args[0]), true);

you have to do this for all referenced files as well.

another solution: you can instantiate and call a workflow components
from java too (Reader)

~Christian


Need professional support for Xtext, Xpand, EMF?
Go to: https://www.itemis.com/en/xtext/
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Re: Resolving imports in standalone generator [message #775149 is a reply to message #775126] Thu, 05 January 2012 12:59 Go to previous messageGo to next message
Lasse  indg is currently offline Lasse indgFriend
Messages: 12
Registered: July 2009
Junior Member
Thank you for your reply.

I have been down this path, but I ran into another problem, so I thought that I would backtrack a little in my original question, to find out if I should have gone down that road in the first place.

Ok, so here is my standalone generator that manually loads all referenced files:

    Injector injector = new SchemaDSLStandaloneSetupGenerated().createInjectorAndDoEMFRegistration();
    Main2 main = injector.getInstance(Main2.class);
    // load the resource
    ResourceSet set = main.resourceSetProvider.get();
    String folderName = args[0];
    Resource resource = null;
    for (String fileName : getSourceFiles(folderName)) {
        if (resource == null) {
            resource = set.getResource(URI.createURI(fileName), true);
        } else {
            Resource modelResource = set.getResource(URI.createURI(fileName), true);
            resource.getContents().addAll(modelResource.getContents());					
        }
    }
    
    // validator disabled for now
    main.fileAccess.setOutputPath("src-gen/");
    main.generator.doGenerate(resource, main.fileAccess);

(I pass in 'src' and it discovers 'src/*.model')

This works for the two files main.model and ref.model above.

However if I introduce ref2.model:
namespace com.mydsl.ref2 r2

element ReferencedEntity2 {
    number : Integer
}

and change ref.model to:
namespace com.mydsl.ref r

import com.mydsl.ref2.ReferencedEntity2

element ReferencedEntity {
    number : Integer
    ref2 : ReferencedEntity2
}


0    [main] ERROR xt.linking.lazy.LazyLinkingResource  - resolution of uriFragment 'xtextLink_::0.1.0.2.1::2::/4' failed.
org.eclipse.emf.common.util.BasicEList$BasicIndexOutOfBoundsException: index=0, size=0
        at org.eclipse.emf.common.util.BasicEList.get(BasicEList.java:352)
        at org.eclipse.xtext.linking.lazy.LazyURIEncoder.resolveShortFragment(LazyURIEncoder.java:92)
        at org.eclipse.xtext.linking.lazy.LazyURIEncoder.decode(LazyURIEncoder.java:80)
        at org.eclipse.xtext.linking.lazy.LazyLinkingResource.getEObject(LazyLinkingResource.java:162)
        at org.eclipse.emf.ecore.resource.impl.ResourceSetImpl.getEObject(ResourceSetImpl.java:219)
        at org.eclipse.emf.ecore.util.EcoreUtil.resolve(EcoreUtil.java:202)
        at org.eclipse.emf.ecore.util.EcoreUtil.resolve(EcoreUtil.java:262)
        at org.eclipse.emf.ecore.impl.BasicEObjectImpl.eResolveProxy(BasicEObjectImpl.java:1475)
        at com.mydsl.schemaDSL.impl.PropertyImpl.getType(PropertyImpl.java:204)


I get the error when I try to access a referenced type:
	def mapType(Property p) {
		var type = p.type // type is a reference to Entity
    }


My guess is that I should read the resources in some different way, but I am stumped on what that way might be.
Re: Resolving imports in standalone generator [message #775297 is a reply to message #775149] Thu, 05 January 2012 18:34 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 13357
Registered: July 2009
Senior Member
Hi,

why do you do this getcontents add all stuff. this is not neccessary. standalone the global scope is built of ALL resources in the resourceset.
so you do not need to shift stuff between resources. and if you have to you have to first resolve all proxies.

~Christian


Need professional support for Xtext, Xpand, EMF?
Go to: https://www.itemis.com/en/xtext/
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Re: Resolving imports in standalone generator [message #775345 is a reply to message #775297] Thu, 05 January 2012 20:16 Go to previous messageGo to next message
Lasse  indg is currently offline Lasse indgFriend
Messages: 12
Registered: July 2009
Junior Member
Thank you very much!
Of course it was incredible simple in the end. I don't know why it caused me so much trouble Grin

Below is my standalone generator for the benefit of others that might have the same request in the future.

/Lasse

package com.mydsl.generator;

import java.io.File;
import java.io.IOException;
import java.util.ArrayList;
import java.util.LinkedHashSet;
import java.util.List;
import java.util.Set;

import org.eclipse.emf.common.util.URI;
import org.eclipse.emf.ecore.resource.Resource;
import org.eclipse.emf.ecore.resource.ResourceSet;
import org.eclipse.xtext.Constants;
import org.eclipse.xtext.generator.IGenerator;
import org.eclipse.xtext.generator.JavaIoFileSystemAccess;
import org.eclipse.xtext.util.CancelIndicator;
import org.eclipse.xtext.validation.CheckMode;
import org.eclipse.xtext.validation.IResourceValidator;
import org.eclipse.xtext.validation.Issue;

import com.google.inject.Inject;
import com.google.inject.Injector;
import com.google.inject.Provider;
import com.google.inject.name.Named;

import com.mydsl.SchemaDSLStandaloneSetupGenerated;

public class Main {

	@Inject 
	private Provider<ResourceSet> resourceSetProvider;
	
	@Inject
	private IResourceValidator validator;
	
	@Inject
	private IGenerator generator;
	
	@Inject 
	private JavaIoFileSystemAccess fileAccess;

	@Inject @Named(Constants.FILE_EXTENSIONS)
	private String fileExtension;
	
	public static void main(String[] args) {
		checkArgs(args);
		try {
			Injector injector = new SchemaDSLStandaloneSetupGenerated().createInjectorAndDoEMFRegistration();
			Main main = injector.getInstance(Main.class);
			main.run(args[0]);
			
		} catch (Exception e) {
			System.err.println("Unexpected error");
			e.printStackTrace();
		}
	}

	private static void checkArgs(String[] args) {
		if (args.length != 1) {
			System.err.println("Name of the source folder expected");
			System.exit(-1);
		} else {
			File folder = new File(args[0]);
			if (!(folder.exists() && folder.isDirectory())) {
				System.err.println("Source folder " + folder.getAbsolutePath() + " not found");
				System.exit(-1);
			}
		}
	}
	
	private void run(String folderName) throws IOException {
		// load the resource
		ResourceSet set = resourceSetProvider.get();
		Set<Resource> resources = new LinkedHashSet<Resource>();
		for (String fileName : getSourceFiles(folderName)) {
			Resource modelResource = set.getResource(URI.createURI(fileName), true);
			resources.add(modelResource);
		}

		// configure and start the generator
		fileAccess.setOutputPath("src-gen/");
		for (Resource resource : resources) {
			// validate the resource
			List<Issue> list = validator.validate(resource, CheckMode.ALL, CancelIndicator.NullImpl);
			if (!list.isEmpty()) {
				for (Issue issue : list) {
					System.err.println(issue);
				}
				System.exit(-2);
			}

			generator.doGenerate(resource, fileAccess);
		}
		
		System.out.println("Code generation finished.");
	}
	
	private List<String> getSourceFiles(String folderName) throws IOException {
		ArrayList<String> result = new ArrayList<String>();
		File folder = new File(folderName);
		for (String s : folder.list()) {
			File f = new File(folder.getAbsolutePath() + '/' + s);
			String relativePath = folderName + '/' + s;
			if (isModelFile(f)) {
				result.add(relativePath);
			}
			else if (f.isDirectory()) {
				result.addAll(getSourceFiles(relativePath));
			}
		}		
		return result;
	}

	private boolean isModelFile(File f) {
		return f.isFile() && f.getName().endsWith("." + fileExtension);
	}
}

Re: Resolving imports in standalone generator [message #885305 is a reply to message #775345] Tue, 12 June 2012 19:28 Go to previous messageGo to next message
Brad Riching is currently offline Brad RichingFriend
Messages: 20
Registered: May 2012
Junior Member
Lasse,

I tried your example because I was running into the same problem you had. However, when I do, I get the following stack trace when I run my compiler at the command line in cygwin:

$ java edu.byu.ee.phdl.Compile fmc_module
1    [main] ERROR xt.linking.lazy.LazyLinkingResource  - resolution of uriFragment 'xtextLink_::0.3.0.3.5::5::/8' failed.
java.lang.StringIndexOutOfBoundsException: String index out of range: 402
        at java.lang.String.substring(String.java:1934)
        at org.eclipse.xtext.nodemodel.impl.AbstractNode.getText(AbstractNode.java:84)
        at org.eclipse.xtext.nodemodel.util.NodeModelUtils.getTokenText(NodeModelUtils.java:351)
        at org.eclipse.xtext.linking.impl.LinkingHelper.getCrossRefNodeAsString(LinkingHelper.java:51)
        at org.eclipse.xtext.linking.impl.DefaultLinkingService.getCrossRefNodeAsString(DefaultLinkingService.java:132)
        at org.eclipse.xtext.linking.impl.DefaultLinkingService.getLinkedObjects(DefaultLinkingService.java:113)
        at org.eclipse.xtext.linking.lazy.LazyLinkingResource.getEObject(LazyLinkingResource.java:175)
        at org.eclipse.xtext.linking.lazy.LazyLinkingResource.resolveLazyCrossReference(LazyLinkingResource.java:143)
        at org.eclipse.xtext.linking.lazy.LazyLinkingResource.resolveLazyCrossReferences(LazyLinkingResource.java:104)
        at org.eclipse.xtext.EcoreUtil2.resolveLazyCrossReferences(EcoreUtil2.java:491)
        at org.eclipse.xtext.validation.ResourceValidatorImpl.resolveProxies(ResourceValidatorImpl.java:127)
        at org.eclipse.xtext.validation.ResourceValidatorImpl.validate(ResourceValidatorImpl.java:62)
        at edu.byu.ee.phdl.Compile.run(Compile.java:81)
        at edu.byu.ee.phdl.Compile.main(Compile.java:117)
Unexpected error
org.eclipse.emf.common.util.WrappedException: java.lang.StringIndexOutOfBoundsException: String index out of range: 402
        at org.eclipse.xtext.linking.lazy.LazyLinkingResource.getEObject(LazyLinkingResource.java:209)
        at org.eclipse.xtext.linking.lazy.LazyLinkingResource.resolveLazyCrossReference(LazyLinkingResource.java:143)
        at org.eclipse.xtext.linking.lazy.LazyLinkingResource.resolveLazyCrossReferences(LazyLinkingResource.java:104)
        at org.eclipse.xtext.EcoreUtil2.resolveLazyCrossReferences(EcoreUtil2.java:491)
        at org.eclipse.xtext.validation.ResourceValidatorImpl.resolveProxies(ResourceValidatorImpl.java:127)
        at org.eclipse.xtext.validation.ResourceValidatorImpl.validate(ResourceValidatorImpl.java:62)
        at edu.byu.ee.phdl.Compile.run(Compile.java:81)
        at edu.byu.ee.phdl.Compile.main(Compile.java:117)
Caused by: java.lang.StringIndexOutOfBoundsException: String index out of range: 402
        at java.lang.String.substring(String.java:1934)
        at org.eclipse.xtext.nodemodel.impl.AbstractNode.getText(AbstractNode.java:84)
        at org.eclipse.xtext.nodemodel.util.NodeModelUtils.getTokenText(NodeModelUtils.java:351)
        at org.eclipse.xtext.linking.impl.LinkingHelper.getCrossRefNodeAsString(LinkingHelper.java:51)
        at org.eclipse.xtext.linking.impl.DefaultLinkingService.getCrossRefNodeAsString(DefaultLinkingService.java:132)
        at org.eclipse.xtext.linking.impl.DefaultLinkingService.getLinkedObjects(DefaultLinkingService.java:113)
        at org.eclipse.xtext.linking.lazy.LazyLinkingResource.getEObject(LazyLinkingResource.java:175)
        ... 7 more


However, when I create a run configuration in Eclipse, give it my main class (from your example) I get expected output in the console window:
Code generation finished.


The program also generates all the output files as expected. As far as I can tell, I have all of the jar dependencies on my classpath for running the standalone application outside of eclipse:
CLASSPATH=.;
C:\Users\brad\eclipse\edu.byu.ee.phdl.xtext\bin;
C:\Program Files\eclipse Indigo\plugins\org.eclipse.xtext_2.2.1.v201112130541.jar;
C:\Program Files\eclipse Indigo\plugins\org.eclipse.emf.ecore_2.7.0.v20120127-1122.jar;
C:\Program Files\eclipse Indigo\plugins\org.eclipse.emf.common_2.7.0.v20120127-1122.jar;
C:\Program Files\eclipse Indigo\plugins\org.eclipse.emf.ecore.xmi_2.7.0.v20120127-1122.jar;
C:\Program Files\eclipse Indigo\plugins\com.google.inject_3.0.0.no_aop.jar;
C:\Program Files\eclipse Indigo\plugins\javax.inject_1.0.0.v20091030.jar;
C:\Program Files\eclipse Indigo\plugins\org.apache.log4j_1.2.15.v201012070815.jar;
C:\Program Files\eclipse Indigo\plugins\org.eclipse.xtext.common.types_2.2.1.v201112130541.jar;
C:\Program Files\eclipse Indigo\plugins\com.google.guava_10.0.1.1.jar;
C:\Program Files\eclipse Indigo\plugins\org.eclipse.xtext.util_2.2.1.v201112130541.jar;
C:\Program Files\eclipse Indigo\plugins\org.eclipse.xtext.xbase.lib_2.2.1.v201112130541.jar;
C:\Program Files\eclipse Indigo\plugins\org.eclipse.emf.mwe.utils_1.2.1.v201112070431.jar;
C:\Program Files\eclipse Indigo\plugins\org.eclipse.emf.mwe.core_1.2.1.v201112070431.jar;
C:\Program Files\eclipse Indigo\plugins\org.apache.commons.logging_1.0.4.v201101211617.jar


I have tried closing my currently running instance of Eclipse to rule out any global registry issues. I end up with the same problem with the linking. My intuition says it has something to do with the URI, since it works inside Eclipse but not from the command line. Do I need to add a line in the main class that sets the platformURI? such as:

new org.eclipse.emf.mwe.utils.StandaloneSetup().setPlatformUri(<cwd>);


where <cwd> is the current working directory (specified by your source folder argument in the main method).

I'm really stumped, as the error reporting doesn't appear to offer me any good direction.

My question is: have you been able to run this outside of the Eclipse platform, truly from the command-line with Eclipse completely closed? If so, would you mind sharing how you did it?
Re: Resolving imports in standalone generator [message #885317 is a reply to message #885305] Tue, 12 June 2012 19:52 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 13357
Registered: July 2009
Senior Member
Sorry from the stuff you posted i cannot tell anything.
did you try the export -> runnable jar file wizard?


Need professional support for Xtext, Xpand, EMF?
Go to: https://www.itemis.com/en/xtext/
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Re: Resolving imports in standalone generator [message #885324 is a reply to message #885317] Tue, 12 June 2012 20:11 Go to previous messageGo to next message
Brad Riching is currently offline Brad RichingFriend
Messages: 20
Registered: May 2012
Junior Member
Christian,

Thank you. That worked for me. I guess I thought exporting into a jar was optional as long as I had all of the dependencies resolved on my classpath. I'm wondering why the xtext standalone howto does not mention this simple step, which would have avoided me several hours of troubleshooting and the hassle of tying up valuable eclipse developer resources.

Best wishes,

Brad
Re: Resolving imports in standalone generator [message #885326 is a reply to message #885324] Tue, 12 June 2012 20:16 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 13357
Registered: July 2009
Senior Member
Hi,

feel free to file an enhancement request against the docs

~Christian


Need professional support for Xtext, Xpand, EMF?
Go to: https://www.itemis.com/en/xtext/
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Re: Resolving imports in standalone generator [message #897077 is a reply to message #775090] Sat, 21 July 2012 13:27 Go to previous messageGo to next message
Hannes Stockner is currently offline Hannes StocknerFriend
Messages: 1
Registered: May 2011
Junior Member
Hello,

I am using xtext 2.3.0 and I copied the code from Lasse indg to my standalone generator.

I still get ""ERROR:Couldn't resolve reference to..."

Using single files without references works fine.

Do you have any idea what the problem could be?

Thanks
Hannes
Re: Resolving imports in standalone generator [message #897080 is a reply to message #897077] Sat, 21 July 2012 13:37 Go to previous message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 13357
Registered: July 2009
Senior Member
https://bugs.eclipse.org/bugs/show_bug.cgi?id=382555

Need professional support for Xtext, Xpand, EMF?
Go to: https://www.itemis.com/en/xtext/
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Previous Topic:can't use any word same than a keyword in my custom Enum rule
Next Topic:Newbie: JVMModelInferrer, extend class from third-party-lib
Goto Forum:
  


Current Time: Wed Feb 19 20:30:14 GMT 2020

Powered by FUDForum. Page generated in 0.05351 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top