Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » TMF (Xtext) » Xtext Standalone: Add/Remove/Modify resources
Xtext Standalone: Add/Remove/Modify resources [message #1774554] Tue, 17 October 2017 08:18 Go to next message
Oliver Libutzki is currently offline Oliver LibutzkiFriend
Messages: 30
Registered: September 2011
Member
Hi everyone,

I use Xtext in a standalone application which parses and interprets my DSL.

I used org.eclipse.xtext.builder.standalone.StandaloneBuilder in order to initialize my DSL files so far. Now I'm faced with the requirement to add/remove/modify resources after the builder has initially picked up and built my dsl files.

I guess switching to org.eclipse.xtext.build.IncrementalBuilder is the right way to go, isn't it? Unfortunately I do not find any documentation (not even JavaDoc) on how to use the IncrementalBuilder?

Is there any documentation and/or examples (unit tests?) how to use it?

Thanks!

Kind regards
Oliver
Re: Xtext Standalone: Add/Remove/Modify resources [message #1774555 is a reply to message #1774554] Tue, 17 October 2017 08:25 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 12029
Registered: July 2009
Senior Member
i dont know any docs but you can have a look what the gradle plugin does https://github.com/xtext/xtext-gradle-plugin/blob/9d689d05f8ad01476bb6867959b96d8e144546d2/xtext-gradle-builder/src/main/java/org/xtext/gradle/builder/XtextGradleBuilder.xtend and what the language server support does. https://github.com/eclipse/xtext-core/blob/1245d9893da2f7cfd16e3147c6907ce3dcb43148/org.eclipse.xtext.ide/src/org/eclipse/xtext/ide/server/ProjectManager.xtend

Need professional support for Xtext, Xpand, EMF?
Go to: https://xtext.itemis.com
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de

[Updated on: Tue, 17 October 2017 08:27]

Report message to a moderator

Re: Xtext Standalone: Add/Remove/Modify resources [message #1774556 is a reply to message #1774555] Tue, 17 October 2017 08:28 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 12029
Registered: July 2009
Senior Member
and i did some experiments figuring out what to do manually some years ago:

package org.xtext.example.mydsl;

import java.io.File;
import java.util.Set;

import org.eclipse.emf.common.util.URI;
import org.eclipse.emf.ecore.util.EcoreUtil;
import org.eclipse.xtext.build.BuildRequest;
import org.eclipse.xtext.build.IncrementalBuilder;
import org.eclipse.xtext.build.IndexState;
import org.eclipse.xtext.mwe.PathTraverser;
import org.eclipse.xtext.resource.IContainer;
import org.eclipse.xtext.resource.IResourceServiceProvider;
import org.eclipse.xtext.resource.XtextResourceSet;
import org.eclipse.xtext.resource.containers.ProjectDescriptionBasedContainerManager;
import org.eclipse.xtext.resource.impl.ChunkedResourceDescriptions;
import org.eclipse.xtext.resource.impl.ProjectDescription;
import org.eclipse.xtext.resource.impl.ResourceDescriptionsData;
import org.eclipse.xtext.service.AbstractGenericModule;
import org.eclipse.xtext.util.UriUtil;
import org.eclipse.xtext.xbase.lib.CollectionLiterals;
import org.eclipse.xtext.xbase.lib.Functions.Function1;

import com.google.common.base.Predicate;
import com.google.common.collect.Lists;
import com.google.inject.Guice;
import com.google.inject.Injector;
import com.google.inject.util.Modules;

public class Main {

    public static void main(String[] args) {

//        Injector injector = Guice.createInjector(new StandaloneBuilderModule());
//        StandaloneBuilder builder = injector.getInstance(StandaloneBuilder.class);
//        builder.setClassPathEntries(ImmutableList.of());
//        builder.setBaseDir("./spielwiese");
//  builder.setSourceDirs(Lists.newArrayList("./spielwiese/src","./spielwiese/src-gen"));
//        ILanguageConfiguration cfg = new ILanguageConfiguration() {
//
//            @Override
//            public boolean isJavaSupport() {
//                // TODO Auto-generated method stub
//                return false;
//            }
//
//            @Override
//            public String getSetup() {
//                return "org.xtext.example.mydsl.MyDslStandaloneSetup";
//            }
//
//            @Override
//            public Set<OutputConfiguration> getOutputConfigurations() {
//                OutputConfiguration config = new OutputConfiguration(IFileSystemAccess.DEFAULT_OUTPUT);
//                config.setOutputDirectory("src-gen");
//                return ImmutableSet.of(config);
//            }
//        };
//        builder.setLanguages(new LanguageAccessFactory().createLanguageAccess(ImmutableList.of(cfg), Main.class.getClassLoader()));
//
//
//        builder.launch();

        Injector injector = new MyDslStandaloneSetup() {

            public Injector createInjector() {
                return Guice.createInjector(Modules.override(new MyDslRuntimeModule()).with(new AbstractGenericModule() {
                    public Class<? extends IContainer.Manager> bindIContainer$Manager() {
                        return ProjectDescriptionBasedContainerManager.class;
                    }
                }));
            }
        }
        .createInjectorAndDoEMFRegistration();
        IResourceServiceProvider resourceServiceProvider = injector.getInstance(IResourceServiceProvider.class);

        Injector injector2 = Guice.createInjector(new AbstractGenericModule() {
            public Class<? extends IContainer.Manager> bindIContainer$Manager() {
                return ProjectDescriptionBasedContainerManager.class;
            }
        });
        IncrementalBuilder incrementalBuilder = injector2.getInstance(IncrementalBuilder.class);

        IndexState indexState = injector2.getInstance(IndexState.class);

        boolean firstRun = true;
        long lastBuild = System.currentTimeMillis();
        while (true) {
            try {
                final boolean firstRunFinal = firstRun;
                final long lastBuildFinal = lastBuild;
                firstRun = false;
                Predicate<URI> isValidPredicate = new Predicate<URI>() {

                    @Override
                    public boolean apply(URI input) {
                        return firstRunFinal || new File(input.toFileString()).lastModified() > lastBuildFinal;
                    }
                };
                Set<URI> allChanged = new PathTraverser().findAllResourceUris("spielwiese/src", isValidPredicate);
                BuildRequest request = new BuildRequest();
                XtextResourceSet resourceSet = injector.getInstance(XtextResourceSet.class);
                request.setResourceSet(resourceSet);
                request.setBaseDir(UriUtil.createFolderURI(new File("./spielwiese/")));
 request.setDirtyFiles(Lists.newArrayList(allChanged));

                ResourceDescriptionsData newIndex = indexState.getResourceDescriptions().copy();

                ProjectDescription description = new ProjectDescription();
                description.setName("demo");

                description.attachToEmfObject(resourceSet);

                ChunkedResourceDescriptions index = new ChunkedResourceDescriptions(CollectionLiterals.emptyMap(), resourceSet);
                index.setContainer("demo", newIndex);

                lastBuild = System.currentTimeMillis();
                Function1<? super URI, ? extends IResourceServiceProvider> languages = new Function1<URI, IResourceServiceProvider>() {

                    @Override
                    public IResourceServiceProvider apply(URI uri) {
                        return IResourceServiceProvider.Registry.INSTANCE.getResourceServiceProvider(uri);
                    }
                };

                request.setState(new IndexState(newIndex, indexState.getFileMappings().copy()));

                indexState = incrementalBuilder.build(request, languages).getIndexState();


                Thread.sleep(5000);
            } catch (InterruptedException e) {
                // TODO Auto-generated catch block
                e.printStackTrace();
            }
        }


    }

}


Need professional support for Xtext, Xpand, EMF?
Go to: https://xtext.itemis.com
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Re: Xtext Standalone: Add/Remove/Modify resources [message #1774559 is a reply to message #1774556] Tue, 17 October 2017 08:50 Go to previous messageGo to next message
Oliver Libutzki is currently offline Oliver LibutzkiFriend
Messages: 30
Registered: September 2011
Member
Great! I highly appreciate your lighting fast, high-quality community work!

So the fundamental idea is to build a BuildRequest and pass it to IncrementalBuilder#build every time I want to rebuild the resources, right?
BuildRequest provides setters for dirtyFiles and deletedFiles, but there is no newFiles setter? How does the builder detect them? In your example you marked all resources as dirty on the first run. Is this the best solution for new resources?

Kind regards
Oliver

Re: Xtext Standalone: Add/Remove/Modify resources [message #1774560 is a reply to message #1774559] Tue, 17 October 2017 09:10 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 12029
Registered: July 2009
Senior Member
i dont know. did you have a look at the ide code? maybe you can store the timestamp of the last build when do a new collection round.

=> i assume the whole watching has to happen outside


Need professional support for Xtext, Xpand, EMF?
Go to: https://xtext.itemis.com
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de

[Updated on: Tue, 17 October 2017 09:14]

Report message to a moderator

Re: Xtext Standalone: Add/Remove/Modify resources [message #1774573 is a reply to message #1774560] Tue, 17 October 2017 12:57 Go to previous messageGo to next message
Oliver Libutzki is currently offline Oliver LibutzkiFriend
Messages: 30
Registered: September 2011
Member
Reading the resources with the IncrementalBuilder works. Now I would like to access EObjects via Xtext's index.

After building I use the indexState to get the first IEObjectDescription. Unfortunately calling getEObjectOrProxy() returns a proxy insted of the resolved EObject.

I wrote a short JUnit Test in order to illustrate this:
import com.google.inject.Inject
import org.eclipse.xtext.junit4.InjectWith
import org.eclipse.xtext.junit4.XtextRunner
import org.eclipse.xtext.junit4.build.AbstractIncrementalBuilderTest
import org.eclipse.xtext.resource.IResourceServiceProvider
import org.eclipse.xtext.resource.IResourceServiceProvider.Registry
import org.junit.Test
import org.junit.runner.RunWith

import static org.junit.Assert.*

@InjectWith(ModulesTestInjectorProvider)
@RunWith(XtextRunner)
class MyIncrementalBuilderTest extends AbstractIncrementalBuilderTest {

	@Inject IResourceServiceProvider.Registry registry

	override protected Registry getLanguages() {
		registry
	}

	@Test def void test() {
		val buildRequest = newBuildRequest [
			dirtyFiles = #[
				'src/SmokeTestModules.module' - '''
					Modules SmokeTestModules {
						Module SmokeTestModule {
							Predicate TruePredicate true
						}
					}
				'''
			]
		]
		val indexState = build(buildRequest)
		val firstElement = indexState.resourceDescriptions.exportedObjects.iterator.next
		println("IEObjectDescription: " + firstElement)
		println("EObjectOrProxy: " + firstElement.EObjectOrProxy)
		assertFalse(firstElement.EObjectOrProxy.eIsProxy)
	}
}


The test fails. This is the console output:
IEObjectDescription: org.eclipse.xtext.resource.persistence.SerializableEObjectDescription@5af6e15e
EObjectOrProxy: <somepackage>.impl.ModuleContainerImpl@64610fa (eProxyURI: inmemory:/src/SmokeTestModules.module#/0/@modulecontainer)


Any idea why the proxy cannot be resolved?

[Updated on: Tue, 17 October 2017 12:58]

Report message to a moderator

Re: Xtext Standalone: Add/Remove/Modify resources [message #1774575 is a reply to message #1774573] Tue, 17 October 2017 13:17 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 12029
Registered: July 2009
Senior Member
where do you try to resolve the proxy? SerializableEObjectDescription is meant to be detached from everything

Need professional support for Xtext, Xpand, EMF?
Go to: https://xtext.itemis.com
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de

[Updated on: Tue, 17 October 2017 13:18]

Report message to a moderator

Re: Xtext Standalone: Add/Remove/Modify resources [message #1774578 is a reply to message #1774575] Tue, 17 October 2017 13:27 Go to previous messageGo to next message
Oliver Libutzki is currently offline Oliver LibutzkiFriend
Messages: 30
Registered: September 2011
Member
I do not resolve the proxy explicitly. but normally the getEObjectOrProxy returns a resolved object automatically (if it's resolveable). As I do not have a context (EObject, Resource or ResourceSet), I cannot call EcoreUtil.resolve.

Is the problems's cause the existence of an instance of org.eclipse.xtext.resource.persistence.SerializableEObjectDescription? After inspecting the implementation it seems to be impossible to get a resolved EObject. Anyway, I wonder why it's a SerializableEObjectDescription...
Re: Xtext Standalone: Add/Remove/Modify resources [message #1774579 is a reply to message #1774578] Tue, 17 October 2017 13:35 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 12029
Registered: July 2009
Senior Member
yes and no. the builder can survive a restart. if you have that index to the disk and load it from there

Need professional support for Xtext, Xpand, EMF?
Go to: https://xtext.itemis.com
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Re: Xtext Standalone: Add/Remove/Modify resources [message #1774581 is a reply to message #1774579] Tue, 17 October 2017 13:39 Go to previous messageGo to next message
Oliver Libutzki is currently offline Oliver LibutzkiFriend
Messages: 30
Registered: September 2011
Member
Christian Dietrich wrote on Tue, 17 October 2017 09:35
yes and no. the builder can survive a restart. if you have that index to the disk and load it from there


Ok, I'm fine with SerializableEObjectDescription, but the missing puzzle piece is how to resolve the proxy?

Got it:
val resolvedObject = EcoreUtil.resolve(firstElement.EObjectOrProxy, buildRequest.resourceSet)

[Updated on: Tue, 17 October 2017 13:41]

Report message to a moderator

Re: Xtext Standalone: Add/Remove/Modify resources [message #1774582 is a reply to message #1774581] Tue, 17 October 2017 13:41 Go to previous messageGo to next message
Christian Dietrich is currently offline Christian DietrichFriend
Messages: 12029
Registered: July 2009
Senior Member
in the test or in general in general: load the resource into a resourceset

Need professional support for Xtext, Xpand, EMF?
Go to: https://xtext.itemis.com
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
Re: Xtext Standalone: Add/Remove/Modify resources [message #1774583 is a reply to message #1774582] Tue, 17 October 2017 13:47 Go to previous message
Oliver Libutzki is currently offline Oliver LibutzkiFriend
Messages: 30
Registered: September 2011
Member
Christian Dietrich wrote on Tue, 17 October 2017 09:41
in the test or in general in general: load the resource into a resourceset

Yeah, afterwards its so trivial. Thanks for your help and patience. Works fine now!
Previous Topic:Linking XCastExpression to a "toX" function
Next Topic:Xtext 2.13.0.RC1 is available
Goto Forum:
  


Current Time: Wed Jun 20 04:14:05 GMT 2018

Powered by FUDForum. Page generated in 0.03048 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top