Home » Modeling » TMF (Xtext) » Problem using the Xtext serializer
Problem using the Xtext serializer [message #1125474] |
Fri, 04 October 2013 14:55 |
Alan DW Messages: 119 Registered: March 2012 |
Senior Member |
|
|
Hello everyone,
I have an XMI file which contains instances of my DSL. I want to deserialize the XMI and store the EObjects in files in my workspace (programmatically), such that the files contain DSL code again. This works very well as long as there are no cross references, but I run into problems when there are some.
I've attached a minimum working example that uses a slight extension of the "Greetings DSL", just to introduce a few cross references and illustrate the problem. The project "MinmalWorkingExample" contains a JUnit test in "src/minimalworkingexample/TestClass" that can be run as a JUnit-Plugin-Test without any setup required. It shows what I'm trying to do.
The error I get when deserializing a model element with cross references back into DSL code is:
java.lang.RuntimeException: No EObjectDescription found in Scope Greetings.person for Model.fragments[1]->Person'John'
It would be easy to state now that this is a scoping problem. Two facts however indicate that this is not the case:
- Using the StandaloneSetup of the DSL fixes the problem, but I can't use it inside Eclipse (this is soooo frustrating -.-)
- I did not implement any custom scoping (or other language customization) at all.
My best guess would be that there is something wrong in the "wiring" of the plug-ins (plugin.xml, build.properties...) but I could not spot any problems (neither did my IDE). I'm using Eclipse Juno SR1 and Xtext 2.4.3.
Any help would be much appreciated!
Alan
|
|
|
Re: Problem using the Xtext serializer [message #1125747 is a reply to message #1125474] |
Fri, 04 October 2013 21:10 |
|
package minmalworkingexample;
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertTrue;
import static org.junit.Assert.fail;
import java.io.ByteArrayInputStream;
import java.io.File;
import java.io.IOException;
import java.util.HashSet;
import java.util.Set;
import javax.inject.Inject;
import org.eclipse.core.resources.IFile;
import org.eclipse.core.resources.IProject;
import org.eclipse.core.resources.IProjectDescription;
import org.eclipse.core.resources.IWorkspaceRoot;
import org.eclipse.core.resources.IncrementalProjectBuilder;
import org.eclipse.core.resources.ResourcesPlugin;
import org.eclipse.core.runtime.CoreException;
import org.eclipse.core.runtime.NullProgressMonitor;
import org.eclipse.core.runtime.jobs.Job;
import org.eclipse.emf.common.util.URI;
import org.eclipse.emf.ecore.EObject;
import org.eclipse.emf.ecore.resource.Resource;
import org.eclipse.emf.ecore.resource.ResourceSet;
import org.eclipse.swt.widgets.Display;
import org.eclipse.xtext.builder.nature.XtextNature;
import org.eclipse.xtext.junit4.InjectWith;
import org.eclipse.xtext.junit4.XtextRunner;
import org.eclipse.xtext.resource.IResourceDescriptions;
import org.eclipse.xtext.ui.XtextProjectHelper;
import org.eclipse.xtext.ui.resource.IResourceSetProvider;
import org.eclipse.xtext.ui.wizard.XtextNewProjectWizard;
import org.junit.AfterClass;
import org.junit.BeforeClass;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.xtext.example.mydsl.MyDslUiInjectorProvider;
import org.xtext.example.mydsl.myDsl.Fragment;
import org.xtext.example.mydsl.myDsl.Greeting;
import org.xtext.example.mydsl.myDsl.Model;
import org.xtext.example.mydsl.myDsl.MyDslFactory;
import org.xtext.example.mydsl.myDsl.Person;
import utils.FileUtils;
import utils.XMIUtils;
@RunWith(XtextRunner.class)
@InjectWith(MyDslUiInjectorProvider.class)
public class TestClass {
private static final String MODEL_FILE_NAME = "resources/model.xmi";
private static final String TARGET_PROJECT_NAME = "TARGET";
@Inject
IResourceSetProvider resourceSetProvider;
@Inject
IResourceDescriptions desc;
@BeforeClass
public static void setUp() {
System.out.println("SET UP START");
// delete our dummy project (that we build during the test) if it exists.
IWorkspaceRoot workspaceRoot = ResourcesPlugin.getWorkspace().getRoot();
IProject targetProject = workspaceRoot.getProject(TARGET_PROJECT_NAME);
if (targetProject.exists()) {
try {
targetProject.delete(true, true, new NullProgressMonitor());
} catch (CoreException e) {
e.printStackTrace();
fail();
}
}
waitForJobs();
System.out.println("SET UP DONE");
}
/**
* This is the main test method.<br>
* It reads the XMI file from "/resources/model.xmi" and tries to save its contents to two different files.
*/
@Test
public void runTest() {
// enable the next line and the JUnit test will work and produce the expected output!
// However, it is strongly discouraged in OSGI-Based environments.
// MyDslStandaloneSetup.doSetup();
EObject model = this.deserializeTestXMI();
assertTrue("Could not deserialize XMI!", model != null && model instanceof Model);
this.partitionAndSerialize((Model) model);
}
private EObject deserializeTestXMI() {
File xmiFile = FileUtils.getBundleResource(Activator.getDefault().getBundle(), MODEL_FILE_NAME);
return XMIUtils.deserializeFromXMIFile(xmiFile);
}
/**
* Partitions the model into files and saves them, using the Xtext serializer.<br>
* All instances of {@link Person} will go into the file <code>Persons.mydsl</code>, all instances of
* {@link Greeting} will go into the file <code>Greetings.mydsl</code>.
*
* @param model
* The model to partition and save
*/
private void partitionAndSerialize(final Model model) {
IProject project = this.createTargetProject();
Set<Person> persons = new HashSet<Person>();
Set<Greeting> greetings = new HashSet<Greeting>();
for (Fragment fragment : model.getFragments()) {
if (fragment instanceof Person) {
persons.add((Person) fragment);
}
if (fragment instanceof Greeting) {
greetings.add((Greeting) fragment);
}
}
assertEquals(2, greetings.size());
assertEquals(2, persons.size());
IFile greetingsFile = project.getFile("Greetings.mydsl");
IFile personsFile = project.getFile("Persons.mydsl");
try {
this.createFile(greetingsFile);
this.createFile(personsFile);
} catch (CoreException e) {
e.printStackTrace();
fail();
}
ResourceSet resSet = this.resourceSetProvider.get(project);
Resource greetingsResource = resSet.getResource(URI.createPlatformResourceURI(greetingsFile.getFullPath().toString()), true);
Resource personsResource = resSet.getResource(URI.createPlatformResourceURI(personsFile.getFullPath().toString()), true);
Model personsModel = MyDslFactory.eINSTANCE.createModel();
personsModel.getFragments().addAll(persons);
Model greetingsModel = MyDslFactory.eINSTANCE.createModel();
greetingsModel.getFragments().addAll(greetings);
greetingsResource.getContents().add(greetingsModel);
personsResource.getContents().add(personsModel);
try {
personsResource.save(null);
project.build(IncrementalProjectBuilder.FULL_BUILD, new NullProgressMonitor());
waitForJobs();
System.out.println(desc.getAllResourceDescriptions());
greetingsResource.save(null);
} catch (IOException e) {
e.printStackTrace();
fail();
} catch (CoreException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
private IProject createTargetProject() {
IWorkspaceRoot workspaceRoot = ResourcesPlugin.getWorkspace().getRoot();
IProject project = workspaceRoot.getProject(TARGET_PROJECT_NAME);
try {
project.create(new NullProgressMonitor());
project.open(new NullProgressMonitor());
IProjectDescription description = project.getDescription();
String[] natures = description.getNatureIds();
String[] newnatures = new String[natures.length + 1];
System.arraycopy(natures, 0, newnatures, 0, natures.length);
newnatures[natures.length] = XtextProjectHelper.NATURE_ID;
description.setNatureIds(newnatures);
System.out.println(newnatures);
project.setDescription(description, new NullProgressMonitor());
XtextNature n = new XtextNature();
n.setProject(project);
n.configure();
} catch (CoreException e) {
e.printStackTrace();
fail();
}
return project;
}
private void createFile(final IFile file) throws CoreException {
file.create(new ByteArrayInputStream(new byte[0]), true, new NullProgressMonitor());
}
/**
* Perform post-test cleanup.
*/
@AfterClass
public static void tearDown() throws Exception {
// Dispose of test fixture.
waitForJobs();
// Add additional teardown code here.
}
/**
* Process UI input but do not return for the specified time interval.
*
* @param waitTimeMillis
* the number of milliseconds
*/
private static void delay(final long waitTimeMillis) {
Display display = Display.getCurrent();
// If this is the UI thread,
// then process input.
if (display != null) {
long endTimeMillis = System.currentTimeMillis() + waitTimeMillis;
while (System.currentTimeMillis() < endTimeMillis) {
if (!display.readAndDispatch()) {
display.sleep();
}
}
display.update();
}
// Otherwise, perform a simple sleep.
else {
try {
Thread.sleep(waitTimeMillis);
} catch (InterruptedException e) {
// Ignored.
}
}
}
/**
* Wait until all background tasks are complete.
*/
private static void waitForJobs() {
while (!Job.getJobManager().isIdle()) {
delay(1000);
}
}
}
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
|
|
| | |
Re: Problem using the Xtext serializer [message #1126247 is a reply to message #1126227] |
Sat, 05 October 2013 10:16 |
Alan DW Messages: 119 Registered: March 2012 |
Senior Member |
|
|
Well, basically, all information is available in the model, it is all there. As far as I know, the serializer draws its information from a kind of registry that is managed internally, and an object gets registered there when a resource (containing the object) is saved and the project is built. Rather than the internal registry, the serializer would have to use the instance model (provided by the XMI) directly. In the end, all it really has to do is to insert the (qualified) name of a referenced object whenever it encounters a non-containment reference. Actually, I'm quite surprised that I'm the first one to stumble upon this problem...
Another solution would be to not use the Xtext serializer alltogether and "manually" write the Model-to-Text transformation in Java/Xtend. I tried that, but I'm also getting some weird exceptions when I just write the DSL text into the files and try to build the project afterwards.
If anyone has a possible solution, please let me know. I'll continue to try things out and do research on the web, maybe I'll find something of use.
[Updated on: Sat, 05 October 2013 10:17] Report message to a moderator
|
|
|
Re: Problem using the Xtext serializer [message #1126309 is a reply to message #1126247] |
Sat, 05 October 2013 11:51 |
Moritz Eysholdt Messages: 161 Registered: July 2009 Location: Kiel, Germany |
Senior Member |
|
|
Alan DW wrote on Sat, 05 October 2013 11:12But what about a scenario where I have cyclic cross references? For example, an EObject in File A references an EObject in File B, but there is also an EObject in B that references one in A? Or transitive cycles (A->B->C->A)? How would you resolve such a situation? It is not so uncommon, after all.
That's no problem for the serializer. The serializer does nothing except call the IScopeProvider for every cross reference. As long as the scope provider can provider a scope for a cross reference and the scope can provide a name for the referenced object, the serializer can save the file.
Christian Dietrich wrote on Sat, 05 October 2013 11:48Sorry have no idea on that. Maybe it is not possible at all (I do not
have the time to dig on that but Xtext indexing is required for the
serialized somehow (I don't know if it is possible to skip that)
As mentioned above, the serializer uses scoping to determine the to-be-serialized names for cross references. However, Xtext's standard scoping implementation for cross-file references is based on the Xtext index. Thereby, serialization indirectly depends on the index.
If you don't like this behavior, you can override
org.eclipse.xtext.serializer.tokens.CrossReferenceSerializer.serializeCrossRef(EObject, CrossReference, EObject, INode, Acceptor)
and put in your own implementation. Maybe one that doesn't make use of scoping and/or the index and instead delegates to the IQualifiedNameProvider.
Alan DW wrote on Sat, 05 October 2013 12:16Well, basically, all information is available in the model, it is all there. As far as I know, the serializer draws its information from a kind of registry that is managed internally, and an object gets registered there when a resource (containing the object) is saved and the project is built. Rather than the internal registry, the serializer would have to use the instance model (provided by the XMI) directly. In the end, all it really has to do is to insert the (qualified) name of a referenced object whenever it encounters a non-containment reference. Actually, I'm quite surprised that I'm the first one to stumble upon this problem...
What you describe as the "internal Registry" sounds like Xtext's index. There is no other "internal Registry" otherwise.
Yes, when the Xtext Builder runs, it collects QualifiedNames for all model elements and stores them in the index. And the Xtext builder runs when you trigger a full build or save a file (autobuild, which is an incremental build).
Is your use case to save a file that cross-references elements from unsaved files?
|
|
|
Re: Problem using the Xtext serializer [message #1127329 is a reply to message #1126309] |
Sun, 06 October 2013 14:37 |
Alan DW Messages: 119 Registered: March 2012 |
Senior Member |
|
|
Hi Moritz,
thanks for your response.
Quote:Is your use case to save a file that cross-references elements from unsaved files?
Yes, it boils down to exactly this problem. I cannot guarantee that all cross-reference targets have been saved (and/or built) because I don't even know the grammar of the language in my use-case (it can be an arbitrary grammar). Of course, I can check all non-containment references in my EObjects and store the ones first that don't have any cross references, build the project, then save the ones which are only cross referencing the EObjects I just saved and built, and so on. But due to the generic nature of the tool I am building, I must consider the case when there are cyclic dependencies. For that reason, I cannot save and build those files first which do not have cross-references.
Do you have any hint on how to do this? If it can be avoided at all, I'd rather not touch on the scoping since I'm totally fine with the default scoping in the editor.
Basically, what happens in my tool is: I get an XMI file that adheres to an Xtext generated Metamodel, and I want to re-build the DSL code from that XMI file. Which model element ends up in which generated file is determined by the user, I have no control over that. In the end, I need to simultaneously save all of them, that's the goal.
[Updated on: Sun, 06 October 2013 14:45] Report message to a moderator
|
|
| |
Re: Problem using the Xtext serializer [message #1389345 is a reply to message #1388544] |
Fri, 27 June 2014 17:19 |
Alan DW Messages: 119 Registered: March 2012 |
Senior Member |
|
|
Benoit Lelandais wrote on Thu, 26 June 2014 11:45Hi Alan,
I have exactly the same problem and find no solution on forums...
Have you finally find a solution ?
Thanks for your help
Benoît
Hi,
I would love to share my solution with you, but unfortunately there is none - in the end, I've given up on using the Xtext serializer. There are certain problems associated with it:
- The runtime performance of the generic serializer can never be as good as the performance of a serializer that is written especially for a single grammar. This is due to the amount of backtracking on the grammar rule tree that the generic serializer has to perform.
- The generic serializer can't do anything about value conversions associated with your grammar. These converters are essentially one-way converters (from concrete syntax to model only).
- The generic serializer was never intended to be used that way. It's purpose is to allow smaller changes (i.e. element name refactorings) to be applied to the model instead of the raw text. It is not intended to serialize entire models from scratch.
So in the end, what I did is I manually wrote a simple model-to-text generator using Xtend templates. Works like a charm and has none of the problems described above. The drawback is of course that this component is tightly coupled with the grammar. If you change the grammar, you must make the corresponding changes in your serializer as well. Plus, since the serializer works for *one* particular grammar, there is no way to support arbitrary grammars with it.
Maybe in the meantime there is a better solution from the Xtext dev team? I'm not really up-to-date with this anymore I'm afraid...
Kind regards,
Alan
|
|
| |
Goto Forum:
Current Time: Wed Sep 18 22:31:32 GMT 2024
Powered by FUDForum. Page generated in 0.03601 seconds
|