Eclipse Community Forums: TMF (Xtext)

Help

Home

Home » Modeling » TMF (Xtext) » Lean serializer

Show: Today's Messages :: Show Polls :: Message Navigator

Lean serializer [message #1806027]

Tue, 30 April 2019 19:16

Adriano Carvalho

Messages: 54
Registered: November 2018

Member

Hello,

I have a model with around 70000 nodes/attributes. If I update the model using the model editor (generated from the metamodel inferred by Xtext), and then save, it takes around 2 minutes to do so.

As expected the Xtext's generated code is trying to preserve the original format of the file. The actual format, comments, and whitespace are not important for me. The non-hidden tokens, on the other hand, are important since they are supposed to feed an external application.

Seen this, I would like to know how can I configure the serializer to be as lean as possible?

I have read that the problem is related to the SemanticSequencer that is configured/generated. In the past there seems to have been a GenericSemanticSequencer that could solve the issue, but this one cannot be found anymore.

I have also read that this may be caused by ambiguities in the grammar. Is it possible that even with a lean serializer I will still have the same performance issues?

Best regards.

Report message to a moderator

Re: Lean serializer [message #1806077 is a reply to message #1806027]

Thu, 02 May 2019 11:10

Karsten Thoms

Messages: 762
Registered: July 2009
Location: Dortmund, Germany

Senior Member

Quote:

I have also read that this may be caused by ambiguities in the grammar.

I hope you don't have ambiguities, and especially don't have backtracking enabled.

Quote:

Is it possible that even with a lean serializer I will still have the same performance issues?

Possibly yes. You should first use a profiler to see why the process takes 2 mins. This should show which part of the serialization takes so much time. It is quite unusual, even for your node model size.

Report message to a moderator

Re: Lean serializer [message #1806100 is a reply to message #1806077]

Thu, 02 May 2019 15:02

Adriano Carvalho

Messages: 54
Registered: November 2018

Member

At the moment I am unable to do profiling. It seems to only work with Junit. However, I see that most of the time (more than 99% for sure) is spent in the {dslName}SemanticSequencer class and that the BacktrackingSemanticSequencer class is involved somehow.

From this I deduce backtracking is enabled. How can I disabled it? What are the consequences of having it disabled? Will I get the same model back?

Report message to a moderator

Re: Lean serializer [message #1806111 is a reply to message #1806100]

Thu, 02 May 2019 19:03

Karsten Thoms

Messages: 762
Registered: July 2009
Location: Dortmund, Germany

Senior Member

You will have enabled backtracking in the generator workflow for your language (.mwe2 file). Unfortunately you have opened Pandora's box with that. This is usually enabled when there are ambiguities in the grammar and one does not know how to create the language without ambiguities.

I can only recommend to get rid of this. This will be a major refactoring. Best cover your language extensively with parser/linker unit tests (let me guess: there are none or not much). Then recreate your grammar without backtracking feature by feature and cover everything by tests.

Report message to a moderator

Re: Lean serializer [message #1806120 is a reply to message #1806111]

Fri, 03 May 2019 05:21

Christian Dietrich

Messages: 14665
Registered: July 2009

Senior Member

You should see the same problem in a unit test
Thus even if you can provide only in unit tests
(Which i don't understand why)
You should be able to measure actually

Twitter : @chrdietrich
Blog : https://www.dietrich-it.de

Report message to a moderator

Re: Lean serializer [message #1806132 is a reply to message #1806120]

Fri, 03 May 2019 09:38

Adriano Carvalho

Messages: 54
Registered: November 2018

Member

Regarding profiling, I only know how to profile unit tests. It seems that profiling running plugins is not possible. Or is it?

I solved all of the ambiguities reported in {MyDsl}SyntacticSequencer class. However, it is still using backtracking.

Is it possible to retrieve a list of ambiguities currently in the grammar?
Once backtracking is enabled it is not automatically disabled and I have to recreate the project?

Thanks.

Report message to a moderator

Re: Lean serializer [message #1806136 is a reply to message #1806132]

Fri, 03 May 2019 10:44

Karsten Thoms

Messages: 762
Registered: July 2009
Location: Dortmund, Germany

Senior Member

Quote:

Is it possible to retrieve a list of ambiguities currently in the grammar?
Once backtracking is enabled it is not automatically disabled and I have to recreate the project?

Disable backtracking and you'll get the ambiguities when regenerating the project. It is not necessary to recreate the project, just disable it in the .mwe2 file. You may work better when creating the grammar one feature by the other to see when you introduce an ambiguity, and then resolve that.

This article might be helpful:
https://blogs.itemis.com/en/debugging-xtext-grammars-what-to-do-when-your-language-is-ambiguous

Report message to a moderator

Re: Lean serializer [message #1806137 is a reply to message #1806132]

Fri, 03 May 2019 10:48

Christian Dietrich

Messages: 14665
Registered: July 2009

Senior Member

BacktrackingSemanticSequencer is the default fallback

public void configureGenericSemanticSequencer(com.google.inject.Binder binder) {
binder.bind(ISemanticSequencer.class).annotatedWith(GenericSequencer.class).to(BacktrackingSemanticSequencer.class);
}
i have doubts that is the problem.

as i said:
create a unit test that reproduces the problem and profile there.
btw what tool do you use to profile

Twitter : @chrdietrich
Blog : https://www.dietrich-it.de

Report message to a moderator

Re: Lean serializer [message #1806140 is a reply to message #1806137]

Fri, 03 May 2019 11:48

Adriano Carvalho

Messages: 54
Registered: November 2018

Member

I've found a solution to my problem: when changing a model with the EMF model editor have the Xtext editor closed.

You are correct. Based on the very same page you've mentioned I used ANTLRWorks 1.5.2 to find and solve all ambiguities and the problem still persisted. Then, using the debugger I realized that the serializer was actually running under 10 seconds. The long running time is caused by what happens after the serializer is run. But this only happens if the Xtext-based editor is also opened.

From my little knowledge of the Eclipse platform I deduce the root cause is in the reconciler. Is this a known problem or is it just me?

Having the Xtext-based editor closed works for me but if you think this is an issue I can provide more information. Just let me know what we need to know.

Thank you very much.

[Updated on: Fri, 03 May 2019 11:51]

Report message to a moderator

Re: Lean serializer [message #1806141 is a reply to message #1806140]

Fri, 03 May 2019 12:06

Christian Dietrich

Messages: 14665
Registered: July 2009

Senior Member

this might be caused by Dirty Editor State and bad Performance in Scoping.
you might want to have a custom CrossReferenceSerializer that does not make use of scope provider too
and the "run after" should be profiled too.
and the reconciliation problems
are basically there in all tool. even java editors with many 10000s of lines.
i dont know what specific case causes this in your case.
it should happen on manual edits too.

Twitter : @chrdietrich
Blog : https://www.dietrich-it.de

Report message to a moderator

Re: Lean serializer [message #1806269 is a reply to message #1806141]

Tue, 07 May 2019 09:06

Adriano Carvalho

Messages: 54
Registered: November 2018

Member

FYI the reverse -- that is, if I have both editors* open and I update the model/text using the Xtext-based editor (instead of the EMF-based editor) -- the performance is good. What could be the reason?

*The Xtext-based editor and the EMF-based editor.

Report message to a moderator

Re: Lean serializer [message #1806284 is a reply to message #1806269]

Tue, 07 May 2019 11:37

Ed Willink

Messages: 7655
Registered: July 2009

Senior Member

Hi

The 'old' way that involved multiple EMF-based editors worked because all the editors shared the same EditingDomain and so the same AdapterFactoryEditingDomainResourceSet.

Xtext makes the situation much more interesting by supporting distinct textual and model representations. Xtext has its distinct XtextResourceSet which I suspect is independent of any AdapterFactoryEditingDomainResourceSet;. Consequently there is a co-ordination issue.

EMF improved, perhaps five years ago, perhaps at the same time as live validation, to be much better at responding to file system changes. You will now find that an EMF-based editor tends to refresh, or at least give a pop-up asking for permission to refresh, when the file system copy changes.

I think that I have sometimes seen an Xtext editor do this too. But I'm not sure because my own Xtext-based editors for OCL and QVTd exhibit very poor update characteristics for file system changes; occasionally it works, often it produces prolific red squiggles. But I know that the problem is a lack of accurate incremental update of the internal model from the external representation. Workaround; restart the editor. I am working on a QVTr-based solution with auto-generated incremental update since I know that I cannot code it manually. Xtext generally used to create new rather than updated models since the developers recognize thatsynchonization via an incremental update is too hard. Therefore you have independent models in memory for the Xtext-based and EMF-based editors. In it possible that work usingEMF-IncQuery has improved this.

I suspect that you are suffering from inadequate synchronization with your/the standard synchronization code.

Regards

Ed Willink

Report message to a moderator