Eclipse Community Forums: TMF (Xtext) » Debugging non-deterministic serialization failure

Home » Modeling » TMF (Xtext) » Debugging non-deterministic serialization failure

Debugging non-deterministic serialization failure [message #1746530]

Mon, 31 October 2016 14:06

Eclipse User

Using Xtext 2.8.4 I have a test case for a quick fix which non-deterministically alternates between passing and "Could not serialize EObject via backtracking."

Does anyone have some hints on how to debug this? What could possibly cause such non-determinism?

thanks,
Stephan

Re: Debugging non-deterministic serialization failure [message #1746531 is a reply to message #1746530]

Mon, 31 October 2016 14:09

Eclipse User

did you have a look what contrained makes this fail? (SequencerDiagnosticProvider)
what is the cause there?

Re: Debugging non-deterministic serialization failure [message #1746532 is a reply to message #1746530]

Mon, 31 October 2016 14:24

Eclipse User

Further observation: during one IDE session the quick fix either consistently works or consistently fails. Apparently s.t. goes havoc during initialization of my plugin? In both cases the error log is free of relevant entries, though.

Re: Debugging non-deterministic serialization failure [message #1746533 is a reply to message #1746532]

Mon, 31 October 2016 14:28

Eclipse User

looks very strange. thus it would be interesting which contraint is affected. do you have grammar inhertiance etc involved?

Re: Debugging non-deterministic serialization failure [message #1746534 is a reply to message #1746533]

Mon, 31 October 2016 14:38

Eclipse User

No grammar inheritance, only cross references to elements from other DSLs.

Here are the details of the exception, not sure if it helps any without the (somewhat complicated) grammar:

Caused by: java.lang.RuntimeException: Could not serialize EObject via backtracking.
Constraint: null ((annotations+=DslAnnotation* incomplete?='incomplete'? | incomplete?='incomplete'? usingDeprecatedAtSyntax?='@') name=ID prototype=[InstanceDefinition|FQN]? entity=[Entity|FQN]? attributeDefinitions+=AttributeDefinition* | (annotations+=DslAnnotation* inherit='inherited' incomplete?='incomplete'? prototype=[InstanceDefinition|ID] | inherit='specialized' prototype=[InstanceDefinition|ID] (attributeDefinitions+=AttributeDefinitionAbbrev | attributeDefinitions+=AttributeDefinitionPlain)) | annotations+=DslAnnotation* inherit='specialized' incomplete?='incomplete'? prototype=[InstanceDefinition|ID] entity=[Entity|FQN]? attributeDefinitions+=AttributeDefinition*) null
Values: inherit(1), prototype(1), incomplete(0-1), resolving(0-1), usingDeprecatedAtSyntax(0-1)
Semantic Object: DomainInstanceModel.bundleDefinitionList[1]->BundleDefinition'b1'.instanceList[0]->InstanceDefinition
Context: SubInstanceDefinitionInBundle

Does this provide any clue where to continue searching?
Stephan

Re: Debugging non-deterministic serialization failure [message #1746535 is a reply to message #1746534]

Mon, 31 October 2016 14:41

Eclipse User

where is the grammar for the "resolving" thing? which attributes/references are actually set in the object you try to serialize?

[Updated on: Mon, 31 October 2016 14:46] by Moderator

Re: Debugging non-deterministic serialization failure [message #1746537 is a reply to message #1746535]

Mon, 31 October 2016 15:11

Eclipse User

"resolving" is marked as transient in EMF, so this shouldn't be looked at at all.

"inherit" is set to the string "inherited" (thus uniquely identifying the alternative)

"prototype" is set as a valid cross reference (from another resource).

Everything else is unset.

Re: Debugging non-deterministic serialization failure [message #1746538 is a reply to message #1746537]

Mon, 31 October 2016 15:17

Eclipse User

Can you Double Check the transient value Service picks up the Emf Value correct

Re: Debugging non-deterministic serialization failure [message #1746539 is a reply to message #1746538]

Mon, 31 October 2016 15:39

Eclipse User

the other point would be the cross ref serializer,
does the name of the prototype be serializable as iD?

unfortunately the error message is not that good in the alternatives case if not.

Re: Debugging non-deterministic serialization failure [message #1746540 is a reply to message #1746538]

Mon, 31 October 2016 15:40

Eclipse User

We actually have our own transient value service (motivated by a tweak concerning another attribute).
That service implements both org.eclipse.xtext.serializer.sequencer.ITransientValueService and org.eclipse.xtext.parsetree.reconstr.ITransientValueService.
Via the former API we answer (as per the super implementation) ValueTransient.PREFERABLY (because the attribute is not of type String and thus answers true from defaultValueIsSerializeable()).
Via the latter API this is reported as 'true' (is transient).

I experimentally added explicit treatment to let this guy consistently answer ValueTransient.YES, but test keeps failing sometimes.

At least that experiment makes "resolving" disappear from the error message.

Re: Debugging non-deterministic serialization failure [message #1746541 is a reply to message #1746540]

Mon, 31 October 2016 15:46

Eclipse User

ok then the second thing is about the scoping for prototype.
your naming sheme seems to differ between branches

Re: Debugging non-deterministic serialization failure [message #1746542 is a reply to message #1746540]

Mon, 31 October 2016 15:50

Eclipse User

We should be looking at this alternative:

annotations+=DslAnnotation* inherit='inherited' incomplete?='incomplete'? prototype=[InstanceDefinition|ID]

The target element for the "prototype" reference has a normal "name" attribute and simple name resolution is granted by the scope provider.

That scope provider is a tad complex, in this particular context it has to travel a few references within the model (same resource).

Re: Debugging non-deterministic serialization failure [message #1746543 is a reply to message #1746542]

Mon, 31 October 2016 15:52

Eclipse User

id debug org.eclipse.xtext.serializer.tokens.CrossReferenceSerializer.getCrossReferenceNameFromScope(EObject, CrossReference, EObject, IScope, Acceptor) and look for valueConverter exceptions

Re: Debugging non-deterministic serialization failure [message #1746544 is a reply to message #1746543]

Mon, 31 October 2016 15:54

Eclipse User

p.s.

for testing purposes i used this grammar


Root:
	prototype=Prototype
	model=Model
;

Prototype:
	name=FQN
;
Model:
"a" 

(
	(
		annotations+=DslAnnotation* incomplete?='incomplete'? 
		| incomplete?='incomplete'? usingDeprecatedAtSyntax?='@'
	) 
	name=ID 
	prototype=[Prototype|FQN]?
	entity=FQN? 
	attributeDefinitions+=AttributeDefinition* 

	| 

	(
		annotations+=DslAnnotation* 
		inherit='inherited' 
		incomplete?='incomplete'? 
		prototype=[Prototype|ID]

		| 

		inherit='specialized' 
		prototype=[Prototype|ID]
		(
		attributeDefinitions+=AttributeDefinitionAbbrev 
		| attributeDefinitions+=AttributeDefinitionPlain
		)
	) 

	| 

	annotations+=DslAnnotation* 
	inherit='specialized' 
	incomplete?='incomplete'? 
	prototype=[Prototype|ID]
	entity=FQN? "#"
	attributeDefinitions+=AttributeDefinition*
) 


"b";

DslAnnotation:
	"@" name=ID "("")"
;

FQN: ID ("."ID)*;

AttributeDefinition:
	"attr" name=ID
;

AttributeDefinitionAbbrev:
	"a" name=ID
;

AttributeDefinitionPlain:
	"ax" name=ID
;

and test


class Main {
	
	def static void main(String[] args) {
		val Injector i = new MyDslStandaloneSetup().createInjectorAndDoEMFRegistration
		val ISerializer s = i.getInstance(ISerializer)
		val p = MyDslFactory.eINSTANCE.createPrototype => [
			name = "a.b.c"
		]
		val m = MyDslFactory.eINSTANCE.createRoot => [
				prototype = p
			model = MyDslFactory.eINSTANCE.createModel => [
				inherit = "inherited"
				prototype = p
				incomplete = false
			]
			
			]
		val rs = i.getInstance(ResourceSet)
		val r = rs.createResource(URI.createURI("dummy.mydsl1"))
		r.contents += m
		println(s.serialize(
			m
		))
	}
	
	
}

Re: Debugging non-deterministic serialization failure [message #1746545 is a reply to message #1746544]

Mon, 31 October 2016 16:07

Eclipse User

No ValueConverterException from CrossReferenceSerializer
(only a bunch from AssignementQuantityAllocator.allowTransient(..) -> .. IDValueConverter.assertValidValue(), but these seem to be part of normal control flow)

Re: Debugging non-deterministic serialization failure [message #1746546 is a reply to message #1746545]

Mon, 31 October 2016 16:10

Eclipse User

hmmmmm.

can you, for test, eliminate the other branches.
this would give you a better error message

Re: Debugging non-deterministic serialization failure [message #1746547 is a reply to message #1746546]

Mon, 31 October 2016 16:16

Eclipse User

> can you, for test, eliminate the other branches.

will do.

Meanwhile I put a print point into the final loop in BacktrackingSemanticSequencer.createSequence(EObject, EObject) to see all 'ti'. In the bad case the output is *much* shorter, much fewer elements in the 'trace'...

Re: Debugging non-deterministic serialization failure [message #1746548 is a reply to message #1746547]

Mon, 31 October 2016 16:23

Eclipse User

removed 3 out of 5 alternatives and couldn't reproduce the serialization bug in 10 test runs.

Re: Debugging non-deterministic serialization failure [message #1746549 is a reply to message #1746548]

Mon, 31 October 2016 16:34

Eclipse User

hmmmm i am kind a out of ideas. the question is: why is the "right path" considered bad.
are you sure there are no relevant errors added to org.eclipse.xtext.serializer.sequencer.BacktrackingSemanticSequencer.SerializableObject.isValueValid(ISemState, int, Object)

Re: Debugging non-deterministic serialization failure [message #1746550 is a reply to message #1746549]

Mon, 31 October 2016 16:38

Eclipse User

I couldn't ever reproduce in any combination of alternatives other than the original full set of 5. Each seems to be necessary to trigger the bug.

Re: Debugging non-deterministic serialization failure [message #1746551 is a reply to message #1746550]

Mon, 31 October 2016 16:41

Eclipse User

btw how do you solve the ambiguity in the first one inside your grammar? is there keywords to distinguish prototype and enitity

Re: Debugging non-deterministic serialization failure [message #1746552 is a reply to message #1746550]

Mon, 31 October 2016 16:45

Eclipse User

Bingo: in the bad case there *is* a 'false' inside isValueValid().
It's the element *before* the one created by the prefix.
state is "inherit='specialized'" (which you can see in one of the other alternatives)

still have to figure out what the values mean hear ...

Re: Debugging non-deterministic serialization failure [message #1746553 is a reply to message #1746552]

Mon, 31 October 2016 16:58

Eclipse User

This looks suspecious (in AssignmentFinder):

	protected Iterable<AbstractElement> findValidValueAssignments(EObject semanticObj,
			Iterable<AbstractElement> assignedElements, Object value) {
		// keywords have precedence over everything else
		for (AbstractElement ass : assignedElements)
			if (ass instanceof Keyword && keywordSerializer.isValid(semanticObj, (Keyword) ass, value, null))
				return Collections.singletonList(ass);

		// now check the remaining assignments
...

In assignedElements there are two keywords with the same value, but one triggers invalid value above. It seems, the loop randomly picks one from the two possible (and necessary) answers.

Re: Debugging non-deterministic serialization failure [message #1746554 is a reply to message #1746553]

Mon, 31 October 2016 17:04

Eclipse User

What are the two keywords, compared with the constants in mydslgrammaraccess.
And where does it take the keywords from?

Re: Debugging non-deterministic serialization failure [message #1746555 is a reply to message #1746553]

Mon, 31 October 2016 17:07

Eclipse User

And here's the fix:

	protected Iterable<AbstractElement> findValidValueAssignments(EObject semanticObj,
			Iterable<AbstractElement> assignedElements, Object value) {
		List<AbstractElement> result = Lists.newArrayList();
		for (AbstractElement ass : assignedElements)
			if (ass instanceof Keyword && keywordSerializer.isValid(semanticObj, (Keyword) ass, value, null))
				result.add(ass);
		// keywords have precedence over everything else
		if (!result.isEmpty())
			return result;

		// now check the remaining assignments
...

Thanks Christian, for asking the right questions to guide me to the culprit!
Stephan

Re: Debugging non-deterministic serialization failure [message #1746557 is a reply to message #1746555]

Mon, 31 October 2016 17:11

Eclipse User

youre welcome. maybe you can backport the current 2.10 code

protected Set<AbstractElement> findValidValueAssignments(EObject semanticObj,
			Iterable<AbstractElement> assignedElements, Object value) {
		Set<AbstractElement> result = Sets.newLinkedHashSet();
		for (AbstractElement ass : assignedElements) {
			if (ass instanceof Keyword && keywordSerializer.isValid(semanticObj, (Keyword) ass, value, null))
				result.add(ass);
			else if (ass instanceof RuleCall) {
				RuleCall rc = (RuleCall) ass;
				if (rc.getRule() instanceof EnumRule) {
					if (enumLiteralSerializer.isValid(semanticObj, rc, value, null))
						result.add(ass);
				} else if (valueSerializer.isValid(semanticObj, rc, value, null))
					result.add(ass);
			}
		}
		return result;
	}

Re: Debugging non-deterministic serialization failure [message #1746558 is a reply to message #1746555]

Mon, 31 October 2016 17:18

Eclipse User

Quote:

What are the two keywords, compared with the constants in mydslgrammaraccess.
And where does it take the keywords from?

Two of them are the same value, but they can be assigned via different rules which makes two occurrences of

  inherit='specialized'

produce two distinct "Keyword"s (you actually have these two occurrences in you're extracted grammar).

good night! Smile

Stephan

Re: Debugging non-deterministic serialization failure [message #1746650 is a reply to message #1746557]

Wed, 02 November 2016 09:39

Eclipse User

Christian Dietrich wrote on Mon, 31 October 2016 22:11

maybe you can backport the current 2.10 code ...

Thanks, that version works as well.

I was briefly worried if more locations in Xtext might work under a wrong assumption: apparently the same keyword can be represented by different Keyword instances, but the old code in findValidValueAssignments() assumed finding one Keyword per value is sufficient.

Anyway, one more indication that migrating to a newer version would have helped / will help.

Stephan

Previous Topic:	XText Get original String from AST
Next Topic:	Syntax coloring / highlighting xtext 2.9

Goto Forum:

-=] Back to Top [=-

Current Time: Mon May 12 10:54:55 EDT 2025

.:: Contact :: Home ::.

Breadcrumbs

Sign up to our Newsletter