Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » TMF (Xtext) » Serialisation time grows exponentially when adding new elements to model instance
Serialisation time grows exponentially when adding new elements to model instance [message #1712047] Wed, 21 October 2015 05:45 Go to next message
Alex Mil is currently offline Alex MilFriend
Messages: 5
Registered: October 2015
Junior Member
Hi all,

we've been experiencing a problem for a while now with our xtext model that we've been able to narrow down a tiny bit further.

Our model instances tend to get quite large so we always thought it was the size of the model that was causing the long time to save (serialise) it.

However, we've reproduced this with the following steps in unit tests:

Unit test 1



  1. Load large model instance from file
  2. Save large model back to file


This saving takes ~8 seconds

Now we do something slightly different:

Unit test 2



  1. Load large model instance from file
  2. Add an arbitrary number of objects to the model in memory, say 100. These objects have cross references to other elements
  3. Save the large model back to file


This will now take between 40 and 50 seconds!

If I take the saved "enlarged" model from unit test 2 and repeat the same steps as in unit test 1 (simply load and then save again, no model changes in memory), the saving goes back to 8-9 seconds!!

Something happens when we "add" elements to an existing model and we are not sure exactly what and why this happens.

Any recommendations?

p.s We are using the latest serialiser which apparently has improvements for large models and even though we saw some improvement, it becomes negligible once we change the model at runtime and then save.


Thanks!
Re: Serialisation time grows exponentially when adding new elements to model instance [message #1712120 is a reply to message #1712047] Wed, 21 October 2015 11:54 Go to previous messageGo to next message
Sebastian Zarnekow is currently offline Sebastian ZarnekowFriend
Messages: 3118
Registered: July 2009
Senior Member
Hi Alex,

I'd recommend profiling the serializer for your language and tweaking it
where it's slow. Please DM me if you need professional help with that.

Best,
Sebastian
--
Looking for professional support for Xtext, Xtend or Eclipse Modeling?
Find help at http://xtext.itemis.com or xtext(@)itemis.com
Blog: zarnekow.blogspot.com
Twitter: @szarnekow
Google+: https://www.google.com/+SebastianZarnekow
Re: Serialisation time grows exponentially when adding new elements to model instance [message #1712637 is a reply to message #1712120] Tue, 27 October 2015 05:27 Go to previous messageGo to next message
Alex Mil is currently offline Alex MilFriend
Messages: 5
Registered: October 2015
Junior Member
Hi,

thanks for your reply, much appreciated. We've worked on reducing the size of our model in an effort to reduce the potential problem space while at the same time reproducing the issue.

Our test projects are available at https://github.com/johnf-repo/xtext


There are two projects using the same model definition:
org.johnf.mydsl1 - this project has no user defined scoping.
org.johnf.mydsl2 - this project has user defined scoping

The model is reduced to having an "entity" with fields that have a type.



Model:
	'workspace' name=ID ';'
	('description' description=STRING ';')?
	(	
			dataTypes+=(Entity)
	)*
	(systems+=System)*
	; 
		
		
System:
	'system' name=ID '{'
		('description' description=STRING ';')?
		(imports+=DefinitionImport)*
		(	
			dataTypes+=(Entity)
		)*
	'}'
	;
		
DefinitionImport:
	'import' definition=[Definition|FQN] ';'
	;
		
Definition:
	DataType;
	
	
Partition:
	Model | System
	;
		
CrossReferenceTarget:
		Definition
	|   Field
	
	;					
terminal DATE returns ecore::EDate: '0'..'9''0'..'9''0'..'9''0'..'9' '-' '0'..'9''0'..'9' '-' '0'..'9''0'..'9';
terminal INT returns ecore::EInt: ('-')? ('0'..'9')+;
terminal DECIMAL returns ecore::EBigDecimal: 
		'-'? ('0'..'9')+ '.' ('0'..'9')+;
      
FQN: // fully qualified name
	ID ('.' ID)*
	;

//
// Data Types
//
DataType:
	BuiltInType | Entity
	;

Entity :
	'entity' name=ID '{'
		('description' description=STRING ';')?
		(fields+=Field)* 
	'}'
	;

	
// Only instantiated "behind the scenes", i.e. no keyword, not called as a rule
BuiltInType:
	name=ID description=STRING
	;
		
Field : 
	name=ID ':' type=[DataType|FQN]
	(	('description' description=STRING ';')?
		('[' lower=INT ',' (upperUnlimited?='*' | upper=INT) ']' ';')?
		// default values: lower=1 and upper=1 set via PostProcessor
	|	';'
	)
	;



The reason for the two models is that the first one shows that there seems to be an issue even without our scoping implementation. In the first model there doesn't seem to be anything custom that could cause this which leads us to believe that it might be an Xtext issue.
The second model shows that using our scoping makes the problem even worse.


Here is a summary of our tests results, these tests are also available in the above projects in github.

org.johnf.mydsl1 - Xtext project without user defined scope


  • Loading an empty model, when adding 4000 entities with no fields, the serialization Took 488ms
  • Loading an empty model, when adding 1000 entities with 3 fields each, the serialization Took 5,551ms
  • Loading a model with 1000 entities with 3 fields each, and without making any modifications, the serialization Took 1,724ms


org.johnf.mydsl2 - Xtext project with user defined scope on field_type

  • Loading an empty model, when adding 4000 entities with no fields, the serialization Took 496ms
  • Loading an empty model, when adding 1000 entities with 3 fields each, the serialization Took 11,569ms
  • Loading a model with 1000 entities with 3 fields each, and without making any modifications, the serialization Took 2,301ms


As you can see, these results show that modifying the model in memory by adding elements (in this case entities) that have cross references, significantly increases the saving time.

Has anybody had experience with this or can help out?

We are at a bit of a loss because this performance issue is becoming a bit of a deal breaker for the use of xtext even though from a development perspective we would love to keep it.

Thanks in advance,

Alex

[Updated on: Tue, 27 October 2015 05:29]

Report message to a moderator

Re: Serialisation time grows exponentially when adding new elements to model instance [message #1713698 is a reply to message #1712637] Fri, 06 November 2015 06:07 Go to previous messageGo to next message
Alex Mil is currently offline Alex MilFriend
Messages: 5
Registered: October 2015
Junior Member
Hi Sebastian,

I've tried PM-ing you but the feature is not enabled for me here. I've asked for it to be enabled, but in the meantime if you have another contact method please let me know.
Re: Serialisation time grows exponentially when adding new elements to model instance [message #1713773 is a reply to message #1712637] Fri, 06 November 2015 15:23 Go to previous message
Sebastian Zarnekow is currently offline Sebastian ZarnekowFriend
Messages: 3118
Registered: July 2009
Senior Member
Hi Alex,

sorry, I didn't follow-up so far. The issue slipped through during the
preparations for EclipseCon and for the 2.9 release.
Please drop me a note at Sebastian.Zarnekow(a_t)itemis.de and I'll try
to have a look or forward your request.

Best,
Sebastian
--
Looking for professional support for Xtext, Xtend or Eclipse Modeling?
Find help at http://xtext.itemis.com or xtext(@)itemis.com
Blog: zarnekow.blogspot.com
Twitter: @szarnekow
Google+: https://www.google.com/+SebastianZarnekow
Previous Topic:Bounds mismatch error using generic extension
Next Topic:True standalone maven/gradle capability in 2.9?
Goto Forum:
  


Current Time: Fri Apr 26 06:33:04 GMT 2024

Powered by FUDForum. Page generated in 0.02910 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top