Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » VIATRA » transforming 10 million elements?
transforming 10 million elements? [message #1262210] Mon, 03 March 2014 11:26 Go to next message
Neil Mackenzie is currently offline Neil MackenzieFriend
Messages: 131
Registered: December 2013
Senior Member
Is there a limit on the size of model that can be dealt with and transformd in Viatra2.

I am considereing an applcation that could require the transfromation of millions of elements in the model instances (the metamodels themselves would only have perhaps 50 elements). Transformation doesn't need to be super quick in this case.

If there is a limit, what is its nature? Is it because elements are stored in memory? Garbage collection problems? Scalability of algorithms? If there are limits due to memory problems could this be solved with technology like bigmemory from terracotta?

Thanks,
Neil
Re: transforming 10 million elements? [message #1262247 is a reply to message #1262210] Mon, 03 March 2014 12:20 Go to previous messageGo to next message
Zoltan Ujhelyi is currently offline Zoltan UjhelyiFriend
Messages: 392
Registered: July 2015
Senior Member
Hi,

in short, there is no theoretical limit, however, all models are stored in-memory, and VIATRA2 stores its models in a not-too-effective internal structure. In general, if your model size reaches 1 million elements, you can expect different kind of slowdowns (e.g. heap is out, garbage collection is slow, etc.).

The algorithms themselves should be scalable, however, the internal model structures can be problematic.

On the other hand, currently we are focusing our efforts on EMF-based queries (and somewhat on transformations). Using these techniques we have managed to efficiently query models of a few million elements (but needed careful query design).

Based on these results we are outlining an experimental EMF transformation API (http://wiki.eclipse.org/VIATRA2/EMF/Transformation_API). However, this API was not yet tested on models of this size, so we don't know how well it would execute.

Cheers,
Zoltán
Re: transforming 10 million elements? [message #1262425 is a reply to message #1262210] Mon, 03 March 2014 16:20 Go to previous messageGo to next message
Istvan Rath is currently offline Istvan RathFriend
Messages: 59
Registered: July 2009
Member
Hi Neil,

as long as you have enough RAM, you should be fine. 10 million elements would require 8GB-ish heaps.

http://incquery.net/performance has some useful tips on getting the best performance out of VIATRA2 and EMF-IncQuery. Also, if you run into any troubles, we are glad to help here in the forum.

cheers
Istvan
Re: transforming 10 million elements? [message #1264795 is a reply to message #1262210] Wed, 05 March 2014 14:08 Go to previous messageGo to next message
Neil Mackenzie is currently offline Neil MackenzieFriend
Messages: 131
Registered: December 2013
Senior Member
thanks, I will take a look.
I am primarily interested in live transformations.

If I understand correctly there is viatra/emf which is more experimental and is part of viatra2 (which can deal with trnsformations), and uses the EVM. and EVM comes from the EMFInquery project is that correct?

I cannot install viatra2 btw, I have tried a number of things. when I do install software from elclipse uing the eclipse link http://download.eclipse.org/modeling/gmt/viatra2/updates/release/ I am told that the site cant be found. I am happy to compile from source also, in fact I would like to do that if it gives me more up to date access to changes.

Regards,
Neil

[Updated on: Wed, 05 March 2014 14:08]

Report message to a moderator

Re: transforming 10 million elements? [message #1264812 is a reply to message #1264795] Wed, 05 March 2014 14:24 Go to previous messageGo to next message
Zoltan Ujhelyi is currently offline Zoltan UjhelyiFriend
Messages: 392
Registered: July 2015
Senior Member
Hi,

sorry for the incorrect links, they were not updated after the project was moved out from the GMT project. The correct URL would be http://download.eclipse.org/viatra2/updates/release - I have also updated the project homepage to incorporate this.

About the project structure:
* The VIATRA2 update site you tried to access contains the older, VPM-based transformation environment. Most documentation available on VIATRA2 corresponds to this version, and focuses on transformations, featuring a custom transformation language.
* The VIATRA2/EMF version is indeed experimental, and thus is not available from this update site, and is also stored in a different repository. Basically, it only reuses some concepts from the old language and the original developers, but indeed relies on EVM from the EMF-IncQuery project. Both the repository URL and the update site url is available on its defining wiki page http://wiki.eclipse.org/VIATRA2/EMF/Transformation_API

I hope I managed to clear it a bit up; if not, feel free to ask for further specification.

Cheers,
Zoltán
Re: transforming 10 million elements? [message #1264842 is a reply to message #1264812] Wed, 05 March 2014 14:50 Go to previous messageGo to next message
Neil Mackenzie is currently offline Neil MackenzieFriend
Messages: 131
Registered: December 2013
Senior Member
Thanks,
I will try and get the live transformations that I am interested in working with VIATRA2.
I will check out the VATRA/EMF also as I may well want to use this in future.
Can you tell me for each of these what is the latest version of eclipse that I can use?
I currently work with Kepler modeling edition.
Regards,
Neil
Re: transforming 10 million elements? [message #1264869 is a reply to message #1264842] Wed, 05 March 2014 15:11 Go to previous message
Zoltan Ujhelyi is currently offline Zoltan UjhelyiFriend
Messages: 392
Registered: July 2015
Senior Member
The older VIATRA2 should work on basically any version of Eclipse; VIATRA2/EMF relies on EMF-IncQuery that needs Xtext 2.4 installed (but that is also a minimum version Smile ).

So, your Kepler version should work fine.

Cheers,
Zoltán
Previous Topic:basic question on transformation of UML to colored petri net
Next Topic:Type Casting
Goto Forum:
  


Current Time: Sat Dec 14 05:27:19 GMT 2024

Powered by FUDForum. Page generated in 0.03861 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top