| transforming 10 million elements? [message #1262210] |
Mon, 03 March 2014 06:26  |
Eclipse User |
|
|
|
Is there a limit on the size of model that can be dealt with and transformd in Viatra2.
I am considereing an applcation that could require the transfromation of millions of elements in the model instances (the metamodels themselves would only have perhaps 50 elements). Transformation doesn't need to be super quick in this case.
If there is a limit, what is its nature? Is it because elements are stored in memory? Garbage collection problems? Scalability of algorithms? If there are limits due to memory problems could this be solved with technology like bigmemory from terracotta?
Thanks,
Neil
|
|
|
| Re: transforming 10 million elements? [message #1262247 is a reply to message #1262210] |
Mon, 03 March 2014 07:20  |
| Eclipse User |
|
|
|
Hi,
in short, there is no theoretical limit, however, all models are stored in-memory, and VIATRA2 stores its models in a not-too-effective internal structure. In general, if your model size reaches 1 million elements, you can expect different kind of slowdowns (e.g. heap is out, garbage collection is slow, etc.).
The algorithms themselves should be scalable, however, the internal model structures can be problematic.
On the other hand, currently we are focusing our efforts on EMF-based queries (and somewhat on transformations). Using these techniques we have managed to efficiently query models of a few million elements (but needed careful query design).
Based on these results we are outlining an experimental EMF transformation API (http://wiki.eclipse.org/VIATRA2/EMF/Transformation_API). However, this API was not yet tested on models of this size, so we don't know how well it would execute.
Cheers,
Zoltán
|
|
|
|
|
| Re: transforming 10 million elements? [message #1264812 is a reply to message #1264795] |
Wed, 05 March 2014 09:24 |
| Eclipse User |
|
|
|
Hi,
sorry for the incorrect links, they were not updated after the project was moved out from the GMT project. The correct URL would be http://download.eclipse.org/viatra2/updates/release - I have also updated the project homepage to incorporate this.
About the project structure:
* The VIATRA2 update site you tried to access contains the older, VPM-based transformation environment. Most documentation available on VIATRA2 corresponds to this version, and focuses on transformations, featuring a custom transformation language.
* The VIATRA2/EMF version is indeed experimental, and thus is not available from this update site, and is also stored in a different repository. Basically, it only reuses some concepts from the old language and the original developers, but indeed relies on EVM from the EMF-IncQuery project. Both the repository URL and the update site url is available on its defining wiki page http://wiki.eclipse.org/VIATRA2/EMF/Transformation_API
I hope I managed to clear it a bit up; if not, feel free to ask for further specification.
Cheers,
Zoltán
|
|
|
|
|
Powered by
FUDForum. Page generated in 0.07094 seconds
Home
).