Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Archived » EMF-IncQuery » Performance differences Plugin/Java API(IncQuery Eclipse Plugin is much more slow than Java API)
Performance differences Plugin/Java API [message #1562277] Tue, 13 January 2015 16:04 Go to next message
gwendal daniel is currently offline gwendal danielFriend
Messages: 5
Registered: January 2014
Junior Member
Hi,

I have experienced an important memory consumption / execution time difference between IncQuery Plugin and Java API:

I am testing IncQuery on a model with about 1.5M elements.

First approach, I load the model in eclipse editor (about 1giga memory used), it took 15 seconds.
Then I load it with the load button of IncQuery view, it increases the used memory by 600Mb and took also around 15 seconds to complete. Finally I load the queries (from my eiq file), it also increases the used memory by about 500Mb.

The second approach is to run the queries from java code, so I start with model loading through EMF API, it used around 1giga and took 15 seconds. Then I create the IncQuery engine
IncQueryEngine engine = IncQueryEngine.on(resource);
, it took few ms and there is no memory overhead (at least not visible). Then I perform a query using its generated class
InvisibleMethodsMatcher matcher = InvisibleMethodsMatcher.on(engine);
and get all the matches
Collection<InvisibleMethodsMatch> matches = matcher.getAllMatches();
.
The matches are retrieved in few seconds and there is no memory overhead.

My point is to know what does the graphical editor that take more time than the API? Does the model need to be loaded from scratch (even if it is loaded in eclipse model editor) ?

Here is the query I ran in both cases, it returns about 4000 objects (import is ok, I just can't post a topic with links in it)
package test

import "eclipse.org/MoDisco/Java/0.2.incubation/java"

pattern invisibleMethods(Met) {
	ClassDeclaration.bodyDeclarations(_Cl, Met);
	MethodDeclaration(Met);
	MethodDeclaration.modifier.visibility(Met,VisibilityKind::^private);
} or
{
	ClassDeclaration.bodyDeclarations(_Cl, Met);
	MethodDeclaration(Met);
	MethodDeclaration.modifier.visibility(Met,VisibilityKind::protected);
}


Gwendal

[Updated on: Tue, 13 January 2015 16:04]

Report message to a moderator

Re: Performance differences Plugin/Java API [message #1562309 is a reply to message #1562277] Tue, 13 January 2015 16:29 Go to previous messageGo to next message
Zoltan Ujhelyi is currently offline Zoltan UjhelyiFriend
Messages: 342
Registered: July 2015
Senior Member
Hi Gwendal,

the main difference between your measurements is wildcard mode, a setting that the Query Explorer component uses to index the model more aggressively as it assumes you will add soon a second pattern with something else to index, and tries to avoid re-traversing the source model. On the other hand, the Java API starts with the idea that you only want to index what you require for your patterns, and ignore everything else. For a bit more details, and how to turn it off for the query explorer see https://wiki.eclipse.org/EMFIncQuery/UserDocumentation/QueryDevelopment#Wildcard_mode.

The reduced runtime and memory usage you experienced running from the Java API was caused by this: these patterns are simple, and for about 4000 return object it does not require very large caches, so it can do it more efficiently.

On the other hand, loading your model into Query Explorer should be instantaneous, unless there are already some patterns loaded, as without any patterns there is no model traversal/indexing done. I would like you to load your model, and try to check whether there is anything that got loaded (e.g. from the target platform). The initialization of these patterns required the traversal of the model. Then when you have added your "real" queries, the corresponding Rete networks were also built, further increasing memory requirement.

I hope this helps. If not, feel free to ask for clarification.


Before I finish, I would add on extra thought: in my experience, if a model starts to reach 1M model elements, the queries needs to be developed carefully, as it is easy to provide queries that need a large amount of memory. If possible, in the query explorer try to use smaller models during query development, and turning off wildcard mode might also help a bit.

Cheers,
Zoltán
Re: Performance differences Plugin/Java API [message #1565477 is a reply to message #1562309] Thu, 15 January 2015 09:53 Go to previous messageGo to next message
gwendal daniel is currently offline gwendal danielFriend
Messages: 5
Registered: January 2014
Junior Member
Hi Zoltán, thank you for the clarification.

I did what you suggest and the loading phase is instantaneous and there is no memory overhead if I turn off wildcard mode.

On the other hand, even if I don't have loaded patterns and I want to load the model (with wildcard mode activated) it will take some time and use memory. Maybe I didn't understand what you explained but I think the network is built even if no pattern is registered.

Cheers,

Gwendal
Re: Performance differences Plugin/Java API [message #1565500 is a reply to message #1565477] Thu, 15 January 2015 10:06 Go to previous messageGo to next message
Zoltan Ujhelyi is currently offline Zoltan UjhelyiFriend
Messages: 342
Registered: July 2015
Senior Member
Hi Gwendal,

the memory usage should be close to minimal if there are no patterns registered. Could you have a look at the Query Registry panel (on the left), and look whether there is any pattern available under the 'Plug-in' category. Maybe those patterns are loaded (even if the top-level checkbox is not checked). In my experience (and also by knowing the code), if no pattern is loaded, the loading is instantaneous for every editor type.

If you clearly have no patterns loaded (e.g. you cannot open the tree in the central part of Query Explorer), then there might something in your editor type that I don't know of, and e.g. makes accessing the EMF ResourceSet problematic. In this case, I am interested in reproducing the issue, for that I would like to have a step-by-step description of what to install, and how to load it.

Cheers,
Zoltán
Re: Performance differences Plugin/Java API [message #1565620 is a reply to message #1565500] Thu, 15 January 2015 11:37 Go to previous messageGo to next message
gwendal daniel is currently offline gwendal danielFriend
Messages: 5
Registered: January 2014
Junior Member
Hi Zoltán,

I have checked and there is no pattern registered when I load the model. Everything work as you describe if I turn off wildcard mode. In the other case I have a memory increase of about 600MB when I load the model in the query explorer (and no pattern in the tree).

Here is the steps to reproduce it:


  • Load jdt.xmi in model explorer (in my case MoDisco model viewer)
  • Uncheck "Plug-in" and "Runtime" pattern groups in the registry (even if there is no one under them)
  • Load it with the "load model" button in the Query editor


I have attached the xmi file I use for my test. To be able to open it in the editor you will need to have the MoDisco plugin installed (as the metamodel used is part of it).
You might have to add a dependency to org.eclipse.gmt.modisco.java.

If you need more information feel free to ask me

Cheers,
Gwendal


Re: Performance differences Plugin/Java API [message #1565913 is a reply to message #1565620] Thu, 15 January 2015 15:07 Go to previous messageGo to next message
Tamas Szabo is currently offline Tamas SzaboFriend
Messages: 12
Registered: June 2013
Junior Member
Hi Gwendal,

I have taken a look at your example and found the problem in the IncQuery code.
Even though you do not have loaded patterns we eagerly prepare the engine for pattern matching. I am not quite sure about what should be the general behavior here, because if we do not do it at that time then you will need to wait later at the first time you load your queries.

Cheers,
Tamas

[Updated on: Thu, 15 January 2015 15:07]

Report message to a moderator

Re: Performance differences Plugin/Java API [message #1574923 is a reply to message #1565913] Tue, 20 January 2015 14:20 Go to previous message
gwendal daniel is currently offline gwendal danielFriend
Messages: 5
Registered: January 2014
Junior Member
Hi Tamas,

I understand the engine has to be prepared, and it is quite the same for me to do it during model loading phase or query loading one.
I just pointed it out after Zoltán's answer because I didn't find the behavior he described.

Anyway thank you guys for the clarifications!

Cheers,
Gwendal
Previous Topic:Incquery &amp; Xtext
Next Topic:Newbie questions - please help me understand
Goto Forum:
  


Current Time: Tue Sep 25 09:20:37 GMT 2018

Powered by FUDForum. Page generated in 0.05793 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top