Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Archived » EMF-IncQuery » How to prevent traversal of objects (part 2)
How to prevent traversal of objects (part 2) [message #1698526] Tue, 16 June 2015 09:12 Go to next message
Per Sterner is currently offline Per SternerFriend
Messages: 91
Registered: October 2011
Member
We have a lot objects which are contained in one resource, but there are some lists in objects which could be ignored for the querying/indexing.

The startup for "instance.prepare(engine);" takes 18 seconds. If I delete the not needed items from the database (>50000 elements), the startup takes 3.2 seconds. We are using CDO.

The original post is here: https://www.eclipse.org/forums/index.php/t/643923/

1. In the documentation https://wiki.eclipse.org/EMFIncQuery/UserDocumentation/API/BaseIndexer#NavigationHelper the NavigationHelper seems to be it? In the NavigationHelper-Class there are functions defined like "registerEClasses" or "registerEStructuralFeatures".

the init-ode "looks like this":
EMFScope scope = new EMFScope(resource);
engine = IncQueryEngine.on(scope);
final Queries instance = Queries.instance();
instance.prepare(engine);


2. For not traversing a specific EReference, how should/can I configure the IncQueryEngine?
Re: How to prevent traversal of objects (part 2) [message #1698547 is a reply to message #1698526] Tue, 16 June 2015 11:05 Go to previous messageGo to next message
Zoltan Ujhelyi is currently offline Zoltan UjhelyiFriend
Messages: 392
Registered: July 2015
Senior Member
Hi,

in general, the Base Indexer (that is indeed implemented with the NavigationHelper class) traverses the entire model, and indexes everything selected with the register* methods. The Queries.instance.prepare(engine) call evaluates all pattern definitions from the selected files, and calls all register* methods that the Rete engine needs for operation (basically, all/most EClass, EDataType and EReference elements mentioned in the description - the truth is a bit more complex, but among these lines), and traverses the model once for all queries.

There are two tasks that might take up the 18 seconds you have measured: (1) the traversal of the model itself (basically, we are following the containment hierarchy to visit every model element), and the (2) index/Rete network building. The latter case usually results in large memory consumption of the pattern matcher as well.

If you want to limit the traversal of the model you can add filters to your engine using a BaseIndexOptions instance: you can filter out resources or model elements as well. However, as the traversal follows the containment hierarchy, filtering out a single model element filters out all elements it has contained. For this reason, you could do something along the following:

BaseIndexOptions options = new BaseIndexOptions().withObjectFilterConfiguration(new IBaseIndexObjectFilter() { «add filter code here»});
EMFScope scope = new EMFScope(«resourceset», options);
IncQueryEngine engine = IncQueryEngine.on(scope);


If not the traversal, but the indexing/Rete building is slow, then you have to try to find out which pattern causes the issue, and refactor/remove the problematic parts. The QueryPerformanceTest abstract class helps to find out whether there is some pattern by measuring the JVM heap size and execution times (http://git.eclipse.org/c/incquery/org.eclipse.incquery.git/tree/tests/org.eclipse.incquery.testing.core/src/org/eclipse/incquery/testing/core/QueryPerformanceTest.xtend), but after that it needs some manual adjustments.

I hope this was helpful. If not, feel free to ask for clarification or any follow-up question that comes up.

Cheers,
Zoltán

[Updated on: Tue, 16 June 2015 11:06] by Moderator

Report message to a moderator

Re: How to prevent traversal of objects (part 2) [message #1698678 is a reply to message #1698547] Wed, 17 June 2015 08:49 Go to previous message
Abel Hegedus is currently offline Abel HegedusFriend
Messages: 197
Registered: September 2015
Senior Member
Zoltan Ujhelyi wrote on Tue, 16 June 2015 13:05
If not the traversal, but the indexing/Rete building is slow, then you have to try to find out which pattern causes the issue, and refactor/remove the problematic parts. The QueryPerformanceTest abstract class helps to find out whether there is some pattern by measuring the JVM heap size and execution times (http://git.eclipse.org/c/incquery/org.eclipse.incquery.git/tree/tests/org.eclipse.incquery.testing.core/src/org/eclipse/incquery/testing/core/QueryPerformanceTest.xtend), but after that it needs some manual adjustments.


See also our wiki for some minor additional details: https://wiki.eclipse.org/EMFIncQuery/UserDocumentation/SDK/QueryHotspotTesting
Previous Topic:How can I keep my EMF model loaded (persistent Indexes)
Next Topic:EVM: Update job - How to find the element that changed?
Goto Forum:
  


Current Time: Sun Dec 08 16:22:35 GMT 2024

Powered by FUDForum. Page generated in 0.03576 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top