Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » EMF-IncQuery » How to prevent traversal of objects
How to prevent traversal of objects [message #1239503] Tue, 04 February 2014 10:22 Go to next message
Christian Mohr is currently offline Christian MohrFriend
Messages: 18
Registered: June 2012
Junior Member
Hi there,

i am trying to work with incquery and a lot of model data (about 500k objects). When accessing the first pattern via API (IncQueryEngine initalized with Root-Model-Object), it trys to cache/traverse all objects. That's not possible, because the model is too big for the memory and even with enough memory, it would take too long.

Is there a way to prevent incquery from caching/traversing given objects, that are not needed for any patterns (e.g. some objects contain about 2-5k other objects, the contained ones are not of interest) or are there any best practices how to design the patterns that only the objects of interest are traversed.

My next question would be if it is possible to persist the cached information in the database? I wouldn't want to traverse all objects again at each startup of the application.

Thanks for any Help or Hints.

Greets

Christian Mohr







Re: How to prevent traversal of objects [message #1239508 is a reply to message #1239503] Tue, 04 February 2014 10:37 Go to previous messageGo to next message
Zoltan Ujhelyi is currently offline Zoltan UjhelyiFriend
Messages: 222
Registered: July 2009
Senior Member
Hi Christian,

in short, there are currently no real solutions to your problem in EMF-IncQuery, only some workarounds.

In order to EMF-IncQuery work (at all because of the used Rete algorithm), a cache needs to be initialized with all model elements in the search scope. This scope can contain relevant elements of a ResourceSet, a Resource or the elements contained in a single model element. Last week we had a meeting when we were talking about introducing a programmable traversal option (https://wiki.eclipse.org/EMFIncQuery/DeveloperMeetingMinutes/Meeting20140129#Action_plan), but the corresponding ticket has not been created yet, and don't have any concrete plans when this will be implemented.

About serializing the caches - it would be a nice idea, yes, but it would require serialization and deserialization support that also has not been implemented yet in our engine.

The only thing I can come up for your problem is to use EObject scope if possible (but if I understood your problem correctly, that might not be the case), or wait for the index to load.

If that is still unacceptable, you could also try program directly over the base indexer component (https://wiki.eclipse.org/EMFIncQuery/UserDocumentation/API/BaseIndexer), that builds much smaller indexes and finishes traversal much faster; however you cannot use the pattern specifications and still need to traverse the model each time.

I hope, something along these line help,
Zoltán
Re: How to prevent traversal of objects [message #1239580 is a reply to message #1239503] Tue, 04 February 2014 14:20 Go to previous message
Istvan Rath is currently offline Istvan RathFriend
Messages: 54
Registered: July 2009
Member
Hi,

To put it in simpler terms:

Quote:

i am trying to work with incquery and a lot of model data (about 500k objects). When accessing the first pattern via API (IncQueryEngine initalized with Root-Model-Object), it trys to cache/traverse all objects. That's not possible, because the model is too big for the memory and even with enough memory, it would take too long.

Is there a way to prevent incquery from caching/traversing given objects, that are not needed for any patterns (e.g. some objects contain about 2-5k other objects, the contained ones are not of interest) or are there any best practices how to design the patterns that only the objects of interest are traversed.


Any search engine would need to traverse your entire model, in order to evaluate global queries. How should the engine know "in advance" that certain subtrees are not of interest?

That being said, EMF-IncQuery provides three matcher scopes (ResourceSet, Resource and EObject) that you can use to restrict traversal. Note however that _caching_ will only be done for those types that are actually contained in your queries, so no unnecessary resources are used.

For more details on related topics and best practices, I highly recommend to read the FAQ:
http://incquery.net/incquery/faq

For specific questions regarding the optimization of patterns, we are glad to help here, if you provide more details. 500k of objects is not too big a model, so with some optimization, I'm certain the execution time can be brought down to an acceptable level.

Quote:
My next question would be if it is possible to persist the cached information in the database? I wouldn't want to traverse all objects again at each startup of the application.


We've already had similar feature requests, and the core infrastructure for this is already present, we just did not wire it to the API yet. I couldn't find the bugzilla, so if you feel like it, please drop us a ticket on this topic.

regards
Previous Topic:Querying Derived Features
Next Topic:IncQuery support for XText 2.5
Goto Forum:
  


Current Time: Fri Dec 19 00:46:12 GMT 2014

Powered by FUDForum. Page generated in 0.10322 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software