Home » Modeling » TMF (Xtext) » Listening for Index-Changes
Listening for Index-Changes [message #1818805] |
Sun, 29 December 2019 14:29 |
Konrad Jünemann Messages: 93 Registered: December 2018 |
Member |
|
|
Hi,
I would like to build an index-based (with index = IResourceDescriptions) cache, that caches calculations for which I have to access the index (example: checking for global uniqueness of items) and that gets refreshed whenever the index changes. I have a working implementation of such da cache that organizes its contents in buckets with one bucket for each supported EClass. When a new A-item gets added to the index, the A bucket is refreshed but not the B-bucket.
All of this works, but the cache gets notified right now by my ResourceDescriptionStrategy (MyDSLResourceDescriptionStrategy). Obviously, this is not the right place, as descriptions will be created for other reasons than putting them to the index, so the cache currently gets refreshed more often than necessary.
My question: What is the right way to listen to changes in the index?
I assume that I would have to register the cache as an IResourceDescription.Event.Listener at an IResourceDescription.Event.Source , however I am at a loss at which concrete implementation I would have to register at the moment.
Currently, I use a helper class to access the index that works kinda as proposed in this book: https://www.amazon.de/Implementing-Domain-Specific-Languages-Xtext-Xtend/dp/1786464969/ref=sr_1_1?__mk_de_DE=%EF%BF%BDM%EF%BF%BD%25u017D%EF%BF%BD%EF%BF%BD&keywords=xtext&qid=1577629071&sr=8-1
It makes use of the visible Container-approach in order to implement class-path mechanisms and looks like this:
@Inject
var ResourceDescriptionsProvider rdp
@Inject
IContainer.Manager cm
/** Returns all eObject descriptions of a provided type that are visible from a provided object o. */
private def searchIndexByName(EObject o, EClass type, QualifiedName fqn) {
val tmp = o.getVisibleContainers.map [ c |
c.getExportedObjects(type, fqn, false)
]
return tmp.flatten
}
/** Returns all containers that are visible from a provided object o. A container is, e.g., an Eclipse project or a class path item. */
private def getVisibleContainers(EObject o) {
if (o?.eResource === null)
return emptyList
val index = rdp.getResourceDescriptions(o.eResource)
val rd = index.getResourceDescription(o.eResource.URI)
if (rd === null) {
// do something else
return emptyList
}
cm.getVisibleContainers(rd, index)
}
As you see, I use the RDProvider in order to return a matching IResourceDescriptions instance whenever accessing the index, so I currently have no "fixed" IResourceDescriptions I could register at. The RDProvider actually returns different kinds of IResourceDescriptions instances, so using the same fixed instance would change the behavior.
What am I missing here? Maybe the container-based approach is outdated or am I doing something else wrong? Ideally I would be notified whenever a new item is added to the index and be able to retrieve the concrete EClass of the added item.
Best regards,
Konrad
[Updated on: Sun, 29 December 2019 14:39] Report message to a moderator
|
|
| | |
Re: Listening for Index-Changes [message #1818829 is a reply to message #1818812] |
Mon, 30 December 2019 07:40 |
|
the question is:
where is the code you have. where do you call this code from.
is it about build changes only or about dirty editors etc too.
the builder state is the impl on the eclipse side.
standalone and in eclipse you then have to inject the ResourceDescriptions with named PERSISTED_DESCRIPTIONS
Twitter : @chrdietrich
Blog : https://www.dietrich-it.de
|
|
| | | | |
Re: Listening for Index-Changes [message #1818845 is a reply to message #1818843] |
Mon, 30 December 2019 10:35 |
Konrad Jünemann Messages: 93 Registered: December 2018 |
Member |
|
|
Hi Christian,
thank you for going into such detail with me.
Caching is deactivated right now, so it is not a caching issue.
I would like to cache computationally expensive operations like, e.g., building a set of a given properties of a certain type of entity which (the property) needs to be unique. This set is checked when a new item is added. Without a cache, all items in the index would have to be traversed every time I need to build this set, as I have no clue whether or not another item has been added to the index. To avoid this recalculation, I would like to cache it, as items are rarely added in general.
The idea to change the scope implementation to one that lazily resolves its contents arose when adding Sirius to my project. Sirius force-resolves *all* references in the model when opening an aird-file, which leads to the creation of a lot of scopes and a lot of accesses to the index - in our case also the containers have to be asked every time. With a large model (~2000 files with a lot of references) this force-resolve alone took over a minute. However, most scopes just refer to the same globally available objects (which they get from the index) under a local name (driven by the local import statements).
It was an enormous performance improvement to thus cache the index-access (as it is the same for almost all scopes in those 2000 files) and just create the aliased IEObjectDescriptions in the lokal scope itself. Resolving went down from 73 s to 3 s.
So, that's where I am coming from. Would you say this is the wrong approach? Surely there is commonly the need to compute something based on the index, which you want to compute again only if the index changes?
[Updated on: Mon, 30 December 2019 10:41] Report message to a moderator
|
|
| |
Re: Listening for Index-Changes [message #1818848 is a reply to message #1818847] |
Mon, 30 December 2019 11:29 |
Konrad Jünemann Messages: 93 Registered: December 2018 |
Member |
|
|
Hi Christian,
so if I am calling IResourceDescriptionsProvider.getResourceDescription(Resource) it will return a NAMED_BUILDER_SCOPE when called during the build and a persistent index when called after the build?
Very intersting link. My cache-implementation looks very similar, but is not bound the the . However, it also binds things to a EClasses by using a Multimap. In my case, the things could be anything computed by using the index, not only the EObjectDescriptions itself. Please find my code below for reference, if it helps you.
I also stumbled upon EObjectDescriptionLookUp during my performance analysis. So far, my understanding is that the performance issues are mainly caused by the following points:
- scopes are repeatatly calculated and always refer to the same IEObjectDescriptions, although they might appear under a different alias from file to file
- scopes are built eagerly, so all items will be retrieved no matter if actually accessed during linking or not
- index access in my implementation is relatively slow (when called repeatatly), mainly because of the Container-thingy I implemented in order to handle classpathes. It is slow, because on every call the following things have to happen:
1. The containers have to be fetched
2. The containers have to be queried by calling (that's where your EObjectDescriptionLookup2 optimization might be handy)
3. The implementation calls in its implementation, in my opinion without the need to do so. This is done at every index access in my case and might be a slow operation. I thus extended the StateBasedContainer as shown below (is just does not check for emptyness).
All together: Yes, I think an optimization like the proposed one might help, but I think it would not work as well as caching the whole computation.
I guess you disagree because you expect issues with maintaining the cache?
Do I also assume correctly that there is currently no known "best practise" for caching index-based computations?
Thanks a lot,
Konrad
class MyStateBasedContainerManager extends StateBasedContainerManager {
override protected getVisibleContainers(List<String> handles, IResourceDescriptions resourceDescriptions) {
if (handles.isEmpty())
return Collections.emptyList();
val List<IContainer> result = Lists.newArrayListWithExpectedSize(handles.size());
for(String handle: handles) {
val IContainer container = createContainer(handle, resourceDescriptions);
// if (!container.isEmpty() || result.isEmpty())
// println("Not calling is Empty()...")
result.add(container);
}
return result;
}
}
/**
* The default implementation of an IndexBackedCache.
*/
@Singleton
class IndexBackedCacheImpl implements IndexBackedCache {
/** Holds a list of all buckets that need to be cleared when an item of a certain type is added to the index. */
val Multimap<EClass, IndexBackedCacheBucket> refreshMap = LinkedListMultimap.create;
/** Maps the bucket that holds the cached values to its type. */
val Map<EClass, IndexBackedCacheBucket> caches = newHashMap
@Inject
Provider<IndexBackedCacheBucket> cacheProv
override void onNewElement(EClass typeOfNewElement) {
val cs = getAffectedBuckets(typeOfNewElement)
for (c : cs) {
c.clear
}
}
override <T> T get(EClass type, Object key, Provider<T> provider) {
var cache = caches.get(type)
if (cache === null) {
cache = create(type);
}
cache.get(key, provider)
}
/** Creates a new Cache bucket of the provided type. */
private def create(EClass type) {
val rslt = cacheProv.get
rslt.type = type
caches.put(type, rslt)
refreshMap.clear
return rslt
}
/** Returns the cache buckets that should be refreshed when a new item of the provided type is added
* to the index. The returned list thus returns all cache buckets for any super types of the provided type. */
private def getAffectedBuckets(EClass type) {
val rslt = refreshMap.get(type)
if (!rslt.nullOrEmpty)
return rslt
rebuildRefreshMapEntry(type)
val r = refreshMap.get(type)
// if (!r.nullOrEmpty)
// println('''«type.name» -> Refresh «r.map[it.type.name].join(", ")»''')
return r
}
/** Rebuilds the entry in the refresh map for the provided type.*/
private def rebuildRefreshMapEntry(EClass type) {
for (e : caches.entrySet) {
val t = e.key
val c = e.value
if (t.isSuperTypeOf(type)) {
refreshMap.put(type, c)
}
}
}
/** An inner bucket, which holds the actual cached values. */
static class IndexBackedCacheBucket {
val Map<Object, Object> items = newHashMap;
@Accessors(#[PUBLIC_GETTER])
var EClass type;
protected new() {
// do nothing
}
def <T> T get(Object key, Provider<T> provider) {
var rslt = items.get(key)
if (rslt !== null || items.containsKey(key)) {
// cache hit
return rslt as T
} else {
// cache miss
val tmp = provider.get
items.put(key, tmp)
return tmp
}
}
def clear() {
// println('''Flushing Bucket «type.name»''')
items.clear
}
}
}
|
|
| |
Goto Forum:
Current Time: Tue Sep 24 15:37:36 GMT 2024
Powered by FUDForum. Page generated in 0.04682 seconds
|