Eclipse Community Forums: TMF (Xtext) » Indexing/Linking order problem

Home » Modeling » TMF (Xtext) » Indexing/Linking order problem

Indexing/Linking order problem [message #1752994]

Tue, 31 January 2017 13:58

Eclipse User

I'm seeing behavior where my scoping is being run before indexing for a given individual file, when I first run my custom Eclipse for my DSL or even just paste in code to a file within it. The result is that if I use the index to lookup EObjects within the current file, I often get out of date broken data or even no data at all.

Can someone explain if this is intended behavior, and what the rationale is? Thanks!

Re: Indexing/Linking order problem [message #1753004 is a reply to message #1752994]

Tue, 31 January 2017 22:58

Eclipse User

Xtext does the thing in two phases

(1) indexing
(2) linking

The reason for that is e.g. Performance

So you should not access scopes in your name provider and or resource description strategy

Can you elaborate more on your use cases to find a solution

Eg

Use node module to fill the user data
Make the user data lazy and access it after indexing only

But therefore we need to know what kind of information you need and when you need it

Re: Indexing/Linking order problem [message #1753005 is a reply to message #1753004]

Tue, 31 January 2017 23:06

Eclipse User

p.s. usually the index queries for scoping are asked through the scope providers.

(so the pseudo-"index" entries for the current file dont come from the actual index but from ImportedNamespaceAwareLocalScopeProvider (internalGetAllDescriptions(Resource))

i filed years ago https://bugs.eclipse.org/bugs/show_bug.cgi?id=382535)

Re: Indexing/Linking order problem [message #1753085 is a reply to message #1753005]

Wed, 01 February 2017 13:13

Eclipse User

Thanks for the fast reply, Christian!

I do not access scopes in my name provider (I use the default one) nor in my resource description strategy (default plus one simple instanceof check). My ScopeProvider extends DelegatingScopeProvider, not ImportedNamespaceAwareLocalScopeProvider. Overall, my project setup resembles the SmallJava example in the book "Implementing Domain Specific Languages with Xtext and Xtend". It is a more complex DSL though - a general purpose language somewhat similar to Xtend, with support for C++ as an output language. I don't yet add any extra data to the index, though I'd like to figure out how to soon so I can support efficient cross-file checking of function overloads/specializations.

Breakpointing both my TestDSLScopeProvider.getScope AND my TestDSLResourceDescriptionStrategy.createEObjectDescriptions, when I paste a bunch of code into eclipse I see the getScope being called FIRST, with the following call stack:

TestDSLScopeProvider.xtend line: 379	
DefaultLinkingService.getScope(EObject, EReference) line: 59	
DefaultLinkingService.getLinkedObjects(EObject, EReference, INode) line: 120	
LazyLinkingResource.getEObject(String, Triple<EObject,EReference,INode>) line: 247	
LazyLinkingResource.getEObject(String) line: 222	
LazyLinkingResource.doResolveLazyCrossReference(InternalEObject, EStructuralFeature) line: 189	
LazyLinkingResource.resolveLazyCrossReference(InternalEObject, EStructuralFeature) line: 148	
LazyLinkingResource.resolveLazyCrossReferences(CancelIndicator) line: 134	
EcoreUtil2.resolveLazyCrossReferences(Resource, CancelIndicator) line: 498	
XtextDocumentReconcileStrategy.postParse(XtextResource, IProgressMonitor) line: 175	
XtextDocumentReconcileStrategy.doReconcile(IRegion) line: 153	
XtextDocumentReconcileStrategy.reconcile(IRegion) line: 67	
XtextReconciler.doRun(XtextResource, IProgressMonitor) line: 449	
XtextReconciler.access$3(XtextReconciler, XtextResource, IProgressMonitor) line: 429	
XtextReconciler$1.process(XtextResource) line: 363	
XtextReconciler$1.process(Object) line: 1	
XtextReconciler$1(IUnitOfWork$Void<T>).exec(T) line: 37	
OutdatedStateManager.xtend line: 71	
XtextDocument$XtextDocumentLocker.modify(IUnitOfWork<T,XtextResource>) line: 428	
XtextDocument.internalModify(IUnitOfWork<T,XtextResource>) line: 162	
XtextReconciler.run(IProgressMonitor) line: 360	
Worker.run() line: 55

Only after all the scoping completes do I see my ResourceDescriptionProvider being called:

TestDSLResouceDescriptionStrategy.xtend line: 27	
DefaultResourceDescription.computeExportedObjects() line: 89	
DefaultResourceDescription$4.get() line: 179	
DefaultResourceDescription$4.get() line: 173	
OnChangeEvictingCache.get(Object, Resource, Provider<T>) line: 77	
DefaultResourceDescription.getLookUp() line: 173	
DefaultResourceDescription(AbstractResourceDescription).getExportedObjects() line: 35	
DefaultResourceDescriptionDelta.internalHasChanges() line: 68	
DefaultResourceDescriptionDelta.haveEObjectDescriptionsChanged() line: 58	
DirtyStateEditorSupport.haveEObjectDescriptionsChanged(IResourceDescription, IResourceDescription$Manager) line: 623	
DirtyStateEditorSupport.announceDirtyState(XtextResource) line: 533	
XtextDocumentReconcileStrategy.postParse(XtextResource, IProgressMonitor) line: 179	
XtextDocumentReconcileStrategy.doReconcile(IRegion) line: 153	
XtextDocumentReconcileStrategy.reconcile(IRegion) line: 67	
XtextReconciler.doRun(XtextResource, IProgressMonitor) line: 449	
XtextReconciler.access$3(XtextReconciler, XtextResource, IProgressMonitor) line: 429	
XtextReconciler$1.process(XtextResource) line: 363	
XtextReconciler$1.process(Object) line: 1	
XtextReconciler$1(IUnitOfWork$Void<T>).exec(T) line: 37	
OutdatedStateManager.xtend line: 71	
XtextDocument$XtextDocumentLocker.modify(IUnitOfWork<T,XtextResource>) line: 428	
XtextDocument.internalModify(IUnitOfWork<T,XtextResource>) line: 162	
XtextReconciler.run(IProgressMonitor) line: 360	
Worker.run() line: 55

As far as I can tell, that means that Xtext is doing linking BEFORE indexing in this case when I paste code into the file, inside of its postParse method first calling resolveLazyCrossReferences and then announceDirtyState. I suppose it's possible that my implementation for some value calculated along the way is somehow forcing scoping to be done early, but I don't see how. Maybe you have ideas of some other places I might be accessing scope related info 'too early' or how I could debug to find out?

I do have a temporary workaround, where I just lookup stuff in the file itself if I detect that it's local to that file, but it seems wrong to have to do so. Maybe I am doing something else wrong or there is a subtle bug in xtext. This is the code for what I'm doing as a workaround:

def getStructForFullyQualifiedName(EObject context, String name) {
	val desc = context.getVisibleStructDescriptions.findFirst[qualifiedName.toString == name && EClass == TestDSLPackage::eINSTANCE.XStruct]
	if (desc == null || context.eResource.URI.trimFragment == desc.EObjectURI.trimFragment) {
		// find it in the context object's containing file instead of the index
		var containingFile = context.containingFile;
		var rv2 = containingFile.defns.findFirst[it.name==name && it.eClass == TestDSLPackage::eINSTANCE.XStruct]
		return rv2 as XStruct;
	}
	var o = desc.EObjectOrProxy
	if (o.eIsProxy)
		o = context.eResource.resourceSet.getEObject(desc.EObjectURI, true)
	o as XStruct
}

Re: Indexing/Linking order problem [message #1753087 is a reply to message #1753085]

Wed, 01 February 2017 13:57

Eclipse User

These stacks are from editor not from build ?!?

Re: Indexing/Linking order problem [message #1753088 is a reply to message #1753087]

Wed, 01 February 2017 14:06

Eclipse User

Yes, they're from running the debug build of the generated Eclipse IDE. Sorry, is there something I'm missing about that? I haven't created a separate compiler-only build yet...

Re: Indexing/Linking order problem [message #1753090 is a reply to message #1753088]

Wed, 01 February 2017 14:17

Eclipse User

No what I mean.

This stacktrace is when you edit in the editor in the runtime eclipse
It is not when you do a build in the runtime eclipse

Re: Indexing/Linking order problem [message #1753098 is a reply to message #1753090]

Wed, 01 February 2017 15:09

Eclipse User

Yes exactly: this is a problem that occurs while *editing* code written in my dsl language. While editing, the index often does not get updated in what I would consider to be the correct order. This causes problems and leaves the editor looking like there are errors in the code, when in fact there are not. Sometimes, adding an extra space resolves these 'errors' because the index gets regenerated. This also occurs when first starting Eclipse - the code is marked with errors that do not exist, because the index is generated late.

Re: Indexing/Linking order problem [message #1753099 is a reply to message #1753098]

Wed, 01 February 2017 15:12

Eclipse User

yes but i dont understand and what i try to ask/tell you.

the local file editing (e.g. scoping) shall not rely on index, since the eobejctdescriptions are created not by the indexer but by ImportedNamespaceAwareLocalScopeProvider

=> what exactly do you query from the index and where do you do it

Previous Topic:	Loading Xtext model without direct dependency to xtext plugin
Next Topic:	Xtext via Equinox without a UI component

Goto Forum:

-=] Back to Top [=-

Current Time: Wed Jul 23 14:00:06 EDT 2025

.:: Contact :: Home ::.

Breadcrumbs

Sign up to our Newsletter