Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » TMF (Xtext) » Odd error using IJvmTypeProvider
icon5.gif  Odd error using IJvmTypeProvider [message #911199] Tue, 11 September 2012 07:16 Go to next message
Aaron Digulla is currently offline Aaron DigullaFriend
Messages: 258
Registered: July 2009
Location: Switzerland
Senior Member
This is part of my DSL:

Import: 'import' name=QualifiedName ';';


In the .i18n (model) file, I have this line:

import java.util.Locale;


This is checked using this code:

import org.eclipse.xtext.common.types.JvmType;
import org.eclipse.xtext.common.types.access.IJvmTypeProvider;
import com.pany.eclipse.i18n_dsl.i18nDsl.I18nDslPackage;
import com.pany.eclipse.i18n_dsl.i18nDsl.Import;
import com.google.inject.Inject;

@SuppressWarnings( "restriction" )
public class ImportChecker {
    
    @Inject
    private IJvmTypeProvider.Factory typeProviderFactory;
    
    public void checkImports( Import imp, I18nDslJavaValidator validator ) {
        // don't check wildcard imports
        String name = imp.getName();
        if( name.endsWith( ".*" ) ) {
            return;
        }
        
        IJvmTypeProvider typeProvider = typeProviderFactory.findOrCreateTypeProvider( imp.eResource().getResourceSet() );
        JvmType jvmType = typeProvider.findTypeByName( name );
        if( jvmType == null ) {
            validator.error( "The import " + name + " cannot be resolved", imp, I18nDslPackage.Literals.IMPORT__NAME );
        }
    }
}


From the code above, this Java code is generated in the same Eclipse project (= same classpath as the .i18n file)

public final static I18nMessage foo( Locale locale ) { ... }


Now the fun part:

1. When I open the .i18n file in Eclipse, there are no problems (warnings or errors).
Then I add a space, delete it again and save.

Suddenly, the import is underlined in red. The error message reads "The import java.util.Locale cannot be resolved"

2. There is no entry in the Problems View!

3. The Java code compiles without error.

Questions:

1. How is it possible that I get an error in the editor but not in the problems view?

2. Why do I get the error at all? If java.util.Locale wasn't on the classpath, I doubt that the generated Java code would compile.

Regards,

A. Digulla
Re: Odd error using IJvmTypeProvider [message #911231 is a reply to message #911199] Tue, 11 September 2012 08:02 Go to previous messageGo to next message
Sebastian Zarnekow is currently offline Sebastian ZarnekowFriend
Messages: 3118
Registered: July 2009
Senior Member
Hi Aaron,

I assume you somehow get a null impl of the JvmTypeProvider. Could you
try to set a breakpoint into the constructor
NullJdtTypeProvider(ResourceSet)?

Regards,
Sebastian
--
Looking for professional support for Xtext, Xtend or Eclipse Modeling?
Go visit: http://xtext.itemis.com

Am 11.09.12 09:16, schrieb Aaron Digulla:
> This is part of my DSL:
>
>
> Import: 'import' name=QualifiedName ';';
>
>
> In the .i18n (model) file, I have this line:
>
>
> import java.util.Locale;
>
>
> This is checked using this code:
>
>
> import org.eclipse.xtext.common.types.JvmType;
> import org.eclipse.xtext.common.types.access.IJvmTypeProvider;
> import com.pany.eclipse.i18n_dsl.i18nDsl.I18nDslPackage;
> import com.pany.eclipse.i18n_dsl.i18nDsl.Import;
> import com.google.inject.Inject;
>
> @SuppressWarnings( "restriction" )
> public class ImportChecker {
> @Inject
> private IJvmTypeProvider.Factory typeProviderFactory;
> public void checkImports( Import imp, I18nDslJavaValidator validator
> ) {
> // don't check wildcard imports
> String name = imp.getName();
> if( name.endsWith( ".*" ) ) {
> return;
> }
> IJvmTypeProvider typeProvider =
> typeProviderFactory.findOrCreateTypeProvider(
> imp.eResource().getResourceSet() );
> JvmType jvmType = typeProvider.findTypeByName( name );
> if( jvmType == null ) {
> validator.error( "The import " + name + " cannot be
> resolved", imp, I18nDslPackage.Literals.IMPORT__NAME );
> }
> }
> }
>
>
> From the code above, this Java code is generated in the same Eclipse
> project (= same classpath as the .i18n file)
>
>
> public final static I18nMessage foo( Locale locale ) { ... }
>
>
> Now the fun part:
> 1. When I open the .i18n file in Eclipse, there are no problems
> (warnings or errors).
> Then I add a space, delete it again and save.
>
> Suddenly, the import is underlined in red. The error message reads "The
> import java.util.Locale cannot be resolved"
>
> 2. There is no entry in the Problems View!
>
> 3. The Java code compiles without error.
>
> Questions:
>
> 1. How is it possible that I get an error in the editor but not in the
> problems view?
>
> 2. Why do I get the error at all? If java.util.Locale wasn't on the
> classpath, I doubt that the generated Java code would compile.
>
> Regards,
>
> A. Digulla
Re: Odd error using IJvmTypeProvider [message #911305 is a reply to message #911231] Tue, 11 September 2012 10:24 Go to previous messageGo to next message
Aaron Digulla is currently offline Aaron DigullaFriend
Messages: 258
Registered: July 2009
Location: Switzerland
Senior Member
Sebastian Zarnekow wrote on Tue, 11 September 2012 10:02
Hi Aaron,

I assume you somehow get a null impl of the JvmTypeProvider. Could you
try to set a breakpoint into the constructor
NullJdtTypeProvider(ResourceSet)?


Thanks for the tip. Unfortunately, I can't debug using the launch config anymore. Eclipse doesn't start after the upgrade to 2.3.0. I get this error:

Error instantiating builder 'org.eclipse.xtext.ui.shared.xtextBuilder'.
Plug-in org.eclipse.xtext.ui.shared was unable to load class org.eclipse.xtext.ui.shared.internal.ExecutableExtensionFactory.
org/eclipse/xtext/ui/guice/AbstractGuiceAwareExecutableExtensionFactory

I deselected all bundles from the target platform, clicked "Add Required Plug-ins", tried again, same error. Sad

I have both org.eclipse.xtext.ui and org.eclipse.xtext.ui.shared 2.3.0 enabled. Any ideas?

Is there some documentation how to create an Eclipse Application launch config that works most of the time?

Note that the very same plugin(s) works perfectly when installed in Eclipse.

Regards,

A. Digulla

[Updated on: Tue, 11 September 2012 10:25]

Report message to a moderator

Re: Odd error using IJvmTypeProvider [message #916778 is a reply to message #911231] Wed, 19 September 2012 12:58 Go to previous messageGo to next message
Aaron Digulla is currently offline Aaron DigullaFriend
Messages: 258
Registered: July 2009
Location: Switzerland
Senior Member
Sebastian Zarnekow wrote on Tue, 11 September 2012 10:02
Hi Aaron,

I assume you somehow get a null impl of the JvmTypeProvider. Could you
try to set a breakpoint into the constructor
NullJdtTypeProvider(ResourceSet)?


Okay, I could create a working launch config again.

The break point is hit. Here is the stack trace:

Thread [Worker-2] (Suspended (breakpoint at line 34 in NullJdtTypeProvider))	
	NullJdtTypeProvider.<init>(ResourceSet) line: 34	
	JdtTypeProviderFactory.createJdtTypeProvider(IJavaProject, ResourceSet) line: 39	
	JdtTypeProviderFactory.createTypeProvider(ResourceSet) line: 32	
	JdtTypeProviderFactory.createTypeProvider(ResourceSet) line: 1	
	JdtTypeProviderFactory(AbstractTypeProviderFactory).findOrCreateTypeProvider(ResourceSet) line: 41	
	ImportChecker.checkImports(Import, I18nDslJavaValidator) line: 22	
	I18nDslJavaValidator.checkImports(Import) line: 367	
	NativeMethodAccessorImpl.invoke0(Method, Object, Object[]) line: not available [native method]	
	NativeMethodAccessorImpl.invoke(Object, Object[]) line: 39	
	DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 25	
	Method.invoke(Object, Object...) line: 597	
	AbstractDeclarativeValidator$MethodWrapper.invoke(AbstractDeclarativeValidator$State) line: 109	
	I18nDslJavaValidator(AbstractDeclarativeValidator).internalValidate(EClass, EObject, DiagnosticChain, Map<Object,Object>) line: 291	
	I18nDslJavaValidator(AbstractInjectableValidator).validate(EClass, EObject, DiagnosticChain, Map<Object,Object>) line: 62	
	CompositeEValidator.validate(EClass, EObject, DiagnosticChain, Map<Object,Object>) line: 126	
	CancelableDiagnostician(Diagnostician).validate(EClass, EObject, DiagnosticChain, Map<Object,Object>) line: 159	
	CancelableDiagnostician(Diagnostician).validate(EObject, DiagnosticChain, Map<Object,Object>) line: 137	
	CancelableDiagnostician.validate(EObject, DiagnosticChain, Map<Object,Object>) line: 36	
	CancelableDiagnostician(Diagnostician).doValidateContents(EObject, DiagnosticChain, Map<Object,Object>) line: 174	
	CancelableDiagnostician.doValidateContents(EObject, DiagnosticChain, Map<Object,Object>) line: 48	
	CancelableDiagnostician(Diagnostician).validate(EClass, EObject, DiagnosticChain, Map<Object,Object>) line: 162	
	CancelableDiagnostician(Diagnostician).validate(EObject, DiagnosticChain, Map<Object,Object>) line: 137	
	CancelableDiagnostician.validate(EObject, DiagnosticChain, Map<Object,Object>) line: 36	
	CancelableDiagnostician(Diagnostician).doValidateContents(EObject, DiagnosticChain, Map<Object,Object>) line: 178	
	CancelableDiagnostician.doValidateContents(EObject, DiagnosticChain, Map<Object,Object>) line: 48	
	CancelableDiagnostician(Diagnostician).validate(EClass, EObject, DiagnosticChain, Map<Object,Object>) line: 162	
	CancelableDiagnostician(Diagnostician).validate(EObject, DiagnosticChain, Map<Object,Object>) line: 137	
	CancelableDiagnostician.validate(EObject, DiagnosticChain, Map<Object,Object>) line: 36	
	CancelableDiagnostician(Diagnostician).validate(EObject, Map<?,?>) line: 120	
	ResourceValidatorImpl.validate(Resource, CheckMode, CancelIndicator) line: 108	
	ValidationJob$1.exec(XtextResource) line: 79	
	ValidationJob$1.exec(Object) line: 1	
	XtextDocument$XtextDocumentLocker(AbstractReadWriteAcces<P>).readOnly(IUnitOfWork<T,P>) line: 32	
	XtextDocument.readOnly(IUnitOfWork<T,XtextResource>) line: 78	
	ValidationJob.createIssues(IProgressMonitor) line: 75	
	ValidationJob.run(IProgressMonitor) line: 64	
	Worker.run() line: 54	


I followed the code and createJdtTypeProvider() is interesting:

		if (javaProject == null)
			//TODO throw a serious exception instead of returning a non working implementation
			return new NullJdtTypeProvider(resourceSet);


I guess the exception would have been better for me Smile

Debugging this further, I got to XtextResourceSetBasedProjectProvider:

Object context = xtextResourceSet.getClasspathURIContext();


This code is invoked when I open a DSL document, when I change it and when I save. The context is null when I open the DSL document and when I type in it. When I save, it contains an instance of JavaProject.

That would explain the odd behavior of the Problems view: It's not updated when the document is loaded or while I make changes. So I get errors in the editor but not in the view. When I save, JavaProject!= null, so the validation passes and the Problem View doesn't show any errors.

The errors in the editor stay because Xtext doesn't refresh them on save (probably on the grounds that the list can't have changed since the last change and saving the document).

So I went further back. The XtextResource resource which is used for org.eclipse.xtext.ui.editor.model.XtextDocument.setInput(XtextResource) has a null classpathURIContext even though the resource comes from a JavaProject. It was created with

XtextResource xtextResource = (XtextResource) resourceForEditorInputFactory.createResource(editorInput);


which eventually calls org.eclipse.xtext.ui.editor.model.ResourceForIEditorInputFactory.getResourceSet(IStorage):

	protected ResourceSet getResourceSet(IStorage storage) {
		if (storage instanceof IFile) {
			return resourceSetProvider.get(((IFile) storage).getProject());
		}
		return resourceSetProvider.get(null);
	}


storage is an IFile and getProject() returns != null. We end up here:

public class SimpleResourceSetProvider implements IResourceSetProvider {

	@Inject
	private Provider<XtextResourceSet> resourceSetProvider;
	
	public ResourceSet get(IProject project) {
		return resourceSetProvider.get();
	}

}


At this point, project is an instance of org.eclipse.core.internal.resources.Project (so not a Java project). When the method returns, classpathURIContext is still null.

Also the setter setClasspathURIContext() is never called. It's only called in org.eclipse.xtext.ui.resource.XtextResourceSetProvider.get(IProject) but this code is only invoked when I save the document. It's not invoked for open/edit.

What now?

Regards,

A. Digulla
Re: Odd error using IJvmTypeProvider [message #916839 is a reply to message #916778] Wed, 19 September 2012 14:37 Go to previous messageGo to next message
Aaron Digulla is currently offline Aaron DigullaFriend
Messages: 258
Registered: July 2009
Location: Switzerland
Senior Member
After some more experiments, I can confirm this is a stable bug. When opening and changing a file, the XtextResourceSet isn't connected to the JavaProject. Only when I save, I get a new resource set with the JavaProject. When I edit the saved file some more (without closing), the JavaProject is null again.
Re: Odd error using IJvmTypeProvider [message #916859 is a reply to message #916778] Wed, 19 September 2012 14:59 Go to previous messageGo to next message
Sebastian Zarnekow is currently offline Sebastian ZarnekowFriend
Messages: 3118
Registered: July 2009
Senior Member
Make sure you use the XtextResourceSetProvider. It's configured in
org.eclipse.xtext.ui.DefaultUiModule.bindIResourceSetProvider().

Why did you override it?

Regards,
Sebastian


Am 19.09.12 14:58, schrieb Aaron Digulla:
> Sebastian Zarnekow wrote on Tue, 11 September 2012 10:02
>> Hi Aaron,
>>
>> I assume you somehow get a null impl of the JvmTypeProvider. Could you
>> try to set a breakpoint into the constructor
>> NullJdtTypeProvider(ResourceSet)?
>
>
> Okay, I could create a working launch config again.
>
> The break point is hit. Here is the stack trace:
>
>
> Thread [Worker-2] (Suspended (breakpoint at line 34 in
> NullJdtTypeProvider))
> NullJdtTypeProvider.<init>(ResourceSet) line: 34
> JdtTypeProviderFactory.createJdtTypeProvider(IJavaProject,
> ResourceSet) line: 39
> JdtTypeProviderFactory.createTypeProvider(ResourceSet) line: 32
> JdtTypeProviderFactory.createTypeProvider(ResourceSet) line: 1
> JdtTypeProviderFactory(AbstractTypeProviderFactory).findOrCreateTypeProvider(ResourceSet) line: 41
> ImportChecker.checkImports(Import, I18nDslJavaValidator) line: 22
> I18nDslJavaValidator.checkImports(Import) line: 367
> NativeMethodAccessorImpl.invoke0(Method, Object, Object[]) line:
> not available [native method]
> NativeMethodAccessorImpl.invoke(Object, Object[]) line: 39
> DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 25
> Method.invoke(Object, Object...) line: 597
> AbstractDeclarativeValidator$MethodWrapper.invoke(AbstractDeclarativeValidator$State) line: 109
> I18nDslJavaValidator(AbstractDeclarativeValidator).internalValidate(EClass, EObject, DiagnosticChain, Map<Object,Object>) line: 291
> I18nDslJavaValidator(AbstractInjectableValidator).validate(EClass,
> EObject, DiagnosticChain, Map<Object,Object>) line: 62
> CompositeEValidator.validate(EClass, EObject, DiagnosticChain,
> Map<Object,Object>) line: 126
> CancelableDiagnostician(Diagnostician).validate(EClass, EObject,
> DiagnosticChain, Map<Object,Object>) line: 159
> CancelableDiagnostician(Diagnostician).validate(EObject,
> DiagnosticChain, Map<Object,Object>) line: 137
> CancelableDiagnostician.validate(EObject, DiagnosticChain,
> Map<Object,Object>) line: 36
> CancelableDiagnostician(Diagnostician).doValidateContents(EObject,
> DiagnosticChain, Map<Object,Object>) line: 174
> CancelableDiagnostician.doValidateContents(EObject,
> DiagnosticChain, Map<Object,Object>) line: 48
> CancelableDiagnostician(Diagnostician).validate(EClass, EObject,
> DiagnosticChain, Map<Object,Object>) line: 162
> CancelableDiagnostician(Diagnostician).validate(EObject,
> DiagnosticChain, Map<Object,Object>) line: 137
> CancelableDiagnostician.validate(EObject, DiagnosticChain,
> Map<Object,Object>) line: 36
> CancelableDiagnostician(Diagnostician).doValidateContents(EObject,
> DiagnosticChain, Map<Object,Object>) line: 178
> CancelableDiagnostician.doValidateContents(EObject,
> DiagnosticChain, Map<Object,Object>) line: 48
> CancelableDiagnostician(Diagnostician).validate(EClass, EObject,
> DiagnosticChain, Map<Object,Object>) line: 162
> CancelableDiagnostician(Diagnostician).validate(EObject,
> DiagnosticChain, Map<Object,Object>) line: 137
> CancelableDiagnostician.validate(EObject, DiagnosticChain,
> Map<Object,Object>) line: 36
> CancelableDiagnostician(Diagnostician).validate(EObject, Map<?,?>)
> line: 120
> ResourceValidatorImpl.validate(Resource, CheckMode,
> CancelIndicator) line: 108
> ValidationJob$1.exec(XtextResource) line: 79
> ValidationJob$1.exec(Object) line: 1
> XtextDocument$XtextDocumentLocker(AbstractReadWriteAcces<P>).readOnly(IUnitOfWork<T,P>) line: 32
> XtextDocument.readOnly(IUnitOfWork<T,XtextResource>) line: 78
> ValidationJob.createIssues(IProgressMonitor) line: 75
> ValidationJob.run(IProgressMonitor) line: 64
> Worker.run() line: 54
>
>
> I followed the code and createJdtTypeProvider() is interesting:
>
>
> if (javaProject == null)
> //TODO throw a serious exception instead of returning a non
> working implementation
> return new NullJdtTypeProvider(resourceSet);
>
>
> I guess the exception would have been better for me :)
>
> Debugging this further, I got to XtextResourceSetBasedProjectProvider:
>
>
> Object context = xtextResourceSet.getClasspathURIContext();
>
>
> This code is invoked when I open a DSL document, when I change it and
> when I save. The context is null when I open the DSL document and when I
> type in it. When I save, it contains an instance of JavaProject.
>
> That would explain the odd behavior of the Problems view: It's not
> updated when the document is loaded or while I make changes. So I get
> errors in the editor but not in the view. When I save, JavaProject!=
> null, so the validation passes and the Problem View doesn't show any
> errors.
>
> The errors in the editor stay because Xtext doesn't refresh them on save
> (probably on the grounds that the list can't have changed since the last
> change and saving the document).
>
> So I went further back. The XtextResource resource which is used for
> org.eclipse.xtext.ui.editor.model.XtextDocument.setInput(XtextResource)
> has a null classpathURIContext even though the resource comes from a
> JavaProject. It was created with
>
>
> XtextResource xtextResource = (XtextResource)
> resourceForEditorInputFactory.createResource(editorInput);
>
>
> which eventually calls
> org.eclipse.xtext.ui.editor.model.ResourceForIEditorInputFactory.getResourceSet(IStorage):
>
>
>
> protected ResourceSet getResourceSet(IStorage storage) {
> if (storage instanceof IFile) {
> return resourceSetProvider.get(((IFile)
> storage).getProject());
> }
> return resourceSetProvider.get(null);
> }
>
>
> storage is an IFile and getProject() returns != null. We end up here:
>
>
> public class SimpleResourceSetProvider implements IResourceSetProvider {
>
> @Inject
> private Provider<XtextResourceSet> resourceSetProvider;
>
> public ResourceSet get(IProject project) {
> return resourceSetProvider.get();
> }
>
> }
>
>
> At this point, project is an instance of
> org.eclipse.core.internal.resources.Project (so not a Java project).
> When the method returns, classpathURIContext is still null.
>
> Also the setter setClasspathURIContext() is never called. It's only
> called in
> org.eclipse.xtext.ui.resource.XtextResourceSetProvider.get(IProject) but
> this code is only invoked when I save the document. It's not invoked for
> open/edit.
>
> What now?
>
> Regards,
>
> A. Digulla


--
Looking for professional support for Xtext, Xtend or Eclipse Modeling?
Go visit: http://xtext.itemis.com
Re: Odd error using IJvmTypeProvider [message #916892 is a reply to message #916859] Wed, 19 September 2012 15:57 Go to previous messageGo to next message
Aaron Digulla is currently offline Aaron DigullaFriend
Messages: 258
Registered: July 2009
Location: Switzerland
Senior Member
Sebastian Zarnekow wrote on Wed, 19 September 2012 16:59
Make sure you use the XtextResourceSetProvider. It's configured in
org.eclipse.xtext.ui.DefaultUiModule.bindIResourceSetProvider().

Why did you override it?


Because computePlatformURIMap is too slow. What do you think of this implementation:

public class PlatformURIMapCache {

    private final static Logger LOG = Logger.getLogger(PlatformURIMapCache.class);
    
    private Map<String, Map<URI, URI>> cache = newHashMap();
    
    public Map<URI, URI> computePlatformURIMap(IJavaProject javaProject) {
        HashMap<URI, URI> hashMap = newHashMap(EcorePlugin.computePlatformURIMap());
        
        try {
            if (!javaProject.exists())
                return hashMap;
            
            IClasspathEntry[] classpath = javaProject.getResolvedClasspath(true);
            for (IClasspathEntry classPathEntry : classpath) {
                processClasspathEntry( hashMap, classPathEntry );
            }
        } catch (JavaModelException e) {
            LOG.error(e.getMessage(), e);
        }
        return hashMap;
    }

    protected void processClasspathEntry( HashMap<URI, URI> hashMap, IClasspathEntry classPathEntry ) {
        
        IPath path = classPathEntry.getPath();
        if (null == path || ! "jar".equals(path.getFileExtension())) {
            return;
        }
        
        try {
            final File file = path.toFile();
            
            if (null == file || ! file.exists()) {
                return;
            }
            
            processJarFile(hashMap, file);
        } catch (IOException e) {
            LOG.error(e.getMessage(), e);
        }
    }

    protected void processJarFile( HashMap<URI, URI> hashMap, final File file ) throws IOException {
        String key = file.getAbsolutePath();
        
        Map<URI, URI> cached = cache.get(key);
        if (null == cached) {
            cached = mapFromJarFile(file);
            cache.put(key, cached);
        }
        
        hashMap.putAll(cached);
    }

    private Map<URI, URI> mapFromJarFile( File file ) throws IOException {
        JarFile jarFile = new JarFile(file);
        try {
            Manifest manifest = jarFile.getManifest();
            
            if (null == manifest) {
                return Collections.emptyMap();
            }
            
            String name = manifest.getMainAttributes().getValue("Bundle-SymbolicName");
            if (null == name) {
                return Collections.emptyMap();
            }
            
            name = stripSemicolon( name );
            if (EcorePlugin.getPlatformResourceMap().containsKey(name)) {
                return Collections.emptyMap();
            }
            
            String p = "archive:" + file.toURI() + "!/";
            URI uri = URI.createURI(p);
            final URI platformResourceKey = URI.createPlatformResourceURI(name + "/", false);
            final URI platformPluginKey = URI.createPlatformPluginURI(name + "/", false);

            
            Map<URI, URI> result = newHashMap();
            result.put(platformResourceKey, uri);
            result.put(platformPluginKey, uri);
        
            return result;
        } finally {
            jarFile.close();
        }
    }

    protected String stripSemicolon( String name ) {
        final int indexOf = name.indexOf(';');
        if (indexOf > 0)
            name = name.substring(0, indexOf);
        return name;
    }

}


Now XtextResourceSetProvider could inject the singleton and call it's computePlatformURIMap().

Regards,

A. Digulla
Re: Odd error using IJvmTypeProvider [message #916895 is a reply to message #916892] Wed, 19 September 2012 16:00 Go to previous messageGo to next message
Aaron Digulla is currently offline Aaron DigullaFriend
Messages: 258
Registered: July 2009
Location: Switzerland
Senior Member
I'm not too happy with file.getAbsolutePath() as the cache key but Path has a very bad hashCode() implementation and I'm not sure how to convert IPath to a useful string which I could use as a key.
Re: Odd error using IJvmTypeProvider [message #916907 is a reply to message #916892] Wed, 19 September 2012 16:11 Go to previous messageGo to next message
Sebastian Zarnekow is currently offline Sebastian ZarnekowFriend
Messages: 3118
Registered: July 2009
Senior Member
So you confirm that the other stuff is working properly if you use the
XtextResourceSetProvider instead of the SimpleResourceSetProvider?

Feel free to bind you own XtextResourceSetProvider if the default impl
is too slow in your environment.

Any kind of caching mechanism needs a means to update the cache, e.g. if
you open / close projects or add more items to the classpath. Thus I
doubt that you sketched impl works for the common case.

Regards,
Sebastian
--
Looking for professional support for Xtext, Xtend or Eclipse Modeling?
Go visit: http://xtext.itemis.com

Am 19.09.12 17:57, schrieb Aaron Digulla:
> Sebastian Zarnekow wrote on Wed, 19 September 2012 16:59
>> Make sure you use the XtextResourceSetProvider. It's configured in
>> org.eclipse.xtext.ui.DefaultUiModule.bindIResourceSetProvider().
>>
>> Why did you override it?
>
>
> Because computePlatformURIMap is too slow. What do you think of this
> implementation:
>
>
> public class PlatformURIMapCache {
>
> private final static Logger LOG =
> Logger.getLogger(PlatformURIMapCache.class);
> private Map<String, Map<URI, URI>> cache = newHashMap();
> public Map<URI, URI> computePlatformURIMap(IJavaProject javaProject) {
> HashMap<URI, URI> hashMap =
> newHashMap(EcorePlugin.computePlatformURIMap());
> try {
> if (!javaProject.exists())
> return hashMap;
> IClasspathEntry[] classpath =
> javaProject.getResolvedClasspath(true);
> for (IClasspathEntry classPathEntry : classpath) {
> processClasspathEntry( hashMap, classPathEntry );
> }
> } catch (JavaModelException e) {
> LOG.error(e.getMessage(), e);
> }
> return hashMap;
> }
>
> protected void processClasspathEntry( HashMap<URI, URI> hashMap,
> IClasspathEntry classPathEntry ) {
> IPath path = classPathEntry.getPath();
> if (null == path || ! "jar".equals(path.getFileExtension())) {
> return;
> }
> try {
> final File file = path.toFile();
> if (null == file || ! file.exists()) {
> return;
> }
> processJarFile(hashMap, file);
> } catch (IOException e) {
> LOG.error(e.getMessage(), e);
> }
> }
>
> protected void processJarFile( HashMap<URI, URI> hashMap, final File
> file ) throws IOException {
> String key = file.getAbsolutePath();
> Map<URI, URI> cached = cache.get(key);
> if (null == cached) {
> cached = mapFromJarFile(file);
> cache.put(key, cached);
> }
> hashMap.putAll(cached);
> }
>
> private Map<URI, URI> mapFromJarFile( File file ) throws IOException {
> JarFile jarFile = new JarFile(file);
> try {
> Manifest manifest = jarFile.getManifest();
> if (null == manifest) {
> return Collections.emptyMap();
> }
> String name =
> manifest.getMainAttributes().getValue("Bundle-SymbolicName");
> if (null == name) {
> return Collections.emptyMap();
> }
> name = stripSemicolon( name );
> if (EcorePlugin.getPlatformResourceMap().containsKey(name)) {
> return Collections.emptyMap();
> }
> String p = "archive:" + file.toURI() + "!/";
> URI uri = URI.createURI(p);
> final URI platformResourceKey =
> URI.createPlatformResourceURI(name + "/", false);
> final URI platformPluginKey =
> URI.createPlatformPluginURI(name + "/", false);
>
> Map<URI, URI> result = newHashMap();
> result.put(platformResourceKey, uri);
> result.put(platformPluginKey, uri);
> return result;
> } finally {
> jarFile.close();
> }
> }
>
> protected String stripSemicolon( String name ) {
> final int indexOf = name.indexOf(';');
> if (indexOf > 0)
> name = name.substring(0, indexOf);
> return name;
> }
>
> }
>
>
> Now XtextResourceSetProvider could inject the singleton and call it's
> computePlatformURIMap().
>
> Regards,
>
> A. Digulla
Re: Odd error using IJvmTypeProvider [message #917528 is a reply to message #916907] Thu, 20 September 2012 07:04 Go to previous messageGo to next message
Aaron Digulla is currently offline Aaron DigullaFriend
Messages: 258
Registered: July 2009
Location: Switzerland
Senior Member
Sebastian Zarnekow wrote on Wed, 19 September 2012 18:11
So you confirm that the other stuff is working properly if you use the
XtextResourceSetProvider instead of the SimpleResourceSetProvider?


Probably. I didn't test it explicitly but it works with my new FastXtextResourceSetProvider, so it's safe to assume your version works as well. Thanks for helping me find the issue so fast.

Sebastian Zarnekow wrote on Wed, 19 September 2012 18:11

Feel free to bind you own XtextResourceSetProvider if the default impl
is too slow in your environment.

Any kind of caching mechanism needs a means to update the cache, e.g. if
you open / close projects or add more items to the classpath. Thus I
doubt that you sketched impl works for the common case.


Please look at the code. You will see that the part which handles the workspace hasn't changed. I'm just caching information from JAR files. Maybe I should generate a key which includes file name, last modification and size but JARs on the classpath will rarely change.

Regards,

A. Digulla

[Updated on: Thu, 20 September 2012 07:16]

Report message to a moderator

Re: Odd error using IJvmTypeProvider [message #917538 is a reply to message #917528] Thu, 20 September 2012 07:13 Go to previous messageGo to next message
Aaron Digulla is currently offline Aaron DigullaFriend
Messages: 258
Registered: July 2009
Location: Switzerland
Senior Member
I created a gist: https://gist.github.com/3754363
Re: Odd error using IJvmTypeProvider [message #917567 is a reply to message #917528] Thu, 20 September 2012 07:50 Go to previous messageGo to next message
Sebastian Zarnekow is currently offline Sebastian ZarnekowFriend
Messages: 3118
Registered: July 2009
Senior Member
Sorry, I missed that at a first glance, though I like the improved
version in the gist better. Can you provide some rough numbers on the
improved time to create a resource set for an average project of yours?

Would you be willing to contribute the code (the gist does not contain
license information)?

Regards,
Sebastian
--
Looking for professional support for Xtext, Xtend or Eclipse Modeling?
Go visit: http://xtext.itemis.com

Am 20.09.12 09:04, schrieb Aaron Digulla:
> Sebastian Zarnekow wrote on Wed, 19 September 2012 18:11
>> So you confirm that the other stuff is working properly if you use the
>> XtextResourceSetProvider instead of the SimpleResourceSetProvider?
>
>
>
> Sebastian Zarnekow wrote on Wed, 19 September 2012 18:11
>> Feel free to bind you own XtextResourceSetProvider if the default impl
>> is too slow in your environment.
>>
>> Any kind of caching mechanism needs a means to update the cache, e.g.
>> if you open / close projects or add more items to the classpath. Thus
>> I doubt that you sketched impl works for the common case.
>
>
> Please look at the code. You will see that the part which handles the
> workspace hasn't changed. I'm just caching information from JAR files.
> Maybe I should generate a key which includes file name, last
> modification and size but JARs on the classpath will rarely change.
>
> Regards,
>
> A. Digulla
Re: Odd error using IJvmTypeProvider [message #917698 is a reply to message #917567] Thu, 20 September 2012 10:43 Go to previous messageGo to next message
Aaron Digulla is currently offline Aaron DigullaFriend
Messages: 258
Registered: July 2009
Location: Switzerland
Senior Member
Sebastian Zarnekow wrote on Thu, 20 September 2012 09:50
Sorry, I missed that at a first glance, though I like the improved
version in the gist better. Can you provide some rough numbers on the
improved time to create a resource set for an average project of yours?


I timed the calls. To my surprise, they weren't that high to begin with. In a big project (188 JARs), the computation takes around 100ms. My patch about halves this figure.

Most of that is probably spent in EcorePlugin.computePlatformURIMap() but I have no idea how to cache that :-/

I also noticed that something expensive happens afterwards:

1. I double click a DSL file
2. I see the log output from my timing code
3. Noticeable lag for 100 to 2000ms. What happens here?
4. Editor opens

Sebastian Zarnekow wrote on Thu, 20 September 2012 09:50
Would you be willing to contribute the code (the gist does not contain
license information)?


I'd love to see this code included. What do you need me to do?

Regards,

A. Digulla
Re: Odd error using IJvmTypeProvider [message #917748 is a reply to message #917698] Thu, 20 September 2012 11:48 Go to previous messageGo to next message
Sebastian Zarnekow is currently offline Sebastian ZarnekowFriend
Messages: 3118
Registered: July 2009
Senior Member
Hi Aaron,

if a reasonable sized projects uses 100ms in that routine but opening a
file (for the second time?) takes 2 secs, I'm reluctant to include the
code. It introduces additional complexity (even though only minor) but
does not address a serious issue. However, please file a ticket and
attach the code. We'll take some measurements with that one and decide
how to continue with the effort to make an editor snappier.

Did you try to profile the open-action?

Regards,
Sebastian
--
Looking for professional support for Xtext, Xtend or Eclipse Modeling?
Go visit: http://xtext.itemis.com

Am 20.09.12 12:43, schrieb Aaron Digulla:
> Sebastian Zarnekow wrote on Thu, 20 September 2012 09:50
>> Sorry, I missed that at a first glance, though I like the improved
>> version in the gist better. Can you provide some rough numbers on the
>> improved time to create a resource set for an average project of yours?
>
>
> I timed the calls. To my surprise, they weren't that high to begin with.
> In a big project (188 JARs), the computation takes around 100ms. My
> patch about halves this figure.
>
> Most of that is probably spent in EcorePlugin.computePlatformURIMap()
> but I have no idea how to cache that :-/
>
> I also noticed that something expensive happens afterwards:
>
> 1. I double click a DSL file
> 2. I see the log output from my timing code
> 3. Noticeable lag for 100 to 2000ms. What happens here?
> 4. Editor opens
>
> Sebastian Zarnekow wrote on Thu, 20 September 2012 09:50
>> Would you be willing to contribute the code (the gist does not contain
>> license information)?
>
>
> I'd love to see this code included. What do you need me to do?
>
> Regards,
>
> A. Digulla
Re: Odd error using IJvmTypeProvider [message #918658 is a reply to message #917748] Fri, 21 September 2012 08:05 Go to previous messageGo to next message
Aaron Digulla is currently offline Aaron DigullaFriend
Messages: 258
Registered: July 2009
Location: Switzerland
Senior Member
Sebastian Zarnekow wrote on Thu, 20 September 2012 13:48
Hi Aaron,

if a reasonable sized projects uses 100ms in that routine but opening a
file (for the second time?) takes 2 secs, I'm reluctant to include the
code. It introduces additional complexity (even though only minor) but
does not address a serious issue. However, please file a ticket and
attach the code. We'll take some measurements with that one and decide
how to continue with the effort to make an editor snappier.


I was surprised as well. After thinking about it for some time, I realized why the timing isn't accurate: I have 16GB of RAM with 4GB file buffer. So the 188 JARs are in memory. When they have to be loaded from disk, that can take several seconds.

After flushing the caches, I get these timings:

1.37s for loading 33 JARs (small project) and 9.62s for 188 JARs (initial load for both cases). This number doesn't change much with the old code. With my patch, the subsequent invocations go down to 50-100ms in both cases.

So I suggest that you include the patch.

Yesterday, I created a version of the cache which duplicates the code from EcorePlugin.computePlatformURIMap() but in my case, it doesn't return many URLs and it uses information which is cached by Eclipse, so the speedup of that code probably isn't that high anyway.

But since it was quite a bit of work to rearrange the EMF code, I've added a switch that disables the cache by default. Maybe it makes a difference for large workspaces but I have only 7 plugins in my workspace, so I can't really tell.

Bug: https://bugs.eclipse.org/bugs/show_bug.cgi?id=390074

I suggest to apply this for the SR1 or SR2. For people with tight memory and large workspaces, this patch is a huge time-saver.

Regards,

A. Digulla

[Updated on: Fri, 21 September 2012 08:06]

Report message to a moderator

Re: Odd error using IJvmTypeProvider [message #918735 is a reply to message #918658] Fri, 21 September 2012 09:39 Go to previous messageGo to next message
Sebastian Zarnekow is currently offline Sebastian ZarnekowFriend
Messages: 3118
Registered: July 2009
Senior Member
Thanks for the patch. The numbers look good to me.

Regards,
Sebastian
--
Looking for professional support for Xtext, Xtend or Eclipse Modeling?
Go visit: http://xtext.itemis.com

Am 21.09.12 10:05, schrieb Aaron Digulla:
> Sebastian Zarnekow wrote on Thu, 20 September 2012 13:48
>> Hi Aaron,
>>
>> if a reasonable sized projects uses 100ms in that routine but opening
>> a file (for the second time?) takes 2 secs, I'm reluctant to include
>> the code. It introduces additional complexity (even though only minor)
>> but does not address a serious issue. However, please file a ticket
>> and attach the code. We'll take some measurements with that one and
>> decide how to continue with the effort to make an editor snappier.
>
>
> I was surprised as well. After thinking about it for some time, I
> realized why the timing isn't accurate: I have 16GB of RAM with 4GB file
> buffer. So the 188 JARs are in memory. When they have to be loaded from
> disk, that can take several seconds.
>
> After flushing the caches, I get these timings:
>
> 1.37s for loading 33 JARs (small project) and 9.62s for 188 JARs
> (initial load for both cases). This number doesn't change much when the
> fast cache is disabled. With my patch, the subsequent invocations go
> down to 50-100ms in both cases.
>
> So I suggest that you include the patch.
>
> Yesterday, I created a version of the cache which duplicates the code
> from EcorePlugin.computePlatformURIMap() but in my case, it doesn't
> return many URLs and it uses information which is cached by Eclipse, so
> the speedup of that code probably isn't that high anyway.
>
> But since it was quite a bit of work to rearrange the EMF code, I've
> added a switch that disables the cache by default. Maybe it makes a
> difference for large workspaces but I have only 7 plugins in my
> workspace, so I can't really tell.
>
> Bug: https://bugs.eclipse.org/bugs/show_bug.cgi?id=390074
>
> I suggest to apply this for the SR1 or SR2. For people with tight memory
> and large workspaces, this patch is a huge time-saver.
>
> Regards,
>
> A. Digulla
Re: Odd error using IJvmTypeProvider [message #918844 is a reply to message #918735] Fri, 21 September 2012 11:56 Go to previous message
Aaron Digulla is currently offline Aaron DigullaFriend
Messages: 258
Registered: July 2009
Location: Switzerland
Senior Member
If you need to test on Linux, use this command to flush all file system caches:

sync ; echo 3 > /proc/sys/vm/drop_caches
Previous Topic:I need both ImportUriGlobalScopeProvider and TypesAwareDefaultGlobalScopeProvider in my DSL
Next Topic:Problem with association between XExpression and logical container
Goto Forum:
  


Current Time: Thu Apr 25 22:35:28 GMT 2024

Powered by FUDForum. Page generated in 0.04498 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top