Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » TMF (Xtext) » Increasing memory consumption
Increasing memory consumption [message #658727] Wed, 09 March 2011 15:53 Go to next message
Roland S. is currently offline Roland S.Friend
Messages: 36
Registered: September 2009
Member
Hi,

I analyzed the memory consumption of Xtext when using our grammar. Each time the projects in my workspace are build, the memory allocation of the parse tree increases. In XtextBuilder.java (line 163) the ResourceSet is cleared, but I could not find code that deletes the parse tree. Could this be the reason for the high memory consumption or is the tree deleted somewhere?

Regards,
Roland
Re: Increasing memory consumption [message #658881 is a reply to message #658727] Thu, 10 March 2011 09:45 Go to previous messageGo to next message
Sebastian Zarnekow is currently offline Sebastian ZarnekowFriend
Messages: 3118
Registered: July 2009
Senior Member
Hi Roland,

the parse tree is part of the resource (from a java memory model point
of view). The garbage collector will reclaim the used memory when the
resourceset is cleared.

Regards,
Sebastian
--
Need professional support for Eclipse Modeling?
Go visit: http://xtext.itemis.com

Am 09.03.11 16:53, schrieb Roland S.:
> Hi,
>
> I analyzed the memory consumption of Xtext when using our grammar. Each
> time the projects in my workspace are build, the memory allocation of
> the parse tree increases. In XtextBuilder.java (line 163) the
> ResourceSet is cleared, but I could not find code that deletes the parse
> tree. Could this be the reason for the high memory consumption or is the
> tree deleted somewhere?
>
> Regards,
> Roland
Re: Increasing memory consumption [message #658886 is a reply to message #658727] Thu, 10 March 2011 10:14 Go to previous messageGo to next message
Roland S. is currently offline Roland S.Friend
Messages: 36
Registered: September 2009
Member
Hi Sebastian,

shouldn't the unload() method inside the LazyLinkingResource be called prior to clearing the resourceset? I think that the garbage collector does not recognize that the resource can be removed from memory.

Regards,
Roland
Re: Increasing memory consumption [message #658892 is a reply to message #658886] Thu, 10 March 2011 10:25 Go to previous messageGo to next message
Ed Willink is currently offline Ed WillinkFriend
Messages: 7655
Registered: July 2009
Senior Member
Hi Sebastian

You might want to check with Eike.

I recall that at EclipseCon 2010 Eike was observing that it required
considerable effort to release a Resource.

Regards

Ed Willink

On 10/03/2011 10:14, Roland S. wrote:
> Hi Sebastian,
>
> shouldn't the unload() method inside the LazyLinkingResource be called
> prior to clearing the resourceset? I think that the garbage collector
> does not recognize that the resource can be removed from memory.
>
> Regards,
> Roland
Re: Increasing memory consumption [message #658907 is a reply to message #658886] Thu, 10 March 2011 11:09 Go to previous messageGo to next message
Sebastian Zarnekow is currently offline Sebastian ZarnekowFriend
Messages: 3118
Registered: July 2009
Senior Member
Hi Roland,

since the complete resource sets resources-list is cleared, there is no
need to do a costly unload prior to the invocation of #clear. However,
you have to make sure that non of your services holds a strong reference
to one of the resources or their contents.

Regards,
Sebastian
--
Need professional support for Eclipse Modeling?
Go visit: http://xtext.itemis.com

Am 10.03.11 11:14, schrieb Roland S.:
> Hi Sebastian,
>
> shouldn't the unload() method inside the LazyLinkingResource be called
> prior to clearing the resourceset? I think that the garbage collector
> does not recognize that the resource can be removed from memory.
>
> Regards,
> Roland
Re: Increasing memory consumption [message #658923 is a reply to message #658907] Thu, 10 March 2011 13:12 Go to previous messageGo to next message
Michel Simeon is currently offline Michel SimeonFriend
Messages: 130
Registered: December 2009
Senior Member
I am not sure I fully understand your discussion, but I suspect I have the same problem.

I am accessing my model "live" using the XText documentation:

private static XtextResource getXtextResource(IEditorPart editor){
IXtextDocument xtextDocument = ( (XtextEditor) editor).getDocument();
XtextResource resource = xtextDocument.readOnly(
new IUnitOfWork<XtextResource, XtextResource>() {
public XtextResource exec(XtextResource state) throws Exception {
return state;
}
});
return resource;
}

Within my main model processing method I call the above to get the resource, use an IResourceValidator to find out if there are errors, and then access the model with:
Model model = (Model) resource.getContents().get(0);

Is it correct that the garbage collector will automatically reclaim memory on exit of my main model processing method ?

Thanks
MS
Re: Increasing memory consumption [message #659135 is a reply to message #658923] Fri, 11 March 2011 08:44 Go to previous messageGo to next message
Jan Koehnlein is currently offline Jan KoehnleinFriend
Messages: 760
Registered: July 2009
Location: Hamburg
Senior Member
Sorry, but this is exactly how NOT to use this API :-)

We introduced the IUnitOfWork to mark a transaction on the resource.
Every access to the model elements should happen within its exec method.
You should never pass EObjects or Resources as a result of an
IUnitOfWork, because that opens the door for memory leaks and
inconsistency in your tooling, as the resource is free to change
completely if you don't hold the lock.

If you want to keep a pointer to a model element, store its URI and
resolve it in the same IUnitOfWork in which you access it.

Am 10.03.11 14:12, schrieb Michel Simeon:
> I am not sure I fully understand your discussion, but I suspect I have
> the same problem.
>
> I am accessing my model "live" using the XText documentation:
>
> private static XtextResource getXtextResource(IEditorPart editor){
> IXtextDocument xtextDocument = ( (XtextEditor) editor).getDocument();
> XtextResource resource = xtextDocument.readOnly(
> new IUnitOfWork<XtextResource, XtextResource>() {
> public XtextResource exec(XtextResource state) throws Exception {
> return state;
> }
> });
> return resource;
> }
>
> Within my main model processing method I call the above to get the
> resource, use an IResourceValidator to find out if there are errors, and
> then access the model with:
> Model model = (Model) resource.getContents().get(0);
>
> Is it correct that the garbage collector will automatically reclaim
> memory on exit of my main model processing method ?
>
> Thanks
> MS
>


--
Need professional support for Eclipse Modeling?
Go visit: http://xtext.itemis.com


---
Get professional support from the Xtext committers at www.typefox.io
Re: Increasing memory consumption [message #659171 is a reply to message #659135] Fri, 11 March 2011 10:53 Go to previous messageGo to next message
Michel Simeon is currently offline Michel SimeonFriend
Messages: 130
Registered: December 2009
Senior Member
Jan,

Thank you for your answer.

I have no problem getting to my model by having it returned directly using IUnitOfWork .

My problem is the following:
- I can get the active editor which is the one with my DSL model;
- I need to find out if there are errors in the editor; I was using the resource to get a IResourceValidator; what is the alternative ?
- I need the IProject where my editor file is located in order to send some output there; I was using resource.getURI() to get there; again, what is the alternative ?

Thanks again for helping
MS

======

I realized I could get the URI and the list of issues from the validator through successive use of IUnitOfWork :

URI uri = xtextDocument.readOnly(
new IUnitOfWork<URI, XtextResource>() {
public URI exec(XtextResource resource) throws Exception {
return resource.getURI();
}
});

List<Issue> issues = xtextDocument.readOnly(
new IUnitOfWork<List<Issue>, XtextResource>() {
public List<Issue> exec(XtextResource resource) throws Exception {
IResourceValidator validator = resource.getResourceServiceProvider().getResourceValidator() ;
return validator.validate(resource, CheckMode.ALL, null);
}
});

Is this the proper way to do it ?

But then you said: Every access to the model elements should happen within its exec method. Does it mean that I should not do :

Model model = xtextDocument.readOnly(
new IUnitOfWork<Model,XtextResource>() {
public Model exec(XtextResource resource) throws Exception {
return (Model)resource.getContents().get(0);
}
});

and then work from there ?

You wrote:
If you want to keep a pointer to a model element, store its URI and resolve it in the same IUnitOfWork in which you access it.

I am not sure I understand. Could you elaborate.

Thanks again
MS

[Updated on: Fri, 11 March 2011 14:13]

Report message to a moderator

Re: Increasing memory consumption [message #659209 is a reply to message #659171] Fri, 11 March 2011 14:31 Go to previous messageGo to next message
Jan Koehnlein is currently offline Jan KoehnleinFriend
Messages: 760
Registered: July 2009
Location: Hamburg
Senior Member
Am 11.03.11 11:53, schrieb Michel Simeon:
> Jan,
>
> Thank you for your answer.
>
> I have no problem getting to my model by having it returned directly
> using IUnitOfWork .

I just meant you should never pass a Resource or an EObject across the
borders of an IUnitOfWork because this is the root of inconsistencies
and memory leaks. URIs are OK.

> My problem is the following:
> - I can get the active editor which is the one with my DSL model;

Was that a question? If so, it depends on the context, e.g. from within
a handler you can call
org.eclipse.xtext.ui.editor.utils.EditorUtils.getActiveXtext Editor(ExecutionEvent)
on the execution event. Otherwise, PlatformUI is your friend.

> - I need to find out if there are errors in the editor; I was using the
> resource to get a IResourceValidator; what is the alternative ?

That will revalidate the model, which is not necessary as the model
should already be validated.
Errors in the editor are stored in its annotation model. You can access
it via
xtextEditor.getDocumentProvider().getAnnotationModel(editorI nput)
Helper methods in org.eclipse.xtext.ui.util.IssueUtil help you to deal
with annotations.
If you save, the annotations are converted into markers. IFile allows
you to access these.

> - I need the IProject where my editor file is located in order to send
> some output there; I was using resource.getURI() to get there; again,
> what is the alternative ?

As editors are usually editing IFiles, get the editorInput, cast it to
IFileEditorInput, get the IFile and its IProject. If you have the
resource URI, you can use the IURI2StorageMapper to find the
corresponding IFile.

> Thanks again for helping
> MS


--
Need professional support for Eclipse Modeling?
Go visit: http://xtext.itemis.com


---
Get professional support from the Xtext committers at www.typefox.io
Re: Increasing memory consumption [message #659298 is a reply to message #659209] Fri, 11 March 2011 21:58 Go to previous messageGo to next message
Michel Simeon is currently offline Michel SimeonFriend
Messages: 130
Registered: December 2009
Senior Member
[quote title=Jan Koehnlein wrote on Fri, 11 March 2011 09:31]Am 11.03.11 11:53, schrieb Michel Simeon:

I just meant you should never pass a Resource or an EObject across the
borders of an IUnitOfWork because this is the root of inconsistencies
and memory leaks. URIs are OK.
--------
Jan,

Can you confirm that if I run all the processing of my Model - i.e instanciating classes defined in various other packages - from within a call
xtextDocument.readOnly(
new IUnitOfWork ....
... alll my processing here ...
return something;
});

then it should be OK.

Thanks again for helping
MS


Re: Increasing memory consumption [message #659479 is a reply to message #659298] Mon, 14 March 2011 08:44 Go to previous messageGo to next message
Jan Koehnlein is currently offline Jan KoehnleinFriend
Messages: 760
Registered: July 2009
Location: Hamburg
Senior Member
As the name suggests, readOnly() is meant for read operations. As other
read operations could be executing concurrently, you should not change
the model (or other resources within the same resource set).

If you want to change the Xtext resource, you should use
IDocumentEditor.process (see
http://koehnlein.blogspot.com/2010/06/semantic-model-access- in-xtext.html)

The catching point is not to pass any EObjects / Resources /
ResourceSets beyond the IUnitOfWork, and not to hold any references to
these.

Am 11.03.11 22:58, schrieb Michel Simeon:
> [quote title=Jan Koehnlein wrote on Fri, 11 March 2011 09:31]Am 11.03.11
> 11:53, schrieb Michel Simeon:
>
> I just meant you should never pass a Resource or an EObject across the
> borders of an IUnitOfWork because this is the root of inconsistencies
> and memory leaks. URIs are OK.
> --------
> Jan,
>
> Can you confirm that if I run all the processing of my Model - i.e
> instanciating classes defined in various other packages - from within a
> call xtextDocument.readOnly(
> new IUnitOfWork ....
> ... alll my processing here ...
> return something;
> });
>
> then it should be OK.
>
> Thanks again for helping
> MS
>
>
>


--
Need professional support for Eclipse Modeling?
Go visit: http://xtext.itemis.com


---
Get professional support from the Xtext committers at www.typefox.io
Re: Increasing memory consumption [message #659487 is a reply to message #659479] Mon, 14 March 2011 09:20 Go to previous message
Michel Simeon is currently offline Michel SimeonFriend
Messages: 130
Registered: December 2009
Senior Member
Thanks again.

The blog article is most useful.

MS
Previous Topic:Xtext and LaunchConfigurationType
Next Topic:[Xtext] implement toString() with the ParseTreeConstructor
Goto Forum:
  


Current Time: Thu Apr 25 21:05:00 GMT 2024

Powered by FUDForum. Page generated in 0.04063 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top