Eclipse Community Forums: EMF "Technology" (Ecore Tools, EMFatic, etc) » [MWE] WorkflowComponent API should support some Resource Cleanup mechanism

Home » Modeling » EMF "Technology" (Ecore Tools, EMFatic, etc) » [MWE] WorkflowComponent API should support some Resource Cleanup mechanism

Show: Today's Messages :: Show Polls :: Message Navigator

[MWE] WorkflowComponent API should support some Resource Cleanup mechanism [message #526244]

Fri, 09 April 2010 11:01

Jochen Schmich

Messages: 29
Registered: July 2009

Junior Member

Hi there,

we've been using MWE for quite a while now in an OSGi-Environment - directly from inside the Eclipse Workbench (Rational Software Architect / -Modeler to be more specific).

We developed a mechanism that allows the user to invoke Workflows in a manner that he is used to - from the context menu of the selected Model Element shown in the Navigator tree. We also use this for other things like directly invoking Xtend scripts on those model elements and so on... but the main purpose is running MWE Workflows.

After running a Generator workflow several times (which occurs while developing those rather often) the heap size will grow and grow until an OutOfMemoryError is thrown or the Workbench just doesn't respond that fast anymore Smile

.

This is caused by the way MWE and many of the Xpand / Xtend-components are designed. Well it's designed as a console based tool and doesn't care too much about the traits that long-running applications cares for - like trying to prevent Memory leaks.

We just can't do it any other way since we need to allow direct model modifications that may be tracked by the EMF Transcation System, too.

We've analyzed a lot of Heap Dumps and patched the source code of some Xpand / Xtend-components in order to prevent those leaks by patching the XSD Typesystem for example (it holds a static HashMap at some point).

We placed a shutdown()-Hook-method in the necessary places and cleared the static HashMaps & Lists unloaded EMF-Resources that have been created in-memory and are still held by the EMF Transaction Manager after the Workflow is finished and so on.

Would it be possible to a) extend the Workflow-API by a method (like "dispose()" of SWT-Widgets) and b) implement basic routines for the existing WorkflowComponents? If it was I can try to be more specific and provide the names and locations all of the Source Code we had to patch.

Report message to a moderator

Re: [MWE] WorkflowComponent API should support some Resource Cleanup mechanism [message #526248 is a reply to message #526244]

Fri, 09 April 2010 11:14

MaximeLecourt

Messages: 108
Registered: February 2010
Location: France

Senior Member

Even if this is the MWE forum, if you are writing about the Xpand/Xtend project, you should do it in the M2T forum.

I've found out that I get more precise answers in that forum, even with questions about the MWE.

One day I shall master M2T, but that day has yet to come...

Report message to a moderator

Re: [MWE] WorkflowComponent API should support some Resource Cleanup mechanism [message #526256 is a reply to message #526248]

Fri, 09 April 2010 11:50

Jochen Schmich

Messages: 29
Registered: July 2009

Junior Member

OK, thanks for the advice. I actually pondered a while about posting this in the Xpand-Forum - I finally posted it here because MWE is currently revamped and my issue would mean an API change in consequence.
How the changed API would be used in the existing components should be discussed in the Xpand forum.
Maybe a moderator could move the topic to the Xpand forum for me if it is considered more of an Xpand-related issue.

Report message to a moderator

Re: [MWE] WorkflowComponent API should support some Resource Cleanup mechanism [message #526888 is a reply to message #526244]

Tue, 13 April 2010 11:52

Sven Efftinge

Messages: 1823
Registered: July 2009

Senior Member

Hi Jochen,

Jochen Schmich schrieb:
> Hi there,
>
> we've been using MWE for quite a while now in an OSGi-Environment -
> directly from inside the Eclipse Workbench (Rational Software Architect
> / -Modeler to be more specific).
>
> We developed a mechanism that allows the user to invoke Workflows in a
> manner that he is used to - from the context menu of the selected Model
> Element shown in the Navigator tree. We also use this for other things
> like directly invoking Xtend scripts on those model elements and so
> on... but the main purpose is running MWE Workflows.
>
> After running a Generator workflow several times (which occurs while
> developing those rather often) the heap size will grow and grow until an
> OutOfMemoryError is thrown or the Workbench just doesn't respond that
> fast anymore :).
>
> This is caused by the way MWE and many of the Xpand / Xtend-components
> are designed. Well it's designed as a console based tool and doesn't
> care too much about the traits that long-running applications cares for
> - like trying to prevent Memory leaks.

There shouldn't be any memory leaks at all.
That is cached state should be associated to some other non static
state. And we shouldn't have any mutable static state.

I know that there are one or two violations of this, which sneaked into
the code for historical reasons (we didn't review contributions good
enough at some point) but the main components and framework of MWE
shouldn't have any static state.

The same applies to Xpand/Xtend.
Please open bugzillas for each occurence of this.

That said the code caches a lot of stuff, but if you start with a new
Workflow each time and release any references to the old state it should
be garbage collected. In Xpand/Xtend it's the ExecutionContext which
holds any cached state.

> We just can't do it any other way since we need to allow direct model
> modifications that may be tracked by the EMF Transcation System, too.
>
> We've analyzed a lot of Heap Dumps and patched the source code of some
> Xpand / Xtend-components in order to prevent those leaks by patching the
> XSD Typesystem for example (it holds a static HashMap at some point).

Please file specific bugzillas.

>
> We placed a shutdown()-Hook-method in the necessary places and cleared
> the static HashMaps & Lists unloaded EMF-Resources that have been
> created in-memory and are still held by the EMF Transaction Manager
> after the Workflow is finished and so on.
>
> Would it be possible to a) extend the Workflow-API by a method (like
> "dispose()" of SWT-Widgets) and b) implement basic routines for the
> existing WorkflowComponents? If it was I can try to be more specific and
> provide the names and locations all of the Source Code we had to patch.

Yes, that would be good.

For MWE2 the IWorkflowComponent interface has an additional call back
method 'public void postInvoke()' which is called after all components
have been invoked.

Regards,
Sven

--
Need professional support for Xtext and EMF?
Go to: http://xtext.itemis.com
Twitter : @svenefftinge
Blog : blog.efftinge.de

Report message to a moderator

Re: [MWE] WorkflowComponent API should support some Resource Cleanup mechanism [message #526994 is a reply to message #526888]

Tue, 13 April 2010 16:15

Jochen Schmich

Messages: 29
Registered: July 2009

Junior Member

Thanks Sven.

Good to hear that. I'll post some bugs as soon as I've assembled all the patches we applied.

I'll also post some links here upon creation of those bugs.

Sorry for not being specific about those bugs at first...

Kind regards,
Jochen

Report message to a moderator

Re: [MWE] WorkflowComponent API should support some Resource Cleanup mechanism [message #529368 is a reply to message #526888]

Sat, 24 April 2010 08:24

Jochen Schmich

Messages: 29
Registered: July 2009

Junior Member

OK, posted some bugs:

310359: XSD TypeSystem causes memory leak in XSDMetaModel

https://bugs.eclipse.org/bugs/show_bug.cgi?id=310359

310360: A WorkflowComponent that disposes formerly created eResources should be introduced

https://bugs.eclipse.org/bugs/show_bug.cgi?id=310360

310361: Some Stdlib-Extensions cause memory leaks

https://bugs.eclipse.org/bugs/show_bug.cgi?id=310361

Kind regards,
Jochen

Report message to a moderator