Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » ATL » [EMFTVM] Reduce initialization time
[EMFTVM] Reduce initialization time [message #1271530] Sat, 15 March 2014 11:55 Go to next message
EtienneB Mising name is currently offline EtienneB Mising name
Messages: 35
Registered: June 2011
Member
Hi, EMFTVM users and developpers!

I would like to ask some questions about performance improvement of model transformations with EMFTVM.

My experience is that the transformation is very fast once it is launched, however it seems to me that the initialization time is quite long... I tried to profile the Java code I have to launch the transformation but I did not find a good explanation using this technique (with jvmmonitor).

I also tried launching several time the same transformation and it seems that the initialization of the second launch is much faster (even though I am using a different ExecEnv object).

I was thinking to do as much as possible of the transformation initialization at initialization time of my tool, but I was wondering if you could give me some information about which initialization steps could be more time consuming than others.

For instance, the ATL/EMFTVM wiki says: "The advantage of reusing an ExecEnv is that you save on loading time, and the JIT has already warmed up."

Is it possible to "warm up" the transformation (or just the JIT part) without runing completly the transformation? Is this "warm up" specific to an ExecEnv object? Could the ExecEnvPool be useful with respect to the initialization time?

Thanks in advance for your ideas or suggestions,
Etienne.
Re: [EMFTVM] Reduce initialization time [message #1272188 is a reply to message #1271530] Mon, 17 March 2014 09:47 Go to previous messageGo to next message
Dennis Wagelaar is currently offline Dennis Wagelaar
Messages: 166
Registered: September 2012
Senior Member

If you're not even reusing the ExecEnv, but already notice speedup on the second run, your initial slow-down is most likely caused by (meta-)model loading. Using an ExecEnvPool allows you to cache transformation module bytecode and metamodels. You can pre-load an ExecEnv instance by getting an ExecEnv instance from your pool, and then immediately return it. The JIT will not yet have worked, but the linker and bytecoder verifier will not be invoked again.

Regarding model loading: can you check how much time is spent for model loading? Do you load models from URI or InputStream? If InputStream, do you wrap the original InputStream in a BufferedInputStream? Can you tweak the XML parser settings for your ResourceFactory (see EMF forums)?


Cheers,
Dennis
Re: [EMFTVM] Reduce initialization time [message #1277874 is a reply to message #1272188] Wed, 26 March 2014 15:28 Go to previous messageGo to next message
EtienneB Mising name is currently offline EtienneB Mising name
Messages: 35
Registered: June 2011
Member
Hi Dennis,

thanks a lot for your answer, it took me some time before to answer because I wanted to experiment a bit the ExecEnv option you mentionned:

Quote:

Using an ExecEnvPool allows you to cache transformation module bytecode and metamodels.


I tried it. Initially, I did not go for this solution because I do not always have the same set of transformation modules, and in some cases I need to load modules dynamically. In order to see if it reduces the init time, I gave a try to this option. Indeed, the init time is significantly reduced (or at least transferred to the initialisation phase of my tool).

However, I also wanted to know if this solution is thread-safe?

Another question I have is: is it thread safe if I load metamodel using the pool but load modules using the exec env? Like this for instance:

// pool init, called at init time of my tool
Metamodel metamodel = EmftvmFactory.eINSTANCE.createMetamodel();
metamodel.setResource(EcorePackage.eINSTANCE.eResource());
// pool (first used bellow) is an attribute of the class; the object is a singleton
pool.registerMetaModel("ECORE", metamodel);



// use pool, potentially multi-threaded
ExecEnv env = pool.getExecEnv(); // a new env object is created in each thread using this call??
ModuleResolver mr = new DefaultModuleResolver("platform:/plugin/my.plugin.id/transformations/", new ResourceSetImpl());
TimingData td = new TimingData();
env.loadModule(mr, "Module");
td.finishLoading();
env.run(td);

pool.returnExecEnv(env);



Using such a setting, launching several transfo in different threads, I experienced difficulties (typically some memory over consumption).

Regarding model loading, I load models using resources obtained from an xtext-based parser. Not sure if it answers your questions.
Best regards and thanks for your help!
Etienne.
Re: [EMFTVM] Reduce initialization time [message #1278747 is a reply to message #1277874] Thu, 27 March 2014 19:57 Go to previous messageGo to next message
Dennis Wagelaar is currently offline Dennis Wagelaar
Messages: 166
Registered: September 2012
Senior Member

EtienneB Mising name wrote on Wed, 26 March 2014 16:28
Hi Dennis,

thanks a lot for your answer, it took me some time before to answer because I wanted to experiment a bit the ExecEnv option you mentionned:

Quote:

Using an ExecEnvPool allows you to cache transformation module bytecode and metamodels.


I tried it. Initially, I did not go for this solution because I do not always have the same set of transformation modules, and in some cases I need to load modules dynamically. In order to see if it reduces the init time, I gave a try to this option. Indeed, the init time is significantly reduced (or at least transferred to the initialisation phase of my tool).

However, I also wanted to know if this solution is thread-safe?


The ExecEnvPool is thread-safe if used right: after use, you can return an ExecEnv to the pool, after which another thread may check it out of the pool. Of course, you may not continue using an ExecEnv after you've returned it to the pool!

You should also not load any additional modules into a single ExecEnv checked out from a pool: the pool will become polluted with different kinds of ExecEnvs, and any ExecEnvs checked out from the pool may have different behaviours!

The way to use an ExecEnvPool in this context would be to instantiate a separate ExecEnvPool for each transformation configuration you want to support.


Cheers,
Dennis
Re: [EMFTVM] Reduce initialization time [message #1278752 is a reply to message #1277874] Thu, 27 March 2014 20:07 Go to previous messageGo to next message
Dennis Wagelaar is currently offline Dennis Wagelaar
Messages: 166
Registered: September 2012
Senior Member

EtienneB Mising name wrote on Wed, 26 March 2014 16:28

Another question I have is: is it thread safe if I load metamodel using the pool but load modules using the exec env? Like this for instance:

// pool init, called at init time of my tool
Metamodel metamodel = EmftvmFactory.eINSTANCE.createMetamodel();
metamodel.setResource(EcorePackage.eINSTANCE.eResource());
// pool (first used bellow) is an attribute of the class; the object is a singleton
pool.registerMetaModel("ECORE", metamodel);



// use pool, potentially multi-threaded
ExecEnv env = pool.getExecEnv(); // a new env object is created in each thread using this call??
ModuleResolver mr = new DefaultModuleResolver("platform:/plugin/my.plugin.id/transformations/", new ResourceSetImpl());
TimingData td = new TimingData();
env.loadModule(mr, "Module");
td.finishLoading();
env.run(td);

pool.returnExecEnv(env);



Using such a setting, launching several transfo in different threads, I experienced difficulties (typically some memory over consumption).


This scenario will not work, because you're loading modules using different ResourceSets into the same instance of an ExecEnv each time. After you call pool.returnExecEnv(env), the pool may return that env again when you call pool.getExecEnv(). The ExecEnvPool achieves its performance gain by reusing ExecEnv instances. You'll find that the ExecEnvPool methods are synchronized, such that the pool acts as a monitor for shared resources.

As said in my previous post, the module loading should be done on the ExecEnvPool itself. Create multiple pools if you need to support different ExecEnv configurations. All ExecEnvs in a single pool should have the same bytecode and metamodels loaded into them, such that they behave the same.


Cheers,
Dennis
Re: [EMFTVM] Reduce initialization time [message #1278756 is a reply to message #1277874] Thu, 27 March 2014 20:11 Go to previous messageGo to next message
Dennis Wagelaar is currently offline Dennis Wagelaar
Messages: 166
Registered: September 2012
Senior Member

EtienneB Mising name wrote on Wed, 26 March 2014 16:28

Regarding model loading, I load models using resources obtained from an xtext-based parser. Not sure if it answers your questions.
Best regards and thanks for your help!
Etienne.


If you're loading xText resources from URI, you probably cannot speed things up. The xText parser (ANTLR generated?) can be slow, depending on the complexity of your grammar. Not much you can do about that... ATL also uses an ANTLR-generated parser, which can take several seconds to parse a large file.

If you're loading the resources from an InputStream, then make sure you wrap the InputStream in a BufferedInputStream before passing it to xText.


Cheers,
Dennis
Re: [EMFTVM] Reduce initialization time [message #1279105 is a reply to message #1278756] Fri, 28 March 2014 08:32 Go to previous message
EtienneB Mising name is currently offline EtienneB Mising name
Messages: 35
Registered: June 2011
Member
Hi Dennis,

thanks for your help. I have very little controle on the way input resources are loaded (I use a third-party tool) but when I launch the transformation, they have already been parsed and loaded.

For the use of exec env pools, I figured out the way I use them was incorrect, I just wanted to check with you to make sure it was a mistake (and to understand why). Now I understand and I will go for your solution; it will ask me to make some major changes in my tool but I reached a point where I beleive it is worth it.

Thanks again, I will come back to you if I see some unexpected results when using the correct solution.
Best regards,
Etienne.

[Updated on: Fri, 28 March 2014 12:36]

Report message to a moderator

Previous Topic:[EMFTVM] Profiling now available
Next Topic:Reflectively obtaining children of an element in ATL
Goto Forum:
  


Current Time: Fri Oct 31 09:19:49 GMT 2014

Powered by FUDForum. Page generated in 0.03490 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software