Home » Modeling » ATL » [EMFTVM] Reduce initialization time
|
Re: [EMFTVM] Reduce initialization time [message #1272188 is a reply to message #1271530] |
Mon, 17 March 2014 09:47 |
|
If you're not even reusing the ExecEnv, but already notice speedup on the second run, your initial slow-down is most likely caused by (meta-)model loading. Using an ExecEnvPool allows you to cache transformation module bytecode and metamodels. You can pre-load an ExecEnv instance by getting an ExecEnv instance from your pool, and then immediately return it. The JIT will not yet have worked, but the linker and bytecoder verifier will not be invoked again.
Regarding model loading: can you check how much time is spent for model loading? Do you load models from URI or InputStream? If InputStream, do you wrap the original InputStream in a BufferedInputStream? Can you tweak the XML parser settings for your ResourceFactory (see EMF forums)?
Cheers,
Dennis
|
|
|
Re: [EMFTVM] Reduce initialization time [message #1277874 is a reply to message #1272188] |
Wed, 26 March 2014 15:28 |
EtienneB Mising name Messages: 35 Registered: June 2011 |
Member |
|
|
Hi Dennis,
thanks a lot for your answer, it took me some time before to answer because I wanted to experiment a bit the ExecEnv option you mentionned:
Quote:
Using an ExecEnvPool allows you to cache transformation module bytecode and metamodels.
I tried it. Initially, I did not go for this solution because I do not always have the same set of transformation modules, and in some cases I need to load modules dynamically. In order to see if it reduces the init time, I gave a try to this option. Indeed, the init time is significantly reduced (or at least transferred to the initialisation phase of my tool).
However, I also wanted to know if this solution is thread-safe?
Another question I have is: is it thread safe if I load metamodel using the pool but load modules using the exec env? Like this for instance:
// pool init, called at init time of my tool
Metamodel metamodel = EmftvmFactory.eINSTANCE.createMetamodel();
metamodel.setResource(EcorePackage.eINSTANCE.eResource());
// pool (first used bellow) is an attribute of the class; the object is a singleton
pool.registerMetaModel("ECORE", metamodel);
// use pool, potentially multi-threaded
ExecEnv env = pool.getExecEnv(); // a new env object is created in each thread using this call??
ModuleResolver mr = new DefaultModuleResolver("platform:/plugin/my.plugin.id/transformations/", new ResourceSetImpl());
TimingData td = new TimingData();
env.loadModule(mr, "Module");
td.finishLoading();
env.run(td);
pool.returnExecEnv(env);
Using such a setting, launching several transfo in different threads, I experienced difficulties (typically some memory over consumption).
Regarding model loading, I load models using resources obtained from an xtext-based parser. Not sure if it answers your questions.
Best regards and thanks for your help!
Etienne.
|
|
|
Re: [EMFTVM] Reduce initialization time [message #1278747 is a reply to message #1277874] |
Thu, 27 March 2014 19:57 |
|
EtienneB Mising name wrote on Wed, 26 March 2014 16:28Hi Dennis,
thanks a lot for your answer, it took me some time before to answer because I wanted to experiment a bit the ExecEnv option you mentionned:
Quote:
Using an ExecEnvPool allows you to cache transformation module bytecode and metamodels.
I tried it. Initially, I did not go for this solution because I do not always have the same set of transformation modules, and in some cases I need to load modules dynamically. In order to see if it reduces the init time, I gave a try to this option. Indeed, the init time is significantly reduced (or at least transferred to the initialisation phase of my tool).
However, I also wanted to know if this solution is thread-safe?
The ExecEnvPool is thread-safe if used right: after use, you can return an ExecEnv to the pool, after which another thread may check it out of the pool. Of course, you may not continue using an ExecEnv after you've returned it to the pool!
You should also not load any additional modules into a single ExecEnv checked out from a pool: the pool will become polluted with different kinds of ExecEnvs, and any ExecEnvs checked out from the pool may have different behaviours!
The way to use an ExecEnvPool in this context would be to instantiate a separate ExecEnvPool for each transformation configuration you want to support.
Cheers,
Dennis
|
|
|
Re: [EMFTVM] Reduce initialization time [message #1278752 is a reply to message #1277874] |
Thu, 27 March 2014 20:07 |
|
EtienneB Mising name wrote on Wed, 26 March 2014 16:28
Another question I have is: is it thread safe if I load metamodel using the pool but load modules using the exec env? Like this for instance:
// pool init, called at init time of my tool
Metamodel metamodel = EmftvmFactory.eINSTANCE.createMetamodel();
metamodel.setResource(EcorePackage.eINSTANCE.eResource());
// pool (first used bellow) is an attribute of the class; the object is a singleton
pool.registerMetaModel("ECORE", metamodel);
// use pool, potentially multi-threaded
ExecEnv env = pool.getExecEnv(); // a new env object is created in each thread using this call??
ModuleResolver mr = new DefaultModuleResolver("platform:/plugin/my.plugin.id/transformations/", new ResourceSetImpl());
TimingData td = new TimingData();
env.loadModule(mr, "Module");
td.finishLoading();
env.run(td);
pool.returnExecEnv(env);
Using such a setting, launching several transfo in different threads, I experienced difficulties (typically some memory over consumption).
This scenario will not work, because you're loading modules using different ResourceSets into the same instance of an ExecEnv each time. After you call pool.returnExecEnv(env), the pool may return that env again when you call pool.getExecEnv(). The ExecEnvPool achieves its performance gain by reusing ExecEnv instances. You'll find that the ExecEnvPool methods are synchronized, such that the pool acts as a monitor for shared resources.
As said in my previous post, the module loading should be done on the ExecEnvPool itself. Create multiple pools if you need to support different ExecEnv configurations. All ExecEnvs in a single pool should have the same bytecode and metamodels loaded into them, such that they behave the same.
Cheers,
Dennis
|
|
|
Re: [EMFTVM] Reduce initialization time [message #1278756 is a reply to message #1277874] |
Thu, 27 March 2014 20:11 |
|
EtienneB Mising name wrote on Wed, 26 March 2014 16:28
Regarding model loading, I load models using resources obtained from an xtext-based parser. Not sure if it answers your questions.
Best regards and thanks for your help!
Etienne.
If you're loading xText resources from URI, you probably cannot speed things up. The xText parser (ANTLR generated?) can be slow, depending on the complexity of your grammar. Not much you can do about that... ATL also uses an ANTLR-generated parser, which can take several seconds to parse a large file.
If you're loading the resources from an InputStream, then make sure you wrap the InputStream in a BufferedInputStream before passing it to xText.
Cheers,
Dennis
|
|
|
Re: [EMFTVM] Reduce initialization time [message #1279105 is a reply to message #1278756] |
Fri, 28 March 2014 08:32 |
EtienneB Mising name Messages: 35 Registered: June 2011 |
Member |
|
|
Hi Dennis,
thanks for your help. I have very little controle on the way input resources are loaded (I use a third-party tool) but when I launch the transformation, they have already been parsed and loaded.
For the use of exec env pools, I figured out the way I use them was incorrect, I just wanted to check with you to make sure it was a mistake (and to understand why). Now I understand and I will go for your solution; it will ask me to make some major changes in my tool but I reached a point where I beleive it is worth it.
Thanks again, I will come back to you if I see some unexpected results when using the correct solution.
Best regards,
Etienne.
[Updated on: Fri, 28 March 2014 12:36] Report message to a moderator
|
|
|
Goto Forum:
Current Time: Tue Apr 23 13:55:45 GMT 2024
Powered by FUDForum. Page generated in 0.03399 seconds
|