Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » Epsilon » Calling ETL from Java with "partial" reset(Trying to merge multiple instances of one meta-model in a single instance of a mother without reloading the latter.)
Calling ETL from Java with "partial" reset [message #1281870] Tue, 01 April 2014 09:39 Go to next message
Philip Makedonski is currently offline Philip Makedonski
Messages: 10
Registered: May 2013
Junior Member
Hi,

I am trying to merge multiple instances of one meta-model in a single instance of a mother without reloading the latter. Let me try to elaborate on this:

I have a model instance of meta-model A and multiple instances of meta-model B. What I want to do is execute the same ETL module on all instances of B and the single instance of A. Typically, a self-contained execution is not a problem:

* 0. load ETL,
* 1. load A,
* 2. load Bi,
* 3. execute ETL,
* 4. reset (dispose of A, Bi, ETL),
* 5. repeat (go to 0.)

But since with time A grows, loading and disposing A for each B comprises a major impact on performance. Thus, I've been trying to modify the workflow at least for a set of B instances (B1 ... Bn), where it becomes:

* 0. load ETL,
* 1. load A,
* 2. load Bi,
* 3. execute ETL,
* 4. dispose of Bi,
* 5. repeat (go to 2),
* 6. dispose of A, ETL,

Now I'm not quite sure whether this is possible at all, looking at the implementations for IModule.reset() does not really help so far, but maybe I am looking at the wrong place. Any advice will be appreciated. Here is some sample code illustrating what I am after:


		//initialise and load ETL source
		module = createModule();
		module.parse(getFile(getSource()));

		//load merged model (A)
		module.getContext().getModelRepository().addModel(getDecentModel(location));
		
		//iterate over models to be merged  (Bi)
		for (int i : list) {
			//load Bi instance 
			IModel famixModel = getFamixModel(location,i);
			module.getContext().getModelRepository().addModel(famixModel);

			//execute ETL
			preProcess();
			result = execute(module);
			postProcess();
			
			//remove the merged instance Bi - neither			
			//module.getContext().getModelRepository().getModelByName("FAMIX").dispose();
			//nor 
			//module.getContext().getModelRepository().removeModel(famixModel);
			//seems to work as the ETL transformations are not executed after the first iteration
			//perhaps I am missing reset the module without disposing of the source or at least of the merged target A
			
			//default approach which works but has potential performance implications
			module.reset();
			//reload
			module.parse(getFile(getSource()));
			module.getContext().getModelRepository().addModel(getDecentModel(location));
		}
		module.getContext().getModelRepository().dispose();


Re: Calling ETL from Java with "partial" reset [message #1282080 is a reply to message #1281870] Tue, 01 April 2014 17:04 Go to previous messageGo to next message
Dimitris Kolovos is currently offline Dimitris Kolovos
Messages: 836
Registered: July 2009
Senior Member
Hi,

Does the following do the trick?

IModel a = ...;
a.load();

for every Bi {
EtlModule module = new EtlModule();
module.parse(...);
IModel b = ...;
b.load();
module.getContext().getModelRepository().addModel(a);
module.getContext().getModelRepository().addModel(b);
module.execute();
b.dispose();
}

a.dispose();

Cheers,
Dimitris
Re: Calling ETL from Java with "partial" reset [message #1282418 is a reply to message #1282080] Wed, 02 April 2014 05:18 Go to previous messageGo to next message
Philip Makedonski is currently offline Philip Makedonski
Messages: 10
Registered: May 2013
Junior Member
Yes, perfectly! Now that I look at it, it seems obvious that this is how it should be done, but it took me quite a while to figure out all the possible ways it shouldn't be done Wink Thanks a lot!

Cheers,

Philip
Re: Calling ETL from Java with "partial" reset [message #1282456 is a reply to message #1282418] Wed, 02 April 2014 06:30 Go to previous messageGo to next message
Dimitris Kolovos is currently offline Dimitris Kolovos
Messages: 836
Registered: July 2009
Senior Member
Great! Glad this helped.

Cheers,
Dimitris
Re: Calling ETL from Java with "partial" reset [message #1357495 is a reply to message #1282456] Thu, 15 May 2014 10:52 Go to previous messageGo to next message
Jens Neuschulz is currently offline Jens Neuschulz
Messages: 3
Registered: May 2014
Junior Member
Hi,

I have a similar problem. I'm trying to transform many source files of one metamodel into many target files of another one metamodel using one ETL. Creating a new ETL Module for every transformation takes a long time and decreases the performance. Using one ETL Module and reloading the source and the target models doesn't work. The content of the source file is changing before every load.
Here is my code:

module = createModule();
module.parse(getFile(getSource()));

for (int i = 0; i < 10000; i++) {
preProcess();
for (IModel model : getModels()) {
module.getContext().getModelRepository().addModel(model);
}
result = execute(module);
postProcess();
module.getContext().getModelRepository().dispose();
}

public List<IModel> getModels() throws Exception {
List<IModel> models = new ArrayList<IModel>();
models.add(createEmfModel("fin5412013", "models/MT541Instance.model", "models/fin5412013.ecore", true, false));
models.add(createEmfModel("sese02300105", "models/sese023Transformed.model", "models/sese02300105.ecore", false, true));
return models;
}

Any help?

Many thanks Jens

Re: Calling ETL from Java with "partial" reset [message #1358293 is a reply to message #1357495] Thu, 15 May 2014 18:52 Go to previous messageGo to next message
Dimitris Kolovos is currently offline Dimitris Kolovos
Messages: 836
Registered: July 2009
Senior Member
Hi Jens,

Off the top of my head, you could remove the old models from the module's model repository and add fresh ones in each iteration. Please let me know if this is unclear and I'll provide some example code.

Cheers,
Dimitris
Re: Calling ETL from Java with "partial" reset [message #1359035 is a reply to message #1358293] Fri, 16 May 2014 02:26 Go to previous messageGo to next message
Jens Neuschulz is currently offline Jens Neuschulz
Messages: 3
Registered: May 2014
Junior Member
Hi Dimitris,

could you provide an example please.

By the way could you provide an example for multi threading please.

Many thanks for your support Jens
Re: Calling ETL from Java with "partial" reset [message #1366277 is a reply to message #1359035] Mon, 19 May 2014 04:32 Go to previous messageGo to next message
Dimitris Kolovos is currently offline Dimitris Kolovos
Messages: 836
Registered: July 2009
Senior Member
Hi Jens,

Sure - I'll try to do this within the next few days.

Cheers,
Dimitris
Re: Calling ETL from Java with "partial" reset [message #1386274 is a reply to message #1366277] Mon, 16 June 2014 06:11 Go to previous messageGo to next message
Jens Neuschulz is currently offline Jens Neuschulz
Messages: 3
Registered: May 2014
Junior Member
Hi Dimitris,

any update on this?

Best regards Jens
Re: Calling ETL from Java with "partial" reset [message #1386639 is a reply to message #1386274] Wed, 18 June 2014 19:16 Go to previous messageGo to next message
Dimitris Kolovos is currently offline Dimitris Kolovos
Messages: 836
Registered: July 2009
Senior Member
Hi Jens,

Apologies for the radio silence - this somehow fell through the cracks. I'll have a look at this within the next couple of days and get back to you.

Cheers,
Dimitris
Re: Calling ETL from Java with "partial" reset [message #1390487 is a reply to message #1386639] Sun, 29 June 2014 10:24 Go to previous message
Dimitris Kolovos is currently offline Dimitris Kolovos
Messages: 836
Registered: July 2009
Senior Member
Hi Jens,

I've had a look at this and it appears that it is not safe to reuse / execute the same ETL module in different threads. I've filed an enhancement request for this: https://bugs.eclipse.org/bugs/show_bug.cgi?id=438490. Please feel free to CC yourself to the request to keep track of any progress on this.

Cheers,
Dimitris
Previous Topic:[ECL] How to match hierarchical models?
Next Topic:[EOL] Type not found when using registered EPackage
Goto Forum:
  


Current Time: Sat Aug 23 01:47:25 EDT 2014

Powered by FUDForum. Page generated in 0.01920 seconds