Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » VIATRA » Viatra evaluation benchmark, execution time
Viatra evaluation benchmark, execution time [message #1815077] Wed, 25 September 2019 09:05 Go to next message
Hana Mkaouar is currently offline Hana MkaouarFriend
Messages: 7
Registered: January 2019
Junior Member
Hi,

I am trying to evaluate Vitra incremental transformation (execution time). For example, I need to take the required time for an update or delete or adding a set of elements in the model. I am following the viatra benchmark specification (https://github.com/viatra/viatra-cps-benchmark/wiki/Benchmark-specification)

You propose this scenario for incremental M2MTransformation test:
Sequence [
EMFResourceInitialization, Generation, Initialization, M2MTransformation
Iteration (5 times) [
Modification
M2MTransformation (distinguished in results from the first run by adding a postfix) ] ]

So I tryied this
transformation.execute // for batch
modifymodel // update an element in the input model
timer.startMeasure
transformation.execute // for incremental transformation
timer.stopMeasure

But, this doesn't work ...

In Viatra plugins I found, for example, the ToolChainPerformanceIncrementalScenario class (https://github.com/viatra/viatra-cps-benchmark/blob/master/tests/com.incquerylabs.examples.cps.performance.tests/src/com/incquerylabs/examples/cps/performance/tests/config/scenarios/ToolChainPerformanceIncrementalScenario.xtend),
in which the iteration contains:

seq.addPhases(
new EMFResourceInitializationPhase("EMFResourceInitialization"),
benchmarkCase.getGenerationPhase("Generation"),
new InitializationPhase("Initialization"),
new M2MTransformationPhase("M2MTransformation1"),
new M2TTransformationPhase("M2TTransformation1"),
new ChangeMonitorInitializationPhase("ChangeMonitorInitialization"), iter)

Why the EMFResourceInitialization, Generation, Initialization phases are repeated every iteration ? It seems like you are repeating the same batch transformation.
How do you take the execution time for an update on an element in the model ? I mean, it is easy to take the batch transformation time

timer.startMeasure
transformation.execute
timer.stopMeasure

But, after the first batch transformation, is it possible to measure the time of model updates ?

Thank you.
Re: Viatra evaluation benchmark, execution time [message #1815080 is a reply to message #1815077] Wed, 25 September 2019 10:00 Go to previous messageGo to next message
Zoltan Ujhelyi is currently offline Zoltan UjhelyiFriend
Messages: 373
Registered: July 2015
Senior Member
Hi Hana,

an incremental transformation can be executed the same way regardless if a new model is to be created or an existing one updated, as the transformation ensures that at the final step the target model will be consistent to the source model. However, if only minor steps need to be executed (e.g. because the source model had only minor changes after the target model was generated), the execution time will be much shorter, as the transformation detects whether any given step needs to be executed or can be ignored.

This means, measuring the model update time can be done the same as the first one: wrapping the transformation execution with the measurement steps.

Hope this is helpful, if not, feel free to ask for clarifications.

Best regards,
Zoli
Re: Viatra evaluation benchmark, execution time [message #1815114 is a reply to message #1815077] Wed, 25 September 2019 15:15 Go to previous messageGo to next message
Abel Hegedus is currently offline Abel HegedusFriend
Messages: 192
Registered: September 2015
Senior Member
Hana Mkaouar wrote on Wed, 25 September 2019 09:05

Why the EMFResourceInitialization, Generation, Initialization phases are repeated every iteration ? It seems like you are repeating the same batch transformation.


They are part of the outer sequence, as well as the iteration, which has three steps defined above.

That inner sequence contains the modification phase and the time for that is also measured (see this example result).

You can find details on modification steps in the benchmark specification including links to case specific implementations that contain the measurement code around modifications (e.g. the client-server case)
Re: Viatra evaluation benchmark, execution time [message #1816028 is a reply to message #1815114] Thu, 17 October 2019 15:32 Go to previous messageGo to next message
Hana Mkaouar is currently offline Hana MkaouarFriend
Messages: 7
Registered: January 2019
Junior Member
Thanks for your responses,

So, if I understood, you mean that I should take the time, as follows:
transformation.execute // for batch
modifymodel // update an element in the input model
timer.startMeasure
transformation.execute // for incremental transformation
timer.stopMeasure

But this method returns almost the same insignificant execution time in all modification cases ( from updating 1 element to updating more than 100 elements) ....
Also I noticed that updates are executed even before the second call for transformation.execute ... even without the second call for transformation.execute, updates are executed ...
Maybe this is due to the synchronous notification mechanism used by Viatra to deal with updates ... isn't it ? Could you please, explain more to me ...

Thank you.
Re: Viatra evaluation benchmark, execution time [message #1816058 is a reply to message #1816028] Fri, 18 October 2019 09:39 Go to previous message
Zoltan Ujhelyi is currently offline Zoltan UjhelyiFriend
Messages: 373
Registered: July 2015
Senior Member
Hi Hana,

as you have noticed, event-driven transformations by default execute model updates together with the model modifications. To measure the modification time, you can do one of two things:

1. You could include the model updates time in the measurement of incremental updates. If your changes are trivial, then the update times would be mostly the transformation time; or you could do some baseline measurements were the transformation is not initialized but you are doing your measurements.
2. You could use a custom scheduler using the setSchedulerFactory() call of the transformation initialization that postpones the transformation execution until the measurements are started (e.g. it never sends a fire event, and after the measurements have started, you could call transformation.getExecutionSchema().startUnscheduledExecution() ). In this case, the update times of the VIATRA index will not be measured, but at least the concrete modifications steps will.

Hope this is helpful,
Zoltán
Previous Topic:Check Class Diagram with Modal Object Diagram Specifications
Next Topic:How to add dependencies to vql files
Goto Forum:
  


Current Time: Fri Aug 14 18:49:54 GMT 2020

Powered by FUDForum. Page generated in 0.02320 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top