| Re: [viatra-dev] Using VIATRA in AADL model transformation |
Question A: do you often have multiple Aadl2AadlTraceSpec instances (traceability model roots) existing at the same time? Specifically in a way that a given aadlElement appears as the leftInstance in multiple different traceability models? Because if not, if you can guarantee that a given element is only ever the left instance in one traceability model (it might be the right instance of another model, if you transform in multiple steps), then there is a lot of simplification you can do! In that case, your patterns do not need to include the extra condition that all the *ref elements must be reachable from the left instances via the *same* traceability model (because there is only ever one traceability model that has those elements as left instances), so instead of find
is_in_trace(aadl2aadlref,_,processsource,processsourceref), you can simply say findis_in_trace(_,_,processsource,processsourceref) in all rows of the pattern body except one (if you want to keep the traceability root as an argument, you need to leave one constraint to determine this variable). I have a hunch that, if the asumptions is true, then this might be a valid workaround to your problem!
is_in_trace(aadl2aadlref, trace and find
is_in_trace(_,
_, are similar. Question B: is the pattern is_in_trace typically one-to-one? In the sense that for a single aadlElement, do you expect there to be more than one aadlrefElement and the other way around?
If yes, and additionally for Question A you also answer yes, then altogether the value of aadlElement uniquely determines all of the other parameters of the pattern is_in_trace. This is information that you may have (based on external domain knowledge), but cannot be automatically learned from the metamodel! And fortunately, there is a way to tell Viatra about this domain knowledge! You can add a @FunctionalDependency annotation to the definition of pattern is_in_trace, something like this:
@FunctionalDependency(forEach=aadlElement, unique=aadlrefElement, unique=trace, unique=aadl2aadlref)
patternis_in_trace(…(Note that if the “traces” is a containment reference and “rightInstance” has multiplicity to-one, then Viatra is smart enough to figure out that “trace” uniquely determines aadlrefElement and aadl2aadlref, but it has no was to know that leftInstance uniquely determines “trace”; really this is the only functional dpenedency rule missing, the rest can be inferred.)
I have the hunch, although no full conviction, that with these annotations, the query plan optimizer will be smart enough to figure out a query plan that does not blow up, even without you having to do the changes (underscores instead of TraceSpec) indicated above in Question A.
In case the above functional dependencies do not always hold, just for certain element types, then you can perhaps subdivide is_in_trace into multiple patterns based on the type of the leftInstance, and have different sets of functional dependencies declared for them; be careful to always use the appropriate one from transformation rules.
If none of the tricks above work:
Question C: could you please tell me how much heap memory you grant to the JVM process that runs the transformation? For instance, -Xmx8G?
Question D: could you please provide a query profile evaluated at a representative / typical state of the model?
Again, it would be very helpful if you could pinpoint which of the (pre-optimization) queries blew up in memory, so that I could take a look at that specific pattern.
This requires an execution of either the relative or the absolute query performance test (linked earlier), but in a saved snapshot of the EMF model where the source model, the target model and the traceability model already have contents of a representative size. When you think you have a triplet of instance models that satisfy these criteria, please either save them to a file (or three files) and then run the query profiling on that file, or run the query profiler on-the-fly from the same process where you run your transformation. If you cannot easily produce such a state when transforming your largest input model, perhaps a slightly smaller input model will allow the un-optimized transformation to run to completion, and still generate a reasonably large source+target+traceability model triplet that you can use for profiling.
Good luck with you project, and wish you all the best with the paper!
_______________________________________________
Gábor Bergmann PhD | Research Expert
From: HANA MKAOUAR
Sent: 2021. április 2., péntek 20:14
To: Viatra project developer discussions
Subject: Re: [viatra-dev] Using VIATRA in AADL model transformation
Hi Gábor,
Thank you for your reply.
1.
About your earlier question regarding the omission of parameters from the big pattern: while I am not familiar with your exact transformation needs, usually traceability connections between the source and target model are (almost) one-to-one. This means whether or not they are included in the pattern *should* make little difference in performance. At least that is my expectation. But ultimately, only actual measurements can tell the truth. (Note that if you have one source/destination element traced to many such “sourceref”/”destinationref” elements, then what I said before does not apply; they may really make a difference in that case)
2.
I have taken a look at the results.txt report that you have attached.
I see that you received an OutOfMemory error… how many gigabytes of Java heap space did you allow the JVM process to use (-Xmx command line switch)?
Remember that if you use the incremental pattern matcher of Viatra, it will maintain both the query results as well as internal auxiliary caches, both of which will take up additional memory on top of the memory required to store the model itself. You have to make sure that Viatra has plenty of heap space to go around; running it with whatever default the JVM or Eclipse chooses is often not enough.
I am using the is_in_trace calls in patterns, to rapidly find the correspondences between AADL element (input model) and AADL refined element (output model) . These calls add two conditions: (1) ensure that the objects to be transformed do indeed exist in the AADL input model, since we have the same meta-model for input and output models, if I ask for a ComponentInstance to match, the engine does not differentiate input or output models, so it can return a ComponentInstance from the output model; (2) also ensure that some objects have already been transformed (since they are added in the trace model), for example the source and destination features should be transformed before the transformation of a port connection.
I eliminated 8 is_in_trace calls, not all the calls because I need some of them to ensure (1). Now the transformation works fine and I no longer have the OutOfMemory problem even for 512 port connections (specifications are here [1]).I had to carefully assign the priorities of the rules to ensure (2), otherwise I get exceptions (because, for example, the source and destination features are not already transformed, when applying the rule of the port connection).
Do you propose other solutions to ensure (2).
Could you please explain why the calls of is_in_trace are too expensive in memory when they are used in the queries.
5.
Also, I took a *very* cursory look at your repo (specifically at [3]), and found that you seem to have two VQL files, one with pattern called find_*, the other with copy_*, however these seem to be the same patterns, just under different names? Or at least I did not manage to find any difference by a single glance.
Why have two duplicate copies of the same pattern? Why not just use the same pattern twice? If you have two different pattern definitions with the same content, and you use both of them over the same model at the same time, you pay twice the memory cost!
There is no use of two different pattern definitions with the same content over the same model at the same time. I used two packages to work on the copying and refinement model transformations separately. They have some common patterns but they are applied in separate tests on different models and not at the same time.
Best regards,
Hana.
From: HANA MKAOUAR
Sent: 2021. március 24., szerda 12:01
To: Viatra project developer discussions
Subject: Re: [viatra-dev] Using VIATRA in AADL model transformation
Hi Zoltan,
sorry, but for clarification, I would like to ask whether you selected these patterns by intuition or by some kind of measurements. I am asking this as the Rete network structure used by VIATRA has some very unusual memory characteristics (e.g. a pattern with more constraints might be more efficient to evaluate than another one with a clear subset of parameters). The measurements in the QueryPerformanceTests (point 2 of my original mail) would provide that information.
OK, this is the Performance test results:
pattern, sequence, matches count, heap before (kb), heap after (kb), used heap (kb), elapsed (ms)
fr.tpt.mem4csd.mtbench.aadl2aadl.viatra.find_component, 10, 0, 39227, 39354, 127, 17
fr.tpt.mem4csd.mtbench.aadl2aadl.viatra.find_connectionref, 9, 0, 39226, 39342, 116, 14
fr.tpt.mem4csd.mtbench.aadl2aadl.viatra.find_feature, 6, 0, 39224, 39631, 407, 75
fr.tpt.mem4csd.mtbench.aadl2aadl.viatra.find_otherconnection, 3, 0, 39214, 39478, 264, 74
fr.tpt.mem4csd.mtbench.aadl2aadl.viatra.find_otherconnection_system, 2, 0, 39207, 39496, 289, 100
fr.tpt.mem4csd.mtbench.aadl2aadl.viatra.find_portconnection_process, 1, 0, 39198, 39702, 504, 209
fr.tpt.mem4csd.mtbench.aadl2aadl.viatra.find_portconnection_system, 5, 0, 39224, 39976, 752, 281
fr.tpt.mem4csd.mtbench.aadl2aadl.viatra.find_subcomponent, 4, 0, 39223, 39415, 192, 29
fr.tpt.mem4csd.mtbench.aadl2aadl.viatra.find_system, 8, 1, 39226, 39272, 46, 5fr.tpt.mem4csd.mtbench.aadl2aadl.viatra.is_in_trace, 7, 0, 39225, 39290, 65, 8
And you can find the total analysis in the attached file.
Furthermore, it is unclear to me how the modified pattern relates to the issue (some more detail would be helpful).
Well, I thought that the problem is caused by the number of parameters, so my idea was to eliminate some parameters in the find_otherconnection_system/find_otherconnection_process patterns.
Hana.
_______________________________________________
viatra-dev mailing list
viatra-dev@xxxxxxxxxxx
To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/viatra-dev
--
Hana MKAOUAR
PostDoc, Telecom Paris
viatra-dev mailing list
viatra-dev@xxxxxxxxxxx
To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/viatra-dev
