| ETL equivalents performance [message #1058136] |
Sun, 12 May 2013 11:15  |
 Joost van Pinxten
Messages: 51
Registered: November 2012 |
Member |
|
|
I've been profiling my ETL tranformation to see why it does not perform as expected; A model containing about 12k elements takes 130 seconds to transform (and the executed rules are quite trivial). The target has roughly the same amount of elements.
The profiler shows that the rules, even there is no (explicit) call to 'allInstances' the execution time linearly increases with the number of executed rules. Looking at the code in the SVN for the EquivalentOperation, the DefaultTransformationStrategy calculates/transforms all the equivalents in a decent way, so my guess is that it might be related to copying some data structure that grows when more rules have been executed.
Is this behavior expected? Or is it possible that I missed an implicit operation that is dependent on the number of elements created in the target model?
FYI; I'm using a set of lazy, extended rules to perform transformations on expressions. Is it possible that all expressions are then combined into the same set/map, such that looking for the equivalent of the transformed expression is about O(N)?
Each transformation rule call starts at about 2ms, then increases up to 10ms after about 150 calls. With the full-blown transformation, starts out at 10ms and ends up at 44; so a factor 4x slower (per rule call!) is something to be looked into.
[Updated on: Sun, 12 May 2013 12:01] Report message to a moderator |
|
|
|
Powered by
FUDForum. Page generated in 0.02681 seconds
] 
Search
Help
Register
Login
Home
