Home » Archived » M2M (model-to-model transformation) » [QVTO] Memory Leak with Intermediate Properties?
| [QVTO] Memory Leak with Intermediate Properties? [message #106066] |
Thu, 04 June 2009 19:40  |
 Alan McMorran
Messages: 55
Registered: July 2009 |
Member |
|
|
Hi,
I think I may have come across a memory leak within the
OperationalTransform when Intermediate Properties are used which
prevents the JVM from garbage collecting objects.
Essentially what I'm doing is having a service run that loads the QVTO
files, creates instances of TransformationExecutor for each and loads
the QVTO then waits for a connection and some data. When this is
received it executes the transform, deals with the output and then goes
back to waiting again.
For most of out transforms this was fine, the JVM garbage collected the
input and output objects after finishing the process, but for one
transform the memory usage grew after every execution until the heap
limit was reached.
I looped the transform 100 times and did some heap dumps using JMAP and
then loaded them into MAT and found that 50% of the memory usage was in
a HashMap that referenced the (input) objects at:
TransformationExecutor
->OperationalTransformationImpl
->IntermediatePropertyModelAdapter
->IntermediatePropertyModelAdapter$ShadowEntry
->IdentifyHashmap
By having the process loop through every object in a resource and
delete all its references and cross-references after we'd dealt with
them it seemed to solve the problem, but in reality what it did was
just reduce the leak being retained and it still grew, albeit far
slower. Another 100+ transforms and a JMAP dump showed only objects of
a single type and it was at that point I realised these objects were of
a class that had an intermediate property defined within that QVTO
(which I realise I should've picked up on before with the class name :-)
For testing I removed the intermediate property and commented out the
lines that required them and ran the transform another 100 times. This
time there was no memory leak and the hashmap was empty.
I therefore think there is a problem with the Intermediate Properties
not being cleared after each execution.
I realise that I'm perhaps not using the runtime code in the way it was
intended, so this is maybe the expected behaviour, so I was hoping
someone could let me know if this is indeed something that should be
fixed (in which case I'll submit a big report) or if I need to find
some other way around this (which has thus far eluded me!)
Thanks in advance for any help.
Alan
|
|
|
| Re: [QVTO] Memory Leak with Intermediate Properties? [message #106080 is a reply to message #106066] |
Thu, 04 June 2009 20:31  |
Radomil Dvorak
Messages: 249
Registered: July 2009 |
Senior Member |
|
|
Hi Alan,
I'm afraid that the only way around this is that we fix that.
It should be simple, so I hope we will get this into RC4.
Regards,
/Radek
On Thu, 04 Jun 2009 21:40:06 +0200, Alan McMorran <alan@mcmorran.co.uk>
wrote:
> Hi,
>
> I think I may have come across a memory leak within the
> OperationalTransform when Intermediate Properties are used which
> prevents the JVM from garbage collecting objects.
>
> Essentially what I'm doing is having a service run that loads the QVTO
> files, creates instances of TransformationExecutor for each and loads
> the QVTO then waits for a connection and some data. When this is
> received it executes the transform, deals with the output and then goes
> back to waiting again.
>
> For most of out transforms this was fine, the JVM garbage collected the
> input and output objects after finishing the process, but for one
> transform the memory usage grew after every execution until the heap
> limit was reached.
>
> I looped the transform 100 times and did some heap dumps using JMAP and
> then loaded them into MAT and found that 50% of the memory usage was in
> a HashMap that referenced the (input) objects at:
>
> TransformationExecutor
> ->OperationalTransformationImpl
> ->IntermediatePropertyModelAdapter
> ->IntermediatePropertyModelAdapter$ShadowEntry
> ->IdentifyHashmap
>
> By having the process loop through every object in a resource and delete
> all its references and cross-references after we'd dealt with them it
> seemed to solve the problem, but in reality what it did was just reduce
> the leak being retained and it still grew, albeit far slower. Another
> 100+ transforms and a JMAP dump showed only objects of a single type and
> it was at that point I realised these objects were of a class that had
> an intermediate property defined within that QVTO (which I realise I
> should've picked up on before with the class name :-)
>
> For testing I removed the intermediate property and commented out the
> lines that required them and ran the transform another 100 times. This
> time there was no memory leak and the hashmap was empty.
>
> I therefore think there is a problem with the Intermediate Properties
> not being cleared after each execution.
>
> I realise that I'm perhaps not using the runtime code in the way it was
> intended, so this is maybe the expected behaviour, so I was hoping
> someone could let me know if this is indeed something that should be
> fixed (in which case I'll submit a big report) or if I need to find some
> other way around this (which has thus far eluded me!)
>
> Thanks in advance for any help.
>
> Alan
>
|
|
|
| Re: [QVTO] Memory Leak with Intermediate Properties? [message #106092 is a reply to message #106080] |
Thu, 04 June 2009 21:19 |
Alan McMorran
Messages: 55
Registered: July 2009 |
Member |
|
|
> I'm afraid that the only way around this is that we fix that.
> It should be simple, so I hope we will get this into RC4.
Thanks Radek, that would be very much appreciated. I'm not familiar
enough with the QVTO execution code itself or I'd have tried to create
a patch! :-)
Do you want me to submit a big on the bugzilla?
Best regards,
Alan
> On Thu, 04 Jun 2009 21:40:06 +0200, Alan McMorran <alan@mcmorran.co.uk> wrote:
>
>> Hi,
>>
>> I think I may have come across a memory leak within the
>> OperationalTransform when Intermediate Properties are used which
>> prevents the JVM from garbage collecting objects.
>>
>> Essentially what I'm doing is having a service run that loads the QVTO
>> files, creates instances of TransformationExecutor for each and loads
>> the QVTO then waits for a connection and some data. When this is
>> received it executes the transform, deals with the output and then goes
>> back to waiting again.
>>
>> For most of out transforms this was fine, the JVM garbage collected the
>> input and output objects after finishing the process, but for one
>> transform the memory usage grew after every execution until the heap
>> limit was reached.
>>
>> I looped the transform 100 times and did some heap dumps using JMAP and
>> then loaded them into MAT and found that 50% of the memory usage was
>> in a HashMap that referenced the (input) objects at:
>>
>> TransformationExecutor
>> ->OperationalTransformationImpl
>> ->IntermediatePropertyModelAdapter
>> ->IntermediatePropertyModelAdapter$ShadowEntry
>> ->IdentifyHashmap
>>
>> By having the process loop through every object in a resource and
>> delete all its references and cross-references after we'd dealt with
>> them it seemed to solve the problem, but in reality what it did was
>> just reduce the leak being retained and it still grew, albeit far
>> slower. Another 100+ transforms and a JMAP dump showed only objects
>> of a single type and it was at that point I realised these objects
>> were of a class that had an intermediate property defined within that
>> QVTO (which I realise I should've picked up on before with the class
>> name :-)
>>
>> For testing I removed the intermediate property and commented out the
>> lines that required them and ran the transform another 100 times. This
>> time there was no memory leak and the hashmap was empty.
>>
>> I therefore think there is a problem with the Intermediate Properties
>> not being cleared after each execution.
>>
>> I realise that I'm perhaps not using the runtime code in the way it was
>> intended, so this is maybe the expected behaviour, so I was hoping
>> someone could let me know if this is indeed something that should be
>> fixed (in which case I'll submit a big report) or if I need to find
>> some other way around this (which has thus far eluded me!)
>>
>> Thanks in advance for any help.
>>
>> Alan
|
|
|
| Re: [QVTO] Memory Leak with Intermediate Properties? [message #106105 is a reply to message #106092] |
Thu, 04 June 2009 22:30 |
Radomil Dvorak
Messages: 249
Registered: July 2009 |
Senior Member |
|
|
Yes, a bugzilla raised would be great!
Thanks,
/Radek
On Thu, 04 Jun 2009 23:19:09 +0200, Alan McMorran <alan@mcmorran.co.uk>
wrote:
>> I'm afraid that the only way around this is that we fix that.
>> It should be simple, so I hope we will get this into RC4.
>
> Thanks Radek, that would be very much appreciated. I'm not familiar
> enough with the QVTO execution code itself or I'd have tried to create a
> patch! :-)
>
> Do you want me to submit a big on the bugzilla?
>
> Best regards,
>
> Alan
>
>> On Thu, 04 Jun 2009 21:40:06 +0200, Alan McMorran
>> <alan@mcmorran.co.uk> wrote:
>>
>>> Hi,
>>> I think I may have come across a memory leak within the
>>> OperationalTransform when Intermediate Properties are used which
>>> prevents the JVM from garbage collecting objects.
>>> Essentially what I'm doing is having a service run that loads the
>>> QVTO files, creates instances of TransformationExecutor for each and
>>> loads the QVTO then waits for a connection and some data. When this
>>> is received it executes the transform, deals with the output and then
>>> goes back to waiting again.
>>> For most of out transforms this was fine, the JVM garbage collected
>>> the input and output objects after finishing the process, but for
>>> one transform the memory usage grew after every execution until the
>>> heap limit was reached.
>>> I looped the transform 100 times and did some heap dumps using JMAP
>>> and then loaded them into MAT and found that 50% of the memory usage
>>> was in a HashMap that referenced the (input) objects at:
>>> TransformationExecutor
>>> ->OperationalTransformationImpl
>>> ->IntermediatePropertyModelAdapter
>>> ->IntermediatePropertyModelAdapter$ShadowEntry
>>> ->IdentifyHashmap
>>> By having the process loop through every object in a resource and
>>> delete all its references and cross-references after we'd dealt with
>>> them it seemed to solve the problem, but in reality what it did was
>>> just reduce the leak being retained and it still grew, albeit far
>>> slower. Another 100+ transforms and a JMAP dump showed only objects
>>> of a single type and it was at that point I realised these objects
>>> were of a class that had an intermediate property defined within that
>>> QVTO (which I realise I should've picked up on before with the class
>>> name :-)
>>> For testing I removed the intermediate property and commented out
>>> the lines that required them and ran the transform another 100
>>> times. This time there was no memory leak and the hashmap was empty.
>>> I therefore think there is a problem with the Intermediate
>>> Properties not being cleared after each execution.
>>> I realise that I'm perhaps not using the runtime code in the way it
>>> was intended, so this is maybe the expected behaviour, so I was
>>> hoping someone could let me know if this is indeed something that
>>> should be fixed (in which case I'll submit a big report) or if I need
>>> to find some other way around this (which has thus far eluded me!)
>>> Thanks in advance for any help.
>>> Alan
>
>
|
|
|
Goto Forum:
Current Time: Fri Apr 26 07:01:22 GMT 2024
Powered by FUDForum. Page generated in 0.03461 seconds |
] 
Search
Help
Register
Login
Home
