Eclipse Community Forums: EMF "Technology" (Ecore Tools, EMFatic, etc) » [TENEO] Possible performance Issue in IdentifierCacheHandler

Home » Modeling » EMF "Technology" (Ecore Tools, EMFatic, etc) » [TENEO] Possible performance Issue in IdentifierCacheHandler(PURGE_TRESHOLD value of 100 causes frequent concurrent purging of the cache)

Show: Today's Messages :: Show Polls :: Message Navigator

[TENEO] Possible performance Issue in IdentifierCacheHandler [message #1597810]

Mon, 02 February 2015 15:21

Lukas Zaruba

Messages: 8
Registered: January 2010

Junior Member

Hi,

first let me thank you for the Teneo project, it is a great piece of work!

and here comes the issue:
Using Teneo under high load we have discovered (using profiling tools and thread dumps) that purging of the cache on each 100 insertions causes frequent concurrent purging of the map.

What I think happens is a thread invokes the

org.eclipse.emf.teneo.hibernate.mapping.identifier.IdentifierCacheHandler.setID(Object, Object)

method, with condition on line 115 resolving as true. It starts purging of the map.
Other threads goes in the same path and also resolves the 115 condition as true because the previous thread haven't finished the job. The second thread also starts purging...

Same scenario can happen with versions cache.

As the map is an instance of ConcurrentHashMap ConcurrentModificationException is not thrown and both threads are purging the map simultaneously. If there is a heavy load and a lot of threads hit the purging before the counter is cleared to zero (line 163), it can heat the CPUs to 100% for a long time.

Our temporary solution was to recompile the class with the configurable (by System.getProperty(String, String) value of the PURGE_TRESHOLD constant which we set to 15000. Since this change has been introduced there were no symptoms of this problem. But I don't think it is the complete solution. Probably only one thread should hit the clearing part in a some kind of the synchronized block, clear the counter and than start the purging. I believe that the configurable value of the PURGE_TRESHOLD is also a valid idea - our application has 20GB of RAM available so I don't mind having the cache of 500MB if it increases the performance.

Can you please give us some feedback on this issue - if it is valid and if maybe there is a better and cleaner solution to the described problem than the one we introduced?

Thank you!

Cheers, LZ

Report message to a moderator

Re: [TENEO] Possible performance Issue in IdentifierCacheHandler [message #1602031 is a reply to message #1597810]

Thu, 05 February 2015 09:22

Martin Taal

Messages: 5468
Registered: July 2009

Senior Member

Hi Lukas,
Yes purging should be single threaded for sure. Can you create a bugzilla for this, and if possible attach a patch for it?

Which version of Teneo are you using?

gr. Martin

On 02-02-15 16:21, Lukas Zaruba wrote:
> Hi,
>
> first let me thank you for the Teneo project, it is a great piece of work!
>
> and here comes the issue:
> Using Teneo under high load we have discovered (using profiling tools and thread dumps) that purging of the cache on
> each 100 insertions causes frequent concurrent purging of the map.
>
> What I think happens is a thread invokes the
> org.eclipse.emf.teneo.hibernate.mapping.identifier.IdentifierCacheHandler.setID(Object, Object) method, with condition
> on line 115 resolving as true. It starts purging of the map.
> Other threads goes in the same path and also resolves the 115 condition as true because the previous thread haven't
> finished the job. The second thread also starts purging...
> Same scenario can happen with versions cache.
>
> As the map is an instance of ConcurrentHashMap ConcurrentModificationException is not thrown and both threads are
> purging the map simultaneously. If there is a heavy load and a lot of threads hit the purging before the counter is
> cleared to zero (line 163), it can heat the CPUs to 100% for a long time.
>
> Our temporary solution was to recompile the class with the configurable (by System.getProperty(String, String) value of
> the PURGE_TRESHOLD constant which we set to 15000. Since this change has been introduced there were no symptoms of this
> problem. But I don't think it is the complete solution. Probably only one thread should hit the clearing part in a some
> kind of the synchronized block, clear the counter and than start the purging. I believe that the configurable value of
> the PURGE_TRESHOLD is also a valid idea - our application has 20GB of RAM available so I don't mind having the cache of
> 500MB if it increases the performance.
>
> Can you please give us some feedback on this issue - if it is valid and if maybe there is a better and cleaner solution
> to the described problem than the one we introduced?
>
> Thank you!
>
> Cheers, LZ

--

With Regards, Martin Taal

Springsite/Elver.org
Office: Hardwareweg 4, 3821 BV Amersfoort
Postal: Nassaulaan 7, 3941 EC Doorn
The Netherlands
Cell: +31 (0)6 288 48 943
Tel: +31 (0)84 420 2397
Fax: +31 (0)84 225 9307
Mail: mtaal@xxxxxxxx - mtaal@xxxxxxxx
Web: www.springsite.com - www.elver.org

Report message to a moderator

Previous Topic:	[EMFStore] Atomic Undo/Redo
Next Topic:	[Teneo] EntityManagerFactoryImpl.transactionType not set

Goto Forum:

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

]

Current Time: Thu Apr 25 09:42:21 GMT 2024

.:: Contact :: Home ::.

Breadcrumbs

Sign up to our Newsletter