Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Eclipse Projects » OM2M » OM2M performance for thousands of resources
OM2M performance for thousands of resources [message #1769690] Wed, 02 August 2017 16:57 Go to next message
ihave question is currently offline ihave questionFriend
Messages: 32
Registered: November 2014
Member
Hello!

My colleagues and I are researching OM2M platform and developing proof of concept plugin and we are encountering a problem with performance. Our plugin creates resources for sensors from large file, so it tries to create 100k of resources one by one. Creation of first few thousands is fast but it slows down more and more as the number of created resources grow. It gets to about 1 per second at 20k and lower after that.

My questions are:
1. Is it reasonable to hope for good performance with number of sensors that high?
2. Is there some kind of online resource, documentation or best practices for this workload?
3. Are there any known deployments of OM2M with large number of sensors?
4. Any other hints?

Thanks in advance!
Re: OM2M performance for thousands of resources [message #1769795 is a reply to message #1769690] Thu, 03 August 2017 13:15 Go to previous messageGo to next message
Mahdi Ben Alaya is currently offline Mahdi Ben AlayaFriend
Messages: 229
Registered: November 2013
Senior Member
Hello Nasuki,

Thank you for your feedback.
I created the following thread on bugzilla to track this issue:
https://bugs.eclipse.org/bugs/show_bug.cgi?id=520517

The platform should handle high workload without issues. I will submit a hot fix to solve this issue. I will keep you informed.
Re: OM2M performance for thousands of resources [message #1769797 is a reply to message #1769795] Thu, 03 August 2017 13:24 Go to previous messageGo to next message
Mahdi Ben Alaya is currently offline Mahdi Ben AlayaFriend
Messages: 229
Registered: November 2013
Senior Member
I submitted a patch on the master branch.
Please test again your scenario from the master branch and tell us about the performance obtained.
Have a good day.
Re: OM2M performance for thousands of resources [message #1769832 is a reply to message #1769797] Thu, 03 August 2017 20:02 Go to previous messageGo to next message
ihave question is currently offline ihave questionFriend
Messages: 32
Registered: November 2014
Member
Hello Mahdi,

Thanks for quick reaction, I'm really impressed!

Unfortunately the issue still occurs. I can't share exact code, but as we started from org.eclipse.om2m.ipe.sample sample plugin the easiest way to demonstrate the problem would be to change for-loop in org.eclipse.om2m.ipe.sample.controller.LifeCycleManager.start() method to create 100k lamp resources instead of 2. I did some profiling and it seems most of the cpu time is eaten somewhere in org.eclipse.persistence packages (I don't know if it's helpful information). Maybe creating all resources in loop is wrong or we should diverge more from sample plugin (we left almost everything as default) - let me know what you think.

Thanks for help!
Re: OM2M performance for thousands of resources [message #1769870 is a reply to message #1769832] Fri, 04 August 2017 10:21 Go to previous messageGo to next message
Mahdi Ben Alaya is currently offline Mahdi Ben AlayaFriend
Messages: 229
Registered: November 2013
Senior Member
Hello Nasuki,

You are welcome.
I will try to reproduce the problem and get back to you soon.

Regards,
Mahdi
Re: OM2M performance for thousands of resources [message #1769923 is a reply to message #1769870] Sat, 05 August 2017 21:32 Go to previous messageGo to next message
Mahdi Ben Alaya is currently offline Mahdi Ben AlayaFriend
Messages: 229
Registered: November 2013
Senior Member
Hello,

I made another fix. It is now merged on the master branch.
Waiting for your feedback.

Regards,
Mahdi
Re: OM2M performance for thousands of resources [message #1769936 is a reply to message #1769923] Sun, 06 August 2017 15:30 Go to previous messageGo to next message
ihave question is currently offline ihave questionFriend
Messages: 32
Registered: November 2014
Member
Hi Mahdi,

Awesome - this fixed our performance issue. Right now I'm able to create 195k resources in 27 minutes, which is great - it starts at about 11k per minute and finishes at about 5k per minute.

Thanks again for all the help and have a great day!
Re: OM2M performance for thousands of resources [message #1769937 is a reply to message #1769936] Sun, 06 August 2017 16:08 Go to previous messageGo to next message
Mahdi Ben Alaya is currently offline Mahdi Ben AlayaFriend
Messages: 229
Registered: November 2013
Senior Member
Hi,

Happy it works for you.
I think you mean 195K lamps. For each lamp, at least 5 resources are created (1 AE, 2 Containers and 2 ContentInstances).
So the total number of resources you created is equal to 975K. Could you confirm this ?

You are welcome.
Mahdi
Re: OM2M performance for thousands of resources [message #1769951 is a reply to message #1769937] Mon, 07 August 2017 01:27 Go to previous message
ihave question is currently offline ihave questionFriend
Messages: 32
Registered: November 2014
Member
Yes, I confirm 5x195k.
Previous Topic:Usage of Postgre DB
Next Topic:Location creation
Goto Forum:
  


Current Time: Fri Oct 11 03:45:25 GMT 2024

Powered by FUDForum. Page generated in 0.09563 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top