Eclipse Community Forums: OM2M » OM2M performance for thousands of resources

Help

Home

Home » Eclipse Projects » OM2M » OM2M performance for thousands of resources

Show: Today's Messages :: Show Polls :: Message Navigator

OM2M performance for thousands of resources [message #1769690]

Wed, 02 August 2017 16:57

ihave question

Messages: 32
Registered: November 2014

Member

Hello!

My colleagues and I are researching OM2M platform and developing proof of concept plugin and we are encountering a problem with performance. Our plugin creates resources for sensors from large file, so it tries to create 100k of resources one by one. Creation of first few thousands is fast but it slows down more and more as the number of created resources grow. It gets to about 1 per second at 20k and lower after that.

My questions are:
1. Is it reasonable to hope for good performance with number of sensors that high?
2. Is there some kind of online resource, documentation or best practices for this workload?
3. Are there any known deployments of OM2M with large number of sensors?
4. Any other hints?

Thanks in advance!

Report message to a moderator

Re: OM2M performance for thousands of resources [message #1769795 is a reply to message #1769690]

Thu, 03 August 2017 13:15

Mahdi Ben Alaya

Messages: 229
Registered: November 2013

Senior Member

Hello Nasuki,

Thank you for your feedback.
I created the following thread on bugzilla to track this issue:
https://bugs.eclipse.org/bugs/show_bug.cgi?id=520517

The platform should handle high workload without issues. I will submit a hot fix to solve this issue. I will keep you informed.

Report message to a moderator

Re: OM2M performance for thousands of resources [message #1769797 is a reply to message #1769795]

Thu, 03 August 2017 13:24

Mahdi Ben Alaya

Messages: 229
Registered: November 2013

Senior Member

I submitted a patch on the master branch.
Please test again your scenario from the master branch and tell us about the performance obtained.
Have a good day.

Report message to a moderator

Re: OM2M performance for thousands of resources [message #1769832 is a reply to message #1769797]

Thu, 03 August 2017 20:02

ihave question

Messages: 32
Registered: November 2014

Member

Hello Mahdi,

Thanks for quick reaction, I'm really impressed!

Unfortunately the issue still occurs. I can't share exact code, but as we started from org.eclipse.om2m.ipe.sample sample plugin the easiest way to demonstrate the problem would be to change for-loop in org.eclipse.om2m.ipe.sample.controller.LifeCycleManager.start() method to create 100k lamp resources instead of 2. I did some profiling and it seems most of the cpu time is eaten somewhere in org.eclipse.persistence packages (I don't know if it's helpful information). Maybe creating all resources in loop is wrong or we should diverge more from sample plugin (we left almost everything as default) - let me know what you think.

Thanks for help!

Report message to a moderator

Re: OM2M performance for thousands of resources [message #1769870 is a reply to message #1769832]

Fri, 04 August 2017 10:21

Mahdi Ben Alaya

Messages: 229
Registered: November 2013

Senior Member

Hello Nasuki,

You are welcome.
I will try to reproduce the problem and get back to you soon.

Regards,
Mahdi

Report message to a moderator

Re: OM2M performance for thousands of resources [message #1769923 is a reply to message #1769870]

Sat, 05 August 2017 21:32

Mahdi Ben Alaya

Messages: 229
Registered: November 2013

Senior Member

Hello,

I made another fix. It is now merged on the master branch.
Waiting for your feedback.

Regards,
Mahdi

Report message to a moderator

Re: OM2M performance for thousands of resources [message #1769936 is a reply to message #1769923]

Sun, 06 August 2017 15:30

ihave question

Messages: 32
Registered: November 2014

Member

Hi Mahdi,

Awesome - this fixed our performance issue. Right now I'm able to create 195k resources in 27 minutes, which is great - it starts at about 11k per minute and finishes at about 5k per minute.

Thanks again for all the help and have a great day!

Report message to a moderator

Re: OM2M performance for thousands of resources [message #1769937 is a reply to message #1769936]

Sun, 06 August 2017 16:08

Mahdi Ben Alaya

Messages: 229
Registered: November 2013

Senior Member

Hi,

Happy it works for you.
I think you mean 195K lamps. For each lamp, at least 5 resources are created (1 AE, 2 Containers and 2 ContentInstances).
So the total number of resources you created is equal to 975K. Could you confirm this ?

You are welcome.
Mahdi

Report message to a moderator