Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Eclipse Projects » OM2M » OM2M - Missing data in NSCL
OM2M - Missing data in NSCL [message #1749730] Sat, 10 December 2016 12:33 Go to next message
Matej AA is currently offline Matej AAFriend
Messages: 23
Registered: February 2016
Junior Member
Hi,

i set up OM2M NSCL on my computer A (8 cores, 16GB RAM) and create simple application in NSCL applications.

I was testing OM2M solution with JMeter:
1. Sending 15k data from PC A, with automatic sending speed (if response time was higher, data was sent slower). All data was in H2.
2. Sending 15k from A, B, C and D with automatic sending speed (if response time was higher, data was sent slower). It took 30min for about 18k but some data in H2 was missing.
3. Sending 12k, 20k, 24k in 32k from A, B, C and D with unlimited sending speed (took about 2min). In H2 was 1/3 of sent data.

Dont know what is worong... Has OM2M some problems with accepting data from multiple clients at high speed? Its possible to parallelize some OM2M components? I also noticed that WEBGUI is realy unresponsive when OM2M contains more then 10k of data. Its realy hard to browse throught all contentInstances.

Thanks,
Matej

[Updated on: Sat, 10 December 2016 12:36]

Report message to a moderator

Re: OM2M - Missing data in NSCL [message #1749786 is a reply to message #1749730] Mon, 12 December 2016 09:28 Go to previous messageGo to next message
Mahdi Ben Alaya is currently offline Mahdi Ben AlayaFriend
Messages: 229
Registered: November 2013
Senior Member
Dear Matej,

Thank you for your feedback.
I opened the following bugs to track and solve these two issues:

Bug 509062 - Missing data from database when using multiple fast clients
https://bugs.eclipse.org/bugs/show_bug.cgi?id=509062

Bug 509061 - Web interface unresponsive with 10k of data
https://bugs.eclipse.org/bugs/show_bug.cgi?id=509061

Regards,
Mahdi

Report message to a moderator

Re: OM2M - Missing data in NSCL [message #1750130 is a reply to message #1749786] Fri, 16 December 2016 10:00 Go to previous messageGo to next message
Mahdi Ben Alaya is currently offline Mahdi Ben AlayaFriend
Messages: 229
Registered: November 2013
Senior Member
Dear Matej,

You are using the term NSCL here.
Are you using OM2M v0.8 which is the implementation of ETSI M2M (deprecated by ETSI and replace by oneM2M) ?

Report message to a moderator

Re: OM2M - Missing data in NSCL [message #1750131 is a reply to message #1750130] Fri, 16 December 2016 10:01 Go to previous messageGo to next message
Mahdi Ben Alaya is currently offline Mahdi Ben AlayaFriend
Messages: 229
Registered: November 2013
Senior Member
If yes then could you please run your tests on OM2M 1.0 (implementation of oneM2M standard) ?

Report message to a moderator

Re: OM2M - Missing data in NSCL [message #1750208 is a reply to message #1750131] Sun, 18 December 2016 11:27 Go to previous messageGo to next message
Matej AA is currently offline Matej AAFriend
Messages: 23
Registered: February 2016
Junior Member
Yes, i am using OM2M v0.8 because when i started writing Master thesis was OM2M v0.8 out. I implemented all of my plugins with OM2M v0.8 support.

[Updated on: Sun, 18 December 2016 11:28]

Report message to a moderator

Re: OM2M - Missing data in NSCL [message #1750209 is a reply to message #1750208] Sun, 18 December 2016 11:39 Go to previous messageGo to next message
Mahdi Ben Alaya is currently offline Mahdi Ben AlayaFriend
Messages: 229
Registered: November 2013
Senior Member
To help me investigating this issue, could you please make a quick test with OM2M 1.0 ? I suggest you to use the last version:
OM2M-DEV-13112016

You can use this code example to quickly create AE, Container and contentInstance resources using JSON and HTTP:
https://wiki.eclipse.org/OM2M/one/App

By the way, there is no big changes between oneM2M and ETSI M2M APIs.

Report message to a moderator

Re: OM2M - Missing data in NSCL [message #1750210 is a reply to message #1750208] Sun, 18 December 2016 11:41 Go to previous messageGo to next message
Mahdi Ben Alaya is currently offline Mahdi Ben AlayaFriend
Messages: 229
Registered: November 2013
Senior Member
"OM2M-DEV-13112016" Is available for download here:
http://wiki.eclipse.org/OM2M/Download

Report message to a moderator

Re: OM2M - Missing data in NSCL [message #1750309 is a reply to message #1749730] Mon, 19 December 2016 19:07 Go to previous messageGo to next message
Matej AA is currently offline Matej AAFriend
Messages: 23
Registered: February 2016
Junior Member
I used JMeter and "OM2M-DEV-13112016". HTTP Request with 1000ms response time.

4PC:
-A: 4core, 1500 data per core. Sum:6000
-B: 2core, 1500 data per core. Sum:3000
-C: 2core, 1500 data per core. Sum:3000
-D: 2core, 1500 data per core. Sum:3000

After 17 mins, i opened H2 database and executed:
SELECT COUNT(*) FROM CIN; Result was 9480.

I tried to DATA in web app and got that exception:

[INFO] - org.eclipse.om2m.core.router.Router
Request handling in the current CSE: /in-cse/in-name/MY_SENSOR/DATA
[INFO] - org.eclipse.om2m.core.router.Router
ResourceController to be used [ContentInstanceController]
[EL Severe]: cache: 2016-12-19 19:43:09.672--ClientSession(1533855376)--MAX TIME 600 seconds EXCEEDED FOR WRITELOCKMANAGER WAIT.  Waiting on Entity type: org.eclipse.om2m.commons.entities.ContainerEntitywith pk: /in-cse/cnt-761411626 currently locked by thread: qtp71587369-76 with the following trace:
                atjava.lang.Object.wait(Native Method)
                atorg.eclipse.persistence.sessions.server.ConnectionPool.acquireConnection(ConnectionPool.java:159)
                atorg.eclipse.persistence.sessions.server.ServerSession.getAccessors(ServerSession.java:538)
                atorg.eclipse.persistence.sessions.server.ServerSession.executeCall(ServerSession.java:557)
                atorg.eclipse.persistence.internal.queries.DatasourceCallQueryMechanism.executeCall(DatasourceCallQueryMechanism.java:242)
                atorg.eclipse.persistence.internal.queries.DatasourceCallQueryMechanism.executeCall(DatasourceCallQueryMechanism.java:228)
                atorg.eclipse.persistence.internal.queries.DatasourceCallQueryMechanism.selectOneRow(DatasourceCallQueryMechanism.java:714)
                atorg.eclipse.persistence.internal.queries.ExpressionQueryMechanism.selectOneRowFromTable(ExpressionQueryMechanism.java:2801)
                atorg.eclipse.persistence.internal.queries.ExpressionQueryMechanism.selectOneRow(ExpressionQueryMechanism.java:2754)
                atorg.eclipse.persistence.queries.ReadObjectQuery.executeObjectLevelReadQuery(ReadObjectQuery.java:545)
                atorg.eclipse.persistence.queries.ObjectLevelReadQuery.executeDatabaseQuery(ObjectLevelReadQuery.java:1168)
                atorg.eclipse.persistence.queries.DatabaseQuery.execute(DatabaseQuery.java:899)
                atorg.eclipse.persistence.queries.ObjectLevelReadQuery.execute(ObjectLevelReadQuery.java:1127)
                atorg.eclipse.persistence.queries.ReadObjectQuery.execute(ReadObjectQuery.java:431)
                atorg.eclipse.persistence.internal.sessions.AbstractSession.internalExecuteQuery(AbstractSession.java:3214)
                atorg.eclipse.persistence.internal.sessions.AbstractSession.executeQuery(AbstractSession.java:1804)
                atorg.eclipse.persistence.internal.sessions.AbstractSession.executeQuery(AbstractSession.java:1786)
                atorg.eclipse.persistence.internal.indirection.NoIndirectionPolicy.valueFromQuery(NoIndirectionPolicy.java:326)
                atorg.eclipse.persistence.mappings.ForeignReferenceMapping.valueFromRowInternal(ForeignReferenceMapping.java:2294)
                atorg.eclipse.persistence.mappings.OneToOneMapping.valueFromRowInternal(OneToOneMapping.java:1860)
                atorg.eclipse.persistence.mappings.ForeignReferenceMapping.valueFromRow(ForeignReferenceMapping.java:2144)
                atorg.eclipse.persistence.mappings.ForeignReferenceMapping.readFromRowIntoObject(ForeignReferenceMapping.java:1471)
                atorg.eclipse.persistence.internal.descriptors.ObjectBuilder.buildAttributesIntoObject(ObjectBuilder.java:461)
                atorg.eclipse.persistence.internal.descriptors.ObjectBuilder.refreshObjectIfRequired(ObjectBuilder.java:4304)
                atorg.eclipse.persistence.internal.descriptors.ObjectBuilder.buildObject(ObjectBuilder.java:1039)
                atorg.eclipse.persistence.internal.descriptors.ObjectBuilder.buildWorkingCopyCloneNormally(ObjectBuilder.java:898)
                atorg.eclipse.persistence.internal.descriptors.ObjectBuilder.buildObjectInUnitOfWork(ObjectBuilder.java:851)
                atorg.eclipse.persistence.internal.descriptors.ObjectBuilder.buildObject(ObjectBuilder.java:734)
                atorg.eclipse.persistence.internal.descriptors.ObjectBuilder.buildObject(ObjectBuilder.java:688)
                atorg.eclipse.persistence.queries.ObjectLevelReadQuery.buildObject(ObjectLevelReadQuery.java:795)
                atorg.eclipse.persistence.queries.ReadObjectQuery.registerResultInUnitOfWork(ReadObjectQuery.java:885)
                atorg.eclipse.persistence.queries.ReadObjectQuery.executeObjectLevelReadQuery(ReadObjectQuery.java:552)
                atorg.eclipse.persistence.queries.ObjectLevelReadQuery.executeDatabaseQuery(ObjectLevelReadQuery.java:1168)
                atorg.eclipse.persistence.queries.DatabaseQuery.execute(DatabaseQuery.java:899)
                atorg.eclipse.persistence.queries.ObjectLevelReadQuery.execute(ObjectLevelReadQuery.java:1127)
                atorg.eclipse.persistence.queries.ReadObjectQuery.execute(ReadObjectQuery.java:431)
                atorg.eclipse.persistence.queries.ObjectLevelReadQuery.executeInUnitOfWork(ObjectLevelReadQuery.java:1215)
                atorg.eclipse.persistence.internal.sessions.UnitOfWorkImpl.internalExecuteQuery(UnitOfWorkImpl.java:2896)
                atorg.eclipse.persistence.internal.sessions.AbstractSession.executeQuery(AbstractSession.java:1804)
                atorg.eclipse.persistence.internal.sessions.AbstractSession.executeQuery(AbstractSession.java:1786)
                atorg.eclipse.persistence.internal.sessions.AbstractSession.executeQuery(AbstractSession.java:1737)
                atorg.eclipse.persistence.internal.jpa.EntityManagerImpl.executeQuery(EntityManagerImpl.java:900)
                atorg.eclipse.persistence.internal.jpa.EntityManagerImpl.findInternal(EntityManagerImpl.java:843)
                atorg.eclipse.persistence.internal.jpa.EntityManagerImpl.find(EntityManagerImpl.java:719)
                atorg.eclipse.persistence.internal.jpa.EntityManagerImpl.find(EntityManagerImpl.java:588)
                atorg.eclipse.om2m.persistence.eclipselink.internal.dao.ContainerDAO.find(ContainerDAO.java:40)
                atorg.eclipse.om2m.persistence.eclipselink.internal.dao.ContainerDAO.find(ContainerDAO.java:1)
                atorg.eclipse.om2m.core.controller.ContentInstanceController.doCreate(ContentInstanceController.java:98)
                atorg.eclipse.om2m.core.controller.Controller.doRequest(Controller.java:76)
                atorg.eclipse.om2m.core.router.Router.doRequest(Router.java:224)
                atorg.eclipse.om2m.binding.http.RestHttpServlet.service(RestHttpServlet.java:130)
                atjavax.servlet.http.HttpServlet.service(HttpServlet.java:790)
                atorg.eclipse.equinox.http.servlet.internal.ServletRegistration.service(ServletRegistration.java:61)
                atorg.eclipse.equinox.http.servlet.internal.ProxyServlet.processAlias(ProxyServlet.java:128)
                atorg.eclipse.equinox.http.servlet.internal.ProxyServlet.service(ProxyServlet.java:76)
                atjavax.servlet.http.HttpServlet.service(HttpServlet.java:790)
                atorg.eclipse.equinox.http.jetty.internal.HttpServerManager$InternalHttpServiceServlet.service(HttpServerManager.java:360)
                atorg.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:684)
                atorg.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:503)
                atorg.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:229)
                atorg.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1086)
                atorg.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:429)
                atorg.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
                atorg.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020)
                atorg.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
                atorg.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
                atorg.eclipse.jetty.server.Server.handle(Server.java:370)
                atorg.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:494)
                atorg.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:982)
                atorg.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1043)
                atorg.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:865)
                atorg.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240)
                atorg.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
                atorg.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:696)
                atorg.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:53)
                atorg.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
                atorg.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
                atjava.lang.Thread.run(Unknown Source)
 (There is no English translation for this message.)
[EL Severe]: cache: 2016-12-19 19:43:09.681--ServerSession(2053940301)--Current object locks:
Thread : qtp71587369-76
Locked Object : org.eclipse.om2m.commons.entities.ContainerEntity@88037c5
PK: /in-cse/cnt-761411626
Depth : 1
Locked Object : org.eclipse.om2m.commons.entities.AeEntity@46d0a766
PK: /in-cse/CAE704876354
Depth : 1

End of locked objects.


I did another test:
Sent 10k data (4 cores, 2500 per core). Throughput at start was around 140s/s at end around 10/s. After i started sending again throughput was 7/s at beginning.

Hmm do you know what could be wrong? Am i sending to much/fast data?

[Updated on: Mon, 19 December 2016 19:08]

Report message to a moderator

Re: OM2M - Missing data in NSCL [message #1750366 is a reply to message #1750309] Tue, 20 December 2016 12:31 Go to previous messageGo to next message
Mahdi Ben Alaya is currently offline Mahdi Ben AlayaFriend
Messages: 229
Registered: November 2013
Senior Member
Hi Matej,

Our team is investigating this issue.
Thank your for the report.

Report message to a moderator

Re: OM2M - Missing data in NSCL [message #1751200 is a reply to message #1750366] Thu, 05 January 2017 15:08 Go to previous messageGo to next message
Mahdi Ben Alaya is currently offline Mahdi Ben AlayaFriend
Messages: 229
Registered: November 2013
Senior Member
Dear Matej,

1) After investigation, we think that the you are checking the database while the data is being persisted in the database.
This could explain why you found only 9480/15000.
This could explain also the error encountered while retrieving the conntent instances from the DATA container.

2) We made recently several fixes in the master branch so we recommend to use this version. We are going to upload new binaries in the wiki soon.

3) You could also try to use H2 database in server mode if it feeds your requirements.
We have the following tutorial for this purpose:
https://wiki.eclipse.org/OM2M/one/Advanced_DB

Thank you for your valuable feedback.

Report message to a moderator

Re: OM2M - Missing data in NSCL [message #1751208 is a reply to message #1751200] Thu, 05 January 2017 16:41 Go to previous message
Matej AA is currently offline Matej AAFriend
Messages: 23
Registered: February 2016
Junior Member
Hi,

no problem, thanks for investigating issue and your reply. If i understand correct, you think that data is missing because H2 write problem?

Report message to a moderator

Previous Topic:Develop Applications
Next Topic:What is the purpose of configuring RequestIdentifier?
Goto Forum:
  

[ Syndicate this forum (XML) ] [ RSS ]

Current Time: Thu Oct 10 20:08:16 GMT 2024

Powered by FUDForum. Page generated in 0.08801 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top

Sign up to our Newsletter

A fresh new issue delivered monthly