Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Eclipse Projects » Kura » persistance in kura(h2 data base )
persistance in kura [message #1778215] Tue, 12 December 2017 04:45 Go to next message
gopal korrapati is currently offline gopal korrapatiFriend
Messages: 48
Registered: October 2017
Member
Hello every one,
Currently I am working persistance part in kura . I am faceing issues to create a data base I am following the bellow link. And I created one of the service like testdb . after that I checked that port through netstat -l that port is in listen mode.(fdbc:h2:mem:testdb) I am not getting where should I create these testdb.after that I sending data by using heater demo bundle where it is stored.one more issue I opened another tab I gave url like:192.168.1.201:9123, here 9123 is my port,it show like these .

https://www.dropbox.com/home?preview=testdb.png

https://www.dropbox.com/home?preview=port.png
  • Attachment: testdb.png
    (Size: 167.17KB, Downloaded 94 times)
Re: persistance in kura [message #1778216 is a reply to message #1778215] Tue, 12 December 2017 04:49 Go to previous messageGo to next message
gopal korrapati is currently offline gopal korrapatiFriend
Messages: 48
Registered: October 2017
Member
These is the link I followed to create a databe in kura.
https://eclipse.github.io/kura/builtin/h2-database-external-connections.html
Re: persistance in kura [message #1778246 is a reply to message #1778216] Tue, 12 December 2017 09:50 Go to previous messageGo to next message
Nicola Timeus is currently offline Nicola TimeusFriend
Messages: 12
Registered: May 2017
Junior Member
Hi,

if you have not already done so, I would recommend you to have a look at this guide as well:
https://eclipse.github.io/kura/builtin/h2-database.html
before reading the guide you mentioned.

Basically Kura provides the following two database-related services:

H2DbService (factory pid org.eclipse.kura.core.db.H2DbService): This represent an actual database instance that can be used to store your data, and it is identified by the following two identifiers:
- a JDBC URL, like the one you mentioned (jdbc:h2:mem:testdb) that can be provided to external applications for connecting to the database.
- the Kura service pid: a Kura specific identifier that can be supplied to other services that run inside Kura (like the DataService, the H2 db wire components) to select a specific db instance.
All data published by an application (like Heater) using a CloudService/DataService is stored into an H2DbService instance (the one selected in the DataService configuration) inside the "DS_MESSAGES" table.

H2DbServer: This is a component that provides access to the Kura database instances (instances of H2DbServices) to processes running outside Kura.
It is basically just a server that provides access to existing db instances trough some network protocol (JDBC, Postgres) and, in addition, is also capable to provide a web console that can be used to inspect the database instances.
If I understand correctly, you are trying to open the H2 web console using your browser, but your H2DbServer instance is configured for JDBC (TCP type), if this is the case the H2DbServer instance must be reconfigured as follows:

- db.server.type: WEB
- db.server.commandline: -webPort 9123 -webAllowOthers -ifExists

then you should be able open the web console, provided that port 9123 is open in the firewall configuration section.
By the way, I'm not able to open the dropbox links you posted, can you please attach the screens like you did for testdb.png?

In any case, you can find more details regarding the H2DbService and H2DbServer components and their configuration in the guide I posted above.

Best Regards,
Nicola
Re: persistance in kura [message #1778252 is a reply to message #1778246] Tue, 12 December 2017 10:29 Go to previous messageGo to next message
gopal korrapati is currently offline gopal korrapatiFriend
Messages: 48
Registered: October 2017
Member
I followed the what ever you given link. but web part I unable to open.
https://www.dropbox.com/personal?preview=port.png

file:///home/gopal/Pictures/port.png
  • Attachment: port.png
    (Size: 120.68KB, Downloaded 80 times)

[Updated on: Tue, 12 December 2017 10:35]

Report message to a moderator

Re: persistance in kura [message #1778263 is a reply to message #1778252] Tue, 12 December 2017 12:22 Go to previous messageGo to next message
gopal korrapati is currently offline gopal korrapatiFriend
Messages: 48
Registered: October 2017
Member
thanks a lot for giving the link.
I sucessfully opened the webpart. there how can I see the data what ever I send through demo heater bundle.in these I gave (jdbc:h2:mem:testdb),what is the diffrence between the (jdbc:h2:mem:testdb) and (jdbc:h2:file:~/data) here (testdb) and (data) is just files I created inside my board home path. please tell me.

file:///home/gopal/Pictures/h2console.png
  • Attachment: h2console.png
    (Size: 162.51KB, Downloaded 76 times)

[Updated on: Tue, 12 December 2017 12:23]

Report message to a moderator

Re: persistance in kura [message #1778268 is a reply to message #1778263] Tue, 12 December 2017 13:23 Go to previous messageGo to next message
Nicola Timeus is currently offline Nicola TimeusFriend
Messages: 12
Registered: May 2017
Junior Member
Hi

For viewing the data published by the heater demo, you need to enter in the Web console the db url of the instance used by the DataService, in this case you should be able to find the data inside the "DS_MESSAGES" table.
In the default configuration, this instance is automatically created, and it has the "H2DbService" kura service pid, (you should see it under Services in the web ui) and its db url should be "jdbc:h2:mem:kuradb". By entering this last URL in the Web console you should be able to see the heater data.

If use the "jdbc:h2:mem" URI you are creating an in-memory (RAM) database instance, that is not persisted on the device storage, so there are no files involved.
In-memory dbs are quite fast and do not wear out flash storage, but all data contained in them is lost when Kura is restarted, the db instance is deleted, or if you change the URL of the db instance to something different.
You can use this for non-critical temporary data that is frequently updated, and it is the default for the Data Service db.

Using the "jdbc:h2:file" URI you get a persistent database instance, that uses a regular file for storing the data.
In this case the data is retained as long as the backing file exists, you can append different parameters to the url string for tuning the db performance and reliability.
You can find more details about the db URIs at the end of the guide I've posted above and in the official H2 documentation, for example here:
http://www.h2database.com/html/features.html#database_url.
In this case it is not necessary to create the files before opening the instance, they will be created automatically.

One more thing about the H2DbServer: if you need to use the TCP/JDBC server and the Web console at the same time, you can create two different H2DbServer instances listening on two different ports.
One instance needs to be configured in TCP mode and the other one in WEB mode.

Best Regards,
Nicola
Re: persistance in kura [message #1778318 is a reply to message #1778268] Wed, 13 December 2017 05:39 Go to previous messageGo to next message
gopal korrapati is currently offline gopal korrapatiFriend
Messages: 48
Registered: October 2017
Member
thanks for reply

know I gave db.connector.url :- jdbc:h2:file:~/data here data is I created a file in my target home path. but demo heater values are not stored in data. demo heater values where it is stored in my target.
if I remove the data file in my target.but table is created in web console,how it is created the table in web console if I remove the data file. but inside the table it showing my demo heater values in payload very big.attached a file please go through that.
  • Attachment: payload.png
    (Size: 166.09KB, Downloaded 98 times)

[Updated on: Wed, 13 December 2017 05:41]

Report message to a moderator

Re: persistance in kura [message #1778363 is a reply to message #1778318] Wed, 13 December 2017 13:29 Go to previous messageGo to next message
ermanno porporato is currently offline ermanno porporatoFriend
Messages: 26
Registered: November 2017
Junior Member
Using Kura, eclipse on windows, and the db h2 in persistence mode, very, very often I have the message:

dbcSQLException: General error: "java.lang.IllegalStateException: The file is locked:

I use one instance of H2DbService and two of H2DbServer, with ports 9123 and 9124

Any advice on this problem is welcome (I do not open issue reports for now having no more precise and since I have not used the db in Kura for long )

thank you
Re: persistance in kura [message #1778370 is a reply to message #1778363] Wed, 13 December 2017 14:17 Go to previous messageGo to next message
mario terzo is currently offline mario terzoFriend
Messages: 3
Registered: November 2017
Junior Member
.

[Updated on: Wed, 13 December 2017 14:20]

Report message to a moderator

Re: persistance in kura [message #1778371 is a reply to message #1778363] Wed, 13 December 2017 14:20 Go to previous messageGo to next message
Nicola Timeus is currently offline Nicola TimeusFriend
Messages: 12
Registered: May 2017
Junior Member
@ermanno porporato
Do you have multiple applications that are trying to open the db file at the same time?
That exception can occur for example if you accidentally start the Kura emulator more than once and end up with multiple instances of it running at the same time.

@gopal korrapati
Heater data is stored in the H2DbService instance attached to the DataService instance you are using.
Basing upon the screen you posted, you managed to find the correct instance.
The metrics published by the heater are not directly visible in the db console because they are encoded using Protobuf and then inserted in a binary column.
The content of the payload column is used as the payload of the MQTT messages sent to the cloud.
Re: persistance in kura [message #1778373 is a reply to message #1778371] Wed, 13 December 2017 14:31 Go to previous messageGo to next message
ermanno porporato is currently offline ermanno porporatoFriend
Messages: 26
Registered: November 2017
Junior Member
[quote title=Nicola Timeus wrote on Wed, 13 December 2017 14:20]@ermanno porporato
Do you have multiple applications that are trying to open the db file at the same time?
That exception can occur for example if you accidentally start the Kura emulator more than once and end up with multiple instances of it running at the same time.

no, in general I have only an eclipse project launched via the emulator and evenely the H2 console But things happen a bit strange, it seems that sometimes the writing on db works and other times no. I see in the doc of H2 that they also recommend to kill the java service in this case.. I continue the test.
it also seems that the variation of some parameters is not taken, for example the 'db.server.enabled' flag from true to false does not seem to work sometimes, also stopping the project, Kura continue to append records to the table.. but I continue to test other examples..
Re: persistance in kura [message #1778378 is a reply to message #1778373] Wed, 13 December 2017 14:57 Go to previous messageGo to next message
Nicola Timeus is currently offline Nicola TimeusFriend
Messages: 12
Registered: May 2017
Junior Member
By stopping the project you mean stopping the emulator using the stop button above the console or just stopping your project's bundle?
Basically every time you start the emulator you will start a new instance of the entire Kura framework, not just your project. If you have more that one instance running some issues can occur.
The fact that the variation of some parameters are not taken and that Kura continues to append records to a table can be symptoms of this.
For this reason if you make some changes to a project it is important that you stop the emulator using the stop button and then restart it again every time before testing.
In order to make sure that there is only one Kura instance running you can try closing and reopening Eclipse, and then check if the issue occurs again.

Thank you for your testing
Nicola
Re: persistance in kura [message #1778433 is a reply to message #1778378] Thu, 14 December 2017 09:06 Go to previous messageGo to next message
ermanno porporato is currently offline ermanno porporatoFriend
Messages: 26
Registered: November 2017
Junior Member
Thanks for the answer. I think the problem is just that sometimes there are more running instances..
Generally in eclipse I start the Eclipse project by launching right click on the script Kura_emulator ...

then I open the Kura console in browser, localhost:8080,
(wires, configuration, ecc..)

at the end I stop with the small red square 'Terminate' in eclipse
I do not understand what do you mean with ' stop button above the console' (??)

The problem is that often, apparently, the situation does not change also closing eclipse..

n.b. we are always working in windows, the script to launch is the script 'home made' by us' modifying the script for linux... (and works well, no problems)

[Updated on: Thu, 14 December 2017 09:26]

Report message to a moderator

Re: persistance in kura [message #1778437 is a reply to message #1778433] Thu, 14 December 2017 09:34 Go to previous messageGo to next message
ermanno porporato is currently offline ermanno porporatoFriend
Messages: 26
Registered: November 2017
Junior Member
Here is an exemple of wires used, 'MODBUS ASSET' receives the data from a modbus simulator, 2 channels, timer set to 5 seconds

(broker activemq, publish and subscribe without problems)

And then the data written in the db, table 'mytable3' (yesterday, now there are problems...)
  • Attachment: gra1.jpg
    (Size: 45.97KB, Downloaded 70 times)
  • Attachment: gra2.jpg
    (Size: 216.73KB, Downloaded 63 times)

[Updated on: Thu, 14 December 2017 10:05]

Report message to a moderator

Re: persistance in kura [message #1778451 is a reply to message #1778437] Thu, 14 December 2017 10:26 Go to previous messageGo to next message
ermanno porporato is currently offline ermanno porporatoFriend
Messages: 26
Registered: November 2017
Junior Member
I made several attempts, including deactivating the bundle heater, turning off the computer, restarting everything and verifying that the modbus simulator works correctly.
Now it seems that everything works, if I stop the project and restart it kura resumes to write the records correctly, even without closing eclipse .. however any advice about this type of problems is always welcome..

[Updated on: Thu, 14 December 2017 10:27]

Report message to a moderator

Re: persistance in kura [message #1778501 is a reply to message #1778451] Fri, 15 December 2017 08:21 Go to previous messageGo to next message
ermanno porporato is currently offline ermanno porporatoFriend
Messages: 26
Registered: November 2017
Junior Member
a report that can be useful: this morning the wire graph above
(message #1778437 ) did not write anything on the db. I removed only connection with the wire component H2DBSTORE, save, re-establish the connection, save, and the writing has begun .. without stopping anything

[Updated on: Fri, 15 December 2017 08:22]

Report message to a moderator

Re: persistance in kura [message #1778571 is a reply to message #1778501] Mon, 18 December 2017 05:00 Go to previous messageGo to next message
gopal korrapati is currently offline gopal korrapatiFriend
Messages: 48
Registered: October 2017
Member
as per your reply, in cloud service I kept payload.encoding :- simple json,then also I got same data in payload.How do I get whatever demo heater sending data in payload.
  • Attachment: payload.png
    (Size: 166.09KB, Downloaded 61 times)

[Updated on: Mon, 18 December 2017 05:02]

Report message to a moderator

Re: persistance in kura [message #1778572 is a reply to message #1778571] Mon, 18 December 2017 05:03 Go to previous messageGo to next message
gopal korrapati is currently offline gopal korrapatiFriend
Messages: 48
Registered: October 2017
Member
How can I get the Exact data in payload in H2 console.Please help me
Re: persistance in kura [message #1778703 is a reply to message #1778572] Wed, 20 December 2017 10:38 Go to previous messageGo to next message
Nicola Timeus is currently offline Nicola TimeusFriend
Messages: 12
Registered: May 2017
Junior Member
Hi

You can try the query in the attached screen
  • Attachment: query.png
    (Size: 375.53KB, Downloaded 55 times)
Re: persistance in kura [message #1778710 is a reply to message #1778703] Wed, 20 December 2017 11:17 Go to previous message
gopal korrapati is currently offline gopal korrapatiFriend
Messages: 48
Registered: October 2017
Member
thanks for your support, now am acquiring according to my requirement.
Previous Topic:Collecting system metrics from Kura devices
Next Topic:Problem, creating new Wire Asset
Goto Forum:
  


Current Time: Fri Sep 21 17:45:35 GMT 2018

Powered by FUDForum. Page generated in 0.03139 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top