Re: [kanto-dev] Software update

[Dirk Van Haerenborgh <dirk.vanhaerenborgh@xxxxxxxx>]

Hi Tina,

I did indeed receive your reply to my question. Thanks for that again!

It's just that I've been very busy with another project so I had to postpone my work with Kanto for a bit.

Recently, I've been working on the base image for our edge device. It'll probably be using libostree for 

managing atomic upgrades. Given how that works, it's required for all binaries to be installed to /usr/bin
rather than /usr/local/bin. You may expect some PRs for that shortly.

Kind regards,
-Dirk

Dirk Van Haerenborgh Software engineer

Aloxy The Beacon, Sint-Pietersvliet 7 2000 Antwerp www.aloxy.io

---

From: kanto-dev <kanto-dev-bounces@xxxxxxxxxxx> on behalf of Gramatova Dr. Konstantina (IOB/PAD-PM1) <Konstantina.Gramatova@xxxxxxxx>
Sent: Tuesday, 9 August 2022 14:00
To: kanto developer discussions <kanto-dev@xxxxxxxxxxx>
Subject: Re: [kanto-dev] Software update

 

Hey Dirk!

I hope all is well on your side 😊

I just wanted to make sure that my last response (see quoted below) has reached you - we had a small kanto-dev mailing list issue that the Eclipse team gracefully helped us to resolve ASAP when it was found 😊 (unfortunately, it took us some time to notice it at first place...)

Any updates on the use cases and your Kanto experience so far?

Looking forward to hearing from you!

Tina

Поздрави / Best regards

Dr. Konstantina Gramatova

Bosch.IO GmbH | Ziegelei 7 | 88090 Immenstaad | GERMANY | www.bosch.io
Tel. +359(2)9055876 | Mobile +359(883)472547 | Fax +359 2 95326-17 | Konstantina.Gramatova@xxxxxxxx

Registered Office: Berlin, Registration Court: Amtsgericht Charlottenburg; HRB 148411 B
Chairman of the Supervisory Board: Stefan Koss; Managing Directors: Dr. Andreas Nauerz, Yvonne Reckling

---

From: kanto-dev <kanto-dev-bounces@xxxxxxxxxxx> on behalf of Gramatova Dr. Konstantina (IOB/PAD-PM1) <Konstantina.Gramatova@xxxxxxxx>
Sent: Tuesday, July 12, 2022 11:46 PM
To: kanto developer discussions <kanto-dev@xxxxxxxxxxx>
Subject: Re: [kanto-dev] Software update

 

Hey Dirk!

Really glad that it turned out to be a small bugger : ))

Regarding your question about inlining the script content into the MQTT message:

There have always been a couple of considerations that have led our decisions regarding the MQTT messages content in general but especially when it comes to potential mass management operations (which SOTA is):

• In order to scale such an operation, you would most probably want to have a single artifact (e.g. a script) downloaded multiple times, rather than sending the artifact's content multiplied 100..0 times being inlined in the messages sent to the respective devices bloating & jamming up the traffic and moving the processing overload on the device instead of handling it in the cloud
  
• Each backend has different limitations when it comes to the size per transferred message and this could potentially bring you a big headache to make sure you can securely transfer the inlined content intact each time

Given that in mind, we have provided the install​ configuration of the component which you can check out in the reference guide (this is the future home of our website that we are currently proofing to finalize the domain migration :). If you have a use case with a fairly simple script, you can directly configure its content (or file path) there, thus, no need to transfer it over-the-air. Indeed, this a fairly static approach and has its own use cases where it could be feasible to apply it so my question would be - what is the use case you would like to fulfill opting for inlining the script content in the message? Given the flexible Vorto model we are using combined with this configuration, I'm sure we will find a way to achieve what you are looking for and we are open to diving into and enabling new use cases 😊

Looking forward to your feedback!

Tina

Поздрави / Best regards

Dr. Konstantina Gramatova

(IOB/PAD-PM1)
Bosch.IO GmbH | Ziegelei 7 | 88090 Immenstaad | GERMANY | www.bosch.io
Tel. +359(2)9055876 | Mobile +359(883)472547 | Fax +359 2 95326-17 | Konstantina.Gramatova@xxxxxxxx

Registered Office: Berlin, Registration Court: Amtsgericht Charlottenburg; HRB 148411 B
Chairman of the Supervisory Board: Stefan Koss; Managing Directors: Dr. Andreas Nauerz, Yvonne Reckling

---

From: kanto-dev <kanto-dev-bounces@xxxxxxxxxxx> on behalf of Dirk Van Haerenborgh <dirk.vanhaerenborgh@xxxxxxxx>
Sent: Tuesday, July 12, 2022 4:07 PM
To: kanto developer discussions <kanto-dev@xxxxxxxxxxx>
Subject: Re: [kanto-dev] Software update

 

Hi Tina,

Thanks for the info. To make a long story short; I had a typo in my device id. After correcting that, I've been able to play around with the software-updater feature.

I like that the default is to just send an install.sh script that contains all install/update instructions. That's really powerful.

What would also be cool is if we could just send all the update instructions (the contents of install.sh) just inline in the mqtt message. Is that something you'd be open to?

Kind regards and thanks for the effort you've put in Kanto thus far.

-Dirk

 

Dirk Van Haerenborgh Software engineer

Aloxy The Beacon, Sint-Pietersvliet 7 2000 Antwerp www.aloxy.io

---

From: kanto-dev <kanto-dev-bounces@xxxxxxxxxxx> on behalf of Gramatova Dr. Konstantina (IOB/PAD-PM1) <Konstantina.Gramatova@xxxxxxxx>
Sent: Tuesday, 5 July 2022 18:22
To: kanto developer discussions <kanto-dev@xxxxxxxxxxx>
Subject: Re: [kanto-dev] Software update

 

Hey Dirk!

Apologies for the late response - got a small vacation time.

You are really on the right track of getting the hang of the concept and how this feature works! I'll try to clear out the open questions you have with the comments on the bullets below:

When starting, the software-updater service will get its configuration from the config file and will immediately fetch it from ditto (if existing).

With configuration I am referring to the shell command for a specific package type. For instance: if the binary packages are debian packages, the shell command should be dpkg -i​.
As such, Kanto would support multiple independent binary types

Yes, the script to be executed can be configured via the file loaded on start up by the service. Not really sure what do you mean by

and will immediately fetch it from ditto (if existing)

Could you elaborate a little bit on that just to make sure I get you right? The only thing "fetched" by the service when it starts up is the device's info populated by the suite-connector component on request locally - e.g. the ID of the thing associated with the provisioned device in Hono, etc. This info is further used for the Ditto payloads composition that the feature uses. Did you refer to this one?

Although I can't find much evidence for it, I suspect that the software-update service also subscribes to the local mqtt broker. Rather than listening on the Thing <thingId>:edge:containers, it will listen on <thingId> itself.
From the source code, I've been trying to construct the payload for it (see attachment).
I've tried sending this via the hono command topic (on kafka) to the gateway, but so far I haven't seen any indication of success

Again - you are right : ) The component does subscribe to the local broker and for the Hono topics related to the initially provisioned Hono device as the SoftwareUpdatable feature it registers and handles is added to it. The flow that you have described regarding the Hono command topic and the target device looks correct. The topic and path of the Ditto message you shared look ok too.

I'm sharing an example payload for you to check as well taken from an actual working scenario:

{

  "topic": "my.ns/gateway/things/live/messages/install",

  "headers": {

    "response-required": true,

    "content-type": "application/json",

    "correlation-id": "p-pdid7z-16h5900127if6e-17az3tz"

  },

  "path": "/features/SoftwareUpdatable/inbox/messages/install",

  "value": {

    "metaData": {},

    "softwareModules": [{

      "metaData": {},

      "softwareModule": {

        "name": "tree",

        "version": "1.8.0-2"

      },

      "artifacts": [{

        "checksums": {

          "SHA256": "9e53230fd20047586d12d3de6f3cadcef51d41a8594d742d0f58c571eda72282",

          "SHA1": "4e285f21c81ae331ed46a5541916cac1b66e36ab",

          "MD5": "2f2dfe898e63dfe44fef9ecbd08ea7ff"

        },

        "download": {

          "HTTPS": {

            "md5url": "https://cdn.....",

            "url": "https://cdn....."

          }

        },

        "filename": "tree_1.8.0-1_amd64.deb",

        "size": 43044

      }]

    }],

    "forced": true,

    "weight": null,

    "correlationId": "f976dd9c-6eb4-4b21-9f04-53e6ae4b0dbe"

  }

}

Could you please compare this one with the one you are testing with because I may be missing something as well when I did it.

Another thing you could do and might be helpful is to enable and check out the software-update component's logs (which I'm sure you have already figured out how and where to do that - via the logLevel and logFile configuration properties).

We're now also running a snapshot of the main branch; the last few commits made the mqtt connection a lot more stable.

Really happy to have this confirmed and that the improvement efforts have the intended effect!

Please, let me know if you still have troubles with getting the flow running via Hono given the clarifications - we'll dig deeper if needed to make sure everything is up and running on your setup!

Looking forward to your feedback!

Tina

Поздрави / Best regards

Dr. Konstantina Gramatova

(IOB/PAD-PM1)
Bosch.IO GmbH | Ziegelei 7 | 88090 Immenstaad | GERMANY | www.bosch.io
Tel. +359(2)9055876 | Mobile +359(883)472547 | Fax +359 2 95326-17 | Konstantina.Gramatova@xxxxxxxx

Registered Office: Berlin, Registration Court: Amtsgericht Charlottenburg; HRB 148411 B
Chairman of the Supervisory Board: Stefan Koss; Managing Directors: Dr. Andreas Nauerz, Yvonne Reckling

---

From: kanto-dev <kanto-dev-bounces@xxxxxxxxxxx> on behalf of Dirk Van Haerenborgh <dirk.vanhaerenborgh@xxxxxxxx>
Sent: Monday, July 4, 2022 6:43 PM
To: kanto developer discussions <kanto-dev@xxxxxxxxxxx>
Subject: Re: [kanto-dev] Software update

 

Hi,

I've been going through the software-update feature some more. I think I'm close to understanding how it works, but if you have time, I'd appreciate it very much if you could review the following statements:

• When starting, the software-updater service will get its configuration from the config file and will immediately fetch it from ditto (if existing).
  With configuration I am referring to the shell command for a specific package type. For instance: if the binary packages are debian packages, the shell command should be dpkg -i​.
  As such, Kanto would support multiple independent binary types
• Although I can't find much evidence for it, I suspect that the software-update service also subscribes to the local mqtt broker. Rather than listening on the Thing <thingId>:edge:containers, it will listen on <thingId> itself.
  From the source code, I've been trying to construct the payload for it (see attachment).
  I've tried sending this via the hono command topic (on kafka) to the gateway, but so far I haven't seen any indication of success

We're now also running a snapshot of the main branch; the last few commits made the mqtt connection a lot more stable.

Kind regards,
-Dirk

Dirk Van Haerenborgh Software engineer

Aloxy The Beacon, Sint-Pietersvliet 7 2000 Antwerp www.aloxy.io

---

From: kanto-dev <kanto-dev-bounces@xxxxxxxxxxx> on behalf of Dirk Van Haerenborgh <dirk.vanhaerenborgh@xxxxxxxx>
Sent: Friday, 1 July 2022 11:49
To: kanto-dev@xxxxxxxxxxx <kanto-dev@xxxxxxxxxxx>
Subject: [kanto-dev] Software update

 

Hi, 

Having no experience whatsoever with Hawkbit (which could be helpful seeing as it uses parts of the hawkbit client framework), we were wondering if you could provide us with a tiny example/demonstrator for how this should be used.

At the moment, all of this seems fairly abstract. How I read it in the source code is that every binary that is downloaded is accompanied with an installer script. So we should be free in deciding how our edge gateway would be updated. For example install snap packages for Ubuntu Core. Is this correct?

Kind regards,
-Dirk

Dirk Van Haerenborgh Software engineer

Aloxy The Beacon, Sint-Pietersvliet 7 2000 Antwerp www.aloxy.io