| Re: [ditto-dev] Exposing digital twin data via MQTT |
|
Hello Stefan, I’ve checked things container logs and I see that event is persisted: 2021-04-15 13:33:50,570 INFO [85cde180-bc11-4f99-bc36-37a735a22fa0] o.e.d.s.t.p.a.ThingPersistenceActor akka://ditto-cluster/system/sharding/thing/15/aptiv.car%3Acar01/pa -
Successfully persisted Event <things.events:featuresModified>. Inside connectivity container there is also information that signal is forwarded: 2021-04-15 13:42:33,367 INFO [b0c6564d-9eec-4c7d-a7bf-f2f203eb3739] o.e.d.s.m.a.AcknowledgementAggregatorActor akka://ditto-cluster/system/sharding/connection/12/mqtt-aptiv-car-connection-123/pa/$N/c1/inboundDispatching/ackr8-b0c6564d-9eec-4c7d-a7bf-f2f203eb3739
- Starting to wait for all requested acknowledgements <[twin-persisted]> for a maximum duration of <PT1M5S>. 2021-04-15 13:42:33,368 INFO [b0c6564d-9eec-4c7d-a7bf-f2f203eb3739] o.e.d.s.m.c.a.ConciergeForwarderActor akka://ditto-cluster/user/connectivityRoot/conciergeForwarder - Forwarding
signal with ID <aptiv.car:car01> and type <things.commands:modifyFeatures> to concierge enforcer But in concierge container I can’t see any logs that occurred after updating digital twin. Today I made also some tests with policy and I added new entry “observer”. I’ve set a new subject and I’ve given access not to thing:/ but to thing:/features and even to a single
property but it didn’t help. What could cause problem with publishing events? Thanks in advance. Regards, Adam Olek From: Maute Stefan (IOC/PAP-DDM-TH) [mailto:Stefan.Maute@xxxxxxxx]
Hello Adam, Regarding to
Ditto protocol specification commands/events section an event should be generated when the features of the thing are modified, am I correct? Yes you are correct a FeatureModifiedEvent is emitted when the feature is updated.
Should I enable an event generation somehow in configuration? No you don’t have to enable anything. An event is generated every time a thing is created/modified.
Maybe you can very that an FeatureModifiedEvent is emitted when checking the logs of the things service. Best regards Von: Olek, Adam <adam.olek@xxxxxxxxx>
Hello Stefan, I’ve applied your advice and now I see that Ditto is sending response but still there is no update about twin modification. This response is just a confirmation of receiving a message in my understanding. I’ve read more documentation and I’ve found something about
communication pattern. So far command is received and processed by Ditto. Later the success information message is sent but the event is not generated after twin modification. Regarding to
Ditto protocol specification commands/events section an event should be generated when the features of the thing are modified, am I correct? Should I enable an event generation somehow in configuration? I’m using Ditto containers with latest tag but today I’ve made a test with 1.5 version and results are the same. As you pleased my current communication configuration: { "targetActorSelection": "/system/sharding/connection", "headers": { "aggregate": false }, "piggybackCommand": { "type": "connectivity.commands:createConnection", "connection": { "id": "mqtt-aptiv-car-connection-123", "connectionType": "mqtt", "connectionStatus": "open", "failoverEnabled": true, "uri": "tcp://IP_ADDRESS:1883", "sources": [{ "addresses": ["aptiv-car/#"], "response-required": true, "payloadMapping": ["customJS"], "authorizationContext": ["nginx:ditto"], "replyTarget": { "address": "/test/mqtt", "expectedResponseTypes": ["response", "error"], "enabled": true }, "qos": 0, "filters": [] }], "targets": [{ "address": "aptiv-car-pub/{{ thing:name }}", "response-required": true, "payloadMapping": ["customJS"], "topics": [ "_/_/things/twin/events", "_/_/things/live/messages" ], "authorizationContext": ["nginx:ditto"], "qos": 0 }], "mappingDefinitions": { "customJS": { "mappingEngine": "_javascript_", "options": { "incomingScript": "function mapToDittoProtocolMsg(headers, textPayload, bytePayload, contentType) {const jsonString = String.fromCharCode.apply(null, new Uint8Array(bytePayload)); const jsonData
= JSON.parse(jsonString); const thingId = jsonData.thingId.split(':'); const value = {transmission: {properties: {rpm: jsonData.rpm, encoder: jsonData.encoder}}}; return Ditto.buildDittoProtocolMsg(thingId[0], thingId[1], 'things', 'twin', 'commands', 'modify',
'/features', headers, value);}", "outgoingScript": "function mapFromDittoProtocolMsg(namespace, id, group, channel, criterion, action, path, dittoHeaders, value, status, extra) {let headers = dittoHeaders; let textPayload = '{}';
let bytePayload = null; let contentType = 'text/plain; charset=UTF-8'; return Ditto.buildExternalMsg(headers, textPayload, bytePayload, contentType);}", "loadBytebufferJS": "false", "loadLongJS": "false" } } } } } } Thanks in advance. Regards, Adam Olek From: Maute Stefan (IOC/PAP-DDM-TH) [mailto:Stefan.Maute@xxxxxxxx]
Hello Adam, Could you please send me the your connection configuration? And tell me which ditto version you are using? If you want to get the response send back, you can configure a replay target in your source. You can specify a topic where the response should be published. Example: "sources": [ { …, "replyTarget": { "address": "/test/mqtt", "expectedResponseTypes": [ "response", "error" ], "enabled": true } } ] FYI: The "response-required": true
in your source and target are ignored when modifying the connection. Best regards Von: Olek, Adam <adam.olek@xxxxxxxxx>
Hello Stefan, Thank you for quick response. Yes, you understand everything correctly. If my understanding of Ditto’s documentation is correct there is such possibility(MQTT
3.1.1 binding). I’ve checked logs as you described and I see that outgoing script is invoked but later message is dropped: { "address": "_responses", "category": "response", "correlationId": "c4deab70-ee4c-4315-ad2c-e74d431ef873", "level": "success", "message": "Mapped outgoing signal with mapper <customJS>.", "thingId": "aptiv.car:car01", "timestamp": "2021-04-13T09:20:28.688999062Z", "type": "mapped" }, { "address": "_responses", "category": "response", "correlationId": "c4deab70-ee4c-4315-ad2c-e74d431ef873", "level": "success", "message": "Signal dropped,
target address unresolved: {{header:reply-to}} - Message headers: [mqtt.qos=0, ditto-read-revoked-subjects=[], correlation-id=c4deab70-ee4c-4315-ad2c-e74d431ef873, ditto-reply-target=0, mqtt.retain=false, mqtt.topic=aptiv-car/car01,
requested-acks=[\"twin-persisted\"], ditto-origin=mqtt-aptiv-car-connection-123, ditto-inbound-payload-mapper=customJS, response-required=false, ditto-originator=nginx:ditto, ditto-auth-context={\"type\":\"pre-authenticated-connection\",\"subjects\":[\"nginx:ditto\"]},
ditto-read-subjects=[\"nginx:ditto\"], ditto-entity-id=aptiv.car:car01, etag=\"hash:28c63cea\", ditto-expected-response-types=[\"response\",\"error\"]] - Message payload: {\"type\":\"things.responses:modifyFeatures\",\"status\":204,\"thingId\":\"aptiv.car:car01\",\"features\":null}", "thingId": "aptiv.car:car01", "timestamp": "2021-04-13T09:20:28.714395319Z", "type": "dropped" } There is information that header:reply-to can’t be resolved. In MQTT 3.1.1 there is no possibility to set header mappings therefore I’ve switched to MQTT 5 where such possibility exists but it didn’t help, it returns
same information. Additionally I see that there is different address from this one I’ve set in connection definition in target section. Is it a correct behavior of Ditto? I’ve also modified my connection definition a little bit: { "targetActorSelection": "/system/sharding/connection", "headers": { "aggregate": false }, "piggybackCommand": { "type": "connectivity.commands:modifyConnection", "connection": { "id": "mqtt-aptiv-car-connection-123", "connectionType": "mqtt", "connectionStatus": "open", "failoverEnabled": true, "uri": "tcp://IP_ADDRESS:1883", "sources": [{ "addresses": ["aptiv-car/#"], "response-required": true, "payloadMapping": ["customJS"], "authorizationContext": ["nginx:ditto"], "qos": 0, "filters": [] }], "targets": [{ "address": "aptiv-car-pub/{{ thing:name }}", "response-required": true, "payloadMapping": ["customJS"], "topics": [ "_/_/things/twin/events", "_/_/things/live/messages" ], "authorizationContext": ["nginx:ditto"], "qos": 0 }], "mappingDefinitions": { "customJS": { "mappingEngine": "_javascript_", "options": { "incomingScript": "function mapToDittoProtocolMsg(headers, textPayload, bytePayload, contentType) {const jsonString = String.fromCharCode.apply(null, new Uint8Array(bytePayload)); const jsonData
= JSON.parse(jsonString); const thingId = jsonData.thingId.split(':'); const value = {transmission: {properties: {rpm: jsonData.rpm, encoder: jsonData.encoder}}}; return Ditto.buildDittoProtocolMsg(thingId[0], thingId[1], 'things', 'twin', 'commands', 'modify',
'/features', headers, value);}", "outgoingScript": "function mapFromDittoProtocolMsg(namespace, id, group, channel, criterion, action, path, dittoHeaders, value, status, extra) {let headers = dittoHeaders; let textPayload = '{}';
let bytePayload = null; let contentType = 'text/plain; charset=UTF-8'; return Ditto.buildExternalMsg(headers, textPayload, bytePayload, contentType);}", "loadBytebufferJS": "false", "loadLongJS": "false" } } } } } } I’ve changed mappingContext to mappingDefinitions, and payloadMapping and response-required fields were set in source and target sections. But also these modifications didn’t help. How can I set header:reply-to? Should it be done inside JS script or in connection definition? Thanks in advance. Regards, Adam Olek From: Maute Stefan (IOC/PAP-DDM-TH) [mailto:Stefan.Maute@xxxxxxxx]
Hello Adam, Thank you for reaching out to the Ditto team. First of all to understand your use case a bit better I have a question: You want to update the thing via MQTT and receive afterwards the changes via MQTT? Is this
correct? Maybe it is a good start to enable the
connection logs and have a look at
the logs for the connection. Another option is to enable the Debug logs of the connectivity service:
https://www.eclipse.org/ditto/installation-operating.html#dynamically-adjust-log-levels
Best regards, Von: ditto-dev <ditto-dev-bounces@xxxxxxxxxxx>
Im Auftrag von Olek, Adam via ditto-dev Dear Ditto team, we're investigating capabilities of Ditto and we've got small problem with exposing data via Ditto connectivity. Our case is: - we've created simple digital twin in Ditto - it is updated with MQTT protocol and it’s work - no update from digital twin is received after modification of digital twin's features Below I'll describe our configuration. Our simple digital twin definition: { "policyId": "my.test:policy", "attributes": { "manufacturer": "Aptiv", "VIN": "0815666337" }, "features": { "transmission": { "properties": { "rpm": 0, "encoder": 0 } } } } Policy definition: { "policyId": "my.test:policy", "entries": { "owner": { "subjects": { "nginx:ditto": { "type": "nginx basic auth user" } }, "resources": { "thing:/": { "grant": ["READ","WRITE"], "revoke": [] }, "policy:/": { "grant": ["READ","WRITE"], "revoke": [] }, "message:/": { "grant": ["READ","WRITE"], "revoke": [] } } } } } Also different subjects were used and it didn’t help. Connection definition: { "targetActorSelection": "/system/sharding/connection", "headers": { "aggregate": false }, "piggybackCommand": { "type": "connectivity.commands:modifyConnection", "connection": { "id": "mqtt-aptiv-car-connection-123", "connectionType": "mqtt", "connectionStatus": "open", "failoverEnabled": true, "uri": "tcp://IP_ADDRESS:1883", "sources": [{ "addresses": ["aptiv-car/#"], "authorizationContext": ["nginx:ditto"], "qos": 0, "filters": [] }], "targets": [{ "address": "aptiv-car-pub/{{ thing:name }}", "topics": [ "_/_/things/twin/events", "_/_/things/live/messages" ], "authorizationContext": ["nginx:ditto"], "qos": 0 }], "mappingContext": { "mappingEngine": "_javascript_", "options": { "incomingScript": "function mapToDittoProtocolMsg(headers, textPayload, bytePayload, contentType) {const jsonString = String.fromCharCode.apply(null, new Uint8Array(bytePayload)); const jsonData =
JSON.parse(jsonString); const thingId = jsonData.thingId.split(':'); const value = {transmission: {properties: {rpm: jsonData.rpm, encoder: jsonData.encoder}}}; return Ditto.buildDittoProtocolMsg(thingId[0], thingId[1], 'things', 'twin', 'commands', 'modify',
'/features', headers, value);}", "outgoingScript": "function mapFromDittoProtocolMsg(namespace, id, group, channel, criterion, action, path, dittoHeaders, value, status, extra) {let headers = dittoHeaders; let textPayload = '{}';
let bytePayload = null; let contentType = 'text/plain; charset=UTF-8'; return Ditto.buildExternalMsg(headers, textPayload, bytePayload, contentType);}", "loadBytebufferJS": "false", "loadLongJS": "false" } } } } } So far outgoingScript is setting an empty paylod, only brackets. I've also experimented with application/json as contentType. After setting the whole setup we are able to update our model with MQTT. Inside Ditto containers logs with information that data was received are visible but there is no information that any data after digital twin update
was sent. Also no such information is visible inside MQTT broker. Should I modify "targets" in connection configuration? Maybe policy is incorrect? How can I check if outgoingScript is executed(it cannot access the filesystem, it cannot print anything)? This email is quite long but I hope I’ve described everything correct and all needed details are included. Regards, Adam Olek |