[volttron-dev] MQTT Historian Questions

["Thompson, Joe" <jthompson@xxxxxxxx>]

Hi Volttron team, 

 

We’re using Volttron 8.2 and are trying to use the MQTTHistorian [github.com] to get data from Voltrron’s Fake Driver to an external MQTT broker and I have a couple questions: 

 

1. Every 5 seconds the Platform Driver is reading 23 points from the fake device. Then the MQTT Historian sends that data to our MQTT broker, but the problem is that it seems to send each of the 23 data points as an individual message. A sample of my log file is attached which shows this behavior. It would be prefferable if each message from the MQTTHistorian -->  Broker was one JSON containing all data points, like what comes out of the Platform Driver. 

Is this possible via some agent configuration? Or will this require some modification of the MQTTHistorian on my end? 

 

2. Is there an easy way to limit the topics that the MQTT Historian forwards to the broker? By default it seems to forward every data point. Maybe this would require some agent modification as well? 

 

Thanks in advance for the help!

 

Joe Thompson

Engineer / Scientist

Electric Power Research Institute

Energy Storage and Distributed Generation 

(912) 663-3407

 

 

*** This email message is for the sole use of the intended recipient(s) and may contain information that is confidential, privileged or exempt from disclosure under applicable law. Unless otherwise expressed in this message by the sender or except as may be allowed by separate written agreement between EPRI and recipient or recipient’s employer, any review, use, distribution or disclosure by others of this message is prohibited and this message is not intended to be an electronic signature, instrument or anything that may form a legally binding agreement with EPRI. If you are not the intended recipient, please contact the sender by reply email and permanently delete all copies of this message. Please be advised that the message and its contents may be disclosed, accessed and reviewed by the sender's email system administrator and/or provider. ***

2023-07-13 14:52:30,001 (platform_driveragent-4.0 10321) platform_driver.driver DEBUG: campus/building/fake next scrape scheduled: 2023-07-13 18:52:35+00:00
2023-07-13 14:52:30,002 (platform_driveragent-4.0 10321) platform_driver.driver DEBUG: scraping device: campus/building/fake
2023-07-13 14:52:30,002 (platform_driveragent-4.0 10321) platform_driver.driver DEBUG: publishing: devices/campus/building/fake/all
2023-07-13 14:52:30,003 () volttron.platform.main DEBUG: Handling subsystem with frames: ['platform.driver', '', 'VIP1', '', '1689273099.000000.8738976235659.000000', 'pubsub', 'publish', 'devices/campus/building/fake/all', {'bus': '', 'headers': {'Date': '2023-07-13T18:52:30.002169+00:00', 'TimeStamp': '2023-07-13T18:52:30.002169+00:00', 'SynchronizedTimeStamp': '2023-07-13T18:52:30.000000+00:00', 'min_compatible_version': '3.0', 'max_compatible_version': ''}, 'message': [{'OutsideAirTemperature1': 50.0, 'SampleLong1': 50, 'SampleBool1': True, 'OutsideAirTemperature2': 50.0, 'SampleLong2': 50, 'SampleBool2': True, 'OutsideAirTemperature3': 50.0, 'SampleLong3': 50, 'SampleBool3': True, 'Heartbeat': True, 'EKG': 5.66553889764798e-16, 'SampleWritableFloat1': 10.0, 'SampleWritableShort1': 20, 'SampleWritableBool1': True, 'SampleWritableFloat2': 10.0, 'SampleWritableShort2': 20, 'SampleWritableBool2': True, 'SampleWritableFloat3': 10.0, 'SampleWritableShort3': 20, 'SampleWritableBool3': True, 'PowerState': 0, 'ValveState': 0, 'EKG_Sin': 5.66553889764798e-16, 'EKG_Cos': 5.66553889764798e-16}, {'Heartbeat': {'units': 'On/Off', 'type': 'integer', 'tz': 'US/Pacific'}, 'EKG': {'units': 'waveform', 'type': 'integer', 'tz': 'US/Pacific'}, 'OutsideAirTemperature1': {'units': 'F', 'type': 'integer', 'tz': 'US/Pacific'}, 'SampleWritableFloat1': {'units': 'PPM', 'type': 'integer', 'tz': 'US/Pacific'}, 'SampleLong1': {'units': 'Enumeration', 'type': 'integer', 'tz': 'US/Pacific'}, 'SampleWritableShort1': {'units': '%', 'type': 'integer', 'tz': 'US/Pacific'}, 'SampleBool1': {'units': 'On / Off', 'type': 'integer', 'tz': 'US/Pacific'}, 'SampleWritableBool1': {'units': 'On / Off', 'type': 'integer', 'tz': 'US/Pacific'}, 'OutsideAirTemperature2': {'units': 'F', 'type': 'integer', 'tz': 'US/Pacific'}, 'SampleWritableFloat2': {'units': 'PPM', 'type': 'integer', 'tz': 'US/Pacific'}, 'SampleLong2': {'units': 'Enumeration', 'type': 'integer', 'tz': 'US/Pacific'}, 'SampleWritableShort2': {'units': '%', 'type': 'integer', 'tz': 'US/Pacific'}, 'SampleBool2': {'units': 'On / Off', 'type': 'integer', 'tz': 'US/Pacific'}, 'SampleWritableBool2': {'units': 'On / Off', 'type': 'integer', 'tz': 'US/Pacific'}, 'OutsideAirTemperature3': {'units': 'F', 'type': 'integer', 'tz': 'US/Pacific'}, 'SampleWritableFloat3': {'units': 'PPM', 'type': 'integer', 'tz': 'US/Pacific'}, 'SampleLong3': {'units': 'Enumeration', 'type': 'integer', 'tz': 'US/Pacific'}, 'SampleWritableShort3': {'units': '%', 'type': 'integer', 'tz': 'US/Pacific'}, 'SampleBool3': {'units': 'On / Off', 'type': 'integer', 'tz': 'US/Pacific'}, 'SampleWritableBool3': {'units': 'On / Off', 'type': 'integer', 'tz': 'US/Pacific'}, 'PowerState': {'units': '1/0', 'type': 'integer', 'tz': 'US/Pacific'}, 'ValveState': {'units': '1/0', 'type': 'integer', 'tz': 'US/Pacific'}, 'EKG_Sin': {'units': '1-0', 'type': 'integer', 'tz': 'US/Pacific'}, 'EKG_Cos': {'units': '1-0', 'type': 'integer', 'tz': 'US/Pacific'}}]}] user_id: platform.driver
2023-07-13 14:52:30,004 (platform_driveragent-4.0 10321) platform_driver.driver DEBUG: finish publishing: devices/campus/building/fake/all
2023-07-13 14:52:30,005 (listeneragent-3.3 10320) __main__ INFO: Peer: pubsub, Sender: platform.driver:, Bus: , Topic: devices/campus/building/fake/all, Headers: {'Date': '2023-07-13T18:52:30.002169+00:00', 'TimeStamp': '2023-07-13T18:52:30.002169+00:00', 'SynchronizedTimeStamp': '2023-07-13T18:52:30.000000+00:00', 'min_compatible_version': '3.0', 'max_compatible_version': ''}, Message: 
[{'EKG': 5.66553889764798e-16,
  'EKG_Cos': 5.66553889764798e-16,
  'EKG_Sin': 5.66553889764798e-16,
  'Heartbeat': True,
  'OutsideAirTemperature1': 50.0,
  'OutsideAirTemperature2': 50.0,
  'OutsideAirTemperature3': 50.0,
  'PowerState': 0,
  'SampleBool1': True,
  'SampleBool2': True,
  'SampleBool3': True,
  'SampleLong1': 50,
  'SampleLong2': 50,
  'SampleLong3': 50,
  'SampleWritableBool1': True,
  'SampleWritableBool2': True,
  'SampleWritableBool3': True,
  'SampleWritableFloat1': 10.0,
  'SampleWritableFloat2': 10.0,
  'SampleWritableFloat3': 10.0,
  'SampleWritableShort1': 20,
  'SampleWritableShort2': 20,
  'SampleWritableShort3': 20,
  'ValveState': 0},
 {'EKG': {'type': 'integer', 'tz': 'US/Pacific', 'units': 'waveform'},
  'EKG_Cos': {'type': 'integer', 'tz': 'US/Pacific', 'units': '1-0'},
  'EKG_Sin': {'type': 'integer', 'tz': 'US/Pacific', 'units': '1-0'},
  'Heartbeat': {'type': 'integer', 'tz': 'US/Pacific', 'units': 'On/Off'},
  'OutsideAirTemperature1': {'type': 'integer',
                             'tz': 'US/Pacific',
                             'units': 'F'},
  'OutsideAirTemperature2': {'type': 'integer',
                             'tz': 'US/Pacific',
                             'units': 'F'},
  'OutsideAirTemperature3': {'type': 'integer',
                             'tz': 'US/Pacific',
                             'units': 'F'},
  'PowerState': {'type': 'integer', 'tz': 'US/Pacific', 'units': '1/0'},
  'SampleBool1': {'type': 'integer', 'tz': 'US/Pacific', 'units': 'On / Off'},
  'SampleBool2': {'type': 'integer', 'tz': 'US/Pacific', 'units': 'On / Off'},
  'SampleBool3': {'type': 'integer', 'tz': 'US/Pacific', 'units': 'On / Off'},
  'SampleLong1': {'type': 'integer',
                  'tz': 'US/Pacific',
                  'units': 'Enumeration'},
  'SampleLong2': {'type': 'integer',
                  'tz': 'US/Pacific',
                  'units': 'Enumeration'},
  'SampleLong3': {'type': 'integer',
                  'tz': 'US/Pacific',
                  'units': 'Enumeration'},
  'SampleWritableBool1': {'type': 'integer',
                          'tz': 'US/Pacific',
                          'units': 'On / Off'},
  'SampleWritableBool2': {'type': 'integer',
                          'tz': 'US/Pacific',
                          'units': 'On / Off'},
  'SampleWritableBool3': {'type': 'integer',
                          'tz': 'US/Pacific',
                          'units': 'On / Off'},
  'SampleWritableFloat1': {'type': 'integer',
                           'tz': 'US/Pacific',
                           'units': 'PPM'},
  'SampleWritableFloat2': {'type': 'integer',
                           'tz': 'US/Pacific',
                           'units': 'PPM'},
  'SampleWritableFloat3': {'type': 'integer',
                           'tz': 'US/Pacific',
                           'units': 'PPM'},
  'SampleWritableShort1': {'type': 'integer', 'tz': 'US/Pacific', 'units': '%'},
  'SampleWritableShort2': {'type': 'integer', 'tz': 'US/Pacific', 'units': '%'},
  'SampleWritableShort3': {'type': 'integer', 'tz': 'US/Pacific', 'units': '%'},
  'ValveState': {'type': 'integer', 'tz': 'US/Pacific', 'units': '1/0'}}]
2023-07-13 14:52:30,195 (mqtt_historianagent-0.2 12211) __main__ DEBUG: publish_to_historian number of items: 24
2023-07-13 14:52:30,195 (mqtt_historianagent-0.2 12211) __main__ DEBUG: payload: 50.0, topic campus/building/fake/OutsideAirTemperature1
2023-07-13 14:52:30,195 (mqtt_historianagent-0.2 12211) __main__ DEBUG: payload: 50, topic campus/building/fake/SampleLong1
2023-07-13 14:52:30,195 (mqtt_historianagent-0.2 12211) __main__ DEBUG: payload: true, topic campus/building/fake/SampleBool1
2023-07-13 14:52:30,195 (mqtt_historianagent-0.2 12211) __main__ DEBUG: payload: 50.0, topic campus/building/fake/OutsideAirTemperature2
2023-07-13 14:52:30,195 (mqtt_historianagent-0.2 12211) __main__ DEBUG: payload: 50, topic campus/building/fake/SampleLong2
2023-07-13 14:52:30,195 (mqtt_historianagent-0.2 12211) __main__ DEBUG: payload: true, topic campus/building/fake/SampleBool2
2023-07-13 14:52:30,195 (mqtt_historianagent-0.2 12211) __main__ DEBUG: payload: 50.0, topic campus/building/fake/OutsideAirTemperature3
2023-07-13 14:52:30,195 (mqtt_historianagent-0.2 12211) __main__ DEBUG: payload: 50, topic campus/building/fake/SampleLong3
2023-07-13 14:52:30,195 (mqtt_historianagent-0.2 12211) __main__ DEBUG: payload: true, topic campus/building/fake/SampleBool3
2023-07-13 14:52:30,195 (mqtt_historianagent-0.2 12211) __main__ DEBUG: payload: true, topic campus/building/fake/Heartbeat
2023-07-13 14:52:30,195 (mqtt_historianagent-0.2 12211) __main__ DEBUG: payload: 5.66553889764798e-16, topic campus/building/fake/EKG
2023-07-13 14:52:30,195 (mqtt_historianagent-0.2 12211) __main__ DEBUG: payload: 10.0, topic campus/building/fake/SampleWritableFloat1
2023-07-13 14:52:30,195 (mqtt_historianagent-0.2 12211) __main__ DEBUG: payload: 20, topic campus/building/fake/SampleWritableShort1
2023-07-13 14:52:30,195 (mqtt_historianagent-0.2 12211) __main__ DEBUG: payload: true, topic campus/building/fake/SampleWritableBool1
2023-07-13 14:52:30,195 (mqtt_historianagent-0.2 12211) __main__ DEBUG: payload: 10.0, topic campus/building/fake/SampleWritableFloat2
2023-07-13 14:52:30,195 (mqtt_historianagent-0.2 12211) __main__ DEBUG: payload: 20, topic campus/building/fake/SampleWritableShort2
2023-07-13 14:52:30,195 (mqtt_historianagent-0.2 12211) __main__ DEBUG: payload: true, topic campus/building/fake/SampleWritableBool2
2023-07-13 14:52:30,195 (mqtt_historianagent-0.2 12211) __main__ DEBUG: payload: 10.0, topic campus/building/fake/SampleWritableFloat3
2023-07-13 14:52:30,195 (mqtt_historianagent-0.2 12211) __main__ DEBUG: payload: 20, topic campus/building/fake/SampleWritableShort3
2023-07-13 14:52:30,195 (mqtt_historianagent-0.2 12211) __main__ DEBUG: payload: true, topic campus/building/fake/SampleWritableBool3
2023-07-13 14:52:30,195 (mqtt_historianagent-0.2 12211) __main__ DEBUG: payload: 0, topic campus/building/fake/PowerState
2023-07-13 14:52:30,195 (mqtt_historianagent-0.2 12211) __main__ DEBUG: payload: 0, topic campus/building/fake/ValveState
2023-07-13 14:52:30,195 (mqtt_historianagent-0.2 12211) __main__ DEBUG: payload: 5.66553889764798e-16, topic campus/building/fake/EKG_Sin
2023-07-13 14:52:30,195 (mqtt_historianagent-0.2 12211) __main__ DEBUG: payload: 5.66553889764798e-16, topic campus/building/fake/EKG_Cos