History

The "History" page allows for the configuration of MQTT Transmission History Stores.

When a Transmitter is configured to use an MQTT Transmission History Store, data is written to the History Store once MQTT Transmission has determined that there is a connection failure. Once a connection with an MQTT server is re-established the History Store will publish the stored messages with a flag set to indicate that the messages are "historical" to prevent confusion with live data values. Determination of a connection failure can be up to 1.5 times the configured keep alive. and, as such, any data published during this time can be lost.

From release 4.0.17, in order to cover data loss during a keep alive timeout scenario, the MQTT Transmission History Store includes a Rolling History Buffer that can be configured in the Advanced Properties configuration section. When the Rolling History Buffer is enabled, all tag changes will be written to the History Store regardless of connection status.

The History tab contains a Main section and an Advanced section.

History - Main

Large Tag Capacities

If using a large tag capacity it is highly recommended to test the system under load in a non-production environment on similar hardware and software that will be used in production. During testing is also important to get the system into a state where the store and forward cache becomes full before beginning to flush. This will ensure that the system is sized appropriately when deployed into a production environment. There are a number of factors involved in determining how large the tag capacity can be including but not limited to system resources such as CPU, RAM (especially when using 'In-Memory'), Disk IOPS (if using 'Disk-Backed'), the nominal tag change rate (e.g. number of tags changing per second in the system), the flush rate, bandwidth availability, whether flushing in order vs asynchronously, etc. Because of the complex interactions of these variables it is highly recommended to test in a controlled environment. Generally any capacity over 2,000,000 is considered large and should be tested before deploying to production.

Review Determining and Testing the settings for an MQTT Transmission History Store

  • Name
    • The name of the History Store.
  • Enabled
    • Checkbox to enable/disable the History Store. Not selected by default.
  • Type
    • The type of History Store.
    • Data stored in an In-Memory History Store will not be persisted across a module configuration change, module disable/enable, module restart or power loss. 
    • Data stored in a Disk-Backed History Store will persist across a module configuration change, module disable/enable, module restart or power loss.
  • Edge Node Tag Capacity
    • Maximum number of Edge Node level tag change events to store per Edge Node before dropping oldest historical Edge Node tag change events. This value is independent of the 'Device Tag Capacity' and only applies to how many 'Edge Node level' tag change events are stored per Edge Node.
    • A tag change event is triggered by either a change in value or quality and results in the tag's Qualified Value (which has three attributes of value, quality and timestamp) being stored. 
  • Device Tag Capacity
    • Maximum number of Device tag change events to store per Device before dropping oldest historical Device tag change events. This value is independent of the 'Edge Node Tag Capacity' and only applies to how many 'Device level' tag change events are stored per Device.
    • A tag change event is triggered by either a change in value or quality and results in the tag's Qualified Value (which has three attributes of value, quality and timestamp) being stored.
  • Flush Quantity
    • The maximum number of tags to publish in a single message upon reestablishing communication.
  • Flush Period
    • The period to wait in milliseconds between publishes when flushing messages upon reestablishing communication

History - Advanced

  • Enable Rolling History Buffer
    • Enable/disable storing data in a rolling buffer to cover data loss in Keep Alive timeout scenario
  • Rolling History Prune Interval
    • The interval, in seconds at which to attempt to prune the rolling history buffer
  • Rolling History Max Age
    • The maximum age allowed of data, in seconds, in the rolling history buffer
    • This should be at least 2 x the Keep Alive timeout
  • Rolling History Prune Quality
    • The number of metrics to prune in a single block


  • No labels

New to MQTT? Start Here!