Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

In order to cover data loss during a keep alive timeout scenario, the MQTT Transmission History Store includes a Rolling History Buffer that can be configured in the Advanced Properties configuration section. When the Rolling History Buffer is enabled, all tag changes will be written to the History Store regardless of connection status.

The Metrics Stored tag for the History Store will update as data is written to and pruned from the Rolling History Buffer. 

The In-Order History configuration parameter of the Transmitter will determine whether the data is flushed in-order (synchronously) or asynchronously.


Note

Data stored in the Rolling History Buffer for an In-Memory History Store will not be persisted across a module configuration change, module disable/enable, module restart or power loss. 

Data stored in the Rolling History Buffer for a Disk-Backed History Store will persist across a module configuration change, module disable/enable, module restart or power loss however you will lose change events that occur during the update or refresh.


Image AddedImage Removed

  • Enable Rolling History Buffer
    • Enable the Rolling History Buffer to store data in a rolling buffer to cover data loss in a keep alive timeout scenario
  • Rolling History Prune Interval
    • Interval at which to attempt to prune the rolling history buffer in seconds
  • Rolling History Max Age
    • The maximum age allowed of the data in the rolling history buffer in seconds
    Rolling History Prune Quantity
    • The quantity of the oldest data over All data older than the Max Age to prune on will be pruned at each prune interval
  • Rolling History Prune Quantity (Deprecated in release 4.0.19)


Tip

It can take MQTT Transmission 1.5 times the configured Server Keep Alive time to determined that there is a connection failure. The Rolling History Max Age should be at least 2x the Transmitter Server Keep Alive timeout to ensure that all tag change events are captured during this period. 

The ideal settings for your Rolling Buffer configuration will be dependent on the rate that your tags are changing. For example, consider these settings:

  • Rolling history max age - 60s (2x the 30s keep alive timeout)

  • Rolling history prune interval - 60s

  • Rolling history prune quantity - 10

The rolling history prune interval is how frequently it checks to see if there is any data that’s older than the max age (60s). If there is, in this example, the prune quantity (10 metrics) will be pruned from the buffer for every prune interval (60s). If there’s 100 metrics in the buffer that are older than 60s, after pruning once, only the oldest 10 will be dropped and there will still be 90 old metrics there.

On a reconnect, these older metrics will be published since they’re still in the buffer. Having a lot of old tags in the buffer will increase the number of duplicates that you get. Keeping a proper balance between the tag change rate and these settings can be important for minimizing duplicates.

...


Note
Using this feature may result in duplicate tag change event data being reported to Ignition as there is the potential for the publish of historical data that was previously successfully received. This will generate errors reported by the StoreAndForward.Sink.HsqlDataStore.DatasourceForwardTransaction logger when inserting the data into a database. These are benign errors and will not result in data loss.

...