Introduction

MQTT Store and Forward allows data to be buffered locally at a client when connections are down to the MQTT Server infrastructure and deliver that data when the connection is restored. This feature is critical in most applications because if we lose connection to the MQTT Server we will lose data if it is not buffered locally. When Store and Forward is enabled and the edge node detects a disconnect to the MQTT Server, tags will be stored locally in a history store. When the edge node can reconnect to the MQTT Server, it will publish any stored tags. The rate at which these stored tags are flushed from the history store is configurable to prevent any delays in the delivery of live data.

Determining the settings for the MQTT Transmission History Store requires understanding the unique system properties at each Edge Node. There are a number of factors involved in determining how much history data can be stored including but not limited to system resources such as CPU, RAM (especially when using 'In-Memory'), Disk IOPS (if using 'Disk-Backed'), the nominal tag change rate (e.g. number of tags changing per second in the system), the flush rate, bandwidth availability, whether flushing in order vs asynchronously, etc.

MQTT Transmission History Store requires the following parameters:

• Type
  • The type of History Store with options of In-Memory and Disk-Backed
  • Data stored in an In-Memory History Store will not be persisted across a module configuration change, module disable/enable, module restart or power loss. 
  • Data stored in a Disk-Backed History Store will persist across a module configuration change, module disable/enable, module restart or power loss.
• History Max Size
  • The maximum number of megabytes history can use before dropping the data
  • In-Memory History Store will use the Ignition Java Heap memory
  • Default is 500
• History Max Age
  
  • Maximum number of minutes to store history before dropping the data.
• Flush Quantity
  
  • The maximum number of tags to publish in a single message upon re-establishing communication.
    
  • Default 10,000
• Flush Period
  • The period to wait in milliseconds between publishes when flushing messages upon re-establishing communication.
  • MQTT Transmission queries the history store and builds up a publish, and publishes it, and then delays by the Flush Period.
    The actual message publish frequency is not deterministic as the time to gather the publish from the history store can vary depending on the frequency tag change events occurring, disk speed, CPU availability, etc. For example, if it takes 5s to gather up and publish that payload, the actual message publish frequency will be it will be 5s + the flush period
  • Default 200

How to test these settings

Testing is the best way to determine your settings for your Edge Gateway(s) and the recommended approach is to:

• Configure a History Store with a large storage time for your tags.
• Disconnect the Edge for a set time period to cause MQTT Transmission to store data to the History Store using one of the methods detailed below:
  • Disable the network connection
  • Disable the broker
  • Use Primary Host ID in both MQTT Engine and in the MQTT Transmission.
    • Removing the Primary Host ID from MQTT Engine will simulate the primary backend application going offline and cause MQTT Transmission to store data to the History Store.
    • Restoring the Primary Host ID to MQTT Engine will simulate that the primary backend application is online and cause MQTT Transmission to flush the stored data
  • Modify the client credentials at the broker preventing the client from connecting

• Monitor the live updates for the count of tag change events and memory used from the Transmission Info History Store metrics.

• Reconnect and monitor the data flushing to ensure that the flush settings (quantity and period) are able to handle the current tag changes that continue to build.

  Whilst the history flush is in progress, all new change events are written to the history store until it has been completely flushed. If the tag change rate at the Edge is faster than the MQTT Transmission Flush Period this can cause a build up of data in history store(s) and prevent the publishing of live data.

  
  

• Extrapolate based on the test