Page History

Summary

MQTT Store and Forward allows data to be buffered locally at a client when connections are down to the MQTT Server infrastructure and deliver that data when the connection is restored. This feature is critical in most applications because if we lose connection to the MQTT Server we will lose data if it is not buffered locally.

...

If the edge node is notified that the state of the primary application is offline, tags will be stored locally until the state of the primary application is online when any stored tags will be published.

Minimizing data loss when using Store and Forward

The MQTT protocol is hosted on top of the TCP protocol, which is connection-oriented, and provides a stable and orderly flow of bytes between two connected parties. Normally when the MQTT Transmission client attempts to send data over TCP, it will receive an error at the TCP level if disconnected and can immediately start to buffer data locally.

However, in some cases, TCP can have half-connection problems. A half-connection is a connection that has been disconnected or not established on one side, while the connection on the other side is still maintained. In this case, the half-connected party may continuously send data, which obviously never reaches the other side. To avoid black holes in communication caused by half-connections, the MQTT protocol provides a Keep Alive mechanism that allows the client and MQTT server to determine whether there is a half-connection problem, and close the corresponding connection.

For MQTT Transmission the Keep Alive is the maximum interval in seconds between any two MQTT protocol control packets sent by the MQTT Transmission client to the data. If the client is idle and has no control packets to send, it will send a PINGREQ protocol packets and expects to receive a PINGRESP packets from the server. If no PINGRESP is received within 1.5 times the Keep Alive interval, the client will close the connection.

This means that, depending on the TCP connection failure, it can take up to 1.5 times the Keep Alive interval for MQTT Transmission to determine that it is disconnected and start to buffer data locally at the Edge and during this time data will be lost. 

For example, a Keep Alive of 30 seconds could result in up to 45 seconds of data lose in the event of a TCP half-connection scenario. Reducing the Keep Alive to 5 seconds would reduce the potential data lose to 8 seconds.

Primary Host

How quickly it determines that there is a disconnect from the server is determined by the MQTT Transmission Server Keep Alive setting.

The short video below describes the benefits and use cases for MQTT Store and Forward within an Ignition system.

MQTT Modules Store and Forward Configuration

MQTT Engine

Setup the Primary Host to be used under MQTT Engine configuration under the Main section on the General tabImage Removed

There are two methods that MQTT Engine can use to process historical events with Ignition:

• Write historical events directly to the database, via the Historian, bypassing the Tag
• Write historical events to the Tag instead of directly to Historian.

Review the MQTT History tutorial to determine the appropriate method for your system and details on the required configuration settings.

MQTT Transmission

Setup the History Store to be used under MQTT Transmission History configuration.

Image Removed

Setup the History Settings to be used under MQTT Transmission Settings configuration.

Image Removed

Additional Resources:

Ignition Video 10: Primary Host ID Setting

...