Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Summary

The Inductive Auotmation Automation platform and MQTT modules can be resilient to failures when configured to use redundancy.  Redundant Ignition systems can be set up and configured to act as failover backups for primary/master Ignition instances.  This tutorial will provide step by step instructions for installing a set of Ignition systems with redundancy on the host/primary Ignition instance as well as redundancy on the MQTT enabled edge nodes.  For this tutorial we will show how to set up a total of six Ignition systems.  These will be:

  • Ignition Primary
    • An Ignition system running as master with MQTT Distributor and MQTT Engine installed.  This is what remote Ignition systems will send data to in normal operation.
  • Ignition Primary Backup
    • An Ignition system running as backup with MQTT Distributor and MQTT Engine installed.  This is what remote Ignition systems will send data to when Ignition Primary fails.
  • Ignition Edge 1
    • An Ignition system running as master with MQTT Transmission installed.  This will send data to Ignition Primary in normal operation.  If Ignition Primary is in a failed state, this will send data to Ignition Primary Backup.
  • Ignition Edge 1 Backup
    • An Ignition system running as backup with MQTT Transmission installed.  This will send data to Ignition Primary in the event that Ignition Edge 1 fails.  If Ignition Primary is in a failed state and Ignition Edge 1 is in a failed state, this will send data to Ignition Primary Backup.
  • Ignition Edge 2
    • An Ignition system running as master with MQTT Transmission installed.  This will send data to Ignition Primary in normal operation.  If Ignition Primary is in a failed state, this will send data to Ignition Primary Backup.
  • Ignition Edge 2 Backup
    • An Ignition system running as backup with MQTT Transmission installed.  This will send data to Ignition Primary in the event that Ignition Edge 2 fails.  If Ignition Primary is in a failed state and Ignition Edge 2 is in a failed state, this will send data to Ignition Primary Backup.

...

  • Network paths
    •  It may make sense to have edge nodes support multiple network paths to the MQTT servers.  For example, Ethernet, cellular, and satellite could all exist as supported network paths on a single Edge gateway.  This will help ensure additional reliability by supporting failover of networks.
  • Primary Ignition and Primary Ignition backup placement
    • This tutorial was created by modeling this exact environment using Amazon AWS EC2 instances in the cloud.  Reliability could be improved by putting Ignition Primary and Ignition Primary Backup in different AWS availability zones or even different AWS regions.  This would allow the primary Ignition with MQTT Distributor and MQTT Engine to continue to operate even in the case of AWS failures.  In the case of on premises installations of Ignition these could be placed in different physical locations and/or on secondary networks.
  • MQTT Servers
    • Additional MQTT Servers can be added so MQTT connections from remote edge nodes remain established.  Additional Chariot MQTT Servers can be used to make the system more robust (httphttps://www.cirrus-link.com/iiot-mqtt-servers/).
  • History enablement in MQTT Transmission
    • MQTT Transmission supports caching of data in the case that it can not establish a connection to any of the configured MQTT Servers.  Once a connection is reestablished, it will begin reporting and flush the stored historical values to prevent data loss in catastrophic failures.

Upon completion of this tutorial you will have a functional system with redundancy/failover support for both remote edge nodes as well as the primary Ignition system that the remote edge nodes are reporting to.

Architecture

Image Modified


Image Modified


Image Modified


Image Modified

Tutorial

Step 1: Download and Install Ignition on Six Systems

Ignition is an Industrial Application Platform that can be used to create SCADA and HMI solutions. A fully functional Ignition system can be downloaded and run in trial mode. Using Ignition as a tool in this way, we can install the Sparkplug MQTT Modules and observe everything working.

Go to the Inductive Automation download page and download the desired Ignition installer for Windows, Linux or MacOS;
https://
inductiveautomation.com/downloads/ignitionarchive.

Once the Ignition installer has been downloaded, follow the instructions provided by Inductive Automation to install and startup Ignition.

(Note: For this test infrastructure, MQTT Distributor will be installed as an Ignition module . Remember on both the Primary Ignition Gateway as well as the Primary Ignition Gateway Backup.  Make sure to either turn off firewalls or at a minimum allow inbound connections to TCP/IP port #1883 and port #88831883, as remote MQTT Clients will need to be able to establish a TCP/IP socket connection to these ports).

...

Go to the Inductive Automation download page again and scroll down to the Third Party modules section. Find the Cirrus Link modules section and download the MQTT Distributor, MQTT Engine, MQTT Transmission modules.
https://
inductiveautomation.com/downloads/ignition.  The download links should look similar to what is shown below.

Image Removed

Step 3: Configure the MQTT Modules

Once you have Ignition installed and running, and the MQTT Distributor, MQTT Engine and MQTT Transmission modules downloaded, browse to the Ignition Gateway console (e.g. http://localhost:8088).  Login using the default credentials of admin/password.  Click on Configuration tab and then click on the Modules tab on the left side of the page.  Scroll to the bottom of the Modules section and click on the Download/Upgrade modules button.  When prompted, select the MQTT Distributor module from the file browser and install it.  Do the same for the MQTT Engine and MQTT Transmission modules.  When complete, the Ignition Gateway Web UI module section should look similar to what is shown below:

Image Removed

Step 4: Use Ignition Designer to Examine the Initial Tag Structure

By default, MQTT Distributor, MQTT Engine, and MQTT Transmission are all configured out of the box to be able to connect locally to each other in the exact same architecture of this tutorial.  In addition, MQTT Engine will startup with outbound tag writes using the Sparkplug NCMD and DCMD messages blocked. For this tutorial we will want to enable tag writes. This is the only configuration change required for this tutorial.  To do so, open the MQTT Engine configuration tab in the Ignition Gateway console and click on the “Advanced Settings” tab. From this configuration screen you can disable command blocking by unchecking “Block Node Commands”, and “Block Device Commands” selections as shown below.  After doing so, save the changes.

Image Removed

With Ignition running and the Distributor, Engine, and Transmission modules loaded now we can open the Ignition Designer to observe some of the MQTT topology.  Regardless of the OS Ignition is running on, there is a “Launch Designer” button on the Ignition Gateway Console. From here you can launch your Designer on any machine.  This is shown below.  The default credentials for the designer are the same as the Gateway Console, admin/password.  Once you have logged into the Designer enter a new project name and open the project. The project name that we used for this tutorial is simply called “Tutorial1”.

Image Removed

After Designer opens, you will see the default Designer screen as shown below.

Image Removed

Once the MQTT Engine module is installed in Ignition, a new “Tag Provider” folder is created called “MQTT Engine”. The MQTT Engine folder will contain both process variable tags from field devices as well as metric and diagnostic information from the MQTT infrastructure itself. At this point in the tutorial we can use the tag browser to start looking at the simple MQTT infrastructure we already have running. In the tag structure below using the default MQTT Distributor and MQTT Engine configurations, you can see that the metrics provided for the MQTT Client connection from MQTT Engine to MQTT Distributor.

Image Removed

Step 5: Configure Transmission to Publish some Tags

Next, we need to create a folder structure where we will create a virtual MQTT Edge device and some tags to be published by MQTT Transmission.  When the Transmission module is installed in Ignition, a folder is automatically created in the Ignition tag structure with the following path:

  • All Providers/default/Edge Nodes

The is the default folder that Transmission will look at to an MQTT Sparkplug session from. For this tutorial, right click on the Edge Nodes folder and create a new folder called Tutorial Group. Then right click on the Tutorial Group folder and create another new folder called Tutorial Edge Node.  Finally, right click on the Tutorial Edge Node folder and create another new folder called Tutorial Device. This folder structure creates the same hierarchy that is described in the Sparkplug B specification of Group ID, Edge ID, and Device ID.

Image Removed

With this folder structure in place, now we can create some memory tags of various data types to publish. Right click on the Tutorial Device folder and select ’New Tag’/’Memory Tag’. In the tag editor change the Name of the tag to “Boolean001”, and change the Data Type to Boolean. Follow this same procedure for new memory tags called “Integer001” of type Integer, “Float001” of type Float, and “String001” of type String. The resulting folder structure should look as follows.

Image Removed

Now that we have a folder structure with some tags we can bring Transmission online.  Make sure that the Ignition Designer has read/write communications turned on by selecting Project/Comm Read/Write.

Image Removed

To bring Transmission online with the folder structure we’ve created, open the folder All Providers/MQTT Transmission/Transmission Control and click on the Refresh Boolean. When this happens, following the Sparkplug B specifications, the Transmission module will create an MQTT Session with the MQTT Server and then publish a NBIRTH message followed by a DBIRTH message.  Note the Boolean tag will not change to true.  This is really a one-shot and as a result, the tag will not change to true.

Then open the All Providers/MQTT Engine/Edge Nodes folder and refresh the Tag Browser.  The MQTT Engine MQTT Client has been subscribed to all Sparkplug B messages just waiting for a new Sparkplug B message to arrive. When we enabled Transmission to create a device structure with some tags, Engine was able to receive this message, and then using the Google Protocol Buffers schema, automatically create a new folder structure populated with the tags and metadata around this new device.  

Image Removed

At this point, memory tag values can be changed to generate events which in turn generates new MQTT messages.

Image Removed

archive.  For each of the Ignition instance, install the following MQTT Modules.

  • Ignition Primary
    • MQTT Distributor and MQTT Engine
  • Ignition Primary Backup
    • MQTT Distributor and MQTT Engine
  • Ignition Edge 1
    • MQTT Transmission
  • Ignition Edge 1 Backup
    • MQTT Transmission
  • Ignition Edge 2
    • MQTT Transmission
  • Ignition Edge 2 Backup
    • MQTT Transmission

Step 3: Configure the MQTT Modules

Once Ignition is installed, the MQTT Modules are installed, and everything is running we can configure the systems.  Since we are going to have a backup for each master system, we only need to do most of the configuration for the master systems.  Later, we can sync the configurations to the backups automatically.  We'll start by configuring the modules and configure the redundancy settings in the next step.

  • Ignition Primary - MQTT Distributor
    • No modifications to the default parameters are required.  However, it is important to make sure the Operation System allows inbound connections on port 1883 and there are no firewalls blocking inbound connections on this port from the remote edge nodes.
  • Ignition Primary - MQTT Engine
    • The only change from defaults is to set a Primary Host ID.  MQTT uses Quality of Service (QOS) levels to ensure messages get delivered.  However, this only ensures delivery between a single MQTT client and the MQTT server.  In other words, it doesn't ensure delivery from one MQTT client to another MQTT client.  Sparkplug introduces the notion of a Primary Host ID which is used to ensure client to client communications.  The only requirement is that it match exactly on both the MQTT Engine and MQTT Transmission configurations.
      Image Added
  • Ignition Edge 1 and Ignition Edge 2 - MQTT Transmission (Configure the same on both Ignition instances)
    • As with the MQTT Engine configuration, the Primary Host ID must be set on the General tab as shown below.
      Image Added
    • Delete the existing default MQTT Transmission Server.
      Image Added

    • Create a new MQTT Server configuration by clicking the link below.

      image2017-9-5_13-45-36.pngImage Added
    • Configure MQTT Transmission to point to the Primary Ignition.  Configure as shown below making sure to change the URL to reflect your network settings.  For example, if you Ignition Primary is at 192.168.1.100 the MQTT Server URL would be: tcp://192.168.1.100:1883.  After setting the parameters as shown below.  Click the 'Save Changes' button at the bottom.
      Image Added
    • Verify the MQTT Server has been created and is shown in the list of MQTT Servers as shown below.

      Image Added
    • Repeat the process of creating a MQTT Server but instead point it to the Ignition Primary Backup MQTT Server.  These are the parameters to use:
      • Name: primary-backup
      • URL: tcp://primary-backup:1883
        • Change 'primary-backup' in the URL to reflect the network address of the Ignition Primary Backup server.
      • Server Type: MQTT Distributor
      • Server Set: Default
      • Username: admin
      • Password: changeme
    • When complete, verify both MQTT Servers appear in the list as shown below.
      Image Added
    • Finally, make sure to set up the same MQTT Transmission configuration in the Ignition Edge 2 instance.

Step 4: Configure Redundancy

The following configuration shows all of the redundancy settings that were used in setting this environment up using Amazon's AWS EC2 instances (virtual machines).  The configuration will vary based on your network configuration.  Additional Ignition redundancy resources can be found at the following links:

https://docs.inductiveautomation.com/display/DOC79/Setting+Up+Redundancy
https://support.inductiveautomation.com/usermanuals/ignition/index.html?redundancy_settings.htm

  • Ignition Primary
    • Select Redundancy on the left navigation bar. Then set the Mode to 'Master' and set the Standby Activity left to 'Warm' as shown below.
      Image Added
    • Set up the Redundancy Network Settings.  The settings here are specific to your network setup.  On many LAN configurations none of these changes are required.  What is shown below was the configuration for setting up all of these components in Amazon's AWS EC2 instances.  The changes were:
      • Uncheck 'Auto-detect network interface'
      • Set the 'Network Bind Interface' to the public IP address of the Ignition Primary EC2 instance.  On a LAN this would be the primary network interface address of the Ignition Primary machine.
      • Uncheck the 'Autodetect HTTP Address' tickbox.
      • Specify two explicit HTTP addresses for clients to use.  These were the public IP addresses of the Ignition Primary and Ignition Primary Backup EC2 instances.  On a LAN, these would be the primary network interface addresses of the Ignition Primary and Ignition Primary Backup machines.  Also note the HTTP port is 8088 which is the default Ignition HTTP port.Image Added
    • Set the Master Node Address.  Note in the configuration below a hostname is being used.  This should be the primary network interface address of the Ignition Primary Gateway.
      Image Added
    • Finally, click the 'Save Changes' button.
  • Ignition Primary Backup
    • Select Redundancy on the left navigation bar. Then set the Mode to 'Backup' and set the Standby Activity left to 'Warm' as shown below.
      Image Added
    • Set up the Redundancy Network Settings.  The settings here are specific to your network setup.  On many LAN configurations none of these changes are required.  What is shown below was the configuration for setting up all of these components in Amazon's AWS EC2 instances.  The changes were:
      • Uncheck 'Auto-detect network interface'
      • Set the 'Network Bind Interface' to the public IP address of the Ignition Primary Backup EC2 instance.  On a LAN this would be the primary network interface address of the Ignition Primary Backup machine.
      • Uncheck the 'Autodetect HTTP Address' tickbox.
      • Specify two explicit HTTP addresses for clients to use.  These were the public IP addresses of the Ignition Primary and Ignition Primary Backup EC2 instances.  On a LAN, these would be the primary network interface addresses of the Ignition Primary and Ignition Primary Backup machines.  Also note the HTTP port is 8088 which is the default Ignition HTTP port.Image Added
    • Set the Master Node Address.  Note in the configuration below a hostname is being used.  This should be the primary network interface address of the Ignition Primary Gateway.
      Image Added
    • Finally, click the 'Save Changes' button.

  • Ignition Edge 1
    • Select Redundancy on the left navigation bar. Then set the Mode to 'Master' and set the Standby Activity left to 'Warm' as shown below.
      Image Added
    • Set up the Redundancy Network Settings.  The settings here are specific to your network setup.  On many LAN configurations none of these changes are required.  What is shown below was the configuration for setting up all of these components in Amazon's AWS EC2 instances.  The changes were:
      • Uncheck 'Auto-detect network interface'
      • Set the 'Network Bind Interface' to the public IP address of the Ignition Edge 1 EC2 instance.  On a LAN this would be the primary network interface address of the Ignition Edge 1 machine.
      • Uncheck the 'Autodetect HTTP Address' tickbox.
      • Specify two explicit HTTP addresses for clients to use.  These were the public IP addresses of the Ignition Edge 1 and Ignition Edge 1 Backup EC2 instances.  On a LAN, these would be the primary network interface addresses of the Ignition Edge 1 and Ignition Edge 1 Backup machines.  Also note the HTTP port is 8088 which is the default Ignition HTTP port.Image Added
    • Set the Master Node Address.  Note in the configuration below a hostname is being used.  This should be the primary network interface address of the Ignition Edge 1 Gateway.
      Image Added
    • Finally, click the 'Save Changes' button.

  • Ignition Edge 1 Backup
    • Select Redundancy on the left navigation bar. Then set the Mode to 'Backup' and set the Standby Activity left to 'Warm' as shown below.
      Image Added
    • Set up the Redundancy Network Settings.  The settings here are specific to your network setup.  On many LAN configurations none of these changes are required.  What is shown below was the configuration for setting up all of these components in Amazon's AWS EC2 instances.  The changes were:
      • Uncheck 'Auto-detect network interface'
      • Set the 'Network Bind Interface' to the public IP address of the Ignition Edge 1 Backup EC2 instance.  On a LAN this would be the primary network interface address of the Ignition Edge 1 Backup machine.
      • Uncheck the 'Autodetect HTTP Address' tickbox.
      • Specify two explicit HTTP addresses for clients to use.  These were the public IP addresses of the Ignition Edge 1 and Ignition Edge 1 Backup EC2 instances.  On a LAN, these would be the primary network interface addresses of the Ignition Edge 1 and Ignition Edge 1 Backup machines.  Also note the HTTP port is 8088 which is the default Ignition HTTP port.Image Added
    • Set the Master Node Address.  Note in the configuration below a hostname is being used.  This should be the primary network interface address of the Ignition Edge 1 Gateway.
      Image Added
    • Finally, click the 'Save Changes' button.
  • Ignition Edge 2
    • Select Redundancy on the left navigation bar. Then set the Mode to 'Master' and set the Standby Activity left to 'Warm' as shown below.
      Image Added
    • Set up the Redundancy Network Settings.  The settings here are specific to your network setup.  On many LAN configurations none of these changes are required.  What is shown below was the configuration for setting up all of these components in Amazon's AWS EC2 instances.  The changes were:
      • Uncheck 'Auto-detect network interface'
      • Set the 'Network Bind Interface' to the public IP address of the Ignition Edge 2 EC2 instance.  On a LAN this would be the primary network interface address of the Ignition Edge 2 machine.
      • Uncheck the 'Autodetect HTTP Address' tickbox.
      • Specify two explicit HTTP addresses for clients to use.  These were the public IP addresses of the Ignition Edge 2 and Ignition Edge 2 Backup EC2 instances.  On a LAN, these would be the primary network interface addresses of the Ignition Edge 2 and Ignition Edge 2 Backup machines.  Also note the HTTP port is 8088 which is the default Ignition HTTP port.Image Added
    • Set the Master Node Address.  Note in the configuration below a hostname is being used.  This should be the primary network interface address of the Ignition Edge 2 Gateway.
      Image Added
    • Finally, click the 'Save Changes' button.

  • Ignition Edge 2 Backup
    • Select Redundancy on the left navigation bar. Then set the Mode to 'Backup' and set the Standby Activity left to 'Warm' as shown below.
      Image Added
    • Set up the Redundancy Network Settings.  The settings here are specific to your network setup.  On many LAN configurations none of these changes are required.  What is shown below was the configuration for setting up all of these components in Amazon's AWS EC2 instances.  The changes were:
      • Uncheck 'Auto-detect network interface'
      • Set the 'Network Bind Interface' to the public IP address of the Ignition Edge 2 Backup EC2 instance.  On a LAN this would be the primary network interface address of the Ignition Edge 2 Backup machine.
      • Uncheck the 'Autodetect HTTP Address' tickbox.
      • Specify two explicit HTTP addresses for clients to use.  These were the public IP addresses of the Ignition Edge 2 and Ignition Edge 2 Backup EC2 instances.  On a LAN, these would be the primary network interface addresses of the Ignition Edge 2 and Ignition Edge 2 Backup machines.  Also note the HTTP port is 8088 which is the default Ignition HTTP port.Image Added
    • Set the Master Node Address.  Note in the configuration below a hostname is being used.  This should be the primary network interface address of the Ignition Edge 2 Gateway.
      Image Added
    • Finally, click the 'Save Changes' button.

Step 5: Create some tags in Edge 1 and Edge 2

In this step we will use Ignition Designer in the Ignition Edge 1 and Edge 2 instances to create some tags.  These tags will be used by MQTT Transmission and the Default Transmitter to push data to MQTT Distributor and MQTT Engine in the Ignition Primary instances.

  • Using Ignition Designer on Edge 1, do the following:
    • Create a new project called 'Test'.
    • In the Tag Browser, confirm there is a folder called 'Edge Nodes'.  If there is not, confirm MQTT Transmission is installed.
    • In the 'Edge Nodes' folder, create a folder called 'Group 1'.
    • In the 'Group 1' folder, create a folder called 'Edge 1'.
    • In the 'Edge 1' folder, create a folder called 'Device 1'.
    • In the 'Device 1' folder, create a Tag called 'Tag 1'.
    • At the end, you should see something similar to what is shown below.

Image Added

  • Using Ignition Designer on Edge 2, do the following:
    • Create a new project called 'Test'.
    • In the Tag Browser, confirm there is a folder called 'Edge Nodes'.  If there is not, confirm MQTT Transmission is installed.
    • In the 'Edge Nodes' folder, create a folder called 'Group 1'.
    • In the 'Group 1' folder, create a folder called 'Edge 2'.
    • In the 'Edge 2' folder, create a folder called 'Device 1'.
    • In the 'Device 1' folder, create a Tag called 'Tag 1'.
    • At the end, you should see something similar to what is shown below

Image Added 

  • Finally, refresh the Transmission runtime.  This is done by clicking the 'MQTT Transmission/Transmission Control/Refresh' Boolean tag.

Image Added

    • After clicking the Boolean tag you may need to 'Enable Read/Write Mode'

Image AddedStep 6: Force Sync of Backup Instances

The next step to to force the backup Ignition instances to receive their configurations from the master Ignition instances.  This will happen automatically eventually but for expediency we're going to force the action.

  • Repeat the following steps for the following Ignition instances: Primary, Edge 1, and Edge 2
    • Browse to the Status menu and then click in the Redundancy box as shown below.

      Image Added 
      • Force the Re-sync via the button below.  This will duplicate the master configuration to the backup Ignition instance it is connected to.
        Image Added
      • Repeat the above Re-sync steps on the Edge 1 and Edge 2 Ignition instances.

Step 7: Verify MQTT Engine is getting data from the MQTT Transmission Edge 1 and Edge 2

Open Ignition Designer on the Ignition Primary instance.  Expand the MQTT Engine tag tree and validate the following tags are present.  If they are present and not stale, they are properly connected.

Image Added

Step 8: Test the Redundancy

In order to test the redundancy, we need to make a few simple dashboards.  It is important to note that this can not be tested with Ignition Designer alone.  Designer can not be opened from an Ignition backup instance since projects get replicated to the backup instances.  So, to show everything working, we'll make some very rudimentary dashboards.

  • Ignition Primary
    • Create the following widgets.
      • Label - Ignition Primary
      • Label - Redundancy State → Label with Tag Path of "[MQTT Engine]/Engine Info/Redundancy State"
      • Label - Redundancy Role → Label with Tag Path of "[MQTT Engine]/Engine Info/Redundancy Role"
      • Label - Edge 1 Tag 1 → Label with Tag Path of "[MQTT Engine]/Edge Nodes/Group 1/Edge 1/Device 1/Tag 1"
      • Label - Edge 2 Tag 1 → Label with Tag Path of "[MQTT Engine]/Edge Nodes/Group 1/Edge 2/Device 1/Tag 1"
      • Label - MQTT Engine Connected → Multi-State Indicator with Tag Path of "[MQTT Engine]/Engine Info/MQTT Clients/Chariot SCADA/Online"
      • Label - Edge 1 Connected → Multi-State Indicator with Tag Path of "[MQTT Engine]/Edge Nodes/Group 1/Edge 1/Node Info/Online"
      • Label - Edge 2 Connected → Multi-State Indicator with Tag Path of "[MQTT Engine]/Edge Nodes/Group 1/Edge 2/Node Info/Online"
    • When complete - it should look similar to what is shown below

Image Added

  • Ignition Edge 1
    • Create the following widgets
      • Label - Ignition Edge 1
      • Label - Redundancy State → Label with Tag Path of "[MQTT Transmission]/Transmission Info/Redundancy State"
      • Label - Redundancy Role → Label with Tag Path of "[MQTT Transmission]/Transmission Info/Redundancy Role"
      • Label - Edge 1 Tag 1 → Label with Tag Path of "[default]/Edge Nodes/Group 1/Edge 1/Device 1/Tag 1"
      • Label - Connected → Multi-State Indicator with Tag Path of "[MQTT Transmission]/Transmission Info/MQTT Clients/Group 1-Edge 1/Online"

Image Added

  • Ignition Edge 2
    • Create the following widgets
      • Label - Ignition Edge 2
      • Label - Redundancy State → Label with Tag Path of "[MQTT Transmission]/Transmission Info/Redundancy State"
      • Label - Redundancy Role → Label with Tag Path of "[MQTT Transmission]/Transmission Info/Redundancy Role"
      • Label - Edge 2 Tag 1 → Label with Tag Path of "[default]/Edge Nodes/Group 1/Edge 2/Device 1/Tag 1"
      • Label - Connected → Multi-State Indicator with Tag Path of "[MQTT Transmission]/Transmission Info/MQTT Clients/Group 1-Edge 2/Online"

Image Added

  • Once all three dashboards have been created, save and publish them and close the Ignition Designer windows.
  • Now open each of the Ignition client 'Test' projects.  With everything running you should see three windows similar to the following.

Image Added

  • At this point we can begin failing Ignition instances.  From the Status → Redundancy page we can use the 'Force Failover' button as shown below.  Of course stopping the actual Ignition instance is another option.

Image Added

  • Primary Ignition Failure: Failing the Ignition Primary instance will cause the following.
    • Ignition Primary will go down and be unreachable
      • This results in all MQTT connections being lost.
    • Ignition Primary Backup will come up and take the place of Ignition Primary
    • MQTT Engine will reconnect on Ignition Primary
    • The MQTT Transmission instances will reconnect to the new MQTT Server (MQTT Distributor) running in Ignition Primary Backup
    • This is all shown in the screenshot below of the Ignition projects.  Note all connections are valid and the 'Redundancy Role' of Ignition Primary is now Backup.

Image Added

  • Edge 1 Failure: Failing the Ignition Edge 1 instance will cause the following.
    • Ignition Edge 1 will go down and be unreachable
      • This results in the MQTT Connection between Ignition Edge 1 and Ignition Primary being lost.
    • Ignition Edge 1 Backup will come up and take the place of Ignition Edge 1.
    • The MQTT Transmission instance on Ignition Edge 1 Backup will connect to the MQTT Server (MQTT Distributor) running in Ignition Primary
    • This is all shown in the screenshot below of the Ignition projects.  Note all connections are valid and the 'Redundancy Role' of Ignition Edge 1 is now Backup.

Image Added

  • Failure of all Master Nodes: Failing all master Ignition instances (Primary, Edge 1, and Edge 2) will cause the following.
    • Ignition Primary, Edge 1, and Edge 2 will all go down and be unreachable
      • This results in all MQTT connections being lost
    • Ignition Primary Backup, Edge 1 Backup, and Edge 2 Backup will all come up and start their MQTT services.
    • The new MQTT Transmission instances on Ignition Edge 1 Backup and Ignition Edge 2 Backup will connect to the new MQTT Server (MQTT Distributor) running in Ignition Primary Backup
    • This is all shown in the screenshot below of the Ignition projects.  Note all connections are valid and the 'Redundancy Role' of all instances is now Backup.

Image Added

To summarize, this tutorial shows how Ignition and the MQTT Modules can be used to create a resilient infrastructure that is able to withstand failures of machines and network connections within the architecture.  As noted earlier, this tutorial shows the basic requirements of configuring failover support with Ignition and the MQTT Modules.  This can be further improved with additional advanced concepts.  Feel free to contact sales@cirrus-link.com for more informationAt this point you can create new tags in the default tag tree to add additional data points.  Any time you add new tags, you must click the 'Transmission Control/Refresh' boolean again to force MQTT Transmission to rescan the tag tree and publish new NBIRTH and DBIRTH messages.  It is also important to note that this architecture would not be used in a real world situations.  A more realistic yet still simple architecture would be to have MQTT Transmission on one Ignition instance and another Ignition instance which had MQTT Distributor and MQTT Engine installed.  This architecture would allow you to replicate tags from one Ignition instance to another using MQTT as the transport.  Also, in this example we used memory tags for simplicity.  However, OPC tags or other tag types could be used instead in the same manner for more real world solutions.