Versions Compared

Old Version 3

changes.mady.by.user Gill Houghton

Saved on

compared with

New Version Current

changes.mady.by.user Gill Houghton

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Prerequisites:

  • Knowledge of Ignition and Module installation process: Cirrus Link Module Installation
  • Install Ignition on an 'host/backend' system
  • This must be a 'full version' of Ignition (i.e. not Ignition Edge)
    • Install MQTT Distributor and MQTT Engine Modules
    Cirrus Link's Ignition Modules can be downloaded from Installation of the following Cirrus Link MQTT modules selecting the version compatible with your Ignition installation.
    Modules can be downloaded from the Ignition Strategic Partner Modules download page.:
    • MQTT Distributor
    • MQTT Engine
    • MQTT Transmission
  • Cirrus Link Module Compatibility with Ignition versions
    • Install Ignition on an 'edge' system
    • This can be a 'full version' of Ignition or Ignition Edge
    •  Install the MQTT Transmission Module

Overview:

Transmission is an MQTT module for Ignition that can publish files using MQTT Sparkplug to be processed by MQTT engine. 

Files are transferred using Sparkplug over MQTT and which files get transferred and when they get transferred can be configured in a number of different ways. The file transfer configuration also defines how quickly a file transfer occurs and how long before a timeout will occur on failure to receive host side acknowledgements which can be adjusted as required based on the size of files you are sending and any bandwidth constraints that you may have.

This tutorial shows how to configure the MQTT modules to publish and receive files from one Ignition system to another.

Step 1: Install MQTT Engine and MQTT Distributor on the host machine

files using both the manual and auto publish methods.

Table of Contents
minLevel3

Install the MQTT modules

Install the three MQTT modules listed in the pre-requisites onto your Install the MQTT Engine and Distributor modules onto your host Ignition system following the Cirrus Link Module Installation guide.

By default, both MQTT Engine is and MQTT Transmission are configured to connect to MQTT Distributor on tcp://localhost:1883 and will show as Connected under the their respective Servers configuration setting in the Ignition UI.

Step 2: Install MQTT Transmission on the edge machine

Install the MQTT Transmission onto your edge Ignition system following the Cirrus Link Module Installation guide.

By default, MQTT Transmission is configured to connect to MQTT Distributor on tcp://localhost:1883. As the MQTT Distributor is installed o a different server, we will need to edit the configuration to allow MQTT Transmission to connect to the host server.

...

The MQTT Server must be accessible to the Ignition system running MQTT Transmission. In order for this to work port 1883 must be open to outside connections. Make sure this is allowed via your operating system's firewall configuration and any anti-virus software you may have installed.

...

.

...

Navigate to the MQTT Transmission > Settings in the left side bar of the Ignition Gateway UI and select the Servers tab.

Change the server URL to point to the server IP address or domain name of the Ignition system running MQTT Distributor and save the configuration. When complete, you should see something similar to the following:Image Removed

Warning
If you do not see '1 of 1' under the 'Connected' status, check the logs to show what may be going wrong and/or to validate the rest of your configuration

 

...

MQTT Transmission will also have an Example Transmitter configured pointing to a set of tags that are configured in the Ignition "default" tag provider. 

Tip
Review the MQTT Transmission Transmitters and Tag Trees tutorial for additional information on how Transmitter configurations interact with Ignition tag trees to create the Sparkplug IDs required.

Now we have This allows the three modules connected, we can setup the file publish configuration.

Step 3: Config MQTT Engine 

...

to automatically connect and provide a starting base for the tutorial.

Configure Primary Host on MQTT Engine and MQTT Transmission

Warning
Primary Host ID must be configured for File Transfer to work

Navigate to the MQTT Engine > Settings in the left side of the Ignition Gateway UI and select the General tab.

Set Primary Host Enabled and add a Primary Host ID as shown below:

Image Added

Navigate to the MQTT Transmission > Settings in the side of the Ignition Gateway UI and select the Sets tab.

Edit the Default Set and add a Primary Host ID as shown below:

Image Added

Config MQTT Engine File Handling

Now we can configure the MQTT Engine module to process the published files.

Navigate to the MQTT Engine > Settings in the left side of the Ignition Gateway UI and select the Namespaces tab. From the Default tab, edit the Sparkplug B namespace shown below:

...

Image Modified

Now select the 'Files' tab and set the following

  • Ignore Files
    • Set this property to unchecked so that publish files will not be ignored: unchecked/false
  • File Host Type:
    • ACTIVE to specify that a message ACK is sent back to MQTT Transmission
  • Base File Directory
    • : Set to some directory file path (i.e. 'C:\My Files' or something similar) where the files will be stored
  • Add Topic Tokens
    • Leave this property as checked to add the Group ID, Edge Node ID and Device ID to the base file directory path: checked/true
  • File Storing Policy
    • Set this policy to : REPLACE_EXISTING_FILE
  • File Attributes Policy
    • Leave this property at the default value of IGNORE
Tip
Review the MQTT Engine Configuration guide for information on all the File properties

When complete, it should look similar to the following. Save the changes after confirming.
Image Modified

...

  • In the example below, the Sparkplug IDs field is 'My MQTT Group/Edge Node faec7e'. This means the corresponding Group ID and Edge Node ID are:
    • Group ID: My MQTT Group
    • Edge Node ID: Edge Node faec7e
      Image Removed

Anchor
configureTransmission
configureTransmission
Configure MQTT Transmission to publish files

Now we can configure the MQTT Transmission module to publish files using a configured Transmitter. Navigate to the MQTT Transmission > Settings in the left side bar of the Ignition Gateway UI and select the Transmitters tab.

Make note of the Sparkplug IDs configured for your transmitter. For our example we have the Group ID as My MQTT Group and the Edge Node ID as Edge Node faec7e

Note
If you have an existing Transmitter configured where the Sparkplug IDs are created dynamically from the Ignition tag path, see MQTT Transmission Transmitters and Tag Trees, you will need to identify the Edge Node Descriptor being used and include the GROUP_ID and EDGE_NODE_ID in the Files configuration.

Image Added

Now on the 

...

'Files' tab, select the Create new Files... link and configure the following fields

    • Name
      • Tag Provider: default
      • Tag Folder Path: files_manual
      • Enable Auto-Publish: unchecked/false
        • Group ID: Use the Group ID you noted in the Transmitter configuration
          • By default a unique name will be set for this File Record. You can edit this if you choose but it must be unique.
        • Tag Provider
          • This is the tag provider where the publish control and information tags will be created. In our example, set to 'default'.
        • Tag Folder Path 
          • This is the tag folder in the specified tag provider where the publish control and information tags will be created.
          • In our examples, set to 'files_manual' if configuring for manual publish or 'files_auto' if configuring for auto publish
        • Enable Auto-Publish
          • This will enable auto_publish.
          • In our examples, leave unchecked if configuring for manual publish and check if configuring for auto publish.  
        • Group ID
          • Use the Group ID from your Transmitters Sparkplug ID. For this example, My MQTT Group
        • Edge Node ID:
          • Use the Edge Node ID
        • you noted in the Transmitter configuration
          • from your Transmitters Sparkplug ID. For this example, Edge Node faec7e 


      Tip
      Review the MQTT Transmission Configuration guide for information on all the File properties

      When complete, it should look

      ...

      similar to the following. Save the changes after confirming.

      Image Modified

      After the configuration is saved, it should look

      ...

      similar to below:

      Image Modified

      ...

      Manual Publish of Files

      With a File record created following the steps in Configure MQTT Transmission to publish files with the 'Enable Auto-Publish' unchecked and the Tag Folder Path set to "files_manual"At this point, everything should be configured to send files from MQTT Transmission to MQTT Engine.

      Open Ignition Designer on the system running MQTT Transmission. You should see a new set of tags there the control and information tags created in the specified tag provider and folder as shown below.

      Image Removed

      :

      Image Added

      The control and information tags created in the folder are:

      Name

      Type

      Description

      Last Published FileStringName of last published file
      Last Published Sequence NumberIntegerSequence number of last published file since last reset of metrics
      Percent CompletedBytePublish completion percent for file being published
      Publish FileBooleanManual trigger to publish file
      Publish File CountLongNumber of files published since last reset of metrics
      Publish File in TransitStringName of current file being published
      Publish Files PathStringFull path to the target file to publish over MQTT
      Publish Operation StatusStringStatus description of current publish operation
      Publish Operation Status CodeIntegerStatus code for current publish operation
      ResetBooleanTrigger to reset publish metrics


      At this point, we just need to tell MQTT Transmission which file to send. Download and unzip this sample file to some location and note that location.

      This sample_file.zip which contains two text files . The two text files which are:

      • sample_file.txt

        • This is a text file with an arbitrary string. But, it could be a file of any type. This is purely for demonstration purposes.

      • sample_file.txt.md5

        • This is a text file that contains only the md5 sum of the sample_file.txt
      Note

      The md5 sum file is only required when using the auto publish method. This is needed to ensure that the file to be transferred is complete and ready to be published.


      Note

      If you are testing with a different file, certain Certain characters have special meanings when used in filenames such as "*" for wildcards, and "\" in filename paths. If a file you are trying to publish contains any of the characters listed below, it will prevent files from being written to the file system.

      " * : < > ? / \ |

      Leading and trailing spaces in filenames and filenames ending in '.' are also not supported.


      With the file to be transferred in place, we can now set the 'Publish File Path' tag.

      This path must be the full path to the target file to publish over MQTT. In this example, we're using the path of '/tmp/transmission/sample_file.txt'. However, on a Windows system that would look something like 'C:\full\path\to\my\sample_file.txt'. When this is set, you should see something like this

      Image Added

      Finally, click the 'Publish File' boolean tag. This will begin the file publish process which will cause the file to be published. After doing so, you should see something similar to what is shown below.

      Image Added

      You should also now see the file to the location you set as the 'Base File Directory' in your MQTT Engine namespace configuration for Sparkplug B.

      Note
      The files will be left on the originating filesystem after a successful publish and will need to be removed manually

      Auto Publish of Files

      With a File record created following the steps in Configure MQTT Transmission to publish files with 'Enable Auto-Publish' checked and the Tag Folder Path set to "files_auto", everything should be configured to send files from MQTT Transmission to MQTT Engine.

      ...

      Open Ignition Designer on the system running MQTT Transmission. You should see the control and information tags created in the specified tag provider and folder as shown below:

      Image Added

      The control and information tags created in the folder are:

      Name

      Type

      Description

      Last Published FileStringName of last published file
      Last Published Sequence NumberIntegerSequence number of last published file since last reset of metrics
      Percent CompletedBytePublish completion percent for file being published
      Publish File CountLongNumber of files published since last reset of metrics
      Publish File in TransitStringName of current file being published
      Publish Files FolderStringFull path to the target folder containing the files to publish over MQTT
      Publish Operation StatusStringStatus description of current publish operation
      Publish Operation Status CodeIntegerStatus code for current publish operation
      ResetBooleanTrigger to reset publish metrics


      At this point, we just need to tell MQTT Transmission which folder contains the file(s) to publish. Download and unzip Unzip this sample file to some location and note that location.

      This sample_file.zip contains two text files which are:

      • sample_file.txt

        • This is a text file with an arbitrary string. But, it could be a file of any type. This is purely for demonstration purposes.

      • sample_file.txt.md5

        • This is a text file that contains only the md5 sum of the sample_file.txt
      Note

      If you a testing with a different file, certain characters have special meanings when used in filenames such as "*" for wildcards, and "\" in filename paths. If a file you are trying to publish contains any of the characters listed below, it will prevent files from being written to the file system.

      ...

      " * : < > ? / \ |

      Leading and trailing spaces in filenames and filenames ending in '.' are also not supported.


      Note

      When transferring files using auto_publish, MQTT Transmission requires two files to be present before it will transfer the target file. The first is the file itself. The second is a file that has the same name as the target file followed by a '.md5' extension. The file containing the md5 sum of the target file will not be transferred.

      The contents of that file must contain the Message Digest Algorithm 5 (or MD5 sum) of the file. The MD5 sum can be calculated using command line utilities on most operating systems or through scripting in Ignition. Here are some examples:

      Linux

      Code Block
      languagebash
      ubuntu@linux-host:~$ md5sum myfile.bin
07180622a24ebf905cf5f770cd54197a  myfile.bin

# In the above example, the md5 sum is: 07180622a24ebf905cf5f770cd54197a

      OSX

      Code Block
      languagebash
      user@osxhost:~$ md5 sample_file.txt
MD5 (sample_file.txt) = 85324ffbcc7d97c478adf53796aff787

# In the above example, the md5 sum is: 85324ffbcc7d97c478adf53796aff787

      Windows

      Code Block
      languagepowershell
      Get-FileHash -Algorithm MD5 .\some_file.iso

Algorithm       Hash                                                                   Path                                                                              
---------       ----                                                                   ----                                                                              
MD5             80FD169D3FDADBC97E66C168F796B1BF                                       C:\temp\some_file.iso

# In the above example, the md5 sum is: 80FD169D3FDADBC97E66C168F796B1BF

      With both files in place, we can now set the 'Publish File Path'. This path must be the full path to the target file to publish over MQTT. In this example, we're using the path of '/tmp/transmission/sample_file.txt'. However, on a Windows system that would look something like 'C:\full\path\to\my\sample_file.txt'. When this is set, you should see something like this

      Image Removed

      Finally, click the 'Publish File' boolean tag. This will begin the file publish process which will cause the file to be published.  When the file is successfully transferred both files will be removed from the originating filesystem.

      After doing so, you should see something similar to what is shown below.

      Image Removed

      You should also now see the file on the Ignition system with MQTT Engine installed. The file will be saved to the location you set as the 'Base File Directory' in your MQTT Engine namespace configuration for Sparkplug B.

      There is also an option to 'automatically' publish files using MQTT Transmission. In the 'File' configuration of MQTT Transmission you can check a tickbox that is 'Enable Auto-Publish'. If this is enabled, MQTT Transmission will create a tag called 'Publish Files Folder' instead of 'Publish File Path' as shown below.

      Image Removed

      Ignition Script
       Code Block
      languagepy
      import hashlib

# File to create md5 sum
file_name = "D:\MyFiles\test_file.txt"

# Open,close, read file and calculate MD5 on its contents 
with open(file_name, 'rb') as file_to_check:
    # read contents of the file
    data = file_to_check.read()    
    # pipe contents of the file through
    md5_returned = hashlib.md5(data).hexdigest()
    
# Save md5 sum file
f = open(file_name + ".md5", "w")
f.write(md5_returned)
f.close()


      When configured this way, MQTT Transmission will scan the folder specified in the tag and will , with a frequency defined by the File Scan Rate, and publish all files that appear in this directory automatically. Just as with the 'manual publish' option, an 
       Note
      When the file is successfully transferred both the file and associated md5 sum file 
      ...
      will be removed from the originating filesystem.

      Viewing the messages in the Ignition logs
      The records published from MQTT Transmission can be be viewed in the Ignition Gateway logs by setting the com.cirruslink.mqtt.engine.gateway.sparkplug.SparkplugPayloadHandler logger to TRACE.
       Tip
      Review the Gateway Loggers from Ignition for details on how to do this.
      Image Added



      Extra Activities
      At this point you have a fully functional system which can be expanded or modified as required. Below are some addition configuration options to try:
      Adjust the file transfer configuration to define how quickly a file transfer occurs and how long before a timeout will occur on failure to receive host side acknowledgements 
      ...
       based on the size of files you are sending and any bandwidth constraints that you may have.
      Additional Resources
      ...
      ...
      ...
      ...
      Setup two Ignition systems - a host system installed with MQTT Engine and MQTT Distributor and and edge system installed with MQTT Transmission
      By default, MQTT Transmission is configured to connect to MQTT Distributor on tcp://localhost:1883. As the MQTT Distributor is installed on a different server, we will need to edit the configuration to allow MQTT Transmission to connect to the host server.
       Note
      The MQTT Server must be accessible to the Ignition system running MQTT Transmission. In order for this to work port 1883 must be open to outside connections. Make sure this is allowed via your operating system's firewall configuration and any anti-virus software you may have installed.
      This configuration is not recommended for production systems. If running this in production, you should be using TLS encryption on port 8883. More information on how to configure TLS can be found here.
      Navigate to the MQTT Transmission > Settings in the left side bar of the Ignition Gateway UI and select the Servers tab.
      Change the server URL to point to the server IP address or domain name of the Ignition system running MQTT Distributor and save the configuration.




       Excerpt Include
      CLD80:FAQ: Ignition Modules
      CLD80:FAQ: Ignition Modules
      nopaneltrue
      ...