Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Before we get started on the requirements for enabling History on MQTT Engine Tags, one should take note of the two ways that MQTT Engine historical event processing can result in historical inserts into the database:

  1. Engine is configured by default, to write historical events directly to the database, via the Historian, bypassing the Tag.
  2. Engine can be configured to write historical events to the Tag instead of directly to the Historian. Reasons for writing directly to the Tag are:
    1. To have Tag Events scripts fire when applicable.
    2. If indirectly referencing MQTT Engine tags.
    3. To have alarms triggered when applicable.

Note: To enable history configuration is required at MQTT Engine, at each MQTT tag, and at the MQTT Transmission module. In addition, a History Store must also be defined for MQTT Transmission.


Note
Store and Forward does not guarantee all data is stored and forwarded. There are some edge cases that are not currently handled with regard to data loss in the event of connection failures related to MQTT keep alive timeouts. This window of potential missed data can be reduced by decreasing MQTT Transmission and MQTT Engine configurable keep alive timeouts.

...


...

Anchor

...

There are caveats to configuration option 2 above (to write historical events directly to the Tag):

If MQTT Engine is configured to write historical events directly to the Tag, history on the Edge (i.e., Transmission side) must be configured to flush history in order. This means that when the Edge side client comes back online and flushes history, it must flush the oldest historical events first (in order) before sending live Tag changes events to Engine. This is because Ignition will ignore writes to the Tag if the timestamp on the Tag change is older that the current value. Please see the screenshots below for context:

...

Requires the [MQTT Transmission Settings -> Transmitters -> [Your Transmitter… Default/Custom] -> Configuration -> In-Order History] setting to be checked/true
Image Removed

...

There are some rules for enabling history on MQTT Engine Tags. They are:

...

historystore
historystore
MQTT Transmission History Store

  1. Define and enable an MQTT Transmission History Store
  2. Config your transmitter to consume the enabled History Store
  3. Store and forward controls whether or not a tag is stored in the configured History Store when Transmission is offline. By default the global setting of "Enable History Storage by Default" is selected for all tags. To override , individual tags will require a custom tag property 'StoreAndForward' (type: boolean) to be created and set to the reverse state of the global setting.

Anchor
enginetags
enginetags
MQTT Engine Tags

  1. History *must* be enabled on the MQTT Engine Tag and the history settings for that Tag tag must be as follows:
    • History Enabled: true
    • Storage Provider: Must be set to an existing Storage Provider
    • Sample Mode: On Change
    • Min Time Between Samples:  0 ms
      • This is required in cases where historical metrics flushed from the Edge have very high resolution; e.g., the historical metrics have timestamps 1 ms apart for a single tag.
  2. Ignition 7 Only: “Timestamp Source” must be set to “Value” (*not* “System”)
    This is required in order for the historical event to have the timestamp when the value changed on the Edge, instead of when the historical event is being processed by Engine. Having the 
    “Timestamp Source” set to “System” will result in historical events being inserted with the wrong (later) timestamp.
    NOTE: the above history settings will ensure the proper historical event processing by MQTT Engine when writing history directly to the database and when writing history directly to the Tag.

...

See the Ignition documentation Configuring Tag History for additional help.

Anchor
historicaleventprocessing
historicaleventprocessing
Historical Event Processing

There are two ways that MQTT Engine historical event processing can result in historical inserts into the database:

  1. Engine is configured to write historical events directly to the database, via the Historian, bypassing the Tag.
  2. Engine is configured to write historical events to the Tag instead of directly to the Historian.

Anchor
historicaleventsdatabase
historicaleventsdatabase
Writing historical events directly to the database, via the Historian, bypassing the tag

The configuration parameters required to write historical events directly to the database, via the Historian, bypassing the Tag are detailed below. 

Note
These are the default settings for both MQTT Engine and MQTT Transmission. Enabling the In-Order History for MQTT Transmission is unnecessary and will result in a waste of resources.  

MQTT Engine

Under the MQTT Engine Settings General Tab, navigate to the Miscellaneous Settings and ensure Store Historical events is selected

Image Added

MQTT Transmission

Under the MQTT Transmission Settings for your transmitter, navigate to the History Settings and ensure that In-Order History is de-selected.

Image Added

Anchor
historicaleventstags
historicaleventstags
Writing historical events directly to tag

There are several reasons why you might need to write directly to the tag which include:

  • To have Tag Events scripts fire when applicable
  • If indirectly referencing MQTT Engine tags
  • To have alarms triggered when applicable

The configuration parameters required to write historical events to the Tag instead of directly to the Historian are detailed below. 

MQTT Engine

Under the MQTT Engine Settings General Tab, navigate to the Miscellaneous Settings and ensure Store Historical events is de-selected

Image Added

Anchor
engine-tag-history-config
engine-tag-history-config

MQTT Transmission

Under the MQTT Transmission Settings for your transmitter, navigate to the History Settings and ensure that In-Order History is selected. This ensures that when the Edge side client comes back online and flushes history, it will flush the oldest historical events first (in order) before sending live Tag changes events to Engine. 

Image Added


Warning
If you do not select In-Order History, data will be lost/ignored at MQTT engine because Ignition will ignore writes to the Tag if the timestamp on the Tag change is older that the current value.


Anchor
engine-tag-persist-history-config
engine-tag-persist-history-config
History Configuration Persistence

Warning
Deleting an MQTT Engine tag will result in a loss of history configuration.

...

However, deleting MQTT Engine tags can be required in certain cases. One example is if UDTs are being propagated from the Edge and the UDT definition on the Edge has been updated. This requires one to delete all instances of the UDT under MQTT Engine Engine and delete delete the corresponding UDT definition so that it can be recreated/updated at MQTT Engine for the Edge side changes to take affect.

There are two options for persisting history configuration in this case and they are as follows:

  1. Use Reference tags to indirectly reference MQTT Engine tags such that history configuration can be persisted on the Reference tag when the underlying MQTT Engine tags are deleted. This is the recommended way of configuring history on MQTT Engine tags (indirectly) so history configuration persists.
  2. Use Ignition scripting to reapply history configuration directly on MQTT Engine tags on demand. This is a bit more cumbersome as there must be some mechanism to "trigger" the script execution. 

...



Anchor

...

indirect-

...

references

...

One can determine the current size of the History Store by examining the History Store metric tags under [MQTT Transmission]Transmission Info/History Store. These tags will show the number of historical metrics stored per edgenode/device and how much memory/disk is being consumed by these metrics. These tags update live (count down) as historical data is flushed. 

Image Removed

indirect-references
History and Indirect References to MQTT Engine Tags

...

...

Configuring history on tags indirectly referencing MQTT Engine tags will work properly if the referencing tag is a Reference tag only. Derived, Expression and OPC tags (expose MQTT Engine tag provider through OPCUA server) will not properly store history when MQTT Engine tags are updated with historical data at a high rate of speed. This is a limitation within the Ignition platform and may be addressed in a future release.

...

If using an Ignition v8.1.4 or later with MQTT Engine - Historical back-fill support was added in Ignition which allows Reference Tags in Ignition to be updated even if history arrives out of order. See the following document for details: MQTT History Back-Fill with Reference Tags


Anchor
flushing-history-metrics
flushing-history-metrics
History Stores Metrics & Flushing History

One can determine the current size of the History Store by examining the History Store metric tags under [MQTT Transmission]Transmission Info/History Store. These tags will show the number of historical metrics stored per edgenode/device and how much memory/disk is being consumed by these metrics. These tags update live (count down) as historical data is flushed. 


Image Added



Excerpt Include
CLD80:FAQ: Ignition Modules
CLD80:FAQ: Ignition Modules
nopaneltrue