IBSNOW: Viewing the data in Snowflake

Data can be view in the databases created via the Setup Assistant during your initial setup:

• Staging database e.g., CL_BRIDGE_STAGE_V2_DB
• Node database e.g., CL_BRIDGE_NODE_V2_DB

Terminology

As the data moves from the Ignition Edge device through MQTT Sparkplug and into Snowflake different terminology will be used with the equivalence shown below:

Staging Database

The Staging database contains two tables: SPARKPLUG_RAW and MQTT_RAW.

These tables are populated by the Snowflake Snowpipe Streaming API as data is streamed in by the IoT bridge.

• SPARKPLUG_RAW

  • This table contains information derived from each Sparkplug message as detailed below

    • GROUP_ID
    • EDGE_NODE_ID
    • DEVICE_ID
    • IINSERTED_AT
    • MESSAGE_TYPE
    • MSG
    • MSG_ID
    • MSG_TOPIC
    • NAMESPACE

• MQTT_RAW

  • This table contains information derived from each MQTT message as detailed below

    • INSERTED_AT
    • INSERTED_AT_DATE
    • MSG
    • MSG_ID
    • MSG_TOPIC

Node Database

The Node database contains five tables: SPARKPLUG_DEVICE_MESSAGES, SPARKPLUG_DEVICE_REGISTRY, SPARKPLUG_EDGE_NODE_REGISTRY and SPARKPLUG_TEMPLATE_DEFINITION_REGISTRY

Data is sourced from SPARKPLUG_RAW table and is inserted by the synch_device_messages() stored procedure. The synch_device_messages() stored procedure is executed on demand by the IoT Bridge for Snowflake.

• SPARKPLUG_DEVICE_MESSAGES

  • Contains contains partially processed data where each row in the table is a single message containing data for each model instance (aka. Device Name)

    • GROUP_ID
    • EDGE_NODE_ID
    • DEVICE_ID
    • INSERTED_AT
    • IS_GOOD
    • IS_HISTORICAL
    • MESSAGE_SEQUENCE
    • MESSAGE_TIMESTAMP
    • MESSAGE_TYPE
    • METRIC_NAME
    • METRIC_TIMESTAMP
    • METRIC_TIMESTAMP_NTZ
    • METRIC_VALUE
    • MSG_ID
    • TEMPLATE_PATH
    • TEMPLATE_REFERENCE
    • TEMPLATE_VERSION

• SPARKPLUG_DEVICE_REGISTRY
  • Shows all of the devices, whether or not they’re alive, and their BIRTH messages
    • GROUP_ID
    • EDGE_NODE_ID
    • DEVICE_ID
    • IS_ALIVE
    • LAST_UPDATE
    • BIRTH_MESSAGE

• SPARKPLUG_EDGE_NODE_REGISTRY
  • Shows all of the edge nodes, whether or not they’re alive, BIRTH messages and BD seq number
    • GROUP_ID
    • EDGE_NODE_ID
    • DEVICE_ID
    • IS_ALIVE
    • LAST_UPDATE
    • BIRTH_MESSAGE
    • BQ_SEQ

• SPARKPLUG_TEMPLATE DEFINITION_REGISTRY
  • Shows the templates (UDT defs), their version, and their definitions
    • TEMPLATE_PATH
    • TEMPLATE_VERSION
    • TEMPLATE_DEFINITION
    • LAST_UPDATE

• SPARKPLUG_TEMPLATE_INSTANCE_REGISTRY
  • Shows all of the template instances (UDT instances), group/edge/device IDs, TEMPLATE_PATH (used to be DEVICE_NAME), TEMPLATE_REFERENCE (UDT def name), and the template definition
    • GROUP_ID
    • EDGE_NODE_ID
    • DEVICE_ID
    • LAST_UPDATE
    • TEMPLATE_PATH
    • TEMPLATE_VERSION
    • TEMPLATE_DEFINITION
    • LAST_UPDATE
    • TEMPLATE_REFERENCE

The IoT Bridge for Snowflake will automatically create a new schema in its “Node” database for every Sparkplug edge node it finds. The Bridge will then create Views within this schema for each model found in the Sparkplug edge node data ingested via the bridge . There are two views per model:

Intermediate View

This view contains the model instance tag data where each row is a single tag change per model instance. In this view, each model instance tag is separated out into its own column with their values. This view will show each individual model instance tag change and the tags that did not changed will be marked as null.

• Example: CL_BRIDGE_NODE_DB.NB_GROUP_EDGE.MOTOR_VW

Asof View

This view is basically the same as the Intermediate view - each model instance tag is separated out into its own column. However, each row in this view has the null values from the Intermediate views removed (unless the tag change really had a null value) and the last known good value for the model instance tag will be replicated in its column. Said another way, each row will have the last known good values for all model instance tags.

• Example: CL_BRIDGE_NODE_DB.NB_GROUP_EDGE.MOTOR_ASOF_VW