Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  • Review our config docs and sanity check your configuration
  • Make sure Ingest user doesn’t have ‘must change password’ set to true
  • Ingest user has been granted the CL_BRIDGE_PROCESS_RL role

I am seeing "Throttled due to memory pressure" warnings in the log

The usual cause for this error is that Snowflake is throttling the data throughput because too much data is trying to be pushed through too few channels.

By default, in the ibsnow.properties configuration file, we use 'snowflake_streaming_channel_scheme = EDGE_ID' which means there is a channel per Sparkplug Edge Node.

We can increase the number of 'streaming channels' available for pushing to Snowflake streaming ingest services by modifying the schema to read 'snowflake_streaming_channel_scheme = DEVICE_ID' which will create a channel on a per Sparkplug Device basis.

By increasing the number of channels, we should be able to increase the throughput and free up the memory backup.