Snowflake Event Ingest vs Ecom Orders, Snowflake

​

At a glance

A dual-axis chart that overlays the rate at which event rows are landing in Snowflake (the ingest pipeline, typically Fivetran, Snowpipe, or a Kafka connector loading order, clickstream, and inventory events) against the rate at which the storefront is taking real orders. The two lines should move together: more shopping activity means more events to load. The dangerous shape is divergence in one specific direction, orders flowing healthily on the storefront while the Snowflake ingest line flattens or drops to zero. That gap means the data warehouse has gone stale: every downstream dashboard, every Vortex Mind investigation, every finance report is now reading yesterday’s world while today’s revenue is happening live. The Fivetran ingest stalled while orders flowing alert exists because a silent ingest stall is invisible from the storefront. Sales look fine; the data backing every decision is quietly hours out of date.

​

Calculation

ingest_rate(t)  = rows loaded into landing tables in bucket t
                  (sum of ROW_COUNT from COPY_HISTORY + Snowpipe deltas)
order_rate(t)   = orders created in bucket t (from linked ecom connector)

divergence(t)   = (order_rate normalised) − (ingest_rate normalised)
alert fires when ingest_rate ≈ 0 for N consecutive buckets
                 AND order_rate stays above its baseline

• A paused source connector is not a Snowflake error. If Fivetran, Airbyte, or a custom job stops pushing rows, Snowflake has nothing to do. COPY_HISTORY simply shows no new entries. There is no failed query, no errored task, nothing for an error-rate card to catch. The absence of work is the symptom, and absence is what this card is built to detect.
• Snowpipe latency is normally seconds, so a flat line is meaningful. Snowpipe auto-ingest typically lands files within a minute or two of arrival in the stage. If the ingest line goes flat for an hour while orders flow, the file is not arriving in the stage at all (an upstream export or a cloud-storage notification problem), not a Snowflake delay.
• COPY_HISTORY reports per-file load outcomes. Each entry carries ROW_COUNT, ROW_PARSED, ERROR_COUNT, and STATUS. A stall shows as no new rows; a partial failure shows as STATUS = 'LOAD_FAILED' or a non-zero ERROR_COUNT. The card reads both, but the alert is specifically about the no-new-rows case while orders continue.
• The order line is the ground truth. Because the storefront connector reads orders directly from the commerce platform’s own API, the order rate is not dependent on Snowflake at all. That independence is the whole point: it gives you a trusted external pulse to compare the warehouse against.
• Account Usage views carry their own latency. COPY_HISTORY in the ACCOUNT_USAGE schema can lag actual loads by up to ~90 minutes. Where lower latency matters, the connector also reads INFORMATION_SCHEMA.COPY_HISTORY (the table function), which is near real-time but limited to the last 14 days and a single table or pipe per call.

​

Worked example

1. Confirm the stall is upstream, not in Snowflake. Query INFORMATION_SCHEMA.COPY_HISTORY for the RAW_SHOPIFY.ORDERS table over the last three hours. No new entries since 08:52 confirms Snowflake received nothing to load: the source stopped pushing. A Snowflake-side failure would instead show LOAD_FAILED rows.
2. Check the source connector. In this case the Fivetran connector’s OAuth token to Shopify had expired at 08:52 and the sync paused with a re-auth prompt. The fix is to re-authorise the connector; Fivetran then backfills the gap on its next sync.
3. Communicate the blast radius while it backfills. Until the backfill completes, every dashboard and report reading RAW_SHOPIFY is missing ~2.5 hours of orders. Anyone making a same-day decision (marketing pacing, inventory reorder, a finance flash) needs to know the warehouse is behind. This is the part teams forget: the data being stale is worse than the data being down, because stale data looks usable.

Cost framing of a silent ingest stall:
  Orders during the stall window (09:00 to 11:30): ~233
  Revenue those orders represent: ~£18,600
  None of it visible in the warehouse until backfill completes.
  Decisions made on stale data during the gap: marketing budget pacing,
  a low-stock reorder trigger, the 11:00 ops standup numbers.
Caught by the alert at 09:15: re-auth and backfill, ~30 minutes to current.
Caught at the next finance close: a day of every report being quietly wrong.

1. A flat ingest line with a healthy order line is the signature of a paused source, not a Snowflake fault. Do not start by debugging warehouses or queries; start at the source connector (Fivetran, Snowpipe stage, Kafka).
2. Stale beats down for danger, because stale is invisible. A warehouse that is fully down gets noticed fast. A warehouse that is silently three hours behind keeps serving plausible-looking numbers that are wrong, and people act on them.
3. The order line is your trusted external pulse. Because it comes from the commerce platform directly, it never depends on Snowflake. Whenever you doubt a warehouse signal, this independent line is the reference to triangulate against.

​

Sibling cards to reference together

​

Reconciling against the source

Recent loads (near real-time, last 14 days): SELECT * FROM TABLE(INFORMATION_SCHEMA.COPY_HISTORY(TABLE_NAME=>'RAW_SHOPIFY.ORDERS', START_TIME=>DATEADD('hour', -6, CURRENT_TIMESTAMP()))); shows per-file ROW_COUNT, STATUS, and ERROR_COUNT. A gap with no rows confirms a stall. Account-wide load history (up to ~90 min latency): SELECT table_name, SUM(row_count), MAX(last_load_time) FROM SNOWFLAKE.ACCOUNT_USAGE.COPY_HISTORY WHERE last_load_time >= DATEADD('hour', -24, CURRENT_TIMESTAMP()) GROUP BY table_name ORDER BY 3 DESC; Snowpipe ingest: SELECT * FROM SNOWFLAKE.ACCOUNT_USAGE.PIPE_USAGE_HISTORY WHERE start_time >= DATEADD('hour', -24, CURRENT_TIMESTAMP()); and SELECT SYSTEM$PIPE_STATUS('your_pipe'); for live pipe state. Managed console: Snowsight under Monitoring -> Copy History gives a visual per-table load timeline; a flat stretch is the same stall this card shows. For the source side, the stall usually originates in the ingest tool’s own console (Fivetran sync history, Airbyte job log), not in Snowflake at all.

​

Known limitations / FAQs

​

Tracked live in Vortex IQ Nerve Centre