At a glance
This dual-axis card plots Elasticsearch search queries per second against storefront traffic (sessions or page views from the connected ecom/analytics source) over the same window. In a healthy store the two lines move together: more shoppers means more searches. The signal you are watching for is divergence. A spike in search QPS with no matching spike in real traffic almost always means something non-human is hitting search: a runaway crawler, a scraper, a misbehaving integration, or an autocomplete loop firing a query on every keystroke. Catching this matters because bot-driven QPS consumes the same search thread pool and JVM heap that paying customers need, so it can degrade or take down search for everyone while generating zero revenue.
Calculation
The Elasticsearch series is a rate computed from a monotonically increasing counter:indices.search.query_total counts query-phase executions, so a single user search that fans out across N shards increments the counter N times. The card normalises for this where shard count is known, but the headline trend is what matters: the shape of the curve, not an exact per-user figure.
The ecom traffic series is taken at face value from the connector (sessions or page views per bucket) and plotted on the second axis with its own scale, which is what “dual axis” means: the two lines can have very different magnitudes and still be compared by shape.
The divergence detector looks at correlation over the 15-minute window. In normal operation search QPS and traffic are tightly correlated (shoppers search as they browse). The alert fires when QPS rises sharply while the traffic line stays flat, breaking that correlation. A flat-traffic, rising-QPS pattern is the classic non-human signature, so the card calls it out as a likely bot/crawler.
Worked example
A fashion retailer on BigCommerce uses a managed Elasticsearch cluster for product search and autocomplete. Storefront traffic is connected via the BigCommerce connector. Snapshot covers the 15 minutes from 13:20 to 13:35 GMT on 22 May 26.
The dual-axis chart shows the blue QPS line climbing steeply away from the flat orange sessions line. The Vortex IQ headline flags QPS spike with no traffic spike (likely bot/crawler). The platform team works the problem:
- Confirm it is not real demand. Sessions are flat at ~500/min, exactly where they have been all afternoon. There is no email send, no paid-media burst, no flash sale in the calendar. Real shoppers did not suddenly start searching 6x harder.
- Identify the source. A quick look at the cluster’s hot threads and the load balancer access logs shows a single user-agent issuing paginated
match_allqueries against the products index, deep-paging through results, a textbook scraper. It is not coming through the storefront search box (those would create sessions), which is exactly why the traffic line never moved. - Assess the collateral damage. During the spike, Search Latency p95 climbed from 120ms to 480ms and HTTP Connection Saturation touched 88%. Real shoppers were being slowed by the scraper. That is the revenue-at-risk: not the bot’s queries (worthless), but the degradation it inflicts on genuine sessions.
- Mitigate. The team rate-limits the offending source at the edge (WAF/CDN), and QPS drops back to ~100 within a minute while sessions stay flat. The two lines reconverge. They then add a standing rule to block deep
match_allpagination from unauthenticated clients.
- Search QPS in isolation is ambiguous; against traffic it becomes diagnostic. A QPS jump could be a genuine demand surge or a bot. The only way to tell at a glance is the second line. If traffic rose too, celebrate; if it did not, investigate.
- Bots cost you through contention, not through their own queries. The danger is the heap, thread pool, and connection capacity the bot steals from paying customers, which is why this card sits in Revenue at Risk despite measuring a non-revenue actor.
- The inverse pattern is also worth a glance. Traffic rising while QPS stays flat can mean search is broken or the search box is not loading, shoppers are landing but cannot search. Same chart, opposite divergence, also actionable.
Sibling cards
Reconciling against the source
Where to look in Elasticsearch and the storefront:Why our number may legitimately differ from a raw stats read:GET /_nodes/stats/indices/searchandGET /<index>/_stats/searchexposequery_totalandquery_time_in_millis; differencingquery_totalover time reproduces the QPS series.GET /_nodes/hot_threadsduring a suspected spike shows which queries are eating CPU, useful for identifying a scraper’s query shape. Your load balancer / CDN / WAF access logs are the authoritative source for “who is sending these queries”; Elasticsearch counts the queries but the edge identifies the client. On the storefront side, reconcile the traffic line against the platform’s own analytics (BigCommerce/Shopify/Adobe reporting, or GA4 real-time) for the same 15 minutes.
Cross-connector reconciliation:
Known limitations / FAQs
Search QPS spiked but so did traffic. Is that an alert? No. When both lines rise together it is genuine demand: an email send, a flash sale, a paid-media burst, or organic interest. The card only flags the case where QPS diverges from traffic. A correlated spike is a capacity question (watch HTTP Connection Saturation and JVM Heap Used %), not a “wrong source” question. Why isquery_total higher than the number of searches I think happened?
Because Elasticsearch counts the query phase per shard. A search that touches 5 shards adds 5 to query_total. The card normalises for this where it can, but for trend purposes the shape of the curve is what matters, not an exact per-user count. If you need exact user-level search counts, instrument the application layer instead.
An autocomplete box fires a query on every keystroke. Does that look like a bot?
It can. Aggressive autocomplete (one Elasticsearch query per keystroke, no debounce) multiplies QPS far faster than sessions grow, producing exactly the divergence this card flags. The fix is on the front end: debounce keystrokes, cache prefixes, and use the search-as-you-type or completion suggester so the query is cheap. The card is doing its job by surfacing the pattern.
The traffic line is flat at zero but QPS is normal. What does that mean?
Either the storefront/analytics connector is not reporting (broken connector, expired token) or you genuinely have no live sessions while a scheduled job still queries search. Check the connector health first; a missing traffic line makes the comparison meaningless and can produce a false bot signal.
Can I see which client is causing the spike from this card?
Not directly. Elasticsearch counts the queries but does not, by default, attribute them to a client IP or user-agent. To identify the source, correlate the spike window with your load balancer, CDN, or WAF access logs, and use GET /_nodes/hot_threads to see the query shape. Once identified, rate-limit or block at the edge.
Is blocking the bot the right response, or should I scale the cluster?
Scale only for real demand. Adding nodes to absorb scraper traffic is paying to serve queries that generate no revenue. The correct response to a divergent (bot) spike is to rate-limit or block the source at the edge. Reserve scaling decisions for correlated spikes where real shoppers are driving the load.
Does this card include indexing load?
No. This card is about the search path only (indices.search.query_total). Indexing throughput is tracked separately by Indexing Rate (docs/sec). Keeping them separate matters because a search spike and an indexing spike have completely different causes and remedies.