Composite, apdex x inverse error-rate x inverse incident-count x SLO compliance. The CXO single-number.
At a glance
A 0, 100 composite that compresses four New Relic operational signals (Apdex, error rate, active incident count, SLO compliance) into one number a non-engineering founder can read at a glance. Designed for the question “is my store backend doing what it should be doing right now, yes or no?”
Calculation
Calculated automatically from your New Relic data. See the At a glance summary above for what the metric tracks and the worked example below for a typical reading.Worked example
A multi-region Shopify Plus brand running a Node.js storefront on Cloud Run, instrumented with the New Relic APM agent. The 5-minute window covers 12:35 to 12:40 on 02 May 26.
Composite = 0.30 x 86 + 0.25 x 68 + 0.20 x 80 + 0.25 x 99.74 = 25.8 + 17.0 + 16.0 + 24.94 = 83.7
Score 84 is healthy (above the 70 alert threshold). Reading the breakdown:
- Error rate is the laggard. At 3.2% it cost 8 composite points (vs an ideal score of 100). With Apdex at 0.86 the user-facing latency feels acceptable but a meaningful slice of requests are failing outright. Open Errors by Transaction and the Apdex calibration: with
t: 0.5the “frustrated” threshold is 4 x 0.5 = 2.0 seconds, so users hitting errors that take >2s to surface are doubly punished in Apdex too. - One open P1. Memory pressure on catalogue. Worth checking whether it correlates with the error rate spike via Deploy Markers vs Latency.
- SLO compliance is comfortable. 99.74% over 7 days, only 0.16% under the 99.9% target. Plenty of error-budget headroom.
Sibling cards merchants should reference together
Reconciling against the vendor’s own dashboard
Where to look in New Relic: New Relic does not surface a single “Operational Health” composite, this card synthesises one from four New Relic-native signals. The closest equivalent screens in one.newrelic.com:- APM & Services for Apdex and error-rate context.
- Alerts & AI > Issues & Activity for active incident count.
- Service Levels for SLO compliance.
- Dashboards > pre-built “Application performance” dashboard for combined view.
Cross-connector reconciliation:
NR APM and Datadog APM are the two leading commercial APM platforms. Probes are different (NR APM agent vs Datadog APM trace agent), so latency numbers can differ by 5, 15ms even on the same service, both are legitimate and the gap itself is sometimes diagnostic (a probe overhead difference, not a real customer-facing change). Browser-side telemetry (NR Browser RUM, GA4 Web Vitals) sample differently again, NR Browser instruments every page load, GA4 samples by session.
Known limitations / merchant FAQs
We have both New Relic and Datadog connected, which Operational Health Score should I trust? Both are legitimate. Practically, pick one as your “system of record” based on which platform owns more of your services. Brands that started on New Relic and added Datadog for infrastructure usually trust the NR composite for backend health and the DD composite for infrastructure health. The two scores should agree within ~5 points; a 15+ point gap is a signal worth investigating (probe coverage difference, agent version drift, or one platform missing a service). My Apdex is 0.92 but the score is only 78, what’s pulling it down? Apdex is only 30% of the formula. With a healthy Apdex but, say, a 4% error rate (the x10 amplifier takes that component to 60), one active P1 (component = 80), and 99.5% SLO compliance (under target so contributes 99.5), the composite lands at 0.30 x 92 + 0.25 x 60 + 0.20 x 80 + 0.25 x 99.5 = 27.6 + 15 + 16 + 24.9 = 83.5. Open Error Rate and Active Incidents to see the actual culprits. How is Apdex calibrated, and why is 0.5 the danger line? Apdex requires a “tolerance threshold”t. Requests below t are satisfied, between t and 4 x t are tolerated, above 4 x t are frustrated. The score = (satisfied + 0.5 x tolerated) / total. So a score of 0.5 means most users are either tolerated or frustrated, the satisfied bucket is below half. In commerce contexts we standardise t = 0.5s for browse pages and t = 1.0s for checkout (longer-tolerance because customers expect a “processing” feel). New Relic’s Apdex documentation covers the math in detail.
Datadog and New Relic disagree on p95 latency by 12ms, who’s right?
Both probes are sampling different segments of the same request. NR’s APM agent instruments at the entry-point (typically the framework middleware), Datadog’s trace agent often instruments slightly earlier or later depending on the integration. A 5, 15ms difference is normal. The 12ms gap itself is not a problem; the trend matching across both platforms is what matters. If NR shows a 50ms increase and DD shows the same 50ms increase, the increase is real regardless of which absolute baseline you trust.
NRQL retention is 8 days, can I still see compliance over a 30-day SLO window?
Yes. SLO compliance is computed on New Relic’s Service Level entity, which maintains rolling SLI/SLO state at the entity layer (separate from raw event retention). The 30-day SLO works even though raw Transaction events past day 8 are aggregated. What you lose at >8 days is the ability to re-query a specific minute of historical data, but the rolled SLI numbers stay accurate.
Do high-cardinality sampling rates affect my score?
Rates yes (sample-corrected automatically), counts no. If your account is in a sampled state during peak traffic, the Apdex score and error rate percentage remain accurate (NR back-calculates from the sample). The active P1 incident count is unaffected (incidents are not sampled). SLO compliance is unaffected (computed on the SLI ratio at ingest time). So the composite stays valid even during sampled periods.
My account aggregates two sub-accounts (production, staging), is the score across both?
Vortex IQ reads the New Relic account configured in your integration. If you’ve connected only the production account, the composite covers production only, which is what most merchants want. To monitor staging separately, connect a second integration (the clone integration flow handles this).
Ingest cost has gotten high, can I drop visibility to save money?
Yes, but tradeoffs apply. Common levers: (a) drop Browser agent on low-traffic pages (saves event ingest), (b) sample Transaction events to 50% on non-checkout endpoints (Apdex stays accurate via sample-correction; raw counts halve), (c) reduce Log ingest by filtering INFO-level entries server-side. The composite stays accurate under all three; what degrades is your ability to drill into individual requests when something goes wrong. We recommend keeping checkout-path instrumentation at 100%, sampling everything else.
The score is at 70 exactly and my pager went off, was the alert noise?
No. The trigger is < 70 (strict less than), so 70.00 should not fire. If you saw an alert at exactly 70 it likely fluttered: a previous reading of 69.x triggered, then the score recovered to 70.x by the next sample. See Recently Flapped Conditions (24h) to confirm. Tune the alert by adding a 5-minute “must stay below 70” window to suppress flutter without losing the real signal.