Alerts

An alert rule watches one metric in your workspace: the error rate of your calls, what they cost, how much of a budget is used, a vendor rejecting your own key, and more. When the metric crosses the rule's threshold, GridRouter opens an alert, shows it as a toast in the dashboard and notifies the rule's destinations: email, your own webhook endpoints, Slack, Discord, Microsoft Teams, PagerDuty or Opsgenie. When the metric recovers, the alert resolves on its own and destinations that take resolve notifications are told.

Manage rules, alerts and destinations in the dashboard under Settings → Alerts & integrations, through the API or with the MCP tools below.

MetricsCopy Anchor Link

Every rule measures one of these. The table is generated from the same definitions the API validates against.

threshold is in the metric's unit. Money is always integer micro-USD, so 5000000 is $5. Rates (error_rate, vendor_failure_spike, p95_latency, cache_hit_rate) wait for min_events calls or lookups in the window before they can fire, so a single failed call in a quiet minute doesn't page anyone.

Default rulesCopy Anchor Link

Every new workspace starts with these rules. They're ordinary rules: edit, disable or delete any of them.

"Key used from a new IP" is informational: it shows as a toast but goes to no destination until you route it to one. A key's first IP is its baseline and never alerts.

Rule settingsCopy Anchor Link

Each destination has its own min_severity (alerts below it are not sent there) and notify_on_resolve (default on).

LifecycleCopy Anchor Link

An alert is triggered, then optionally acknowledged, then resolved. Its timeline records every step: when it triggered, each notification sent or failed, acknowledgement and resolution (the newest 60 entries, always keeping the trigger).

• Dedupe. While an alert is open, further breaches of the same rule and subject don't open another one. They raise the alert's count and update its value and last_seen_at.
• Grouping. Some metrics open one alert per subject instead of one per rule: vendor_failure_spike and vendor_key_failing per vendor, budget_used per budget (the workspace's or one key's), new_ip per key and security_event per signal kind. The subject is on the alert as subject.
• Cooldown. After an alert resolves, the same rule and subject can't open a new alert for cooldown_s. The budget rules default to a day, so a budget that hovers around 80% alerts once a day at most.
• Auto-resolve. A live rule reports that it has cleared once it has stayed on the right side of its threshold for a minute. A scheduled rule resolves on the next five-minute run that's no longer breached. An alert that nothing has confirmed for a while resolves too: 5 to 30 minutes for live rules (the window, within that range), 15 minutes for scheduled rules, and the rule's window for event rules.
• Acknowledge. POST /v1/alerts/{id}/acknowledge marks a triggered alert as being handled. PagerDuty and Opsgenie are acknowledged too; chat and email destinations only hear about triggers and resolutions.
• Resolve. POST /v1/alerts/{id}/resolve closes an alert now, and the rule's cooldown starts from there. Changing what a rule measures (metric, threshold, window, min_events, scope or kinds), disabling it or deleting it resolves its open alerts.

MuteCopy Anchor Link

POST /v1/alerts/rules/{id}/mute with { "minutes": 60 } (or { "until": "<ISO time>" }) snoozes a rule (minutes goes up to 30 days). It keeps evaluating: its alerts still open and resolve, show in history marked muted and still show as toasts, but nothing is sent to destinations or webhook endpoints. { "minutes": 0 } unmutes it.

Live and scheduled evaluationCopy Anchor Link

Where a rule runs depends on its metric and window:

• Live. error_rate, spend, vendor_failure_spike, vendor_key_failing and billing_mismatch with windows up to an hour run in your workspace's live tail hub on every call, so they fire within seconds.
• Scheduled. p95_latency, cache_hit_rate, budget_used, delivery_backlog and any live metric with a window over an hour are evaluated every five minutes, against your call log (hourly rollups for whole-hour windows), your budgets and the webhook delivery log.
• Event. new_ip and security_event fire when the signal arrives.

Every rule shows its state: ok, firing, muted, disabled or no_data. no_data means the rule couldn't be evaluated, with the reason in state.note; for example, in a deployment without the call-log store, scheduled call metrics report no_data while every other rule keeps working.

ToastsCopy Anchor Link

Alerts with toast on appear as toasts in the dashboard when they trigger and when they resolve. They arrive over the live tail as alert events, so any client that tails kinds=alert sees them too:

curl -N "https://api.gridrouter.io/v1/logs/live?format=sse&kinds=alert&replay=0" \
  -H "Authorization: Bearer $GRID_API_KEY"

Create a ruleCopy Anchor Link

Creating and changing rules needs a key with keys:write. This rule pages on-call when a third of email-finding calls fail over ten minutes, and only notifies one destination:

curl https://api.gridrouter.io/v1/alerts/rules \
  -H "Authorization: Bearer $GRID_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Email finding failing",
    "metric": "vendor_failure_spike",
    "threshold": 0.33,
    "window_s": 600,
    "min_events": 25,
    "severity": "critical",
    "scope": { "type": "capability", "value": "people.email.find" },
    "all_destinations": false,
    "destination_ids": ["dst_4f8k2m9q4w8x1c5v"]
  }'

The response is the rule with its id, its evaluator and its state. POST /v1/alerts/rules/{id}/test sends a test alert to every destination the rule notifies and reports each result; nothing is recorded as a real alert.

API and MCPCopy Anchor Link

Every operation is a REST route and an MCP tool of the same name. Reads need logs:read, rule and alert changes need keys:write, and destinations need credentials:write because they hold secrets.

Reads, with logs:read:

• alerts_summary: GET /v1/alerts/summary
• alerts_rules_list and alerts_rules_get: GET /v1/alerts/rules and GET /v1/alerts/rules/{id}
• alerts_list and alerts_get: GET /v1/alerts?status=open and GET /v1/alerts/{id}
• alerts_destinations_list and alerts_destinations_get: GET /v1/alerts/destinations and GET /v1/alerts/destinations/{id}

Rule and alert changes, with keys:write:

• alerts_rules_create: POST /v1/alerts/rules
• alerts_rules_update and alerts_rules_delete: PUT and DELETE /v1/alerts/rules/{id}
• alerts_rules_mute and alerts_rules_test: POST /v1/alerts/rules/{id}/mute and POST /v1/alerts/rules/{id}/test
• alerts_acknowledge and alerts_resolve: POST /v1/alerts/{id}/acknowledge and POST /v1/alerts/{id}/resolve

Destination changes, with credentials:write:

• alerts_destinations_create: POST /v1/alerts/destinations
• alerts_destinations_update and alerts_destinations_delete: PUT and DELETE /v1/alerts/destinations/{id}
• alerts_destinations_test: POST /v1/alerts/destinations/{id}/test

alerts_list takes status (open, triggered, acknowledged, resolved or all), rule_id and limit (up to 200). A PUT changes only the fields you send.

LimitsCopy Anchor Link