Skip to main content

Alerts

Alerts are emitted by the monitoring engine when a Rule trigger condition is met for a monitored resource. The engine evaluates every active rule once per minute and opens at most one alert per (rule, resource) pair — repeated triggers update the existing alert rather than creating a new one.

Accessing Alerts​

Navigate to Monitoring > Alerts for the full list. The Overview page summarizes alert activity with summary widgets and the most recent activity.

Alerts List​

The alerts table presents every alert across the tenant.

Columns:

ColumnDescription
Alert NameAlert identifier combining rule type and affected resource (for example, Disk utilization for web-01)
CreatedTimestamp the alert first opened
Last seenWhen the rule last found the alert's condition, and how many times it has found it since the alert opened, for example "30/10/2026 10:31 · 31 times". For a count of 1 it reads "· once". Acknowledging or resolving the alert does not change it
SeverityCritical / High / Medium / Low — current level after any overrides
Alert Rule NameRule that produced the alert
Resource GroupDirectors / Devices / Targets
StatusOpen / Acknowledged / Resolved

Controls:

  • Search by alert, rule or resource name — free-text search, ignoring case, across the alert name (such as "Memory utilization for db-01"), the alert type as the list shows it, the rule name and the resource name. An alert raised on one log source is also found by the log source's name or its device's. %, _ and \ are matched as typed, not as wildcards. The same search runs on Monitoring > Overview
  • Severity filter — All, Critical, High, Medium, Low
  • Status filter — All, Open, Acknowledged, Resolved
  • Alert Type filter — All, Director, Device, Target
  • Row checkboxes — select alerts to Acknowledge or Resolve them together (see Acting on several alerts)
  • Pagination at the foot of the table

Severity and Status​

Severity levels rank the urgency of an alert. A multi-severity rule sets the level from its configured thresholds; a single-severity rule applies the same level to every trigger.

SeverityBadge
Criticalred
Highred (lighter)
Mediumorange
Lowyellow

Status values describe where the alert sits in its lifecycle.

StatusDescription
OpenAlert was opened by a trigger and has not been acknowledged or resolved
AcknowledgedAn operator has taken ownership; the alert is still active but is being worked
ResolvedThe alert is closed, either manually or automatically by the engine

Alert Detail​

Clicking an alert's name opens its details in a side panel, which also shows the latest value and the threshold for a threshold rule. See alert details in that panel opens the alert detail page. The detail page's main panel surfaces:

  • Current status badge and severity badge
  • An alert summary line naming the rule type and resource (for example, "Disk utilization alert for web-01"), followed by a state-dependent description
  • Rule Name and Rule Type. An event storm rule's type reads Event storm · Anomaly detection, and a volume rule that uses anomaly detection carries the same · Anomaly detection marker.
  • A See alert rule details link that opens the rule's detail page
  • For a rule that compares a value with a threshold: Value when triggered, the value that raised the alert over the rule's time period (for example, 97.2% over 5 minutes), and Threshold (for example, Above 95%). For the rule types that record it, the value also says how it was worked out: Averaged 97.2% over 5 minutes (CPU, memory and disk), Peaked at 1.2 GB in 5 minutes (a queue's size, a lag, and the timing rules), or Stayed at or above 82.3% at every reading for 5 minutes (a queue's percentage: every reading in the time period reached the value). Other threshold rules, such as the volume rules, keep the plain wording. When the alert records where the value was measured, Measured on names the Director host, and the disk or queue, for example dir-02 · /data or dir-01 · sending queue (persistent). When the latest value came from another place, a second Measured on row follows it. Older alerts, raised before the platform recorded how and where a value was measured, show the value without this wording and no Measured on row. While the alert is open or acknowledged, Latest value above the threshold (or Latest value below the threshold) gives the most recent value that still crossed the threshold. If the severity has been raised since, a second Threshold beside it gives the threshold that value was compared with.
  • Acknowledged By and Acknowledgement Note (shown once the alert is acknowledged)
  • Resolved By and Resolution Note (shown once the alert is resolved)

A measured value is rounded to one decimal place. It keeps more decimals, up to six, when one would put it level with its threshold or on the other side of it: 95.04 against Above 95% reads 95.04%, not 95%. A data volume is written in the largest unit that keeps it at one or more, such as 312.3 MB, while the threshold keeps the unit the rule was written in. A Director's queue usage rule in Size mode is the exception: its value and its threshold are both written in the largest unit, for example Peaked at 600 MB against Above 500 MB. Rules whose own sentence already gives their readings, such as baseline, anomaly detection and event storm rules, show none of these rows, and neither do status rules, which measure no value.

When an API token acknowledged or resolved the alert, Acknowledged By or Resolved By shows API Token: <name>, as the audit log does: a token acts as no user.

When a rule action resolved the alert because its ticket was closed, Resolved By shows Rule action: <name> and the resolution note names the ticket, for example Closed in Jira: OPS-123. See Rule Actions.

A side info panel lists the Created and Last seen timestamps (Last seen with its count, as in the list), plus Acknowledged and Resolved timestamps when those states have been reached.

Below the panel, a timeline section records every state change — the original trigger, severity upgrades, acknowledgements, and the resolution. Its columns are Event Time, Event, Severity, and Trigger Condition. For a threshold rule, the Trigger Condition ends with the value the rule measured, for example The measured value was 97.2%. When the alert says where the value was measured, the sentence names it, and it says when the value is an average or the highest reading: The average value measured on dir-02 · /data was 91.4%.

Acknowledging and Resolving​

Both actions are available from the alerts list row menu, from the Actions menu on the alert detail page, and for several alerts at once from the list's batch bar (see Acting on several alerts). The menu items are state-conditional: Acknowledge alert appears while the alert is Open, Remove acknowledgement while it is Acknowledged, and Resolve alert as long as it is not Resolved. These actions are gated by the alert-edit permission; opening the detail page itself requires the alert-read permission.

Both actions are also available from the REST API, from PowerShell and from Cloud Shell, so a script or an automation can acknowledge and resolve alerts too.

Acknowledge​

Acknowledge alert moves the alert from Open to Acknowledged and records the operator and timestamp. The acknowledgement note is optional. Remove acknowledgement returns the alert to Open.

Resolve​

Resolve alert moves the alert to Resolved and closes the lifecycle. The resolution note is required, captured in the timeline alongside the operator and timestamp. While an alert is Open or Acknowledged, its rule raises no new alert for the same resource. The existing alert counts each repeat instead. If your team tracks alerts somewhere else, such as in Jira, resolve them here too.

Resolution is final — a resolved alert cannot be reopened, but a new trigger against the same (rule, resource) pair will open a fresh alert.

Acting on several alerts​

The alerts list has a checkbox on every row. Selecting one or more alerts replaces the table toolbar with a batch bar, so a backlog can be acknowledged or resolved in one step instead of one alert at a time.

  1. Tick the alerts on the list. The selection is kept across pages, up to 500 alerts per action.
  2. The batch bar shows N alerts selected with Acknowledge, Resolve, and Cancel.
  3. Acknowledge opens Acknowledge N alerts. The note is optional, as it is for a single alert.
  4. Resolve opens Resolve N alerts. The note is required, as it is for a single alert.
  5. The note is entered once and saved on every selected alert. The result is reported in one notification:
    • N alerts acknowledged (or resolved) when every selected alert moved
    • X of Y alerts acknowledged (or resolved) when some were skipped, followed by how many had already changed state

Permissions. The checkboxes appear only for users with the alert-edit permission, the same permission as the single-alert actions. Without it the list has no checkboxes.

Resolved alerts cannot be selected. Their checkbox is shown but disabled.

Alerts the action no longer applies to are skipped, not failed. If another operator moved one of the selected alerts first, to a state the chosen action does not apply to, the action leaves that alert as it is and counts it in the notification; the rest of the selection is still actioned. Bulk Acknowledge skips an alert that is already Acknowledged, and bulk Resolve skips one that is already Resolved. An alert another operator acknowledged is still resolved by bulk Resolve.

Every actioned alert gets its own timeline entry and audit log entry, exactly as if it had been acted on alone.

There is no bulk Remove acknowledgement and no bulk Stop monitoring log source. Those actions stay per alert.

Alert Lifecycle​

Evaluation tick​

The monitoring engine re-evaluates every active rule once a minute. Threshold and metric checks run against the most recent observation window. Status and no-data checks measure how long a resource has been silent — see below. A gradual decline rule judges whole days in UTC, so its comparison changes only once a day, after midnight UTC, when the day that has just ended is read — see Gradual decline.

Status and no-data rules​

Director status, Device status, Target status and No data received rules fire once a resource has been silent for longer than the rule's Time period. For a status rule, silent means not connected; for No data received, it means no data has arrived. The silence is counted from the latest of three moments:

  • the resource's last signal — the last time it was seen connected, or the last data it sent
  • when the resource was added
  • when the resource first connected

A resource therefore alerts only after a full time period has passed without a signal. A Target that drops for a single evaluation does not alert straight away; it alerts once it has stayed disconnected for longer than the period.

A resource that has never connected is not evaluated by these rules at all. A Director that has been created but not yet installed, or a Device or Target that has never been reported connected, raises no status or no-data alert, however long it stays that way. Its monitoring starts the first time it connects.

This applies to custom rules of these four types as well as to the built-in ones.

note

The first connection is recorded by the monitoring engine when it sees the resource connected. Recording started with the platform release that introduced it, and earlier connections were not backfilled. A resource that was offline when the platform was upgraded counts as never connected until it next connects.

Severity override​

When a multi-severity rule fires at a higher level against a resource that already has an Open alert, the existing alert's severity is upgraded in place — the timeline gains an entry recording the change, but no new alert is created. Triggers at the same or a lower severity update the last-triggered timestamp and increment the occurrence count without changing the severity.

Auto-resolve​

Auto-resolve is time-based, for the rule types that accept a Resolve after period. The second way is Resolve the alert when the resource is back to normal: a rule that uses it closes its alert once the resource has been back to normal for 10 minutes in a row — see Resolve condition. Four other kinds of rule also close their alerts when the condition clears, all with Resolve alerts automatically on: a volume rule that uses Anomaly detection closes its alert once the resource has been back within its normal for 10 minutes (or one time period, if longer), an event storm rule closes its alert once traffic has been back to normal for 10 minutes (or one time period, if longer), a gradual decline rule closes its alert once the resource's last 3 days are lower by less than half the rule's percentage, and a forecast rule (Queue filling up, Disk filling up, Delivery falling behind) closes its alert once nothing has been forecast to reach its limit within the warning for 10 minutes — see Rules. When a Resolve after period elapses since the alert first triggered, the alert closes on its own.

Rule typeAuto-resolve
Backpressure, Director status, Device status, Target status, No data received, Queue usage, Delivery lagCloses automatically once the configured Resolve after period has elapsed, or once the resource has been back to normal for 10 minutes when the rule uses Resolve the alert when the resource is back to normal
Crash detection, Bus unhealthy, Bus restart, Consumer lag, JetStream storage utilizationCloses automatically once the configured Resolve after period has elapsed
Processor / memory / disk utilizationCloses on its own once the resource has been back to normal for 10 minutes, when the rule uses Resolve the alert when the resource is back to normal. Otherwise it never closes on its own: resolve it by hand once the condition has passed
High and low data volume, High and low event volume, Total ingest amountNever closes on its own, unless it is a volume rule that uses Anomaly detection with Resolve alerts automatically on. Resolve it by hand once the condition has passed
Event stormCloses on its own once traffic has been back to normal for 10 minutes (or one time period, if longer), when the rule's Resolve alerts automatically is on. With it off, resolve it by hand
Gradual declineCloses on its own once the last 3 days are lower by less than half the rule's percentage (under 15% lower with Balanced), when the rule's Resolve alerts automatically is on. With it off, resolve it by hand
Queue filling up, Disk filling up, Delivery falling behindCloses on its own once nothing has been forecast to reach its limit within the warning for 10 minutes, when the rule's Resolve alerts automatically is on. With it off, the alert stays Open until someone resolves it

Status and no-data alerts close when the resource reconnects or its data resumes only when their rule uses Resolve the alert when the resource is back to normal, and then once the resource has been back to normal for 10 minutes in a row. With a Resolve after period, they close once that period has passed, whether or not the condition has cleared. With Do not resolve the alert, they stay Open until someone resolves them. The built-in status and no-data rules use Resolve the alert when the resource is back to normal, so their alerts close once the resource is back to normal. With that option, missing data never closes an alert, and an idle target's alert stays open — see Resolve condition.

An alert of any rule type can also be resolved through the rule action that opened a Jira issue or a Freshservice or Freshdesk ticket for it. With Resolve alerts when the issue is closed on a Jira action, or Resolve alerts when the ticket is resolved or closed on a Freshservice or Freshdesk action, VirtualMetric resolves an Open or Acknowledged alert once the newest issue the action opened or updated for it is closed, or once the newest ticket is resolved or closed. Someone still closes the issue, or resolves or closes the ticket; the alert follows. See Rule Actions.

Auto-resolved alerts carry the system-generated resolution note This alert was automatically resolved by the system after the defined resolution period expired. and are flagged in the timeline. An alert that a rule with Resolve the alert when the resource is back to normal, a volume rule using Anomaly detection, an event storm rule, a gradual decline rule or a forecast rule closed because its condition cleared carries This alert was automatically resolved by the system because the triggering condition is no longer met. instead, and its timeline row reads Alert has been auto-resolved as the condition cleared.

warning

The utilization rules close when the triggering condition stops being met only with Resolve the alert when the resource is back to normal. The volume rules close then only when they use Anomaly detection with Resolve alerts automatically on. Otherwise, a CPU spike that has long since passed leaves its alert Open until someone resolves it, so a busy list of these alerts is a backlog of past events, not a picture of the present.

Multi-resource fan-out​

A rule scoped to All directors, All devices, or All targets — with or without exceptions — fans out to every resource it matches. If five Directors breach the rule simultaneously, five independent alerts open — one per resource — and each follows its own acknowledge / resolve lifecycle.