Stats: Overview
This Overview page summarizes data collection, processing, and delivery metrics across the DataStream deployment. Use it to monitor pipeline throughput, resource utilization, and performance at the Director, Cluster, or global level.
Accessing Stats
Navigate to
Filters
The page header provides two filter controls that affect every section on the page.
Time Range
Select a time period using the
Presets:
| Preset | Period |
|---|---|
| Today | Current calendar day |
| This week | Current calendar week |
| This month | Current calendar month |
| Last 3 month | Rolling 3-month window |
Date & Time Range: Set a custom start and end date with time selection. The maximum allowed range is 3 months.
Entity Selection
Filter metrics by scope using the entity dropdown.
| Option | Scope |
|---|---|
| All Directors | Aggregated metrics across all Directors and Clusters |
| Director name | Metrics for a single Director instance |
| Cluster name (Cluster) | Metrics for a specific Cluster |
Page Header
The header displays three summary values for the selected entity and time period:
| Metric | Description |
|---|---|
| Data Collected | Total raw data volume received from devices |
| Data Processed | Total data volume after pipeline processing |
| Data sent to targets | Total data volume delivered to target destinations |
Data Overview
Three cards display data flow metrics from collection through delivery.
Collected raw data
Displays a pie chart of collected data volume broken down by device type. The card header shows total data size and event count for the selected period.
Click
Processed Data
Displays a grouped bar chart showing the top 5 pipelines processing the most data. Each pipeline shows
Data sent to targets
Displays a gauge chart showing the distribution of processed data across target destinations, split between two segments labeled SIEM and Others. The card header shows total data size and event count.
Data Collection and Transformation
Two cards provide operational health and performance metrics.
Directors
The card is titled Directors (or Cluster when a Cluster entity is selected). It displays the status of Directors, Agent Devices, and Stream Devices with enabled and disabled counts for each. When components have connection errors, a warning indicator appears with a
When a single Director is selected, an inner panel labeled
| Metric | Description |
|---|---|
| CPU Utilization | Current CPU usage percentage, averaged across connected replicas |
| RAM Utilization | Current RAM usage percentage, averaged across connected replicas |
Resource Utilization is only available for individual Director selection, not for All Directors or Cluster views.
A Serverless or Orchestrated Director can run more than one replica, and every replica reports under the same Director. The panel combines them as follows:
| Director state | CPU and RAM shown |
|---|---|
| One connected replica | That replica's own CPU and RAM |
| Several connected replicas | The arithmetic mean of their CPU, and the mean of their RAM |
| No connected replica | 0 for both |
A replica counts as connected while its last heartbeat is less than 90 seconds old. An older replica is left out of the mean.
Three replicas at 10%, 20% and 90% CPU show 40%. When the figure looks healthy but a workload is struggling, check the CPU and memory of each replica's pod or container.
When a Cluster is selected, the inner panel is labeled
Pipeline Performance
Displays a line chart plotting latency, errors, and dropped logs over time for the selected period. The card header shows three summary values:
| Metric | Description |
|---|---|
| Latency | Average pipeline processing latency in milliseconds, with a status tag |
| Pipeline Errors | Total error events during the selected period |
| Dropped Logs | Total dropped log events during the selected period |
Every enabled route receives every payload and keeps only the records of its own devices. Earlier versions counted another route's records as received and then dropped on each route that did not carry them. On a Director with more than one route this inflated Dropped Logs and the route-level volume behind Processed Data, and the inflation grew with each route added. Both figures now count a record only on the route that carries its device. After the upgrade, expect Dropped Logs to fall, often to zero, and Processed Data to fall with it. The traffic has not changed: the earlier figures were overcounts.
A route that selects its devices with an if expression reading only device fields, such as device.type == "eventhubs", now counts the same way for records that carry a device id: a record that if rejects is outside this route's scope, so this route counts it as neither received nor dropped. After the upgrade, such a route shows only its own records as received, and its Dropped Logs and Processed Data fall, often to zero for drops. The Advanced Routes Log Stream figure falls with them. Again the traffic has not changed: the earlier figures were overcounts.
A record that any other if expression rejects, one that reads record content alone or beside device fields, is still counted as a drop on that route, because it was offered to the route. So is a record with no device id, whatever the if reads. See route configuration for which expressions count which way.
Latency status thresholds:
| Tag | Range |
|---|---|
| Normal | 0--20 ms |
| Higher | 20--35 ms |
| Critical | 35+ ms |