Microsoft Sentinel

Microsoft Azure SIEM

Synopsis

Creates a target that ingests log messages into Microsoft Sentinel workspace tables using Data Collection Rules (DCRs). Supports automatic table selection, field normalization, and filtering options.

tip

For more details on Microsoft Sentinel integration, refer to Microsoft Sentinel Overview and Microsoft Sentinel Integration. For Director Proxy deployment, see VirtualMetric Director Proxy. For cost-optimized ingestion with extended retention, see Microsoft Sentinel data lake.

Schema

- name: <string>
  description: <string>
  type: sentinel
  pipelines: <pipeline[]>
  status: <boolean>
  properties:
    tenant_id: <string>
    client_id: <string>
    client_secret: <string>
    function_app: <string>
    function_token: <string>
    rule_id: <string>
    endpoint: <string>
    streams:
      - name: <string>
        rule_id: <string>
    stream: <string[]>
    buffer_size: <numeric>
    batch_size: <numeric>
    keep_phantom_fields: <boolean>
    drop_unknown_stream_events: <boolean>
    cache:
      timeout: <numeric>
    field_format: <string>
    detection_rules:
      - name: <string>
        query: <string>
        language: <string>
    unmatched_tables: <enum>
    debug:
      status: <boolean>
      dont_send_logs: <boolean>

Configuration

The following fields are used to define the target:

Core Settings

Field	Required	Default	Description
name	Y		Target name
description	N	-	Optional description
type	Y		Must be sentinel
pipelines	N	-	Optional post-processor pipelines
status	N	true	Enable/disable the target

Authentication

Field	Required	Default	Description
tenant_id	N*	-	Azure tenant ID (required for direct authentication)
client_id	N*	-	Azure client ID (required for direct authentication)
client_secret	N*	-	Client secret (required for direct authentication)
function_app	N*	-	Director Proxy endpoint URL (required for proxy forwarding)
function_token	N*	-	Director Proxy authentication token (required with function_app)

* = Conditionally required. Use either direct authentication (tenant_id, client_id, client_secret) OR Director Proxy forwarding (function_app, function_token), OR omit all credentials to use Managed Identity.

Managed Identity Authentication

Azure targets support Managed Identity authentication for credential-free access when Director is deployed on Azure infrastructure.

How it works: When tenant_id, client_id, and client_secret are omitted from the configuration, the target automatically uses Azure's DefaultAzureCredential, which attempts authentication in the following order:

1. Environment variables (AZURE_TENANT_ID, AZURE_CLIENT_ID, AZURE_CLIENT_SECRET)
2. Workload Identity (for Kubernetes deployments)
3. Managed Identity (system-assigned or user-assigned)
4. Azure CLI credentials
5. Azure PowerShell credentials

Deployment requirement: Director must run on Azure infrastructure that supports Managed Identity:

• Azure Virtual Machines
• Azure App Service
• Azure Container Instances
• Azure Kubernetes Service (AKS)
• Azure Functions

Required permissions: The Managed Identity must be granted the appropriate Azure RBAC roles documented in each target's IAM Permissions section.

note

Managed Identity eliminates credential management overhead and is the recommended authentication method for Azure-hosted Director deployments.

Stream Configuration

Field	Required	Default	Description
endpoint	Y		DCE Address or DCE Resource ID
rule_id	N		Default Data Collection Rule (DCR) ID
streams	N	-	Detailed stream configurations
stream	N	-	Legacy string array of stream names
buffer_size	N	1048576	Encoding buffer size in bytes (1MB), applied per destination stream
batch_size	N	1000	Maximum messages per batch, counted per destination stream. Must be greater than 0
keep_phantom_fields	N	false	Keep fields not defined in DCR schema
drop_unknown_stream_events	N	true	Silently drop events for undefined streams
cache.timeout	N	300	Stream cache timeout in seconds
field_format	N	-	Data normalization format. See applicable Normalization section

Detection Rules

A detection rule set assigned to this target turns its smart pipeline on: the target compiles the set with the Smart Engine processor and ships only the events those detections could need, field- and value-reduced.

In the web interface these entries are written for you — assign rules from Content Management > Detections, or from this target's own Detection Rules step and tab, and the assignment is regenerated on every configuration update. See Detections.

Field	Required	Default	Description
detection_rules	N	-	Detection rules to reduce against. Omit, or leave empty, to ship everything the pipelines produce
detection_rules[].name	Y		Rule name, used to report which detections matched
detection_rules[].query	Y		The rule's full source, inline. Not a reference to a rule stored elsewhere
detection_rules[].language	N	kql	Rule language: kql or yaral
unmatched_tables	N	keep	What to do with an event no assigned detection is scoped to: keep or drop

An entry missing either name or query is skipped, and the skipped count is reported in the debug log. If every entry is skipped the smart pipeline does not run at all, which ships everything rather than nothing — reduction fails open throughout.

unmatched_tables accepts keep and drop case-insensitively. Any other value is treated as keep and reported as a warning, so a typo is visible rather than a silent no-op.

warning

drop discards every event that no assigned detection is scoped to. A detection scoped to a table only applies to events that declare that table, so events with no declared table are never matched by a table-scoped rule and are governed entirely by this setting. Vet the assigned rule set against your traffic before switching it.

Scheduling

warning

Unlike most target types, sentinel does not flush once per notification when left unconfigured. With neither interval nor cron set it flushes every 30 seconds.

See Scheduling and Pool Behavior for the interval, cron and max_items fields shared by all targets, and for how to override or opt out of this default.

Debug Options

Field	Required	Default	Description
debug.status	N	false	Enable debug logging
debug.dont_send_logs	N	false	Process logs but don't send to target (testing)

Automatic Table Selection

When streams is not specified, tables are automatically selected based on input type:

Input Type	Target Table
Windows Event Log	Custom-WindowsEvent
Windows Application Log	Custom-WindowsEvent
Windows System Log	Custom-WindowsEvent
Windows Security Log	Custom-SecurityEvent
Syslog	Custom-Syslog
Linux Audit Report	Custom-CommonSecurityLog
Windows Audit Report	Custom-CommonSecurityLog

Available Tables

Standard Tables (Prefix: Custom-)

WindowsEvent SecurityEvent CommonSecurityLog Syslog

ASim Tables (Prefix: Custom- or Microsoft-)

ASimAuditEventLogs ASimAuthenticationEventLogs ASimDhcpEventLogs ASimDnsActivityLogs ASimFileEventLogs ASimNetworkSessionLogs ASimProcessEventLogs ASimRegistryEventLogs ASimUserManagementActivityLogs ASimWebSessionLogs

Details

The Microsoft Sentinel target enables direct ingestion into Microsoft Sentinel tables with flexible configuration options. It supports using the SystemS3 field to route messages to specific stream tables, using the format Custom-TableName.

Deployment Models

The target supports two deployment models:

Direct Authentication - Director connects directly to Azure using service principal credentials (tenant_id, client_id, client_secret). This model requires Director to have network connectivity to Azure endpoints and credentials for the target subscription.

Director Proxy Forwarding - Director sends processed data to VirtualMetric Director Proxy (Azure Function) deployed in customer environment. Director Proxy uses Azure Managed Identity for credential-free access to Microsoft Sentinel, eliminating the need to share Azure credentials with Director.

The Director Proxy model is particularly valuable for MSSP deployments where customers maintain complete control over Azure credentials while enabling centralized data processing and routing by the MSSP's Director infrastructure.

IAM Permissions

When using Service Principal or Managed Identity authentication, the following Azure roles are required for runtime data ingestion:

Azure Role	Scope	Purpose
Monitoring Metrics Publisher	Data Collection Rule	Send data via DCR ingestion API
Monitoring Reader	Resource Group	List DCEs and DCRs for stream auto-discovery

If the setup functions (DCE/DCR/table creation) are used, additional roles are required:

Azure Role	Scope	Purpose
Monitoring Contributor	Resource Group	Create and update DCEs and DCRs
Log Analytics Contributor	Log Analytics Workspace	Create and update custom log tables

The target automatically detects table schemas and can clean messages to remove phantom fields that aren't defined in the schema when keep_phantom_fields is set to false.

warning

Disabling keep_phantom_fields may result in data loss for undefined fields.

Data is buffered per destination stream until batch_size records or buffer_size bytes are reached for that stream, or the flush schedule fires. Whichever limit binds first triggers the upload; the record that would have exceeded it rides the next one rather than being dropped. Because both limits are per stream, a target fanning out to several streams accumulates up to that much for each of them before a flush. Each stream type also has its own ceiling in the Log Analytics ingestion API.

warning

Enabling drop_unknown_stream_events silently discards unmatched events.

Field Normalization

The field_format property allows normalizing log data to standard formats:

• csl - Common Security Log
• asim - Advanced Security Information Model

Field normalization is applied before the logs are sent to Sentinel, ensuring consistent indexing and search capabilities.

Preconfigured Schemas

The target includes built-in schema definitions for standard tables like:

• Syslog
• Common Security Log
• Security Event
• Windows Event
• ASim tables (various types)

These predefined schemas ensure proper column mapping and validation when sending data to Sentinel.

warning

Large buffer sizes or batch sizes increase memory usage, and both are allocated per destination stream — a target writing to many streams multiplies the cost.

Autodiscovery

Director provides an autodiscovery feature that automatically configures Data Collection Rules and their associated streams.

The required permissions are:

• For Data Collection Rules

  Role	Scope
  Monitoring Metrics Publisher	Each DCR with name starting with vmetric

• For Resource Groups

  Role	Scope
  Monitoring Reader	Resource Group containing your DCE

warning

Always assign the Monitoring Reader role at the Resource Group level, not at the Subscription level.

The feature uses Resource IDs to discover DCRs and their configurations. It then automatically detects table schemas and validates fields, and prevents phantom fields through schema validation.

Autodiscovery adapts to environment changes automatically.

Examples

Basic

Configuration using Resource ID-based autodiscovery:

targets:
  - name: auto_sentinel
    type: sentinel
    properties:
      tenant_id: "00000000-0000-0000-0000-000000000000"
      client_id: "00000000-0000-0000-0000-000000000000"
      client_secret: "your-client-secret"
      endpoint: "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/myResourceGroup/providers/Microsoft.Insights/dataCollectionEndpoints/myDCE"

Managed Identity

Using Managed Identity Authentication instead of App Registration:

targets:
  - name: managed_identity_sentinel
    type: sentinel
    properties:
      endpoint: "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/myResourceGroup/providers/Microsoft.Insights/dataCollectionEndpoints/myDCE"

Director Proxy

Configuration using Director Proxy for credential-free forwarding:

targets:
  - name: proxy_sentinel
    type: sentinel
    properties:
      function_app: "https://my-director-proxy.azurewebsites.net/api/Sentinel"
      function_token: "your-proxy-authentication-token"
      endpoint: "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/myResourceGroup/providers/Microsoft.Insights/dataCollectionEndpoints/myDCE"

Filtered

Using specific stream filtering and custom cache timeout:

targets:
  - name: filtered_autodiscovery
    type: sentinel
    properties:
      tenant_id: "00000000-0000-0000-0000-000000000000"
      client_id: "00000000-0000-0000-0000-000000000000"
      client_secret: "your-client-secret"
      endpoint: "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/myResourceGroup/providers/Microsoft.Insights/dataCollectionEndpoints/myDCE"
      streams:
        - name: "Custom-WindowsEvent"
        - name: "Custom-SecurityEvent"
      cache:
        timeout: 300  # 5 minutes
      keep_phantom_fields: false
      drop_unknown_stream_events: true

High-Volume

Optimization for high-volume ingestion:

targets:
  - name: optimized_sentinel
    type: sentinel
    pipelines:
      - normalization
    properties:
      tenant_id: "00000000-0000-0000-0000-000000000000"
      client_id: "00000000-0000-0000-0000-000000000000"
      client_secret: "your-client-secret"
      endpoint: "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/myResourceGroup/providers/Microsoft.Insights/dataCollectionEndpoints/myDCE"
      buffer_size: 5242880  # 5MB
      batch_size: 5000
      field_format: "asim"
      streams:
        - name: "Custom-ASimProcessEventLogs"
        - name: "Custom-ASimNetworkSessionLogs"

With Debugging

Configuration with debug options:

targets:
  - name: debug_sentinel
    type: sentinel
    properties:
      tenant_id: "00000000-0000-0000-0000-000000000000"
      client_id: "00000000-0000-0000-0000-000000000000"
      client_secret: "your-client-secret"
      endpoint: "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/myResourceGroup/providers/Microsoft.Insights/dataCollectionEndpoints/myDCE"
      debug:
        status: true
        dont_send_logs: true  # Test mode that doesn't actually upload