Skip to main content

Microsoft Fabric Eventhouse

Microsoft Azure Observability

Synopsis​

Creates a Microsoft Fabric Eventhouse target that writes records into a KQL database of an eventhouse.

An eventhouse takes data in the same way as Azure Data Explorer, so this target is the Azure Data Explorer target under a second name. Every Azure Data Explorer setting means the same here. This page describes what reads differently for an eventhouse, and links to the Azure Data Explorer page for the rest.

Schema​

- name: <string>
  description: <string>
  type: fabriceventhouse
  pipelines: <pipeline[]>
  status: <boolean>
  properties:
    tenant_id: <string>
    client_id: <string>
    client_secret: <string>
    function_app: <string>
    function_token: <string>
    endpoint: <string>
    database: <string>
    table: <string>
    schema: <string>
    format: <string>
    compression: <string>
    flush_immediately: <boolean>
    timeout: <numeric>
    batch_size: <numeric>
    max_size: <numeric>
    field_format: <string>
    tables:
      - table: <string>
        schema: <string>
        format: <string>
        compression: <string>
    debug:
      status: <boolean>
      dont_send_logs: <boolean>

Configuration​

The following fields are used to define the target:

FieldRequiredDefaultDescription
nameY-Target name
descriptionN-Optional description
typeY-Must be fabriceventhouse
pipelinesN-Optional post-processor pipelines
statusNtrueEnable/disable the target

Eventhouse​

FieldRequiredDefaultDescription
endpointY-The eventhouse's Query URI, such as https://trd-1a2b3c4d.z0.kusto.fabric.microsoft.com, or its Ingestion URI. Fabric shows both in the eventhouse's details, and either works: the target adds or removes the ingest- part itself. It must start with https://. In the console: Query URI
databaseY-The KQL database. In the console: KQL database
tableY*-The table that every record goes to. Set table, or at least one entry in tables: the director refuses a configuration with neither. In the console: Table
tenant_id, client_id, client_secretN-The service principal to sign in as. Each takes plain text, an environment variable or a secret. When one of the three is missing, the target signs in through Azure's default credential chain, which on an Azure host is usually the managed identity of the director (see Managed Identity Authentication). This applies to a direct connection only: when the target sends through a director proxy (function_app), the proxy's managed identity writes to the eventhouse and these fields are ignored. In the console: Tenant ID, Client ID and Client secret, shown while Managed identity is off

* = Set table, or at least one entry in tables

Other Settings​

Every other setting works as on the Azure Data Explorer page:

FieldRequiredDefaultDescription
tablesN-Up to ten tables. Each entry requires table, and may set its own schema, format and compression: an entry without format or compression uses the root one. An entry whose format is avro or parquet requires a schema. A catch-all table counts toward the ten, so with one set, tables takes at most nine entries. See Table Routing and Catch-All Behavior
schemaN-The schema of table. Required with avro and parquet
formatNjsonjson, multijson, avro or parquet. See Formats. In the console: Format
compressionNzstdSee Compression. In the console: Compression
flush_immediatelyNtrueAsk the eventhouse to ingest each batch at once, without waiting to gather more
timeoutN30The seconds that one call to the eventhouse may take
batch_sizeN100000Maximum number of messages per batch
max_sizeN32MBMaximum batch size in bytes
field_formatN-Data normalization format. See applicable Normalization section
function_app, function_tokenN-The director proxy to send through. See Errors when using Director Proxy and Proxy
debug.status, debug.dont_send_logsNfalseDebug logging, and processing records without sending them

In the console, the form is the Azure Data Explorer form in Fabric's words. Its steps are:

  • Azure settings — Managed identity, Tenant ID, Client ID, Client secret, Director proxy and the connection settings
  • Eventhouse settings — Query URI, KQL database, Table, Format, Compression and Schema

Managed Identity Authentication​

Azure targets support Managed Identity authentication for credential-free access when Director is deployed on Azure infrastructure.

How it works: When tenant_id, client_id, and client_secret are omitted from the configuration, the target automatically uses Azure's DefaultAzureCredential, which attempts authentication in the following order:

  1. Environment variables (AZURE_TENANT_ID, AZURE_CLIENT_ID, AZURE_CLIENT_SECRET)
  2. Workload Identity (for Kubernetes deployments)
  3. Managed Identity (system-assigned or user-assigned)
  4. Azure CLI credentials
  5. Azure PowerShell credentials

Deployment requirement: Director must run on Azure infrastructure that supports Managed Identity:

  • Azure Virtual Machines
  • Azure App Service
  • Azure Container Instances
  • Azure Kubernetes Service (AKS)
  • Azure Functions

Required permissions: The Managed Identity must be granted the appropriate Azure RBAC roles documented in each target's IAM Permissions section.

note

Managed Identity eliminates credential management overhead and is the recommended authentication method for Azure-hosted Director deployments.

Scheduling​

See Scheduling and Pool Behavior for interval and cron fields shared by all targets.

Details​

Where to Find the Values in Fabric​

  • The eventhouse's details show the Query URI and the Ingestion URI.
  • The KQL databases are listed under the eventhouse.

What the Identity Needs​

  • The ingestor role on the KQL database. For a service principal, an administrator of the database runs a command such as:

    .add database <db> ingestors ('aadapp=<app id>;<tenant id>')

  • The Contributor role on the Fabric workspace also gives the ingestor role, but it gives admin rights on every item in the workspace as well. For ingestion only, grant the ingestor role.

  • When the target signs in as the managed identity of the director, give the role to that identity. When it sends through a director proxy, give the role to the proxy's managed identity instead.

  • The target always names its table, so it never lists the tables of the database, and it needs no viewer role.

How the Target Writes​

  • The target tries streaming ingestion first. When streaming is not available, it queues the batch for ingestion instead. An eventhouse has streaming ingestion on by default.

Examples​

Service Principal​

A service principal, with JSON records going into one table:

targets:
  - name: fabric_eventhouse
    type: fabriceventhouse
    properties:
      endpoint: "https://trd-1a2b3c4d.z0.kusto.fabric.microsoft.com"
      database: "Telemetry"
      table: "SensorReadings"
      tenant_id: "00000000-0000-0000-0000-000000000000"
      client_id: "00000000-0000-0000-0000-000000000000"
      client_secret: "$secret{id=1001}"
      format: json

Managed Identity​

The managed identity of the director:

targets:
  - name: fabric_eventhouse_mi
    type: fabriceventhouse
    properties:
      endpoint: "https://trd-1a2b3c4d.z0.kusto.fabric.microsoft.com"
      database: "Telemetry"
      table: "SensorReadings"

The Azure Data Explorer page's examples for several tables, Parquet and normalized records hold here too, with type: fabriceventhouse.

Troubleshooting​

Configuration the Target Refuses​

Cause: the director refuses a configuration that it cannot use, with the Azure Data Explorer target's messages. One of them names the eventhouse:

  • endpoint is required; invalid endpoint URL for Microsoft Fabric Eventhouse (the Query URI must start with https://); database is required
  • at least one table must be configured (either 'table' or 'tables')
  • invalid table configuration: schema is required for parquet format, and the same for avro
  • maximum of 10 tables allowed

Fix: correct the setting the message names.

Sign-in and Permission Errors​

Sign-in and permission errors are the Azure Data Explorer target's: see Which permission is missing? and "Error while getting token" with an AADSTS code on the Azure Data Explorer page. A missing ingestor role is the usual cause.