Proofpoint On Demand

Pull

Synopsis

Director polls the source system's API at configured intervals to retrieve log and telemetry data rather than receiving pushed events. The device establishes a persistent WebSocket connection to Proofpoint's On Demand (POD) log stream service and receives email security event data. Supports both message and maillog data types with token authentication.

Schema

- id: <numeric>
  name: <string>
  description: <string>
  type: proofpoint
  tags: <string[]>
  pipelines: <pipeline[]>
  status: <boolean>
  properties:
    endpoint: <string>
    cluster: <string>
    token: <string>
    type: <string>
    workers: <numeric>
    reuse: <boolean>
    tls:
      ca_name: <string>
      cert_name: <string>
      key_name: <string>
      passphrase: <string>
      server_name: <string>
      min_tls_version: <string>
      max_tls_version: <string>
      insecure_skip_verify: <boolean>

Configuration

The following fields are used to define the device:

Device

Field	Required	Default	Description
id	Y	-	Unique numeric identifier
name	Y	-	Device name
description	N	-	Optional description
type	Y	-	Must be proofpoint
tags	N	-	Array of labels for categorization
pipelines	N	-	Array of preprocessing pipeline references
status	N	true	Enable/disable the device

Connection

Field	Required	Default	Description
endpoint	N	"wss://logstream.proofpoint.com:443/v1/stream"	Proofpoint WebSocket endpoint URL
cluster	Y	-	Proofpoint cluster identifier
token	Y	-	Authentication token for Proofpoint API (resolvable via ${ENV_VAR} or $secret{...})
type	N	"message"	Data type to consume (message or maillog)

Performance

Field	Required	Default	Description
workers	N	1	Number of worker processes
reuse	N	true	Enable multi-worker mode

Details

TLS

TLS is always on and cannot be turned off — the endpoint is a wss:// URL, and a plaintext ws:// or schemeless endpoint is rejected outright, since the bearer token travels in the Authorization header. There is no tls.status here; the block below only shapes a connection that is already encrypted.

Proofpoint's public endpoints present a certificate from a public CA, so the defaults need no configuration. The fields exist for a deployment behind a TLS-inspecting proxy, or one that requires a client certificate.

Field	Required	Default	Description
tls.ca_name	N	-	CA bundle used to verify the server certificate. When unset, the host trust store is used; when set, it replaces the host trust store rather than adding to it
tls.cert_name	N*	-	Client certificate for mutual TLS
tls.key_name	N*	-	Client private key for mutual TLS
tls.passphrase	N	-	Passphrase for an encrypted private key
tls.server_name	N	-	SNI hostname override for the TLS handshake
tls.min_tls_version	N	tls1.2	Minimum accepted TLS version (tls1.0, tls1.1, tls1.2, tls1.3)
tls.max_tls_version	N	-	Maximum accepted TLS version. When unset, the highest mutually supported version is negotiated
tls.insecure_skip_verify	N	false	Skip server certificate verification. Use only for testing

* = Mutual TLS requires both tls.cert_name and tls.key_name.

note

TLS material fields (cert_name, key_name, ca_name, client_ca_name) accept any of the following:

• File name — resolved relative to the service root directory. Nested paths such as certs/prod/server.pem are supported.
• Absolute path — honored only if it resolves inside the service root. Any path that escapes the root is refused.
• Inline PEM content — used verbatim when the value contains -----BEGIN.
• Environment variable — ${ENV_VAR}.
• Vault reference — $secret{id=...} or $secret{store=...,ref=...}.

WebSocket Connection

The device establishes a persistent WebSocket connection to Proofpoint's On Demand log stream service. The connection URL includes query parameters for cluster ID and data type. Bearer token authentication is used in the Authorization header.

Data Types

Proofpoint supports two log data types:

• message: Email processing logs including connection metadata, envelope details, message headers, and filter module results
• maillog: Mail transfer agent logs including SMTP transactions and delivery status

Token Security

Use ${ENV_VAR} or $secret{...} token resolution to keep the token out of the configuration file entirely. The resolved value is used as the bearer token at runtime.

Connection Management

The device handles WebSocket connection lifecycle including automatic reconnection on failure. Query parameters are encoded in the connection URL including cluster ID (cid) and data type (type).

Examples

Basic Configuration

Creating a basic Proofpoint On Demand consumer for email processing logs...	- id: 1 name: proofpoint-pod type: proofpoint properties: cluster: "your-cluster-id" token: "your-api-token" type: "message"
Device receives Proofpoint email events in real-time...	{ "guid": "abc123-def456", "ts": "2025-01-02T10:30:00.000000", "connection": { "ip": "192.0.2.10", "host": "mail.example.com" }, "envelope": { "from": "sender@example.com", "rcpts": ["user@company.com"] }, "msg": { "header": { "subject": "Quarterly Report", "from": ["sender@example.com"] } }, "filter": { "modules": { "spam": { "verdict": "notspam" } } } }

Maillog Collection

Collecting mail routing and delivery logs...	- id: 3 name: proofpoint-maillog type: proofpoint properties: cluster: "your-cluster-id" token: "your-api-token" type: "maillog"
Device receives mail transfer agent log events including delivery status...	{ "ts": "2025-01-02T10:30:00.000000", "data": "smtp;250 2.0.0 OK", "sm": { "from": "sender@example.com", "to": ["dest@company.com"] }, "pps": { "cid": "cluster01", "agent": "m0042.ppops.net" }, "id": "msg-20250102-001" }

High-Volume Processing

Multi-worker mode for high message rates...

- id: 4 name: proofpoint-highvolume type: proofpoint properties: cluster: "your-cluster-id" token: "your-api-token" type: "message" reuse: true workers: 4

Custom Endpoint

Connecting to a regional or custom endpoint...

- id: 5 name: proofpoint-custom type: proofpoint properties: endpoint: "wss://logstream-eu.proofpoint.com:443/v1/stream" cluster: "eu-cluster-id" token: "your-api-token" type: "message"

Pipeline Processing

Applying custom processing to email security events...

- id: 6 name: proofpoint-pipeline type: proofpoint pipelines: - email_enrichment - geo_lookup properties: cluster: "your-cluster-id" token: "your-api-token" type: "message"

note

Pipelines are processed sequentially and can modify or drop events before ingestion.