Datadog

SIEM Platform

Synopsis

The Datadog target sends your data to Datadog over HTTPS. It sends logs to the Datadog Cloud SIEM logs intake API. It also sends the metrics, traces and APM stats of Datadog Agents back to Datadog, in Datadog's own formats. The target handles batching and delivery. A log must already be a complete JSON document in Datadog's log schema when it reaches the target. For the records of a Datadog Agent device, the target rebuilds what the Agent sent, so you do not format them.

Schema

- name: <string>
  description: <string>
  type: datadog
  pipelines: <pipeline[]>
  status: <boolean>
  properties:
    api_key: <string>
    site: <string>
    url: <string>
    metrics_url: <string>
    traces_url: <string>
    batch_size: <integer>
    timeout: <integer>
    field_format: <string>
    debug:
      status: <boolean>
      dont_send_logs: <boolean>

Configuration

The following fields are used to define the target:

Field	Required	Default	Description
`name`	Y		Target name
`description`	N	-	Optional description
`type`	Y		Must be `datadog`
`pipelines`	N	-	Optional post-processor pipelines
`status`	N	`true`	Enable/disable the target

Connection

Field	Required	Default	Description
`api_key`	Y*	-	Datadog API key, sent as the `DD-API-KEY` header. Takes an environment variable (`$env{DATADOG_API_KEY}`) or a vault secret (`$secret{id=...}`) in place of the value. The target also sends it with the series, sketches, traces and APM stats of Datadog Agents
`site`	N	`datadoghq.com`	Datadog site: `datadoghq.com` (US1), `us3.datadoghq.com`, `us5.datadoghq.com`, `datadoghq.eu`, `ap1.datadoghq.com`, `ddog-gov.com`. It also decides where Agent data goes: `api.{site}` for series and sketches, and `trace.agent.{site}` for traces and APM stats
`url`	N*	built from `site`	Explicit logs intake URL. Overrides `site` for logs only. Default: `https://http-intake.logs.{site}/api/v2/logs`. Takes a `$env{...}` or `$secret{...}` reference as `api_key` does. It does not change where Agent data goes. Use `metrics_url` and `traces_url` for that
`metrics_url`	N	built from `site`	The address that the series and sketches of Datadog Agents go to. It replaces `https://api.{site}`, so use it for a proxy in front of Datadog. The target adds `/api/v2/series` and `/api/beta/sketches` to the end of it. It must be an http or https URL with a host, and it can hold no query and no fragment. It takes a `$env{...}` or `$secret{...}` reference as `api_key` does. YAML only
`traces_url`	N	built from `site`	The address that the traces and APM stats of Datadog Agents go to. It replaces `https://trace.agent.{site}`, so use it for a proxy in front of Datadog. The target adds `/api/v0.2/traces` and `/api/v0.2/stats` to the end of it. It follows the same rules as `metrics_url`. YAML only
`max_bytes`	N	`5242880` (5 MB)	Maximum size of a single request body or batch of logs, in bytes. It does not apply to Agent data, which follows Datadog's own request limits

note

max_bytes caps the size of a single request body or batch. It behaves differently from max_size on the file-writing targets, and the difference is easy to get wrong:

An explicit max_bytes: 0 IS honoured and removes the ceiling on targets that have no ceiling of their own. The value is read with Int64, which returns what you wrote — unlike max_size, which is read with MustInt64 and silently substitutes its default for a zero. Where a target has a fixed ceiling of its own, its page says so and a zero is clamped to that ceiling instead.
A negative value is rejected at configuration time with max_bytes must not be negative.

The ceiling is validated when the target is built and enforced again on every batch, so a value the receiving service will refuse fails early rather than per request.

* = Conditionally required/optional. Either api_key (endpoint built from site) or an explicit url is required. A url without an api_key passes validation, but no DD-API-KEY header is then sent and the intake API typically refuses the request, so set api_key as well unless the address you point at supplies the key itself. Without api_key, the target sends series and sketches only when metrics_url is set. It sends traces and APM stats only when traces_url is set. It drops the Agent data of any path that is off, and counts it as dropped.

Batch Configuration

Field	Required	Default	Description
`batch_size`	N	`1000`	Maximum events per POST, serialized as a JSON array. For Agent data, it is the most records of each kind (series, sketches, traces, APM stats) that the target holds before it sends them
`timeout`	N	`30`	HTTP client timeout, in seconds. It applies to the requests for Agent data too

Processing

Field	Required	Default	Description
`field_format`	N	-	Optional target-side normalization format applied before delivery. Typically left empty when the paired content pack is in the route, since the payload then already arrives in Datadog's JSON schema. Without the pack, this is the only normalization applied. See applicable Normalization section

Scheduling

See Scheduling and Pool Behavior for interval and cron fields shared by all targets.

Debug Options

Field	Required	Default	Description
`debug.status`	N	`false`	Enable debug logging
`debug.dont_send_logs`	N	`false`	Process logs but don't send to target (testing). It holds back Agent data too

Details

For logs, this target is transport-only: it does not render Datadog's log schema itself. It expects each event to already be a complete JSON document when it reaches the target, buffers events in memory, and delivers them as one HTTP POST — a JSON array ([{...},{...}]) — once batch_size is reached or during finalization. Datadog Agent data is the exception, because the target builds Datadog's own formats for it (see Datadog Agent Data).

Endpoint and Authentication

For logs, an explicit url always wins. Otherwise the endpoint is built from site as `https://http-intake.logs.{site}/api/v2/logs`. When api_key is set, it is sent as the DD-API-KEY request header; configuration validation rejects a target that has neither api_key nor url. Both api_key and url resolve an environment variable or a vault secret reference ($env{NAME}, or the ${NAME} form the examples on this page use, and $secret{id=...}) when the target starts, and a plain value is sent exactly as stored. The url setting does not change where Agent data goes (see Datadog Agent Data).

Datadog Agent Data

The datadog target also sends the records of a Datadog Agent device back to Datadog. The device receives what Datadog Agents send. The target sends the logs to the logs intake, as before. It sends the series, sketches, traces and APM stats in Datadog's own formats, with its own api_key.

Where each kind goes. The series and sketches go to the metrics API. The traces and APM stats go to the trace intake. Sketches have an address of their own, so a distribution keeps its whole sketch.

Kind	Address	Replaced by
Series	`https://api.{site}/api/v2/series`	`metrics_url`
Sketches	`https://api.{site}/api/beta/sketches`	`metrics_url`
Traces	`https://trace.agent.{site}/api/v0.2/traces`	`traces_url`
APM stats	`https://trace.agent.{site}/api/v0.2/stats`	`traces_url`

How the requests look. Each body is compressed with gzip. Series, sketches and traces go as protobuf. APM stats go as MessagePack. Each request carries the target's api_key in the DD-API-KEY header. This is useful to know for a proxy.

Where the address comes from. The target builds both addresses from site. metrics_url replaces the address for series and sketches, and traces_url replaces the address for traces and APM stats. Use them for a proxy in front of Datadog. The target removes a closing / from the address and adds the path of each kind to the end, so an address that has a path keeps that path. An http:// address sends the API key in clear text, so use https://.

When there is no api_key. Datadog refuses a request that has no key. So the target sends series and sketches only when metrics_url is set, and traces and APM stats only when traces_url is set. A proxy at that address can add the key. An api_key or a URL that resolves to an empty value counts as not set.

A path that is off. The target drops the records of a path that is off and counts them as dropped. It warns at most once a minute, and the warning describes one record. Nothing is retried. The warning is described under Troubleshooting.

Which records take this path. A record takes it only when it comes from a Datadog Agent device and is a series, a sketch, a trace or APM stats. Every other record goes to the logs intake as before. The logs of an Agent go back as the Agent sent them, as long as the target has no field_format.

Size limits. The target keeps each request within Datadog's limits:

A request of series or sketches holds at most 512,000 bytes after compression and 5,242,880 bytes before it. The target packs the records into as few requests as these limits allow. It splits a request that is too large in two, as often as it takes.
A request of traces holds at most 3,200,000 bytes before compression. This is the limit of the trace agent. The target splits a request that is too large in the same way.
Each APM stats payload goes as one request.
The target rejects a series, sketch or stats record of more than 5,242,880 bytes. Director treats it like an oversized log record (see Troubleshooting).
The target drops any other single record that no request can hold. It counts the record as dropped and reports it in a warning.
max_bytes does not apply to these requests. batch_size sets how many records of each kind the target holds before it sends them. timeout applies to every request.

What happens when a request fails. The target sends the logs and each kind of Agent data as separate requests. It tries every kind, even when an earlier kind fails. Director delivers again only what failed, and skips what already landed.

Datadog may refuse a request for good, with the status 400, 413 or 422. The target then drops the records of that request and counts them as dropped. It warns at most once a minute and goes on. Director does not deliver those records again. For logs, Director gives up after four deliveries instead.
Any other failure makes Director deliver the records again later. A wrong key, a rate limit, an outage and a network error are among the causes. The other kinds in the same delivery still go through.

Counts. Each record that Datadog accepts counts as an event out of the target. Each dropped record counts as dropped.

Batching and Retry

Events buffer until batch_size is reached, then POST as a single request. A failed POST does not retry the batch internally: the buffer is cleared unconditionally after Finalize, and delivery failure is surfaced to the sender pool, which handles redelivery through the JetStream layer. Retaining the batch locally in addition to that redelivery would duplicate events at the destination. Error responses include up to the first 1 MB of the response body for troubleshooting. Agent data follows the same rule, kind by kind (see Datadog Agent Data).

note

For logs, the target does not render Datadog's JSON schema. It only batches and delivers the payload it receives. Format the log as Datadog-compatible JSON before it reaches this target. The records of a Datadog Agent device are the exception, because the target builds Datadog's formats for them.

Examples

Basic Configuration

Sending JSON events to Datadog Cloud SIEM logs intake on the default US1 site. The same target also sends the metrics, traces and APM stats of Datadog Agents to the US1 site. It needs nothing more.

targets:
  - name: datadog
    type: datadog
    properties:
      api_key: "${DATADOG_API_KEY}"

EU Site

Routing to the Datadog EU site instead of the US1 default. The site also decides where Agent data goes: api.datadoghq.eu and trace.agent.datadoghq.eu.

targets:
  - name: datadog-eu
    type: datadog
    properties:
      api_key: "${DATADOG_API_KEY}"
      site: datadoghq.eu

Explicit Endpoint Override

Overriding the intake URL directly instead of deriving it from site. This changes the logs address only. Agent data still goes to the addresses for site, unless you set metrics_url and traces_url.

targets:
  - name: datadog-custom
    type: datadog
    properties:
      url: "https://http-intake.logs.datadoghq.com/api/v2/logs"
      api_key: "${DATADOG_API_KEY}"
      batch_size: 500

Agent Data Through a Proxy

Sending the metrics, traces and APM stats of Datadog Agents through a proxy in front of Datadog. The target adds the path of each kind to these addresses. Logs still go to the intake for site.

targets:
  - name: datadog-proxy
    type: datadog
    properties:
      api_key: "${DATADOG_API_KEY}"
      metrics_url: "https://metrics-proxy.example.com"
      traces_url: "https://traces-proxy.example.com"

Proxy That Adds the Key

A proxy that adds the API key itself, so the target has no api_key. Both addresses are set, so the target sends all four kinds of Agent data to the proxy. Without them, the target would drop that data.

targets:
  - name: datadog-keyless-proxy
    type: datadog
    properties:
      url: "https://datadog-proxy.example.com/api/v2/logs"
      metrics_url: "https://datadog-proxy.example.com"
      traces_url: "https://datadog-proxy.example.com"

Troubleshooting

This section covers the errors you are most likely to see with the datadog target, what causes each one, and how to fix it. The sumologic and rapid7 targets behave the same way and report the same errors, so these entries apply to them too. The entries about Agent data apply to the datadog target only, because only this target sends Agent data. Their own pages list what differs.

Where to look:

Director logs. Target errors are tagged with the target name. The part after Reason:, or after the last colon, is the actual cause. See Target Delivery Errors for how Director logs and retries target failures.
The target's connection status in the web interface. It shows the same reason as the log line, prefixed with connection failed for <target name>:.
Director writes the warnings about Agent data to its log. Each warning describes one record, and the target writes it at most once a minute. To see how many records went, check the dropped count of the target.

A refusal from the intake API is logged as datadog api returned status <code>: followed by the response body, up to its first 1 MB. The status code is the quickest way to find the setting at fault.

Which setting is wrong?

Status	Setting to check	What it means
`400`	The pipeline that feeds the target	The body is not in the schema the intake API expects
`401`	`api_key`	No usable key reached the intake API
`403`	`api_key`, `site`	The key was read and refused, or it belongs to another site
`404`	`url`	The path is wrong, or the source behind it no longer exists
`413`	`max_bytes`	The request body is larger than the intake API accepts
`429`	`batch_size`, and how much traffic you route here	You are over the ingestion quota
`5xx`	Nothing on your side	The intake API is failing, and Director keeps retrying

Which status a vendor picks for a given mistake is not guaranteed, so read the quoted response body as well as the code.

"datadog target requires api_key (or an explicit url)"

ValidateConfig failed for target "datadog": datadog target requires api_key (or an explicit url)

Cause: neither api_key nor url is set. An ${DATADOG_API_KEY} reference that resolves to an empty value counts as not set.

Fix: set api_key to a Datadog API key. An Application key is a different credential and the intake API does not accept it.

Nothing is sent and nothing is lost while this message repeats. The configuration is re-checked about every 30 seconds, so the target starts on its own once the value is there.

"metrics_url must name a host", "traces_url is not a valid URL", and the other URL checks

The target checks metrics_url and traces_url when it validates the configuration. It refuses a configuration with one of these messages, where <name> stands for metrics_url or traces_url:

<name> is not a valid URL
<name> must start with http:// or https://
<name> must name a host
<name> must not hold a query or a fragment

Cause: the value breaks the rule in the message. The message names the setting and never the value, because the value may hold a credential.

Fix: set the URL to an http or https address with a host and no query or fragment, such as https://proxy.example.com.

Nothing is sent and nothing is lost while the message repeats. The configuration is re-checked about every 30 seconds, as for the api_key message above.

A URL that is a $env{...} or $secret{...} reference is checked when the target starts, once the target has read it. The same four messages appear there. If the target cannot read the reference, the message is failed to resolve <name>: ..., and the rest gives the reason. The target does not start. The message has the same form for api_key and url.

"batch_size must be greater than 0", "max_bytes must not be negative", "timeout must not be negative"

Cause: one of the three numeric settings is negative.

Fix: use a positive value, or remove the setting to take the default. Note that 0 is read as "not set" rather than as a value, so batch_size: 0 becomes 1000 and timeout: 0 becomes 30. The exception is max_bytes: 0, which is honored and removes the ceiling. Keep batch_size at or below 1000, which is the largest array the intake API typically accepts.

Nothing is sent while a value is invalid, and nothing is lost.

"api returned status 401" or "403"

[Error] [director] [target-<target id>] [datadog] Sender worker 1 execute() failed for mem://<payload id>: target broken: failed to finalize target cache: datadog api returned status 401: {"errors":["Missing API key"]}

Cause: 401 means no usable key reached the intake API. The common way to produce it here is to set url and leave api_key empty. Validation accepts that pair, but no DD-API-KEY header is then sent. 403 typically means the key was read and refused, because it was revoked, mistyped, or is an Application key rather than an API key. Requests for Agent data fail with the same statuses. Their messages name the kind: datadog metrics api returned status <code>: <response>, datadog traces api returned status <code>: <response> and datadog stats api returned status <code>: <response>. It also happens to Agent data when metrics_url or traces_url is set, api_key is empty and the proxy adds no key.

Fix: set api_key even when you set url. The only case that does not need one is an address of your own that adds the key on the way. On sumologic and rapid7 the equivalent mistake is a collector URL that lost characters when it was pasted, because there the credential is part of the path. For a 404 on Agent data, check metrics_url or traces_url instead of url.

No data is lost. Both statuses are retried until you fix them. The payload comes back about every five seconds, and the queue grows in the meantime. For Agent data, the other kinds in the same delivery still go through.

"403" from a key that works elsewhere

Cause: the key belongs to a different Datadog site than the one this target sends to. site defaults to datadoghq.com, so a target that never sets site goes to US1 whatever region the key came from. A key from another site is typically refused with 403.

Fix: set site to the site of the organization that issued the key. The accepted values are listed under Connection above. If you set url instead, site is ignored and the host in url decides where the batch goes.

No data is lost, and the batch is retried until the key and the site agree.

"no such host", "i/o timeout", "connection refused", or "unsupported protocol scheme"

[Error] [director] [target-<target id>] [datadog] Sender worker 2 execute() failed for natsobj://vm-sender/<payload id>: target broken: failed to finalize target cache: failed to send request: Post "https://http-intake.logs.example.com/api/v2/logs": dial tcp: lookup http-intake.logs.example.com: no such host

Cause: the request never reached the intake API. The tail of the message says which step failed. Requests for Agent data fail the same way. The message then shows the address of that kind.

no such host is a misspelled site, or a typo in the host part of url. Dropping the last letter of datadoghq.com is the usual one.
i/o timeout or connection refused is egress. Outbound HTTPS on port 443 to the intake host is blocked, or the network requires a proxy that is not configured.
proxyconnect tcp means HTTPS_PROXY points at a proxy that is down or wrong.
unsupported protocol scheme "" means url was pasted without https://.
invalid character in a host name means url holds a stray space or quote. The address is not parsed when the configuration is validated, so a malformed url only fails here, at the first delivery.
failed to compress the series, failed to compress the sketch, failed to compress the traces or failed to compress the stats, followed by the reason, is a rare failure to compress the body of an Agent data request. Director delivers the records again.

Fix: correct the address, allow outbound 443 from the Director host to the intake host, or set HTTP_PROXY, HTTPS_PROXY and NO_PROXY for the Director service.

No data is lost. All of these are retried until fixed, so the queue grows until the address or the route is corrected.

"x509: certificate signed by unknown authority"

... failed to finalize target cache: failed to send request: Post "https://http-intake.logs.datadoghq.com/api/v2/logs": tls: failed to verify certificate: x509: certificate signed by unknown authority

Cause: something between Director and the intake API is terminating TLS with its own certificate, and the authority that signed it is not trusted on the Director host.

Fix: this target has no TLS settings. There is no property for a certificate authority file and no option to skip verification, so the trust has to be granted at the operating system level. Install the inspecting authority's root certificate in the Director host's trust store, or exclude the intake host from TLS inspection. Restart Director after changing the trust store.

No data is lost, and the batch is retried until the handshake succeeds.

warning

The scheme is not checked. An http:// address is accepted and sends the batch, and any credential in that address, in clear text. Always use https://.

"context deadline exceeded (Client.Timeout exceeded while awaiting headers)"

Cause: the whole request, including connecting and sending the body, did not finish inside timeout seconds. The default is 30. A multi-megabyte body on a slow link, or a slow intake API, reaches it.

Fix: raise timeout, or make the batches smaller by lowering batch_size or max_bytes.

No data is lost, but duplicates are possible. The intake API may have stored the batch before Director stopped waiting for the response, and the payload is redelivered, so those events arrive twice.

"record rejected by target": the only errors that lose data

[Error] [director] [target-<target id>] [datadog] Sender worker 3 execute() failed for natsobj://vm-sender/<payload id>: record rejected by target: datadog api returned status 400: {"errors":["Invalid message"]}
[Error] [director] [target-<target id>] [datadog] Sender worker 3 deterministic failure for natsobj://vm-sender/<payload id> after 4 attempts — dropping (giving up): record rejected by target: datadog api returned status 400: {"errors":["Invalid message"]}

Cause: two different problems carry this prefix.

A rejected body, logged with status 400, 413 or 422. 400 means the payload is not what the intake API expects, because the matching content pack pipeline was not applied on the route, or because a raw non-JSON message reached the target. 413 means the body was larger than the intake API accepts, which happens when max_bytes is raised above the vendor's own limit or set to 0. 422 means the document was understood and refused on its content.
An oversized single record, logged as record rejected by target: record size <n> exceeds configured max_bytes <m>. One event on its own is larger than the ceiling. Windows events with very large message fields, and base64 blobs, are the usual source.

Fix: for a rejected body, route the events through the matching Datadog content pack pipeline and put max_bytes back to its default. For an oversized record, trim the event in the pipeline by dropping or truncating the offending field. Raising max_bytes past the vendor's own limit only turns the rejection into a 413.

For logs, these two are the only errors on this target that lose data. They are treated as permanent, so Director gives up after four deliveries. A rejected body loses the whole batch, up to batch_size events. An oversized record loses only that record: on the fourth delivery it is dropped alone, logged as Target "<name>" rejected record <n> of <payload> on its last delivery attempt — dropping that record and delivering the rest of the payload: <reason>, and the records before and after it in the same payload are delivered. The Agent data entries below can lose data too.

In the Agent data entries below, <name> stands for metrics_url or traces_url, <kind> for series, sketch, trace or stats, <code> for a status code, <response> for the answer of Datadog, and N for a number.

"The target has no api_key and no metrics_url" or "traces_url"

A <kind> record of the datadog_agent device was dropped. The target has no api_key and no <name>, so a request would reach Datadog with no key and be refused. Set api_key, or set <name> when a proxy in front of Datadog adds the key. Records dropped for this reason are reported once a minute.

<kind> is series or sketch when <name> is metrics_url. <kind> is trace or stats when <name> is traces_url.

Cause: the target has no api_key, and the URL for that kind of data is not set.

Fix: set api_key. Or set the URL named in the message, when a proxy in front of Datadog adds the key.

The target drops the records of that path and counts them as dropped. Nothing is retried. Logs are not affected.

"could not be read and was dropped"

A <kind> record of the datadog_agent device could not be read and was dropped: <reason>. Dropped records of this kind are reported once a minute.

Reason in the message	What it means
`the record has no datadog_agent object`	The record holds no Agent data
`the record has no datadog_agent.<member> object`	The record lacks the data of its own kind. `<member>` is `series`, `sketch`, `trace` or `apm_stats`
`the series has no metric name`	A series needs a metric name. Datadog would refuse the whole request without one
`the sketch has no metric name`	A sketch needs a metric name
`a sketch has a different number of bin keys and bin counts`	The sketch is not well formed
`the trace chunk has no spans`	A trace chunk needs at least one span

A reason can also be the text of a decoding error, when the data of the record cannot be read.

Cause: the record says it is a series, sketch, trace or APM stats, but its data is missing or broken.

Fix: check any pipeline between the device and the target. It must leave the Agent data of the record as the device wrote it.

The target drops the record and counts it as dropped. Nothing is retried.

"Datadog refused a request of N records for good"

Datadog refused a request of N <kind> records for good, so they were dropped: record rejected by target: datadog <metrics|traces|stats> api returned status <code>: <response>. Refused requests are reported once a minute.

In the message, metrics stands for series and sketches, traces for traces and stats for APM stats. <code> is 400, 413 or 422. <response> is the answer of Datadog, as for logs.

Cause: Datadog understood the request and refused it for good. The status meanings are the ones in the table under Which setting is wrong?.

Fix: read the response. It says what Datadog did not accept. max_bytes changes nothing here, because it applies to logs only.

The records of that request are lost. The target drops them, counts them as dropped and goes on with the rest. Director does not deliver them again.

"more than a request to Datadog holds"

A <kind> record of N bytes is more than a request to Datadog holds, so it was dropped. Dropped records of this kind are reported once a minute.

Cause: one record is larger than a request can hold. The target splits requests, not records.

Fix: make the record smaller at its source, such as the points of one series or the spans of one trace chunk.

The target drops that record and counts it as dropped. The records around it still go.

A series, sketch or stats record of more than 5,242,880 bytes fails earlier, with this message, where <kind> is series, sketch or stats:

record rejected by target: a <kind> record of N bytes is over the 5242880 bytes that a request holds

Director treats it as it treats an oversized log record (see "record rejected by target" above): it gives up after four deliveries, drops that record alone and delivers the rest.

"429 Too Many Requests"

... target broken: failed to finalize target cache: datadog api returned status 429: {"errors":["Too Many Requests"]}

Cause: you are sending faster than the account's ingestion quota allows.

Fix: send fewer and larger requests by raising batch_size, reduce how many routes write to this target, or raise the quota with the vendor. The target does not back off on its own and does not read a Retry-After header, so it keeps offering the batch at the redelivery pace until the limit clears.

No data is lost, and the batch is retried until it is accepted.

The target looks healthy but nothing arrives

Check these in order.

debug.dont_send_logs is enabled. Events are processed and then discarded. The payload is marked delivered, the queue drains, and the target's event-out counters stay flat while the route's counters climb. Nothing is written to the log unless debug.status is also enabled, in which case startup logs Log sending is disabled for this target. It holds back Agent data too. Remove the setting.
field_format is set and the mapping failed. A normalization failure here is ignored rather than reported, so the event is delivered without being normalized and nothing is logged. Leave field_format empty when the payload already arrives in the vendor's schema.
The events arrived but were not parsed. On sumologic and rapid7 the body is newline-delimited, so the receiver typically accepts arbitrary lines with a success status and stores them as unparsed text. Delivery counters climb and no error is logged. Check the vendor's own parse errors, and route the events through the matching content pack pipeline.
interval or cron is set. The target then flushes on that schedule instead of on every payload, so expect a delay before anything is sent. See Scheduling above.
Agent data is dropped. The target has no api_key and no metrics_url or traces_url. The dropped count of the target climbs, and the warning in the entry above appears in the log.
The kind is turned off on the device. A Datadog Agent device drops a kind of data that is turned off, so that data never reaches this target. See the device page.

Log lines and connection status can contain the full endpoint URL

When a request fails before a response arrives, for example on a DNS, connection, proxy, TLS or timeout error, the message includes the full URL the batch was posted to. The same text is shown as the target's connection status in the web interface.

... failed to send request: Post "https://<endpoint>.collection.sumologic.com/receiver/v1/http/<collector code>": tls: failed to verify certificate: x509: certificate signed by unknown authority

On datadog the credential travels in the DD-API-KEY header, so the URL holds no secret. For requests to metrics_url and traces_url, the message shows the address without any user name or password in it. On sumologic and rapid7 the credential is part of the URL path: the Sumo Logic collector code, and the Rapid7 ingestion token. For those two, a line like the one above carries a working ingestion credential, and anyone who can read the log or the connection status can write data into your collector.

warning

Treat Director logs and connection status for sumologic and rapid7 targets as sensitive. Restrict who can read them, and remove the URL before attaching a log line to a ticket, an email or a chat message. If such a line has already been shared, rotate the credential: create a new HTTP Source or ingestion endpoint, point url at the new one, and delete the old one.

Synopsis​

Schema​

Configuration​

Connection​

Batch Configuration​

Processing​

Scheduling​

Debug Options​

Details​

Endpoint and Authentication​

Datadog Agent Data​

Batching and Retry​

Examples​

Basic Configuration​

EU Site​

Explicit Endpoint Override​

Agent Data Through a Proxy​

Proxy That Adds the Key​

Troubleshooting​

Which setting is wrong?​

"datadog target requires api_key (or an explicit url)"​

"metrics_url must name a host", "traces_url is not a valid URL", and the other URL checks​

"batch_size must be greater than 0", "max_bytes must not be negative", "timeout must not be negative"​

"api returned status 401" or "403"​

"403" from a key that works elsewhere​

"no such host", "i/o timeout", "connection refused", or "unsupported protocol scheme"​

"x509: certificate signed by unknown authority"​

"context deadline exceeded (Client.Timeout exceeded while awaiting headers)"​

"record rejected by target": the only errors that lose data​

"The target has no api_key and no metrics_url" or "traces_url"​

"could not be read and was dropped"​

"Datadog refused a request of N records for good"​

"more than a request to Datadog holds"​

"429 Too Many Requests"​

The target looks healthy but nothing arrives​

Log lines and connection status can contain the full endpoint URL​