Skip to main content

AWS IoT SiteWise

Amazon AWS Observability

Synopsis​

Creates an AWS IoT SiteWise target that sends metrics to asset properties and data streams. Each value goes to the alias that a template gives.

Schema​

- name: <string>
  description: <string>
  type: awsiotsitewise
  pipelines: <pipeline[]>
  status: <boolean>
  properties:
    region: <string>
    endpoint: <string>
    alias: <string>
    key: <string>
    secret: <string>
    session: <string>
    timeout: <numeric>
    batch_size: <numeric>
    max_retries: <numeric>
    retry_delay: <numeric>
    disable_retries: <boolean>
    debug:
      status: <boolean>
      dont_send_logs: <boolean>

Configuration​

The following fields are used to define the target:

FieldRequiredDefaultDescription
nameYTarget name
descriptionN-Optional description
typeYMust be awsiotsitewise
pipelinesN-Optional post-processor pipelines
statusNtrueEnable/disable the target

Connection​

FieldRequiredDefaultDescription
regionY-The AWS region of the SiteWise service, such as eu-west-1. The target sends to https://data.iotsitewise.<region>.amazonaws.com. In the console: Region
endpointN-Another address, such as a VPC endpoint, a FIPS endpoint or an endpoint in China. It must be an https URL with a host. It takes no user, query or fragment, but it can have a path. In the console: Endpoint

Data​

FieldRequiredDefaultDescription
aliasN/{device}/{metric}The template of the alias that each value goes to. See The Alias. In the console: Property alias

AWS Credentials​

FieldRequiredDefaultDescription
keyN-The access key ID. Set it with secret. In the console: Access key ID, when Credentials is Access key
secretN-The secret access key. Set it with key. In the console: Secret access key
sessionN-The session token of temporary credentials. In the console: Session token

key, secret and session can each be plain text, an environment variable or a secret. Leave out key and secret to sign with the credentials of the host (see Credentials).

Performance​

FieldRequiredDefaultDescription
timeoutN30As on the OTLP page
batch_sizeN1000The most records in one batch. The target splits a batch into as many requests as its values need (see Requests and Answers)
max_retriesN3As on the OTLP page
retry_delayN1As on the OTLP page
disable_retriesNfalseAs on the OTLP page

Other Settings​

  • headers and the tls settings work as on the OTLP page.
  • cumulative_to_delta has no effect on this target.

Scheduling​

See Scheduling and Pool Behavior for interval and cron fields shared by all targets.

Debug Options​

The debug options work as on the OTLP page.

The Console Form​

The form has one step, AWS IoT SiteWise settings, in four sections:

  • Connection settings: Region and Endpoint (Optional).
  • Data settings: Property alias, filled with /{device}/{metric}.
  • Authentication: Credentials, which is Host's role (the default) or Access key. Access key shows Access key ID and Secret access key, both required, and Session token (Optional), each with the Manual / Environment variable / Vault picker.
  • Performance.

The Credentials choice belongs to the form, not to the YAML. Host's role sends no credentials, and Access key sends the key fields. The form has no TLS step, because AWS's endpoints use public certificates.

Details​

What the Target Sends​

Only metrics:

  • the readings of metric devices;
  • the metrics that the OpenTelemetry device receives.

The number points of gauges and sums are sent. Histograms, summaries and exponential histograms are not, because SiteWise stores single values only.

A record that is not a metric, such as a log line, is counted as dropped, with a warning at most once a minute. So route only metrics to this target.

The Values​

Every value goes as a double, with the quality GOOD and the time of the reading.

  • So an alias must name a DOUBLE asset property, or a data stream that no asset property uses. An INTEGER, BOOLEAN or STRING property refuses the values.
  • A gauge and a cumulative counter go with their value. A delta counter goes as its change in each interval, because SiteWise keeps no running total.

The Alias​

alias is a template:

  • {device} is the name of the device, {metric} the name of the metric, and {attr.<key>} the value of an attribute of the reading, or else of its resource, such as service.instance.id or host.name.
  • The default /{device}/{metric} gives aliases such as /router1/cpu.load. The template /{attr.site}/{device}/{metric} gives /plant1/router1/cpu.load.
  • A template with an unknown placeholder, an empty {attr.} or a brace without its partner stops the target from starting. The error names the character where the problem is.
  • An alias names an asset property, or a data stream that no asset property uses.
  • SiteWise accepts a value for an alias that no asset property uses only when its storage settings accept data streams without an asset property. It then creates the data stream, which can be associated with an asset property later.

One Series to One Alias​

A series is one metric of one instance of a device or service, from one instrumentation scope, with one set of point attributes.

  • Instances are told apart by service.name, service.namespace, service.instance.id and host.name. Other resource attributes, such as host.uptime, change from one reading to the next and do not make a new series.
  • When two series give the same alias, the target sends the first one and drops the values of the other, with a warning, because they would mix in one property. Add {attr.service.instance.id} to the template to give each instance its own alias.
  • The target keeps up to 100,000 aliases. A series that sends nothing for an hour gives up its alias. When 100,000 series hold an alias and none of them is stale, the target sends the values of a new alias without this check, so that no value is lost. Two series can then mix their values in that alias.
  • Each Director keeps its own list, so two Directors can write to one alias without a warning.

What Is Dropped Before Sending​

The target drops a value that SiteWise would refuse:

  • a value that is NaN, an infinity, or beyond -10^10 and 10^10;
  • a time more than 7 days old or more than 10 minutes ahead;
  • a point without a timestamp, because its time would differ each time the batch is delivered;
  • a reading that lacks an attribute that the alias names;
  • an alias that is empty, longer than 1,000 characters or holds a control character.

A warning with the number of values and the reason is logged at most once a minute. These values are counted in the warning, not as dropped records, because one record can hold many values.

Requests and Answers​

  • A request holds up to 10 entries, and an entry holds up to 10 values of one alias.
  • SiteWise answers 200 and lists the entries it refused. Throttled entries, and entries that hit a temporary error, are sent again in the next attempt.
  • When SiteWise refuses values for good, such as an invalid request, a time it does not take, a limit or an alias it does not know, the values are dropped. The warning names the error, the alias and the start of SiteWise's message.
  • A refused permission, and any error the target does not know, make Director deliver the batch again later. SiteWise overwrites a value that has the same time and quality, so a delivery that repeats adds no duplicates.
  • An empty 200 answer confirms nothing, so the batch is delivered again.
  • A 429 or 5xx answer, or a failed connection, is retried. A 400 drops the values of the request and logs the answer. Any other answer, such as a 403, makes Director deliver the batch again later.
  • batch_size counts records. The target splits a batch into as many requests as its values need.

Quotas​

By default SiteWise takes 5,000 values a second for each account and region, and 10 entries a second for each alias. Over them, SiteWise throttles, and the target sends the values again. The quotas are on the Service Quotas page of the AWS console.

Sign-in​

The target signs each request with AWS Signature Version 4 for the service iotsitewise. The identity needs iotsitewise:BatchPutAssetPropertyValue for every alias that the template gives. A policy that allows only some aliases makes Director deliver every batch that holds any other alias again.

Credentials​

Two ways:

  • An access key: set key and secret, and session for temporary credentials. The target reads them when it starts, and it does not renew a session token when the token expires.
  • The credentials of the host: leave out key and secret. The target then finds credentials the same way the AWS SDK does: the environment, then a shared profile, then the role of the host, such as an EC2 instance profile or an EKS service account role.

For temporary credentials, use the role of the host, or a shared profile that assumes a role, signs in with IAM Identity Center or runs a credential process. The AWS SDK renews these.

A key, a secret or a session token that is set but resolves to nothing stops the target from starting, so an empty secret never makes the target sign as the host by mistake.

Workers​

The target keeps no running totals, so it can run several workers in parallel.

Examples​

The Role of the Host​

Sending with the role of the host and the default alias...

targets:
  - name: aws_iot_sitewise
    type: awsiotsitewise
    properties:
      region: "eu-west-1"

An Alias per Site and Instance​

An alias for each site and instance, with an access key whose secret comes from a secret store...

targets:
  - name: aws_iot_sitewise_plant
    type: awsiotsitewise
    properties:
      region: "eu-west-1"
      alias: "/{attr.site}/{device}/{attr.service.instance.id}/{metric}"
      key: "${AWS_ACCESS_KEY_ID}"
      secret: "$secret{aws-sitewise-secret-key}"

Troubleshooting​

Configuration Errors​

Director refuses a configuration it cannot use, with one of these messages:

  • region is required for AWS IoT SiteWise, for example eu-west-1
  • endpoint for AWS IoT SiteWise must be an https URL with a host and no user, query or fragment, such as https://data.iotsitewise.eu-west-1.amazonaws.com
  • alias template is empty: write a path such as /{device}/{metric}
  • alias template has an unknown placeholder {name} at character 10: use {device}, {metric} or {attr.<key>}
  • alias template has an empty attribute name in {attr.} at character N: write the name after attr., as in {attr.site}
  • alias template has a { at character N with no closing }: close every placeholder with }, and the messages for a } with no opening { and for a { inside a placeholder
  • set both key and secret, or neither
  • the secret of the AWS IoT SiteWise target resolved to nothing: check the environment variable or the secret it names: the setting is set, but its environment variable or secret is empty. The same message names the key or the session token.
  • failed to resolve the key of the AWS IoT SiteWise target: ...: the environment variable or the secret could not be read.
  • no AWS credentials to sign the requests with: set key and secret, or give the host an AWS role

Warnings While It Runs​

These appear in the Director log, at most once a minute:

  • [OTLP] dropped N records that are not metrics in the last minute: AWS IoT SiteWise takes only metrics: the route sends log records or spans to this target.
  • [OTLP] dropped N values in the last minute: AWS IoT SiteWise <reason>, where the reason is one of:
    • takes no value outside ±10^10, NaN or an infinity
    • takes no timestamp older than 7 days or more than 10 minutes ahead
    • takes no value without a timestamp
    • has no value for an attribute the alias names
    • cannot take the alias: the alias is empty, too long or holds a control character.
    • stores no histograms or summaries
    • already writes this alias for another series, <alias>: two series give the same alias. Add an attribute to the template that tells them apart.
    • refused the value: <error>, <message>, alias <alias>: SiteWise refused the values for good. ResourceNotFoundException means that no asset property has the alias and SiteWise does not accept data streams without an asset property. InvalidRequestException can mean that the property is not DOUBLE.
    • refused the request: ...: SiteWise answered 400 to the whole request.

Errors While It Runs​

  • AWS IoT SiteWise refused an entry: AccessDeniedException: the identity lacks iotsitewise:BatchPutAssetPropertyValue for that alias. The batches are delivered again once this is fixed.
  • status 403: the credentials are wrong or expired, such as a session token set by hand, or region does not match. The batches are delivered again once this is fixed.
  • status 429, or entries refused as throttled: the account or an alias is over its quota. The target sends them again.