Skip to main content

Monitor and Verify

Your first DataStream pipeline should now be processing data. This page covers how to verify the data flow and check Director health.

Check Director Status​

  1. Navigate to Director Detail

    • Go to Fleet Management > Directors
    • Click on your Director's name in the table

  2. Verify on Director Details Tab (first tab)

    • Status: Should show Enabled

  3. Verify on Connection Details Tab (second tab)

    • Connection Status: Should show Connected with green indicator
    • IP Address: Should display your server's IP address

Verify Data Flow​

Check Device Activity​

info

The steps below use Syslog as the example because it is the device type configured in the Getting Started tutorials. If you configured a different device type, navigate to that device's tab instead.

  1. Go to Fleet Management > Devices > Syslog
  2. Find your device in the table
  3. Confirm the device status shows Enabled and your Director is assigned

Examine Target Output​

  1. Navigate to the file location you specified in your File target
  2. Look for JSON files matching the file name template you configured
  3. Open a recent file to confirm it contains processed data
note

The output format depends on the pipeline template you selected from Content Hub. Field names and structure vary by pipeline.

Check Route Status​

  1. Go to Routes > Quick Routes
  2. Verify your route shows a solid connection line between device and target
  3. Confirm the pipeline node appears on the connection line with your pipeline name

Watch Data in Flight​

The Director detail page carries two tabs for observing the Director directly:

  1. Open the Live Data tab and start a capture to record messages passing through the Director. Use it to confirm events are arriving and to see their shape at each processing stage.
  2. Open the Console Logs tab and start a capture to record the Director's own diagnostic messages.

Both are bounded capture sessions rather than a continuous stream, and both need the Director to be Enabled and Connected.

For capture locations, filters, severities and export options, see Live Data and Console Logs.

Troubleshooting​

No Data Appearing​

  1. Check data sources: Verify log sources are sending to the correct IP address and port (514 for syslog)
  2. Test connectivity: Confirm firewall rules allow inbound traffic on port 514
  3. Verify device status: Ensure the device is Enabled in Fleet Management > Devices
  4. Verify route: Confirm the route is saved with a solid connection line in Quick Routes
  5. Check Director: Verify the Director shows Connected status
  6. Capture console logs: On the Director's Console Logs tab, run a capture with Error and Warning selected while your source is sending — see Console Logs

Data Looks Incorrect​

  1. Verify you selected the correct pipeline template for your log format
  2. Review the pipeline configuration in My Pipelines
  3. Compare raw input against the pipeline's expected input format

What's Next?​

Your basic DataStream pipeline is operational. Now you can expand and enhance your data processing capabilities.

Next: Next Steps to learn how to customize your pipeline, add more data sources, and scale your deployment.