Content Hub: Overview

The Content Hub provides a centralized repository of professionally-developed pipeline templates designed to accelerate DataStream deployment and standardize data processing workflows. These templates contain pre-built parsing, transformation, and enrichment logic for popular security devices, network equipment, and enterprise applications.

What Are Templates?

A template is a pre-configured pipeline definition that includes all the processing logic needed to handle data from a specific source type. Each template contains field extraction rules, normalization mappings, enrichment configurations, and output formatting—ready to deploy without manual configuration.

Templates are complete pipeline packages. They define how raw log data is parsed, which fields are extracted, how values are normalized to standard schemas, and what enrichments are applied. When installed, a template becomes a fully functional pipeline within DataStream.

Why Use Templates?

Templates eliminate the need to build pipelines from scratch. Instead of manually configuring parsers, field mappings, and transformation rules for each data source, users can install a tested template and begin processing data immediately.

Building a pipeline manually for a complex data source can take hours or days. Installing a template takes seconds.

Templates also ensure consistency and quality. Each template is developed by VirtualMetric's engineering team, tested against real-world data, and maintained with regular updates. This professional development cycle produces reliable configurations that handle edge cases and follow industry best practices.

For organizations processing data from multiple vendors and platforms, templates provide standardized normalization across all sources. Data from Palo Alto firewalls, Cisco switches, and Azure services all map to consistent field names and schemas.

Template Library

The Template Library contains production-ready pipeline configurations for common data sources and use cases. Each template is professionally developed, tested, and maintained to ensure reliable data processing.

Professional Template Development

All templates in the Content Hub are developed and maintained by VirtualMetric's engineering team.

Each template undergoes rigorous testing across multiple device firmware versions. Validation includes real-world log samples and edge cases to ensure reliable parsing. Templates are optimized for high-volume data processing and receive regular updates to support new device features and log formats.

Templates comply with industry standards by supporting multiple security schemas including ASIM, OCSF, ECS, CIM, and UDM. Field mapping and normalization remain consistent across vendors. Integration with popular SIEM and analytics platforms is built-in, along with compliance support for regulatory and audit requirements.

Coverage spans major security vendors such as Palo Alto, Cisco, Fortinet, and Check Point. Network infrastructure devices including switches, routers, and load balancers are supported. Cloud platform integrations cover Azure, AWS, and Google Cloud, alongside enterprise applications and databases.

Template Categories

Security device templates cover firewall logs with advanced threat detection parsing. They also handle intrusion detection and prevention system events. Endpoint protection and antivirus solutions are supported, along with network access control and authentication systems.

Infrastructure templates handle network device logs and SNMP data processing. Server operating system event collection is included. Application performance and error log analysis is supported, as is database audit and transaction log processing.

Cloud platform templates address cloud service activity and audit log processing. Container and orchestration platform monitoring is available. Serverless function execution and error tracking is supported, along with cloud security and compliance event analysis.

Interface Navigation

The Content Hub interface enables browsing, searching, and evaluating templates before installation. Templates can be filtered by device type, vendor, and target platform to quickly find relevant configurations.

Content Discovery

The Content Hub interface provides multiple ways to discover relevant templates through search and filtering capabilities.

A global search field matches template names and descriptions. Search results appear in real-time with highlighting of matching terms. Search history is preserved, with suggested queries for common use cases.

Available filter categories include:

• Device Type (single-select): Filter by device category (firewall, router, server, etc.)
• Target (multi-select): Filter by supported destination platforms (Sentinel, Splunk, etc.)
• Device Vendor (multi-select): Filter by manufacturer (Cisco, Palo Alto, Microsoft, etc.)

Filter selections display visual feedback with highlighting and count indicators. Dynamic count updates show template matches for each filter option. Clear filter removal options maintain visual selection states.

Template Cards

Each template card displays essential information for evaluation. The card shows template name and version information, along with supported device types and vendor compatibility. Target platform integration capabilities are indicated, as is the installation status (Available or Installed).

Visual indicators help identify template status at a glance. A blue Installed badge appears on already-installed templates. Icons represent device types and target platforms. Template complexity indicators (Simple/Advanced/Enterprise) and last update timestamps with version history are also displayed.

Template Details

Clicking on any template card opens detailed documentation across four specialized views.

General Overview Tab

Template metadata provides complete template description and use case documentation. Supported device models and firmware versions are listed. Prerequisites and dependency requirements are specified, along with installation and configuration guidance.

Technical specifications cover supported log formats and parsing capabilities. Output schema and field mapping documentation is included. Performance characteristics and resource requirements are documented, along with the integration compatibility matrix.

Processor documentation provides a complete list of processing components used in the template. Individual processor documentation includes configuration examples. Links to detailed processor reference documentation are provided, along with recommendations for customization.

Pipeline Overview Tab

Pipeline architecture displays a visual representation of data processing flow. Parent and child pipeline relationships are shown. Processing stage documentation includes dependencies.

The read-only pipeline view shows complete YAML configuration with syntax highlighting and structure visualization. Processing logic explanation and comments are included. Child pipeline navigation with cross-references enables exploration of the full pipeline structure.

Log Transformation Overview Tab

Sample data processing demonstrates real-world log samples showing input data format. A step-by-step transformation process demonstration is provided. Final output format with field mapping examples shows the complete transformation.

The before and after comparison shows raw log data in original format alongside processed output with normalized fields. Schema compliance validation results are displayed. Performance metrics with processing statistics complete the view.

License Details Tab

Template licensing information includes complete license text for the specific template. Elastic License 2.0 terms and conditions are presented. Copyright notices and attribution requirements are specified, along with usage restrictions and compliance guidelines.

The read-only display shows full license text in code editor format. Detailed terms specific to the template version are included. Reference to general licensing framework documentation is provided.

Template Installation Process

Installing a template creates a local copy that can be customized for specific organizational requirements. The installation process handles dependencies automatically and provides immediate access to the installed pipeline.

Template Actions

Template detail pages provide different actions based on installation status.

For uninstalled templates, the Install Template button is available in the header. Clicking triggers a dependency check and installation process. Template installation requires PIPELINE_CREATE permission.

note

Template installation requires PIPELINE_CREATE permission. Users without this permission will not see the install button.

For installed templates, an Installed badge (blue tag) is displayed next to the template name. The Actions menu provides management operations.

Manage Dependencies allows updating optional and required dependencies (only shown if dependencies exist). See Installed Pipeline navigates to the installed pipeline detail page.

Installation Workflow

Before installation, users can thoroughly evaluate templates through the detail view tabs. Complete pipeline logic review is available without installation. Sample data transformation testing allows verification of expected output. License terms review and resource requirement estimation help inform the installation decision.

Dependency Management

When installing a template with dependencies, a modal appears for dependency selection.

Required dependencies include essential processing components that must be installed for the template to function. Core libraries, shared processing modules, and schema definitions are included. A checkbox selection confirms installation. Already installed dependencies appear with an "Installed" indicator and a disabled checkbox.

info

Required dependencies are mandatory. The template will not function correctly without them.

Optional dependencies include enhanced processing features and advanced transformations. Integration modules for specific target platforms and performance optimization components are available. These appear as user-selectable checkboxes. Already installed dependencies are indicated and disabled.

Dependency list features include Show More and Show Less buttons for lists exceeding 5 items. Clickable dependency names open dependency details. Visual distinction differentiates installed from available dependencies.

An installation progress indicator displays during processing.

For installed templates with dependencies, use the Manage Dependencies action to install additional optional dependencies or view currently installed dependencies.

Installation Completion

Upon successful template installation, a success toast message confirms the installation. The user is automatically redirected to the installed pipeline detail page at the /pipelines/{id}/general-overview route. This provides immediate access to pipeline configuration and customization.

After installation, the template remains visible in Content Hub with an Installed badge. The template detail page provides access to the installed pipeline via the See Installed Pipeline action. Full pipeline editing capabilities are available in the Pipeline management section.

Post-Installation Management

After installation, templates become editable pipelines within DataStream. Users can customize processing logic, modify field mappings, and integrate templates with existing workflows.

Template Integration

Once installed, templates become fully integrated into the DataStream platform.

Customization capabilities include full editing access to installed pipeline configurations. Custom field mapping and transformation rule modification is supported. Integration with existing processing workflows is straightforward.

Child pipeline creation and management is available.

Status tracking provides installation status indicators throughout the Content Hub interface. A version tracking and update notification system alerts users to new versions.

Usage statistics and performance monitoring are available. Change history with configuration audit trails enables review of modifications.

Template Updates

When Content Hub templates are updated by VirtualMetric, installed pipelines can receive these updates while preserving custom modifications. A merge workflow enables selective update acceptance while retaining organization-specific changes.

tip

Template updates preserve your customizations. The merge workflow lets you review changes and decide which updates to accept while keeping organization-specific modifications intact.

Update Notification System

Installed templates with available updates display visual indicators throughout the platform.

In the pipeline overview, a renew icon (circular arrow) is displayed on pipeline cards. Hovering over the icon shows an "Update available" tooltip. Clicking the card navigates to the pipeline detail view.

In the pipeline detail view, a yellow warning alert banner is displayed at the top of the page content. The banner shows an "Update available" title and explains that the template source has updates. The Review Update action button provides access to the update review interface. The Review Update option is also available in the pipeline actions menu.

Review Update Interface

Navigate to update review through multiple entry points:

• Click Review Update action button on warning alert banner
• Select Review Update from pipeline Actions menu
• Click renew icon on pipeline overview card (redirects to detail, then review)

The review update page is structured with two panels.

The left panel contains the pipeline tree with a resizable sidebar (adjustable width 292-600px). The main pipeline appears as the root with an update indicator if applicable. Child pipelines are nested with individual update indicators.

Renew icons are displayed on tree items with available updates. Clicking tree items displays specific pipeline changes. The selected pipeline is highlighted in the tree.

The right panel contains the merge editor with a side-by-side diff view showing original vs updated YAML comparison. Syntax highlighting provides YAML color coding for readability. Line-by-line changes show visual indicators for additions, deletions, and modifications.

The original content (left) is read-only, while the modified content (right) is editable.

Manual editing allows modification of the updated version before accepting. The editor respects the user's light/dark theme preference.

Update Application Workflow

To review and apply changes:

1. Select Pipeline - Click a pipeline or child pipeline in the tree navigation to view changes for that specific component. Navigate between multiple pipelines with updates as needed.

2. Examine Differences - Review the side-by-side comparison of changes. Identify new features, modifications, and removals. Understand the impact of template updates before proceeding.

3. Customize Updates - Edit the right-side (modified) content as needed to merge custom configurations with template updates. Preserve organization-specific modifications. Adjust updated content before application.

4. Accept Changes - Click the Accept Changes button to apply updates. A success notification confirms the update application. The pipeline refreshes with updated content. The process repeats for additional pipelines with updates.

When all pipeline updates are applied, a blue info notification banner confirms "All updates installed" and indicates no pending updates. A button allows navigation back to the pipeline detail view. Renew icons are removed from overview cards.

Update Scenarios

For individual pipeline updates, the main pipeline has an update while child pipelines remain unchanged. The update only affects root pipeline content. Child pipelines remain on their current version.

For child pipeline updates, one or more child pipelines have updates while the main pipeline may or may not have updates. Each child pipeline is reviewed and updated independently.

For complete template updates, the main pipeline and all child pipelines have updates. Navigate through the tree reviewing each component. Apply updates systematically across the entire template structure.

For partial updates, accept some pipeline updates and skip others. The Review Update interface remains accessible to return later and apply remaining updates. This provides flexibility for staged rollout of template changes.

The Content Hub accelerates DataStream deployment by providing professionally-developed, tested, and maintained pipeline templates. These templates can be quickly installed and customized for specific organizational requirements.