Skip to content

Conversation

@vladimir-dd
Copy link
Contributor

@vladimir-dd vladimir-dd commented Nov 24, 2025

Replaces standalone processors with processor groups.

Before:

processors {
      filter {
        id      = "filter-1"
        include = "service:my-service"
        inputs  = ["source-1"]
      }
      parse_json {
        id      = "parser-1"
        include = "service:my-service"
        field   = "message"
        inputs  = ["filter-1"]
      }
}

After:

processors {
      processour_group {
        id = "group-1" 
        include = "*"
        inputs  = ["source-1"]
        enabled = true
        
        processor {
          id      = "filter-1"
          include = "service:my-service"
          enabled = true
          filter {}
        }
        
        processor {
          id      = "parser-1"
          include = "service:my-service"
          enabled = true
          parse_json {
            field   = "message"
          }
        }
      
}

Important context:

  • individual processor's don't have inputs anymore, only processor groups;
  • processor block is required to maintain the order of processors(TF tracks the order only for the blocks of the same type). See TestAccDatadogObservabilityPipeline_multipleProcessorGroups as an example test with reordering.
  • required enabled field is added to both processors and processor groups;
  • processor blocks(filter, parse_json etc.) must be ListNestedBlock with the size validation(<=1), because of the way TF handles optional blocks. If we use SingleNestedBlock(which makes more sense) for an optional block, which contains any required field, TF considers this block as required. This is a recommended workaround(see Disallow use of SingleNestedBlock in resource schemas hashicorp/terraform-provider-aws#35813 for example).

@vladimir-dd vladimir-dd changed the title WIP: add processor groups [datadog_observability_pipeline] Add processor groups instead of standalone processors Nov 26, 2025
@vladimir-dd vladimir-dd marked this pull request as ready for review November 26, 2025 15:05
@vladimir-dd vladimir-dd requested a review from a team as a code owner November 26, 2025 15:05
@vladimir-dd vladimir-dd requested a review from a team November 26, 2025 15:06
Copy link

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants