Skip to content

Conversation

@bezarsnba
Copy link

@bezarsnba bezarsnba commented Nov 2, 2025

Problem

The current PodMonitor templates have hardcoded metricRelabelings that cannot be customized. This prevents users from:

  • Adding organizational labels for cost allocation and compliance
  • Reducing metric cardinality to control storage costs

Somes scenarios require custom metric labels to meet regulatory compliance or internal governance policies.

Solution

Add new configuration options to the Helm chart:

  • podMonitor.*.metricRelabelings - Replace default relabelings entirely
  • podMonitor.*.additionalRelabelings - Extend existing relabelings

Changes Made

  • Updated podmonitor.yaml template to support configurable relabelings
  • Added new fields to values.yaml with documentation and examples
  • Created comprehensive test scenarios in debug.values.yaml
  • Maintained full backward compatibility

Example Usage

podMonitor:
  # Enable PodMonitor creation
  enabled: true

  # Global PodMonitor settings
  scrapeInterval: "15s"
  scrapeTimeout: "10s"
  controller:
    enabled: true
    namespaceSelector: |
      matchNames:
        - {{ .Release.Namespace }}
        - linkerd-viz
    metricRelabelings:
      - sourceLabels: [__name__]
        regex: 'controller_.*_duration_seconds_bucket'
        action: drop
    additionalRelabelings:
      - sourceLabels: [__meta_kubernetes_pod_label_environment]
        targetLabel: environment
        action: replace
      - action: labeldrop
        regex: 'pod_template_hash|controller_revision_hash'

  serviceMirror:
    enabled: true
    metricRelabelings:
      - sourceLabels: [__name__]
        regex: 'service_mirror_.*_errors_total'
        action: drop

    additionalRelabelings:
      - sourceLabels: [__meta_kubernetes_pod_label_source_cluster]
        targetLabel: source_cluster
        action: replace

  proxy:
    enabled: true
    metricRelabelings:
      - sourceLabels: [le]
        regex: '2\.[0-9]+|3\.[0-9]+|4\.[0-9]+|5\.[0-9]+'
        action: drop
    additionalRelabelings:
      - action: labeldrop
        regex: '__tmp_.*'
      - sourceLabels: [__meta_kubernetes_pod_label_pod_template_hash]
        targetLabel: deployment_generation
        action: replace

Testing

✅ Helm template renders correctly
✅ Deploys successfully to Kubernetes
✅ Prometheus processes custom relabelings
✅ Backward compatibility confirmed

**Problem**
Currently, PodMonitor resources for linkerd-control-plane components
(controller, serviceMirror, and proxy-injector) have fixed
metricRelabelings that cannot be customized. This limits operational
flexibility for:
- Adding organizational/environment-specific labels
- Implementing cardinality control strategies
- Customizing metric routing and processing

**Solution**
- Added configurable metricRelabelings to PodMonitor templates
- Added additionalRelabelings configuration option to extend default rules
- Maintained backward compatibility with existing configurations
- Enhanced values.yaml with comprehensive configuration examples
- Created debug infrastructure for testing and validation

**Validation**
- Verified helm template rendering with various configurations
- Tested additionalRelabelings functionality with sample rules
- Ensured backward compatibility with existing deployments
- Added comprehensive debug.values.yaml for development testing

Signed-off-by: Beza <[email protected]>
@bezarsnba bezarsnba requested a review from a team as a code owner November 2, 2025 19:24
@bezarsnba bezarsnba changed the title feat(helm): add configurable relabelings to PodMonitor resources feat(helm linkerd-control-plane): add configurable relabelings to PodMonitor resources Nov 2, 2025
@bezarsnba bezarsnba changed the title feat(helm linkerd-control-plane): add configurable relabelings to PodMonitor resources feat(helm): add configurable relabelings to PodMonitor resources linkerd-control-plane Nov 3, 2025
@bezarsnba bezarsnba changed the title feat(helm): add configurable relabelings to PodMonitor resources linkerd-control-plane feat(helm): add configurable relabelings to PodMonitor resources Nov 3, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant