Skip to content

Conversation

davidpepm
Copy link

Pull Request Summary: Add Bucket and Task Creation Support to Helm Chart

Overview

This pull request enhances the Helm chart for InfluxDB by introducing the ability to create aggregated buckets and tasks for data aggregation. The feature addresses performance considerations by enabling the creation of pre-aggregated data.

Key Changes

  1. New Aggregated Buckets

    • Adds support for creating the following buckets:
      • default_agg_topic
      • default_agg_stats
      • default_agg_kpi
    • Each bucket is configured with a retention policy of "forever" (0s).
  2. Data Aggregation Task

    • Introduces a scheduled task to aggregate data into the new buckets.
    • The task runs every 6 hours and processes data from the default bucket into the aggregated buckets.
  3. Helm Chart Modifications

    • values.yaml: Includes new configuration to toggle bucket and task creation using the createBucketsAndTask.enabled flag.
    • ConfigMap: Stores the aggregation task script for maintainability.
    • Job: Automates bucket and task creation during deployment via a Kubernetes Job.

Testing and Validation

  • I am unable to set up the required Kubernetes environment (e.g., Minikube) for testing the implementation locally. Assistance is needed to validate these changes in a test environment.

Next Steps

  • Review the changes and test the setup to confirm functionality.
  • Merge this pull request into the main branch after testing is successful.

Related Issue

This pull request addresses [Issue #691].
Including the keyword Closes #691 in the pull request description will link the issue and close it upon merging.


Request for Assistance

  • Testing: Help is needed to validate the changes in a Kubernetes environment since I lack the setup and expertise in tools like Minikube.
  • Merge: Once reviewed and validated, either grant me permission to merge or let someone with access proceed with the merge.

Thanks for reviewing this!

- Added configuration for `default_agg_topic`, `default_agg_stats`, and `default_agg_kpi` buckets with retention policy set to forever.
- Added support for task creation to aggregate data into the new buckets.(the flux query to be added)
- Introduced `createBucketsAndTask.enabled` flag to toggle this feature.
- Implemented task script to populate data into these buckets with a 6-hour interval.
- Introduced a ConfigMap template to store the task script (`task.flux`) from values.yaml.
- Ensures the task script can be used by the job for bucket and task creation.
- Added a Job template to create aggregated buckets and the task using the provided ConfigMap script.
- Ensures buckets and tasks are initialized during deployment.
- Supports toggling the feature via `createBucketsAndTask.enabled`.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Extend InfluxDB2 Helm Chart for Aggregated Buckets and Task Creation to Support Grafana Performance Optimization

2 participants