#20463 - Retention for test case results and profile data #23988
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
github-actions
bot
added
Ingestion
safe to test
There was a problem hiding this comment.
Pull Request Overview
This PR implements retention policies for test case results and profile data to manage data growth over time. The default retention period is set to 1440 days (approximately 4 years) for both data types, allowing users to configure these values as needed.
Key Changes:
- Added retention configuration fields for test case results and profile data to the Data Retention Application schema
- Implemented cleanup methods in the DataRetention class to delete old test case results and profile data based on retention periods
- Added database migration scripts to set default retention values for existing installations
Reviewed Changes
Copilot reviewed 10 out of 10 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| DataRetentionApplication.json (UI schema) | Added testCaseResultsRetentionPeriod and profileDataRetentionPeriod fields with 1440-day defaults |
| DataRetentionApplication.md | Added documentation sections for the new retention period configuration fields |
| dataRetentionConfiguration.json (spec) | Added retention period fields to the application configuration schema definition |
| EntityProfileResourceTest.java | Added test to verify profile data deletion by timestamp cutoff |
| TestCaseResourceTest.java | Added test to verify test case results deletion by timestamp cutoff |
| DataRetentionApplication.json (service config) | Updated default application configuration with new retention period values |
| EntityTimeSeriesDAO.java | Added deleteTestCaseResultsBeforeCutOff method for time-series data cleanup |
| DataRetention.java | Implemented cleanup logic for test case results and profile data with DAO references |
| postDataMigrationSQLScript.sql (postgres) | Added migration to set default retention values for existing DataRetentionApplication |
| postDataMigrationSQLScript.sql (mysql) | Added migration to set default retention values for existing DataRetentionApplication |
openmetadata-service/src/main/resources/json/data/app/DataRetentionApplication.json
Show resolved
Hide resolved
manerow
previously approved these changes
Oct 23, 2025
Jest test CoverageUI tests summary
|
|
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Describe your changes:
Fixes #20463
implement retention for test case results and profile data. The default retention period is 1440 days (or 4 years worth of data) -- configurable by users.
Type of change:
Checklist:
Fixes <issue-number>: <short explanation>Improvement