-
Notifications
You must be signed in to change notification settings - Fork 131
Open
Labels
good second issueafter you have solved good first issueafter you have solved good first issuehacktoberfestIssues open for Hacktoberfest contributorsIssues open for Hacktoberfest contributorsimprovementimprovements in existing flowsimprovements in existing flows
Description
Description
Our application supports multiple external catalogs for metadata management (Glue, REST-based catalogs, Hive, and JDBC), but it currently lacks a consistent and robust mechanism to validate the configuration parameters required for each specific catalog type. This can lead to runtime failures and a poor user experience due to misconfigurations.Problems Identified
- No standardized validation framework for catalog-specific connection parameters.- Configuration errors are only caught at runtime during connection attempts, not at initialization.
- Each catalog type (Glue, REST, Hive, JDBC) has unique required fields and validation rules that are not being enforced.
- Inconsistent user experience; the validation error messages are not clear or actionable.
Proposed Solution
- Implement a catalog validation function that uses a strategy pattern, with a dedicated validator for each supported catalog type.- Perform validation immediately upon check command
- Return clear, catalog-specific error messages that guide the user toward fixing their configuration.
Targeted Catalogs
- AWS Glue- REST-based Catalogs (Polaris, Lakekeeper, Unity, S3 Tables, Nessie)
- Hive Metastore
- JDBC-based Catalogs
Code where action needs to be taken
olake/destination/iceberg/config.go
Lines 26 to 29 in 794965f
// TODO: add validation for each catalog properly | |
type Config struct { | |
// S3-compatible Storage Configuration | |
Region string `json:"aws_region,omitempty"` |
This issue was generated by todo-issue based on a TODO
comment in 794965f. It's been assigned to @hash-data because they committed the code.
Metadata
Metadata
Assignees
Labels
good second issueafter you have solved good first issueafter you have solved good first issuehacktoberfestIssues open for Hacktoberfest contributorsIssues open for Hacktoberfest contributorsimprovementimprovements in existing flowsimprovements in existing flows