Skip to content

add validation for each catalog properly #478

@github-actions

Description

@github-actions

Description

Our application supports multiple external catalogs for metadata management (Glue, REST-based catalogs, Hive, and JDBC), but it currently lacks a consistent and robust mechanism to validate the configuration parameters required for each specific catalog type. This can lead to runtime failures and a poor user experience due to misconfigurations.

Problems Identified

- No standardized validation framework for catalog-specific connection parameters.
- Configuration errors are only caught at runtime during connection attempts, not at initialization.
- Each catalog type (Glue, REST, Hive, JDBC) has unique required fields and validation rules that are not being enforced.
- Inconsistent user experience; the validation error messages are not clear or actionable.

Proposed Solution

- Implement a catalog validation function that uses a strategy pattern, with a dedicated validator for each supported catalog type.
- Perform validation immediately upon check command
- Return clear, catalog-specific error messages that guide the user toward fixing their configuration.

Targeted Catalogs

- AWS Glue
- REST-based Catalogs (Polaris, Lakekeeper, Unity, S3 Tables, Nessie)
- Hive Metastore
- JDBC-based Catalogs

Code where action needs to be taken

// TODO: add validation for each catalog properly
type Config struct {
// S3-compatible Storage Configuration
Region string `json:"aws_region,omitempty"`


This issue was generated by todo-issue based on a TODO comment in 794965f. It's been assigned to @hash-data because they committed the code.

Metadata

Metadata

Assignees

Labels

good second issueafter you have solved good first issuehacktoberfestIssues open for Hacktoberfest contributorsimprovementimprovements in existing flows

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions