Skip to content

Add possibility of configuration for control group Id, Coordinator and Worker prefix in databricks iceberg connector #342

@ayyap-2k

Description

@ayyap-2k

Proposed Change
Feature Request / Improvement
Summary:
Databricks Iceberg Kafka Connect should be able to allow configuration of the naming of structures that need to be compliant with ACL policies in the environment where it is being installed.

Motivation
Currently, Databricks Iceberg does not allow to have custom names for control group id, transactional id, Coordinator, Worker in the kafka-connect since the names are hard coded, which causes problem in the using of the connector because it is blocked by ACL policies that are expecting a specific prefix in the structures names.

This is a problem specially for cases where the ACL policies can not be changed to accommodate the kafka-connect names.

Proposed Implementation:

Introduce a new sink property that define the prefix
Add method to IcebergSinkConfig to retrieve the value of the property from the sink configuration
Use the prefix as the name when constructing instances of classes that extend Channel (e.g., Coordinator or Worker)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions