Skip to content

[FEATURE] Move OpenSearchSparkSqlDialect out of test scope for reuse #4820

@dai-chen

Description

@dai-chen

Is your feature request related to a problem?

Yes. The custom Calcite dialect OpenSearchSparkSqlDialect is currently defined under the test source folder and therefore cannot be reused by other modules or external projects from production code: https://github.com/opensearch-project/sql/blob/main/ppl/src/test/java/org/opensearch/sql/ppl/calcite/OpenSearchSparkSqlDialect.java.

What solution would you like?

  1. Move OpenSearchSparkSqlDialect from ppl/src/test/java/... into a non-test source set (for example somewhere under ppl/src/main/java/... or another appropriate shared module).
  2. Reuse it in unified query transpile API in api module in future.

What alternatives have you considered?

  • Copying the class into the PPL–Spark project: This works short term but creates code duplication and the risk of divergence whenever the dialect behavior changes in the SQL plugin.
  • Contributing all fixes to upstream Calcite: This is a slower process and may not capture all OpenSearch-/PPL-specific behavior we currently rely on.

Do you have any additional context?

This change is intended to support ongoing PPL unification work where PPL queries are parsed and planned via Calcite and then executed on Spark. Find more background, see the meta issue opensearch-project/opensearch-spark#1136.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    Status

    Not Started

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions