Skip to content

[FEATURE]: Data Contract Implicit Checks to Include Schema Validation #1016

@vb-dbrks

Description

@vb-dbrks

Is there an existing issue for this?

  • I have searched the existing issues

Problem statement

the data contracts to dqx check generator should also generate schema validation checks based on what's defined in the ODCS data contracts

Proposed Solution

  1. Add a schema validation rule type: for each ODCS schema, generate one dataset-level rule that calls has_valid_schema with the expected schema derived from the contract (column names and Unity Catalog physicalTypes).
  2. Use strict mode only: exact column set, order, and types (no permissive/type-widening).
  3. Require every property to have Unity Catalog physicalType (e.g. STRING, INT, DECIMAL(10,2), ARRAY<STRING>). No ODCS→Unity mapping; missing or invalid physicalType raises an error.
  4. Make it opt-in by default: new parameter generate_schema_validation=True; set to False to skip these rules.
  5. Emit rules with user_metadata.rule_type: "schema_validation" and the expected DDL string so they can be filtered or reported separately.

Additional Context

No response

Metadata

Metadata

Assignees

Labels

DQX CoreFeature/Bug Related to DQX Core functionalityenhancementNew feature or request

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions