Hip: Schema validation with external dependencies

hips/hip-9999.md

@@ -0,0 +1,118 @@
+---
+hip: 9999
+title: "Schema Validation with External Dependencies"
+authors: [ "Benoit Tigeot <[email protected]>" ]
+created: "2025-10-08"
+type: "feature"
+status: "draft"
+---
+
+## Abstract
+
+A recent change in Helm's JSON schema validation library (see issue helm/helm#31170) has made schema validation stricter, causing failures when schemas contain unresolvable URN or URL references. While the immediate issue can be addressed by [making the compiler less strict](https://github.com/helm/helm/issues/31170), it highlights a need for a formal mechanism that allows `values.schema.json` to reference other schemas. In line with discussions for Helm v4, the goal is to enforce stricter schema validation. This proposal introduces a method for `values.schema.json` to reference other JSON schema files. It suggests the creation of a new file, `schema-dependencies.json`, in the chart's root directory to define the additional requirements.
+
+## Motivation
+
+Currently, `values.schema.json` is self-contained, which limits the ability to reuse common schema definitions across multiple charts. For instance, an organization may have a standard definition for a "CPU" or "Liveness Probes" resource that they want to use in all of their charts. Without the ability to reference external schemas, chart authors are forced to duplicate these definitions in every chart, leading to inconsistencies and maintenance overhead.
+
+This proposal solves this problem by providing a mechanism to reference external schemas, making it easier to create and maintain complex charts with shared schema definitions.
+
+## Rationale
+
+Several approaches were considered:
+
+1.  **Convention-based `schemas` directory**: This approach, while simple, was deemed too implicit and less flexible than an explicit mapping file. Also a folder only for validations doesn't seem the right approach.
+
+2.  **`schema-dependencies.json` file**: This approach was chosen as it provides an explicit, flexible, and decoupled way to manage schema dependencies. It also permits referencing remote definitions.
+
+## Specification
+
+A new file named `schema-dependencies.json` will be recognized in the root of a chart. This file will contain a JSON object that maps a URI prefix to a resolution target. This target can be either a relative path to a local file within the chart, or a full URL for a remote schema.
+
+This allows for aliasing both local and remote schemas.
+
+### Example
+
+**1. Define local and remote mappings:**
+
+The `schema-dependencies.json` file can define prefixes for both local paths and remote URLs.
+
+```json
+// schema-dependencies.json
+{
+  "local-defs": "schemas/",
+  "k8s-api": "https://kubernetesjsonschema.dev/v1.28.2/_definitions.json"
+}
+```
+
+**2. Use the prefixes in `values.schema.json`:**
+
+```json
+// values.schema.json
+{
+  "title": "My Chart Values",
+  "type": "object",
+  "properties": {
+    "resources": {
+      "$ref": "local-defs/resources.json#/definitions/resourceRequirements"
+    },
+    "livenessProbe": {
+      "$ref": "k8s-api#/definitions/io.k8s.api.core.v1.Probe"
+    }
+  }
+}
+```
+
+### Resolution
+
+When Helm validates the schema, the custom loader will:
+1.  Check if a `$ref` value starts with a prefix defined in `schema-dependencies.json`.
+2.  If a match is found (e.g., `k8s-api`): It replaces the prefix with the mapped target. For a remote target, the result is a full URL (`https://...`) that is then fetched. For a local target, it resolves to a file within the chart.
+3.  If no prefix matches, the `$ref` is passed to Helm's default loader. This allows you to use both your custom aliases and direct URLs (like `https://...`) in the same `values.schema.json` file without needing to create an alias for every remote schema.
+
+## Backwards compatibility
+
+This is a breaking change in v4. In v4, charts that use `values.schema.json` to reference other schemas (e.g., via a `$ref` to a non-standard URI) without a corresponding `schema-dependencies.json` file will fail validation.
+
+## Security implications
+
+To prevent security risks, such as accessing arbitrary files on the filesystem, the paths in `schema-dependencies.json` must be relative and will be constrained to the chart's directory. Any path attempting to traverse outside the chart will be rejected.
+
+Additionally, while schemas can still be fetched from accessible remote URLs via HTTP/S, the initial implementation of this feature will not support authentication for these remote endpoints. Fetching from private or authenticated sources is out of scope for this proposal and could be considered in the future.
+
+## How to teach this
+
+The Helm documentation will be updated to describe the `schema-dependencies.json` file, its format, and how to use it to reference external schemas in `values.schema.json`. Examples will be provided to illustrate its usage.
+
+## Reference implementation
+
+No reference implementation exists yet. One should be proposed before the draft is accepted.
+
+## Implementation Strategy
+
+The implementation can be localized within the Helm codebase by leveraging the intended extension points of the `jsonschema` library. The approach is inspired by the `--map` [functionality](https://github.com/santhosh-tekuri/jsonschema/commit/ed65924244c7fad4784aebfafcad7aa1ece6d2c2) in the library's own CLI, but would be more powerful.
+
+1.  **Extending the Library's Loader Pattern**: The `jsonschema` library is designed to be extensible with custom loaders. Its CLI uses a custom loader to map URL prefixes to local directories. We will adopt and extend this pattern in Helm to create a loader that can map prefixes to **both local paths and remote URLs**.
+
+2.  **Minimal Changes to Helm**: The implementation would be localized almost entirely within Helm's `pkg/chart/common/util/jsonschema.go` file. The logic for the new custom loader would be:
+
+    *   Before compiling the schema, check for `schema-dependencies.json` and parse the mappings.
+    *   For each `$ref`, check if it starts with a mapped prefix.
+    *   If it matches, inspect the corresponding target value from the mapping:
+        *   If the target is a local path, resolve the file from within the chart.
+        *   If the target is a remote URL (`https://...`), construct a new full URL for the schema and fetch it.
+    *   If no prefix matches, Helm's default loader will then attempt to handle the URL.
+    *   Pass this new loader to the `jsonschema.Compiler`.
+
+## Rejected ideas
+
+- **Convention-based `schemas` directory**: This idea was rejected in favor of a more explicit and flexible mapping file.
+
+## Open issues
+
+- https://github.com/helm/helm/issues/31170
+
+## References
+
+- [santhosh-tekuri/jsonschema](https://github.com/santhosh-tekuri/jsonschema)
+- [silencing schema loading error](https://github.com/helm/helm/pull/31240)