Fully integrate JSON mapping into the relational model#38038
Fully integrate JSON mapping into the relational model#38038AndriySvyryd merged 7 commits intomainfrom
Conversation
There was a problem hiding this comment.
Pull request overview
This PR advances EF Core’s relational JSON support by modeling JSON elements (objects/arrays/scalars) directly in the relational model and by replacing string JSONPATH usage in partial updates with a structured JsonPath, enabling providers (e.g., PostgreSQL) to consume path components more flexibly.
Changes:
- Introduces
JsonPath/JsonPathSegmentand updates JSON partial update SQL generation to use the structured path instead of a string. - Adds relational-model JSON element types (
IRelationalJsonElement+ object/array/scalar), JSON element mappings, and populates JSON element trees on JSON columns. - Extends
FindColumnAPIs to acceptIPropertyBase(including navigations and complex properties) and updates compiled model/scaffolding baselines and tests accordingly.
Reviewed changes
Copilot reviewed 46 out of 47 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| test/EFCore.Tests/Infrastructure/JsonPathTest.cs | Adds unit tests for JsonPath and JsonPathSegment formatting/behavior. |
| test/EFCore.SqlServer.FunctionalTests/Scaffolding/Baselines/ComplexTypes/DbContextModelBuilder.cs | Updates scaffolding baseline to emit JSON element tree initialization. |
| test/EFCore.SqlServer.FunctionalTests/Scaffolding/Baselines/BigModel_with_JSON_columns/DbContextModelBuilder.cs | Updates scaffolding baseline to emit JSON element tree initialization (big model). |
| test/EFCore.Sqlite.Tests/Migrations/SqliteMigrationAnnotationProviderTest.cs | Adjusts test for updated column mapping typing and removes BOM. |
| test/EFCore.Sqlite.FunctionalTests/Scaffolding/Baselines/BigModel_with_JSON_columns/DbContextModelBuilder.cs | Updates scaffolding baseline to emit JSON element tree initialization (SQLite). |
| test/EFCore.Relational.Tests/Metadata/RelationalModelTest.cs | Adds tests asserting JSON element trees/mappings and FindColumn behavior. |
| test/EFCore.Relational.Specification.Tests/Scaffolding/CompiledModelRelationalTestBase.cs | Adds assertions around JSON element metadata in compiled model scenarios. |
| src/EFCore/Infrastructure/JsonPathSegment.cs | Introduces structured JSON path segment (property vs array placeholder). |
| src/EFCore/Infrastructure/JsonPath.cs | Introduces structured JSON path (segments + ordinals) with formatting helpers. |
| src/EFCore.SqlServer/Update/Internal/SqlServerUpdateSqlGenerator.cs | Uses JsonPath.AppendTo and structured root detection for JSON updates. |
| src/EFCore.SqlServer/Query/Internal/SqlServerQueryableMethodTranslatingExpressionVisitor.cs | Switches JSON container column lookup to FindColumn. |
| src/EFCore.Sqlite.Core/Update/Internal/SqliteUpdateSqlGenerator.cs | Uses JsonPath.AppendTo and structured root detection for JSON updates. |
| src/EFCore.Relational/Update/ModificationCommand.cs | Builds structured JsonPath for JSON partial updates using element metadata/mappings. |
| src/EFCore.Relational/Update/IColumnModification.cs | Changes JsonPath from string to JsonPath?. |
| src/EFCore.Relational/Update/ColumnModificationParameters.cs | Changes JsonPath from string to JsonPath? and updates ctor signature. |
| src/EFCore.Relational/Update/ColumnModification.cs | Changes JsonPath from string to JsonPath?. |
| src/EFCore.Relational/Query/RelationalQueryableMethodTranslatingExpressionVisitor.ExecuteUpdate.cs | Updates container column resolution to FindColumn with null handling. |
| src/EFCore.Relational/Metadata/RelationalAnnotationNames.cs | Adds JsonElementMappings annotation name. |
| src/EFCore.Relational/Metadata/JsonValueType.cs | Adds enum to capture scalar JSON value kind. |
| src/EFCore.Relational/Metadata/IView.cs | Broadens FindColumn parameter type to IPropertyBase. |
| src/EFCore.Relational/Metadata/ITableBase.cs | Broadens FindColumn parameter type to IPropertyBase. |
| src/EFCore.Relational/Metadata/ITable.cs | Broadens FindColumn parameter type to IPropertyBase. |
| src/EFCore.Relational/Metadata/IStoreFunction.cs | Broadens FindColumn parameter type to IPropertyBase. |
| src/EFCore.Relational/Metadata/ISqlQuery.cs | Broadens FindColumn parameter type to IPropertyBase. |
| src/EFCore.Relational/Metadata/IRelationalJsonScalar.cs | Adds public interface for scalar JSON elements. |
| src/EFCore.Relational/Metadata/IRelationalJsonObject.cs | Adds public interface for object JSON elements. |
| src/EFCore.Relational/Metadata/IRelationalJsonElement.cs | Adds public interface for JSON elements (name/path/column/mappings). |
| src/EFCore.Relational/Metadata/IRelationalJsonArray.cs | Adds public interface for array JSON elements. |
| src/EFCore.Relational/Metadata/Internal/View.cs | Implements FindColumn(IPropertyBase) with navigation/complex-property support. |
| src/EFCore.Relational/Metadata/Internal/TableBase.cs | Implements FindColumn(IPropertyBase) with navigation/complex-property support. |
| src/EFCore.Relational/Metadata/Internal/Table.cs | Implements FindColumn(IPropertyBase) with navigation/complex-property support. |
| src/EFCore.Relational/Metadata/Internal/StoreStoredProcedure.cs | Updates FindColumn override for broadened IPropertyBase signature. |
| src/EFCore.Relational/Metadata/Internal/StoreFunction.cs | Updates FindColumn override for broadened IPropertyBase signature. |
| src/EFCore.Relational/Metadata/Internal/SqlQuery.cs | Updates FindColumn override for broadened IPropertyBase signature. |
| src/EFCore.Relational/Metadata/Internal/RelationalModel.cs | Builds JSON element trees, adds JSON element mappings, and sets JsonElement on JSON columns. |
| src/EFCore.Relational/Metadata/Internal/RelationalJsonScalar.cs | Implements internal scalar JSON element. |
| src/EFCore.Relational/Metadata/Internal/RelationalJsonObject.cs | Implements internal object JSON element with ordered properties. |
| src/EFCore.Relational/Metadata/Internal/RelationalJsonElement.cs | Implements internal base type for JSON elements (path/nullable/mappings). |
| src/EFCore.Relational/Metadata/Internal/RelationalJsonArray.cs | Implements internal array JSON element with element-type linkage. |
| src/EFCore.Relational/Metadata/Internal/JsonElementMapping.cs | Implements internal mapping between model property and JSON element. |
| src/EFCore.Relational/Metadata/Internal/ColumnBase.cs | Adds JsonElement property slot to columns. |
| src/EFCore.Relational/Metadata/IJsonElementMapping.cs | Adds public interface for property-to-JSON-element mappings. |
| src/EFCore.Relational/Metadata/IColumnBase.cs | Exposes JsonElement on columns (default null for non-JSON columns). |
| src/EFCore.Relational/Metadata/Conventions/RelationalRuntimeModelConvention.cs | Removes JSON element mapping annotations from runtime annotations dictionary (consistent with other mapping annotations). |
| src/EFCore.Relational/Extensions/RelationalPropertyExtensions.cs | Generalizes mapping accessors to IPropertyBase and adds GetJsonElementMappings(). |
| src/EFCore.Relational/Design/Internal/RelationalCSharpRuntimeAnnotationCodeGenerator.cs | Emits compiled-model code to recreate JSON element trees on columns. |
92c7cbb to
3f990fc
Compare
roji
left a comment
There was a problem hiding this comment.
Here's a first review @AndriySvyryd.
- From this PR specifically it's difficult to see the immediate, concrete value (in terms of simplifying/improving code in update/query). Are you planning more significant work on update (since the current changes seem minimal and slightly orthogonal too)?
- I'm noting that we're kinda redoing a version of JSON Schema here, though we have our own concerns (our model is fully traversable in all directions, contains mapping to IPropertyBase...).
- I'd want to understand why JSON scalars have a only string/number/bool typing, as opposed to our full known type mapping.
- Ideally we'd have JSON-mapped primitive collections using the same JSON modeling in the relational model, what do you think?
- I can't help but think of other non-JSON document-style mappings (e.g. PG arrays, composite types...). Since the relational model isn't extensible (I think?), the PG provider can't integrate that into the relational model and has to make do with the current way. Which maybe raises the question of how useful/necessary JSON modeling really is in the relational model... We can discuss.
...Core.Relational/Query/RelationalQueryableMethodTranslatingExpressionVisitor.ExecuteUpdate.cs
Show resolved
Hide resolved
...Core.Relational/Query/RelationalQueryableMethodTranslatingExpressionVisitor.ExecuteUpdate.cs
Show resolved
Hide resolved
src/EFCore.SqlServer/Update/Internal/SqlServerUpdateSqlGenerator.cs
Outdated
Show resolved
Hide resolved
This will be used when implementing the features linked from #36646. Currently it indeed doesn't reduce much of the complexity because we only support simple JSON mappings.
Those are all the possible JSON scalar types, I'm not sure yet whether this property will be useful, only adding it for completeness. You can get the type mapping from the property mapping.
Sure, though I'm not sure yet how it would be used
It's extensible in the same way as the regular model - everything is IAnnotatable |
src/EFCore.Relational/Metadata/Internal/RelationalJsonElement.cs
Outdated
Show resolved
Hide resolved
Then maybe introduce this later based on need, rather than up-front for completeness?
But isn't the creation of the relational model all in private static methods in RelationalModel.cs? How would I go about adding new things (or even tweaking existing things)? |
c14353a to
2315125
Compare
2315125 to
6b035db
Compare
| SELECT "o0"."OwnedReferenceLeaf" ->> 'SomethingSomething' AS "c", "o0"."Date" | ||
| FROM ( | ||
| SELECT "o"."value" ->> 'Date' AS "Date", "o"."value" ->> 'Enum' AS "Enum", "o"."value" ->> 'Fraction' AS "Fraction", "o"."value" ->> 'Id' AS "Id", "o"."value" ->> 'OwnedReferenceLeaf' AS "OwnedReferenceLeaf" | ||
| SELECT "o"."value" ->> 'Date' AS "Date", "o"."value" ->> 'OwnedReferenceLeaf' AS "OwnedReferenceLeaf" |
There was a problem hiding this comment.
@roji Should we be at all concerned about this change or just count it as a win?
src/EFCore.Relational/Design/Internal/RelationalCSharpRuntimeAnnotationCodeGenerator.cs
Show resolved
Hide resolved
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
API review baseline changesThis PR modified one or more
|
Change partial update JSON path to be structured instead of a string JSONPATH
Fixes #36646
Fixes #32185