Skip to content

Commit 0c4dc2a

Browse files
kcp-ci-botkcp-ci-bot
authored
Merge pull request #19 from kcp-dev/re-enable-mutations
✨ Add basic support for mutating object contents during syncing
2 parents 491adf3 + 5d7e49e commit 0c4dc2a

File tree

18 files changed

+459
-164
lines changed

18 files changed

+459
-164
lines changed

‎deploy/crd/kcp.io/syncagent.kcp.io_publishedresources.yaml‎

Lines changed: 156 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -149,6 +149,84 @@ spec:
149149
type: object
150150
x-kubernetes-map-type: atomic
151151
type: object
152+
mutation:
153+
description: |-
154+
Mutation allows to configure "rewrite rules" to modify the objects in both
155+
directions during the synchronization.
156+
properties:
157+
spec:
158+
items:
159+
properties:
160+
delete:
161+
properties:
162+
path:
163+
type: string
164+
required:
165+
- path
166+
type: object
167+
regex:
168+
properties:
169+
path:
170+
type: string
171+
pattern:
172+
description: |-
173+
Pattern can be left empty to simply replace the entire value with the
174+
replacement.
175+
type: string
176+
replacement:
177+
type: string
178+
required:
179+
- path
180+
type: object
181+
template:
182+
properties:
183+
path:
184+
type: string
185+
template:
186+
type: string
187+
required:
188+
- path
189+
- template
190+
type: object
191+
type: object
192+
type: array
193+
status:
194+
items:
195+
properties:
196+
delete:
197+
properties:
198+
path:
199+
type: string
200+
required:
201+
- path
202+
type: object
203+
regex:
204+
properties:
205+
path:
206+
type: string
207+
pattern:
208+
description: |-
209+
Pattern can be left empty to simply replace the entire value with the
210+
replacement.
211+
type: string
212+
replacement:
213+
type: string
214+
required:
215+
- path
216+
type: object
217+
template:
218+
properties:
219+
path:
220+
type: string
221+
template:
222+
type: string
223+
required:
224+
- path
225+
- template
226+
type: object
227+
type: object
228+
type: array
229+
type: object
152230
naming:
153231
description: |-
154232
Naming can be used to control how the namespace and names for local objects
@@ -251,6 +329,84 @@ spec:
251329
kind:
252330
description: ConfigMap or Secret
253331
type: string
332+
mutation:
333+
description: |-
334+
Mutation configures optional transformation rules for the related resource.
335+
Status mutations are only performed when the related resource originates in kcp.
336+
properties:
337+
spec:
338+
items:
339+
properties:
340+
delete:
341+
properties:
342+
path:
343+
type: string
344+
required:
345+
- path
346+
type: object
347+
regex:
348+
properties:
349+
path:
350+
type: string
351+
pattern:
352+
description: |-
353+
Pattern can be left empty to simply replace the entire value with the
354+
replacement.
355+
type: string
356+
replacement:
357+
type: string
358+
required:
359+
- path
360+
type: object
361+
template:
362+
properties:
363+
path:
364+
type: string
365+
template:
366+
type: string
367+
required:
368+
- path
369+
- template
370+
type: object
371+
type: object
372+
type: array
373+
status:
374+
items:
375+
properties:
376+
delete:
377+
properties:
378+
path:
379+
type: string
380+
required:
381+
- path
382+
type: object
383+
regex:
384+
properties:
385+
path:
386+
type: string
387+
pattern:
388+
description: |-
389+
Pattern can be left empty to simply replace the entire value with the
390+
replacement.
391+
type: string
392+
replacement:
393+
type: string
394+
required:
395+
- path
396+
type: object
397+
template:
398+
properties:
399+
path:
400+
type: string
401+
template:
402+
type: string
403+
required:
404+
- path
405+
- template
406+
type: object
407+
type: object
408+
type: array
409+
type: object
254410
origin:
255411
description: '"service" or "kcp"'
256412
type: string

‎docs/publish-resources.md‎

Lines changed: 0 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -150,7 +150,6 @@ spec:
150150
name: "cert-$remoteNamespaceHash-$remoteNameHash"
151151
```
152152

153-
<!--
154153
### Mutation
155154

156155
Besides projecting the type meta, changes to object contents are also nearly always required.
@@ -182,7 +181,6 @@ spec:
182181
- regex: ...
183182
template: ...
184183
delete: ...
185-
rudi: ...
186184
```
187185

188186
#### Regex
@@ -218,19 +216,6 @@ delete:
218216
This mutation simply removes the value at the given path from the document. JSON path is the
219217
usual path, without a leading dot.
220218

221-
#### Rudi
222-
223-
```yaml
224-
rudi:
225-
script: |
226-
(set! .metadata.name (concat "-" [$localObj.metadata.name "foo"]))
227-
```
228-
229-
This mutation runs a [Rudi](https://github.com/xrstf/rudi) program on the document. Note that in
230-
Rudi, in contrast to all the other mutation types, JSON paths _do_ begin with leading dot if you
231-
refer to the main document. Just `metadata.name` would be invalid.
232-
-->
233-
234219
### Related Resources
235220

236221
The processing of resources on the service cluster often leads to additional resources being

‎go.mod‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,6 @@ require (
1717
github.com/tidwall/gjson v1.18.0
1818
github.com/tidwall/sjson v1.2.5
1919
go.uber.org/zap v1.27.0
20-
go.xrstf.de/rudi v0.0.9
2120
k8c.io/reconciler v0.5.0
2221
k8s.io/api v0.31.2
2322
k8s.io/apiextensions-apiserver v0.31.2

‎go.sum‎

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -359,8 +359,6 @@ go.uber.org/multierr v1.11.0/go.mod h1:20+QtiLqy0Nd6FdQB9TLXag12DsQkrbs3htMFfDN8
359359
go.uber.org/zap v1.10.0/go.mod h1:vwi/ZaCAaUcBkycHslxD9B2zi4UTXhF60s6SWpuDF0Q=
360360
go.uber.org/zap v1.27.0 h1:aJMhYGrd5QSmlpLMr2MftRKl7t8J8PTZPA732ud/XR8=
361361
go.uber.org/zap v1.27.0/go.mod h1:GB2qFLM7cTU87MWRP2mPIjqfIDnGu+VIO4V/SdhGo2E=
362-
go.xrstf.de/rudi v0.0.9 h1:ACTYQJcq8Kk24yPGDhApMVwGH5JrgBLQdvkrwJXUt3c=
363-
go.xrstf.de/rudi v0.0.9/go.mod h1:ERo0X1RhWc5J8FFlNWx9i0j3ZEvrRD/YXqVvo+q1rfo=
364362
golang.org/x/crypto v0.0.0-20180904163835-0709b304e793/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
365363
golang.org/x/crypto v0.0.0-20181029021203-45a5f77698d3/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
366364
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=

‎internal/controller/sync/controller.go‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -93,7 +93,7 @@ func Create(
9393
}
9494

9595
// create the syncer that holds the meat&potatoes of the synchronization logic
96-
mutator := mutation.NewMutator(nil) // pubRes.Spec.Mutation
96+
mutator := mutation.NewMutator(pubRes.Spec.Mutation)
9797
syncer, err := sync.NewResourceSyncer(log, localManager.GetClient(), virtualWorkspaceCluster.GetClient(), pubRes, localCRD, apiExportName, mutator, stateNamespace, agentName)
9898
if err != nil {
9999
return nil, fmt.Errorf("failed to create syncer: %w", err)

‎internal/mutation/mutation.go‎

Lines changed: 1 addition & 30 deletions
Original file line numberDiff line numberDiff line change
@@ -28,7 +28,6 @@ import (
2828
"github.com/Masterminds/sprig/v3"
2929
"github.com/tidwall/gjson"
3030
"github.com/tidwall/sjson"
31-
"go.xrstf.de/rudi"
3231

3332
syncagentv1alpha1 "github.com/kcp-dev/api-syncagent/sdk/apis/syncagent/v1alpha1"
3433
)
@@ -46,11 +45,6 @@ func ApplyResourceMutations(value any, mutations []syncagentv1alpha1.ResourceMut
4645
}
4746

4847
func ApplyResourceMutation(value any, mut syncagentv1alpha1.ResourceMutation, ctx *TemplateMutationContext) (any, error) {
49-
// for Rudi scripts we can skip all the JSON encoding/decoding
50-
if mut.Rudi != nil {
51-
return applyResourceRudiMigration(value, *mut.Rudi, ctx)
52-
}
53-
5448
// encode current value as JSON
5549
encoded, err := json.Marshal(value)
5650
if err != nil {
@@ -82,33 +76,10 @@ func applyResourceMutationToJSON(jsonData string, mut syncagentv1alpha1.Resource
8276
case mut.Regex != nil:
8377
return applyResourceRegexMutation(jsonData, *mut.Regex)
8478
default:
85-
return "", errors.New("must use either Rudi, regex, template or delete mutation")
79+
return "", errors.New("must use either regex, template or delete mutation")
8680
}
8781
}
8882

89-
func applyResourceRudiMigration(value any, mut syncagentv1alpha1.ResourceRudiMutation, ctx *TemplateMutationContext) (any, error) {
90-
program, err := rudi.Parse("myscript", mut.Script)
91-
if err != nil {
92-
return nil, fmt.Errorf("invalid script: %w", err)
93-
}
94-
95-
funcs := rudi.NewBuiltInFunctions()
96-
vars := rudi.NewVariables()
97-
98-
if ctx != nil {
99-
vars.
100-
Set("localObj", ctx.LocalObject).
101-
Set("remoteObj", ctx.RemoteObject)
102-
}
103-
104-
processed, _, err := program.Run(value, vars, funcs)
105-
if err != nil {
106-
return nil, fmt.Errorf("script failed: %w", err)
107-
}
108-
109-
return processed, nil
110-
}
111-
11283
func applyResourceDeleteMutation(jsonData string, mut syncagentv1alpha1.ResourceDeleteMutation) (string, error) {
11384
jsonData, err := sjson.Delete(jsonData, mut.Path)
11485
if err != nil {

‎internal/mutation/mutation_test.go‎

Lines changed: 0 additions & 83 deletions
Original file line numberDiff line numberDiff line change
@@ -187,89 +187,6 @@ func TestApplyResourceMutation(t *testing.T) {
187187
},
188188
expected: `{"spec":[1,3]}`,
189189
},
190-
191-
// Rudi
192-
193-
{
194-
name: "Rudi: empty script",
195-
inputData: `{"spec":{"secretName":"foo"}}`,
196-
mutation: syncagentv1alpha1.ResourceMutation{
197-
Rudi: &syncagentv1alpha1.ResourceRudiMutation{
198-
Script: `.`,
199-
},
200-
},
201-
expected: `{"spec":{"secretName":"foo"}}`,
202-
},
203-
{
204-
name: "Rudi: set one new key",
205-
inputData: `{"spec":{"secretName":"foo"}}`,
206-
mutation: syncagentv1alpha1.ResourceMutation{
207-
Rudi: &syncagentv1alpha1.ResourceRudiMutation{
208-
Script: `(set! .foo "bar")`,
209-
},
210-
},
211-
expected: `{"foo":"bar","spec":{"secretName":"foo"}}`,
212-
},
213-
{
214-
name: "Rudi: update existing key",
215-
inputData: `{"spec":{"secretName":"foo"}}`,
216-
mutation: syncagentv1alpha1.ResourceMutation{
217-
Rudi: &syncagentv1alpha1.ResourceRudiMutation{
218-
Script: `(set! .spec.secretName "bar")`,
219-
},
220-
},
221-
expected: `{"spec":{"secretName":"bar"}}`,
222-
},
223-
{
224-
name: "Rudi: remove a key",
225-
inputData: `{"spec":{"secretName":"foo"}}`,
226-
mutation: syncagentv1alpha1.ResourceMutation{
227-
Rudi: &syncagentv1alpha1.ResourceRudiMutation{
228-
Script: `(delete! .spec.secretName)`,
229-
},
230-
},
231-
expected: `{"spec":{}}`,
232-
},
233-
{
234-
name: "Rudi: result value is ignored, only document counts",
235-
inputData: `{"spec":{"secretName":"foo"}}`,
236-
mutation: syncagentv1alpha1.ResourceMutation{
237-
Rudi: &syncagentv1alpha1.ResourceRudiMutation{
238-
Script: `(delete! .spec.secretName) false`,
239-
},
240-
},
241-
expected: `{"spec":{}}`,
242-
},
243-
{
244-
name: "Rudi: local object becomes $localObj",
245-
inputData: `{"spec":{"secretName":"foo"}}`,
246-
mutation: syncagentv1alpha1.ResourceMutation{
247-
Rudi: &syncagentv1alpha1.ResourceRudiMutation{
248-
Script: `(set! .spec $localObj.local)`,
249-
},
250-
},
251-
ctx: &TemplateMutationContext{
252-
LocalObject: map[string]any{
253-
"local": "data",
254-
},
255-
},
256-
expected: `{"spec":"data"}`,
257-
},
258-
{
259-
name: "Rudi: remote object becomes $remoteObj",
260-
inputData: `{"spec":{"secretName":"foo"}}`,
261-
mutation: syncagentv1alpha1.ResourceMutation{
262-
Rudi: &syncagentv1alpha1.ResourceRudiMutation{
263-
Script: `(set! .spec $remoteObj.remote)`,
264-
},
265-
},
266-
ctx: &TemplateMutationContext{
267-
RemoteObject: map[string]any{
268-
"remote": "data",
269-
},
270-
},
271-
expected: `{"spec":"data"}`,
272-
},
273190
}
274191

275192
for _, testcase := range testcases {

‎internal/sync/syncer_related.go‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -150,7 +150,7 @@ func (s *ResourceSyncer) processRelatedResource(log *zap.SugaredLogger, stateSto
150150
// sure we can clean up properly
151151
blockSourceDeletion: relRes.Origin == "kcp",
152152
// apply mutation rules configured for the related resource
153-
mutator: mutation.NewMutator(nil), // relRes.Mutation
153+
mutator: mutation.NewMutator(relRes.Mutation),
154154
}
155155

156156
requeue, err = syncer.Sync(log, sourceSide, destSide)

0 commit comments

Comments
 (0)