"invalid object type" applying v1 NetworkPolicy with Calico

[SanjayVas]

Describe the bug
I get an error status with message invalid object type: networking.k8s.io/v1, Kind=NetworkPolicy when calling Kubectl.apply from io.kubernetes.client.extended.kubectl on a V1NetworkPolicy object loaded from YAML. This specifically happens when calling a minikube instance with the Calico networking plugin. If it's using the default Kindnet plugin (which now supports NetworkPolicy), there's no error.

Note that there's also no error when applying the same YAML file using the kubectl CLI.

Client Version
24.0.0

Kubernetes Version
1.32.0

Java Version
Kotlin 2.1.0 on Java 17

To Reproduce

1. Start minikube with --cni=calico
2. Load YAML file containing a v1 NetworkPolicy object using io.kubernetes.client.util.Yaml
3. Call Kubectl.apply with that object.

Expected behavior
NetworkPolicy object is applied (in this case, created) successfully.

Server (please complete the following information):

• OS: Linux (Debian testing)
• Environment: Local w/minikube

Additional context
Full log message:

io.kubernetes.client.extended.kubectl.exception.KubectlException: io.kubernetes.client.openapi.ApiException: Message: class V1Status {
    apiVersion: v1
    code: 400
    details: null
    kind: Status
    message: invalid object type: networking.k8s.io/v1, Kind=NetworkPolicy
    metadata: class V1ListMeta {
        _continue: null
        remainingItemCount: null
        resourceVersion: null
        selfLink: null
    }
    reason: BadRequest
    status: Failure
}
HTTP response code: 400
HTTP response body: null
HTTP response headers: null
        at io.kubernetes.client.extended.kubectl.KubectlApply.executeServerSideApply(KubectlApply.java:102)
        at io.kubernetes.client.extended.kubectl.KubectlApply.execute(KubectlApply.java:66

[brendandburns]

kubectl apply is a relatively complicated client-side operation. It wouldn't surprise me if there were bug/differences between how we implemented it and how kubectl implements it.

The best way to debug would be to add --v=10 to the kubectl command line call and then check that against the calls that the Java client is making. In particular, we are using server-side apply, I'm not sure if that is the default in kubectl or not. If it is not, it is probably worth testing to see if server side apply works.

Note also that the kubectl package in this library is provided mostly for learning/convenience. You are ultimately probably better off making direct calls into the API instead of relying on emulation of kubectl calls.

[SanjayVas]

In particular, we are using server-side apply, I'm not sure if that is the default in kubectl or not. If it is not, it is probably worth testing to see if server side apply works.

I confirmed that server-side apply works with kubectl.

Note also that the kubectl package in this library is provided mostly for learning/convenience. You are ultimately probably better off making direct calls into the API instead of relying on emulation of kubectl calls.

In this particular case, I'm specifically trying to apply a yaml file containing many objects as a List object. I do this more than once in the same cluster for making changes, so I am in fact relying on things like the create/update behavior.

[SanjayVas]

Relevant logs from kubectl with v=10:

I0626 17:14:20.894283 1108668 helper.go:269] "Request Body" body=<
        {"apiVersion":"networking.k8s.io/v1","kind":"NetworkPolicy","metadata":{"labels":{"app.kubernetes.io/name":"system-api-server-network-policy","app.kubernetes.io/part-of":"halo-cmms"},"name":"system-api-server-network-policy","namespace":"default"},"spec":{"egress":[{"ports":[{"port":8443,"protocol":"TCP"}],"to":[{"podSelector":{"matchLabels":{"app":"gcp-kingdom-data-server-app"}}}]}],"ingress":[{"ports":[{"port":8443}]}],"podSelector":{"matchLabels":{"app":"system-api-server-app"}}}}
 >
I0626 17:14:20.894353 1108668 round_trippers.go:473] curl -v -XPATCH  -H "Accept: application/json" -H "Content-Type: application/apply-patch+yaml" -H "User-Agent: kubectl/v1.32.4 (linux/amd64) kubernetes/4cb5f07" 'https://127.0.0.1:32801/apis/networking.k8s.io/v1/namespaces/default/networkpolicies/system-api-server-network-policy?fieldManager=kubectl&fieldValidation=Strict&force=false'
I0626 17:14:20.901173 1108668 round_trippers.go:560] PATCH https://127.0.0.1:32801/apis/networking.k8s.io/v1/namespaces/default/networkpolicies/system-api-server-network-policy?fieldManager=kubectl&fieldValidation=Strict&force=false 201 Created in 6 milliseconds
I0626 17:14:20.901224 1108668 round_trippers.go:577] HTTP Statistics: GetConnection 0 ms ServerProcessing 6 ms Duration 6 ms
I0626 17:14:20.901244 1108668 round_trippers.go:584] Response Headers:
I0626 17:14:20.901261 1108668 round_trippers.go:587]     X-Kubernetes-Pf-Flowschema-Uid: e89fce5b-145b-4987-9b12-ea2044cc740d
I0626 17:14:20.901273 1108668 round_trippers.go:587]     X-Kubernetes-Pf-Prioritylevel-Uid: 90a1986b-a828-40e7-a22c-33d5cf053379
I0626 17:14:20.901286 1108668 round_trippers.go:587]     Content-Length: 981
I0626 17:14:20.901298 1108668 round_trippers.go:587]     Date: Thu, 26 Jun 2025 17:14:20 GMT
I0626 17:14:20.901309 1108668 round_trippers.go:587]     Audit-Id: 74e1c565-ba6a-4b3a-9227-50f0a214f4e8
I0626 17:14:20.901320 1108668 round_trippers.go:587]     Cache-Control: no-cache, private
I0626 17:14:20.901329 1108668 round_trippers.go:587]     Content-Type: application/json
I0626 17:14:20.901382 1108668 helper.go:269] "Response Body" body=<
        {"kind":"NetworkPolicy","apiVersion":"networking.k8s.io/v1","metadata":{"name":"system-api-server-network-policy","namespace":"default","uid":"feb03579-41a4-4fd5-9a59-c4602e152a4b","resourceVersion":"670","generation":1,"creationTimestamp":"2025-06-26T17:14:20Z","labels":{"app.kubernetes.io/name":"system-api-server-network-policy","app.kubernetes.io/part-of":"halo-cmms"},"managedFields":[{"manager":"kubectl","operation":"Apply","apiVersion":"networking.k8s.io/v1","time":"2025-06-26T17:14:20Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:labels":{"f:app.kubernetes.io/name":{},"f:app.kubernetes.io/part-of":{}}},"f:spec":{"f:egress":{},"f:ingress":{},"f:podSelector":{}}}}]},"spec":{"podSelector":{"matchLabels":{"app":"system-api-server-app"}},"ingress":[{"ports":[{"protocol":"TCP","port":8443}]}],"egress":[{"ports":[{"protocol":"TCP","port":8443}],"to":[{"podSelector":{"matchLabels":{"app":"gcp-kingdom-data-server-app"}}}]}],"policyTypes":["Ingress","Egress"]}}
 >
networkpolicy.networking.k8s.io/system-api-server-network-policy serverside-applied

[brendandburns]

Ah, yes, I'm pretty sure it's the fact that it's a list object. If you look at the output from kubectl above you'll see that it is sending a single NetworkPolicy resource not a list.

afaik, calling patch on a list isn't accepted anywhere in the Kubernetes API, you have to patch individual resources.

My bet is that kubectl is deserializing the list and then applying each resource one at a time. The code in KubectlApply.java definitely doesn't do that. You could validate this by introspecting the kubectl code.

For now, I think your best bet would be to use the Yaml class to load the list and then use a for loop to iterate over the resources and call server-side apply (or patch) directly on each resource.

Alternately, we'd take a PR to add support to optionally load from a list instead of a single resource.

[SanjayVas]

For now, I think your best bet would be to use the Yaml class to load the list and then use a for loop to iterate over the resources and call server-side apply (or patch) directly on each resource.

That's what my code is already doing. The difference is that it's calling Kubectl.apply on each KubernetesObject from the list rather than the specific patch method for that resource type. It fails when it gets to the NetworkPolicy item in the list.

[brendandburns]

At this point, I think we need to print out the body of the request that the Java client is sending, without looking at the differences between it and what kubectl is sending it is hard to figure out what is going wrong.

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle rotten
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten