Mutlicluster f5-cis connection broken between clusters deletes all standalone loadbalancers from secondary cluster posted on BIGIP

[achaudh25]

Setup Details

CIS Version : 2.20.1
Build: f5networks/k8s-bigip-ctlr:2.20.1
BIGIP Version: Big IP v17.1.1.3
AS3 Version: 3.52
Agent Mode: AS3
Orchestration: K8S
Pool Mode: Cluster/Nodeport
Additional Setup details: Rancher/Calico network

Description

Multicluster f5-cis connection broken between clusters deletes all load balancers from secondary cluster posted on BIGIP and when connection is fixed it recreates them.

Steps To Reproduce

1. Created load balancers on Cluster A and Cluster B standalone in same partition
2. We broke the connection between Cluster A and Cluster B by providing wrong kubeconfig and that leads deletion of Load balancers from Cluster B posted on BIGIP.
3. Provided correct kubeconfig fix the connection all LB's were posted back.

Expected Result

f5-cis shouldn't delete anything due to connection error that is already posted.

[lavanya-f5]

@achaudh25 Could you please share cis deployment yaml and global configmap?

[lavanya-f5]

@achaudh25 Couldn't reproduce the issue on 2.20.1 build

Steps followed

• Deploy cis multicluster in standalone mode, deploy svc type LB in cluster1 and cluster2 and both the virtual servers are created on BIGIP
• Update the cluster2 secret with wrong kubeconfig(changed server ip configuration in kubeconfig) --> cis throws connection errors in informer setup. no declaration posted to bigip and existing config remains same
• change something in cluster1 svc changes are pushed to bigip, but cluster2 vs config stays same, pool members not removed

Please share cis manifests and full logs to debug this issue

[achaudh25]

@lavanya-f5 will add it teams chat thanks

[achaudh25]

@lavanya-f5 we re-tried to reproduce the issue again and there is one more thing we did while updating the kubeconfig is change of root certificates. When we changed root certificates with expired kubeconfig on A and B sides they are not able to talk to each other and also deleted the pools from B side when we try to add new LB in same partition.

[lavanya-f5]

Created [CONTCNTR-5772] for internal tracking