Data Plane / Gateway Pods all restart when controller restarts

[ryan-dyer-sp]

Describe the bug
I am seeing that every time a control plane (nginx-gateway) pod is restarted all of the data plane / gateway (nginx) pods restart as well. It is my understanding they should only be rebooting when their configuration is being changed ie. via an nginxproxy update, a new image version, etc.

To Reproduce
Steps to reproduce the behavior:

1. kubectl delete pod
2. watch over the next minute or two as all pods created as part of gateways get restarted.

Expected behavior
Only expect data plane pods to reboot during nginx upgrades or changes to nginxproxy or other scenarios. This existing one leads to a lot of fragility in the nginx's as even when running multiple gateway and controller replicas, a node going out (autoscaling) which has the CP can trigger all gateway pods to be redeployed which leads to brief outages.

Your environment

• Version of the NGINX Gateway Fabric - nginx-gateway-fabric@sha256:a5ef983e5d5124aba3b46c511e6409bc43cfcf1faa28a5aebeb17d6c9e7c5bd1
• Version of Kubernetes - 1.31
• Kubernetes platform (e.g. Mini-kube or GCP) - EKS
• Details on how you expose the NGINX Gateway Fabric Pod - LB

Additional context
Original mention of this issue : #4068

[nginx-bot]

Hi @ryan-dyer-sp! Welcome to the project! 🎉

Thanks for opening this issue!
Be sure to check out our Contributing Guidelines and the Issue Lifecycle while you wait for someone on the team to take a look at this.