MongoDB operator design discussion

[antoineozenne]

I would create a discussion, but it seems this repo doesn't allow doing it.

In the chart, I see the file database-roles.yaml in which the RBAC are namespace scoped, and not cluster scoped. If I correctly understand, it's because the MongoDB instance itself needs access to the secrets and pods. This isn't very practical for us, as it creates coupling between the operator and the MongoDB instances. We use one operator to manage multiple MongoDB instances. We then have to choose between these two solutions:

• Create the RBACs at the same time as the MongoDB instance (by including them directly in the chart that deploys the MongoDB instance). The problem is that this requires knowing precisely which RBACs to create. With each operator update, we'll have to check if the RBACs necessary for MongoDB to function correctly have changed, and if so, update them in the chart.
• Provide the operator with a complete list of the namespaces for the MongoDB instances we want to create (with operator.watchNamespace) so it can create the RBACs itself. The problem is that every time we want to deploy a new MongoDB instance, we have to redeploy the operator to update the list...

Taking a step back, I feel like that's not the real issue. What really bothers me is that the operator doesn't seem to fully manage the MongoDB instance's lifecycle, especially secrets. The MongoDB instance itself reads the secrets, and that seems like an unusual design choice for Kubernetes operators. Typically, the operator handles everything, including reading secrets containing database passwords.

This is an open discussion to understand this design choice.