In my setup I do not have an easy way to restart keep after updating providers so I need a way to trigger a re-provisioning at runtime.

Having API endpoints for this is a common approach. This way it can be achieved via a script that updates the files in the provision directory and then triggers re-provisioning via an API call.

I propose these endpoints:

POST /providers/provision
POST /workflows/provision

The ProvidersService.provision_providers and WorflowStore.provision_workflows functions are already designed in a way that they can be re-run any time after startup and update the provisioned providers and workflows.