prometheus output with strings-as-labels does not handle duplicate labels under different paths

[vadimberezniker]

e.g. if the raw data contains these kvs:

      "/network-instances/network-instance/protocols/protocol/bgp/neighbors/neighbor/state/messages/received/last-notification-error-code": "openconfig-bgp-types:CEASE",
      "/network-instances/network-instance/protocols/protocol/bgp/neighbors/neighbor/state/messages/sent/last-notification-error-code": "openconfig-bgp-types:HOLD_TIMER_EXPIRED",

Both will be added to a prometheus metric under the last-notification-error-code label causing this metric to be dropped silently since the prometheus http handler is registered with ContinueOnError.
It would be nice if the prometheus exporter at least logged a warning when duplicate labels are encountered and removed any duplicates so that at least some data was pushed to prometheus instead of dropping the metric entirely.

[karimra]

In #808 I changed both Prometheus outputs to resolve duplicate labels by progressively prepending parent pathElem names from the path until the names are unique.

[vadimberezniker]

That's great, thank you.

The only thing I'd be worried about is the fact that I think the key-value pairs can change over time, for example certain attributes are only present when something is an error state (like last-notification-error-code in the example above). That could theoretically cause the prometheus label to change only when some state changes on the device. This could potentially cause problems with monitoring and alerting rules.
I don't have a specific example in mind, just something worth considering.

We'll pick up the new version and remove the hack I added to our config.