-
Notifications
You must be signed in to change notification settings - Fork 17
Description
Currently it's possible to configure the gem to include the :error tag in failed jobs. It's implemented in the server middlewere here
It's also possible to include custom_tags, but this is evaluated before the perform method is called.
I would need a sort of combination of these two features - I want to include custom tags when an error occurrs. My specific use case is that we have an SLI which compares the ratio between failed and total jobs. It looks something like this
sli:
events:
errorQuery: sum(rate(sidekiq_jobs_failed_total{queue="my_queue"}[{{.window}}])) > 0 or vector(0)
totalQuery: sum(rate(sidekiq_jobs_executed_total{queue="my_queue"}[{{.window}}])) > 0 or vector(1)
However, there are some errors which are expected, and I'd like to exclude those based on a tag
errorQuery: sum(rate(sidekiq_jobs_failed_total{queue="my_queue", ignore_error_in_sli=false}[{{.window}}])) > 0 or vector(0)
Currently the only way implement something like this would be to use the :error tag and include all the possible error class names.
errorQuery: sum(rate(sidekiq_jobs_failed_total{queue="my_queue", error!~="ErrorClass1|ErrorClass2|ErrorClass3"}[{{.window}}])) > 0 or vector(0)
This however is quite cumbersome if there are a lot of error classes, and I'd prefer not to define the errors in the manifest files as we'd like to use the list elsewhere in the code (single source of truth).
If this sounds like a reasonable feature then I can work on a PR.