history: Use initial timestamp for history retention

[oxzi]

The history retention is currently based on the final or closing timestamp, if such a timestamp exists. This change also takes the initial timestamp into account as a fallback.

As reported in the Community Forum¹ and in an ongoing PR², there might be (historical) history entries without a closing timestamp. Thus, these entries will never be rotated. Furthermore, based on the documentation, it might be expected that the initial timestamp is used for retention, not a later one.

This change does not cover the SLA retention, to ensure no misleading SLA reports are being generated when a dangerously short SLA retention period was defined.

Footnotes

1. https://community.icinga.com/t/comment-history-and-retention-on-a-busy-system/15318/3 ↩

2. https://github.com/Icinga/icingadb/pull/913 ↩

[julianbrost]

The history retention is currently based on the final or closing timestamp, if such a timestamp exists. This change switches to the initial timestamp, which always exists.

This will always be done? So in case I have configured the history retention to be 30 days, and have a downtime, say created 42 days in the past. There would then be no indication of that downtime in the history (fine, it's older than 30 days), but if it now expires or gets cancelled, the downtime end history would be written but immediately deleted because it's start is older than 30 days? So after the next history retention run, there would be no sign in the history that a downtime ended like an hour ago? That sounds like it could result in misleading history: notifications missing, that would be explained by a later downtime end event.

Hence, would it be possible to use something like coalesce(end_time, entry_time) (or max() instead of coalesce(), should result in the same behavior, max() could easily be extended to more timestamp columns if necessary) to determine if something is due for removal. The idea being that in the aforementioned situation, the downtime start history entry would be removed as before, but after after it's inserted again for the end event, it's kept alive for 30 more days. Also, doing so might need some additional considerations regarding SQL indices to keep it fast (could be possible that this change also needs some changes there, I haven't checked).

This change does not cover the SLA retention, to ensure no misleading SLA reports are being generated when a dangerously short SLA retention period was defined.

I think it would be a good idea to add a comment there that the difference is intentional and why it's done. Otherwise, that could be mistaken for an oversight in the future and be "fixed".

[oxzi]

Thanks for your ideas on this PR.

Honestly, I am not even sure if I am in favor of the changes I have suggested here. This PR tries to address those forgotten entries which #913 cannot clean up anymore. My motivation was the Community Forum post with some 24k outdated comments which should have been deleted.

Hence, would it be possible to use something like coalesce(end_time, entry_time) (or max() instead of coalesce(), should result in the same behavior, max() could easily be extended to more timestamp columns if necessary) to determine if something is due for removal.

This should work, at least for downtimes. Unfortunately, both acknowledgements and comments do not necessarily need to have a expiry or remove date. Thus, you can still end up with some object being still acknowledged while the history does not say why anymore.

This change does not cover the SLA retention, to ensure no misleading SLA reports are being generated when a dangerously short SLA retention period was defined.

I think it would be a good idea to add a comment there that the difference is intentional and why it's done. Otherwise, that could be mistaken for an oversight in the future and be "fixed".

Good point.

[julianbrost]

Unfortunately, both acknowledgements and comments do not necessarily need to have a expiry or remove date. Thus, you can still end up with some object being still acknowledged while the history does not say why anymore.

That doesn't sound like a problem though. If the acknowledgement is still active, there's still the corresponding comment object that will show since when it was acknowledged. If that was years ago, I don't think there's anything wrong with no corresponding event showing in the history if the retention period is less than that.