feat: add exhaust node resources fault

[preespp]

Added a new fault: exhaust-node-resources and a corresponding incident spec (Incident 297) from issue #297

[rohanarora]

Pree (@preespp), Dropped a few comments for you to look at. Please let us know if you have any follow-up questions.

cc: @Red-GV

[preespp]

Thank you for your comment. Below are what I changed based on what we discussed in last Thursday's meeting.

• Changed node label from 'itbench-fault: exhaust-node' to 'node-role.kubernetes.io/app: "true"'
• Introduced anti-affinity so that observability/tool-related pods are deployed to separate nodes, preventing them from being affected by the resource hog.
• Ensured the targeted workload is scheduled on a different node to avoid pending pods.
• Updated incident metadata and alerts to reflect resource pressure correctly.
• Changed tasks to label/unlabel nodes before/after resource hog deployment to maintain cluster state.

[Red-GV]

Few changes I'm noticing on this round of the review. Sorry if I didn't mention them earlier.

[Red-GV]

Just one minor thing in the ground truth and I think we're good here.

@rohanarora I'm going to recommend that we table the node labeling for now until we add observability node labels to install the tools onto. That also means we should probably avoid smoke testing this one for now because it might render the cluster inoperable.

[Red-GV]

@preespp Not seeing any of the alerts go off for this one. I suggest doing two things for this PR:

1. Look at Incident 26/27 for how to enable a ChaosMesh fault rather than manually orchestrating it.
2. Make a new fault that removes resource limits on a workload. Technically, you can make two incidents with that. Though I think that runs a bit into #336 with that.