test: Add Flaky Test Plan

[philipphofmann]

Add an extra test plan, Sentry_Flaky, that reruns failing tests 3 times before the test officially fails.

This has the advantage of still running flaky tests in CI over completely disabling them. It's also more efficient than rerunning the entire test suite when a test fails and helps surface new flaky tests.

#skip-changelog

Closes #6535

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 86.950%. Comparing base (1b899b8) to head (57cf4ba).
⚠️ Report is 2 commits behind head on main.
✅ All tests successful. No failed tests found.

Additional details and impacted files

@@              Coverage Diff              @@
##              main     #6534       +/-   ##
=============================================
+ Coverage   86.939%   86.950%   +0.010%     
=============================================
  Files          451       451               
  Lines        27488     27488               
  Branches     11975     11973        -2     
=============================================
+ Hits         23898     23901        +3     
+ Misses        3544      3540        -4     
- Partials        46        47        +1     

see 12 files with indirect coverage changes

---

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1b899b8...57cf4ba. Read the comment docs.

[github-actions]

Performance metrics 🚀

	Plain	With Sentry	Diff
Startup time	1214.43 ms	1253.14 ms	38.71 ms
Size	23.75 KiB	1.00 MiB	1005.07 KiB

Baseline results on branch: main

Startup times

Revision	Plain	With Sentry	Diff
ef2c9b3	1222.12 ms	1253.96 ms	31.84 ms
be882e4	1199.35 ms	1231.20 ms	31.86 ms
effeafa	1225.88 ms	1249.78 ms	23.90 ms
debf3e9	1227.45 ms	1235.71 ms	8.26 ms
e36128b	1213.04 ms	1248.60 ms	35.56 ms
d8b2303	1231.53 ms	1249.89 ms	18.35 ms
48f53bc	1218.71 ms	1253.02 ms	34.31 ms
d7461dc	1233.69 ms	1255.29 ms	21.60 ms
c585b1e	1232.88 ms	1249.45 ms	16.58 ms
fc6557e	1226.40 ms	1249.88 ms	23.48 ms

App size

Revision	Plain	With Sentry	Diff
ef2c9b3	23.75 KiB	933.03 KiB	909.29 KiB
be882e4	23.75 KiB	946.69 KiB	922.94 KiB
effeafa	23.74 KiB	926.64 KiB	902.89 KiB
debf3e9	23.75 KiB	1.01 MiB	1006.46 KiB
e36128b	23.75 KiB	963.19 KiB	939.44 KiB
d8b2303	23.75 KiB	908.02 KiB	884.27 KiB
48f53bc	23.75 KiB	1014.71 KiB	990.96 KiB
d7461dc	23.75 KiB	874.45 KiB	850.70 KiB
c585b1e	23.75 KiB	879.95 KiB	856.21 KiB
fc6557e	23.75 KiB	866.68 KiB	842.93 KiB

Previous results on branch: test/flaky-test-plan

Startup times

Revision	Plain	With Sentry	Diff
5734b6d	1213.11 ms	1248.23 ms	35.13 ms

App size

Revision	Plain	With Sentry	Diff
5734b6d	23.75 KiB	1.00 MiB	1005.07 KiB

[philprime]

LGTM, thank you for tackling the flakiness