feat: add enhanced retry options with delay and condition support #8548

naaa760 · 2025-09-07T10:34:56Z

Description

This PR implements enhanced retry options for Vitest tests, extending the current simple retry mechanism with advanced configuration capabilities.

fixes: #8482

What This PR Solves

The current Vitest retry mechanism only supports a fixed number of retries on any kind of error. This PR adds:

New Retry Features:

retryStrategy: Choose when retries are executed ("immediate", "test-file", "deferred")
retryDelay: Configurable delay between retries with exponential backoff
retryCondition: Conditional retry based on error type (string patterns, RegExp, or custom functions)

Backward Compatibility:

Fully maintains existing retry: number syntax
No breaking changes to current API
All existing tests continue to work

Implementation Details

Files Modified:

Configuration types in packages/vitest/src/node/types/config.ts
Core retry logic in packages/runner/src/run.ts
CLI support in packages/vitest/src/node/cli/cli-config.ts
Utility functions in packages/vitest/src/utils/retry.ts (new file)

Key Features:

Exponential backoff delay mechanism
Flexible error condition matching
TypeScript type safety
CLI configuration support
Comprehensive test coverage

sheremet-va

Thank you for your PR. Looks good, but needs some polishing.

Please, add this change to the documentation and add some tests that work with this feature

sheremet-va · 2025-09-08T15:31:35Z

packages/runner/src/types/runner.ts

  testTimeout: number
  hookTimeout: number
-  retry: number
+  retry: number | {


I see that the type is repeated often. Maybe it should be a separate type in ./tasks?

sheremet-va · 2025-09-08T15:32:36Z

packages/runner/src/run.ts

      catch (e) {
        failTask(test.result, e, runner.config.diffOptions)
+
+        const error = test.result.errors?.[0]?.error as Error


Can you explain this line, please? Why do we need to go to test.result.errors?.[0]?.error when e is available already?

sheremet-va · 2025-09-08T15:33:18Z

packages/runner/src/run.ts

+        }
      }

-      // update retry info


Let's not remove comments not related to the changes made

sheremet-va · 2025-09-08T15:33:29Z

packages/runner/src/run.ts

    }
  }

-  // if test is marked to be failed, flip the result


Let's not remove comments not related to the changes made

sheremet-va · 2025-09-08T15:34:10Z

packages/vitest/src/node/cli/cli-config.ts

    description:
-      'Retry the test specific number of times if it fails (default: `0`)',
-    argument: '<times>',
+      'Retry configuration. Can be a number or JSON string with advanced options (default: `0`)',


It cannot be a JSON string, the value would be defined as --retry.count=5 --retry.strategy=immediate

sheremet-va · 2025-09-08T15:35:42Z

packages/vitest/src/node/cli/cli-config.ts

      'Stop test execution when given number of tests have failed (default: `0`)',
    argument: '<number>',
  },
  retry: {


Please, add a test for CLI input to test/core/cli-test.test.ts

sheremet-va · 2025-09-08T15:37:10Z

packages/vitest/src/node/cli/cli-config.ts

+        argument: '<milliseconds>',
+      },
+      condition: {
+        description: 'Condition for retrying: RegExp pattern or custom function',


I think this needs an explanation where this pattern is applied. According to types, you can also pass down a string, but this message doesn't mention it

sheremet-va · 2025-09-08T15:37:29Z

packages/vitest/src/node/types/config.ts

-   * @default 0
-   */
-  retry?: number
+  retry?: number | {


Let's keep comments, please

sheremet-va · 2025-09-08T15:38:00Z

packages/vitest/src/utils/retry.ts

@@ -0,0 +1,70 @@
+import type { SerializedConfig } from '../runtime/config'


This whole file should be in packages/runner. There is no reason to keep it in another package

sheremet-va · 2025-09-08T15:39:19Z

packages/vitest/src/utils/retry.ts

+export function calculateBackoffDelay(
+  baseDelay: number,
+  retryCount: number,
+  algorithm: 'linear' | 'exponential' = 'linear'


I am a bit confused why this needs to be customizable when we always pass down exponential. Should this check be removed?

netlify · 2025-09-09T09:57:28Z

✅ Deploy Preview for vitest-dev ready!

Built without sensitive environment variables

Name	Link
🔨 Latest commit	`2b66c7c`
🔍 Latest deploy log	https://app.netlify.com/projects/vitest-dev/deploys/68bff9699eb5c20008b8d0a5
😎 Deploy Preview	https://deploy-preview-8548--vitest-dev.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

netlify · 2025-09-09T10:00:37Z

✅ Deploy Preview for vitest-dev ready!

Built without sensitive environment variables

Name	Link
🔨 Latest commit	`d4cc802`
🔍 Latest deploy log	https://app.netlify.com/projects/vitest-dev/deploys/68bffa550d544a00083fbe2d
😎 Deploy Preview	https://deploy-preview-8548--vitest-dev.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

netlify · 2025-09-09T10:02:00Z

✅ Deploy Preview for vitest-dev ready!

Built without sensitive environment variables

Name	Link
🔨 Latest commit	`8c98cdb`
🔍 Latest deploy log	https://app.netlify.com/projects/vitest-dev/deploys/68bffa6e53cc050008637ad6
😎 Deploy Preview	https://deploy-preview-8548--vitest-dev.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

sheremet-va · 2025-09-22T12:59:50Z

The tests are failing.

flx-sta · 2025-09-22T16:06:42Z

packages/runner/src/types/tasks.ts

-  /**
-   * Times to retry the test if fails. Useful for making flaky tests more stable.
-   * When retries is up, the last test error will be thrown.
-   *
-   * @default 0
-   */
-  retry?: number


Shouldn't the doc-comment be replaced/extended instead of removed completely?

naaa760 added 9 commits September 7, 2025 13:45

Extended main config interface

ff657ae

Updated serialized config

2bca65e

CLI configuration

ce7f779

new utility functions

b2ff23d

Core retry logic implementation

4d3432b

enhanced task options

1f113ff

updated runner config

889c5df

updated runner config

7dc0db2

update

e99f667

sheremet-va requested changes Sep 8, 2025

View reviewed changes

added RetryConfig type

2b66c7c

naaa760 force-pushed the retry-options branch from 7d78989 to 2b66c7c Compare September 9, 2025 09:54

naaa760 added 4 commits September 9, 2025 15:25

uses RetryConfig type

d38b495

updated retry type

6c02f41

restored JSDoc comments

02b9403

fixed error handling and comments

877ee32

naaa760 added 4 commits September 9, 2025 15:27

fixed CLI descriptions

87edabf

moved and simplified

42fcbca

added retry CLI tests

d4cc802

Merge branch 'main' into retry-options

8c98cdb

flx-sta reviewed Sep 22, 2025

View reviewed changes

		@@ -0,0 +1,70 @@
		import type { SerializedConfig } from '../runtime/config'

Uh oh!

feat: add enhanced retry options with delay and condition support #8548

Are you sure you want to change the base?

feat: add enhanced retry options with delay and condition support #8548

Conversation

naaa760 commented Sep 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

What This PR Solves

New Retry Features:

Backward Compatibility:

Implementation Details

Uh oh!

sheremet-va left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

netlify bot commented Sep 9, 2025

✅ Deploy Preview for vitest-dev ready!

Uh oh!

netlify bot commented Sep 9, 2025

✅ Deploy Preview for vitest-dev ready!

Uh oh!

netlify bot commented Sep 9, 2025

✅ Deploy Preview for vitest-dev ready!

Uh oh!

sheremet-va commented Sep 22, 2025

Uh oh!

flx-sta Sep 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

naaa760 commented Sep 7, 2025 •

edited

Loading

flx-sta Sep 22, 2025 •

edited

Loading