add-option-to-continue-on-script-error

[luisggc]

Continue-on-Error Enhancements

Overview

• Introduced optional flags to allow deployments to continue after script failures.
• Full error messages are recorded and failed scripts are listed upon completion.
• New flags are documented in the README so users can enable them via the CLI when needed.

Configuration

• Added configuration fields to capture continue-on-error behavior.
• Default behavior remains fail-fast unless explicitly enabled.
• Exposed CLI options for:
  • versioned scripts
  • repeatable scripts
  • always scripts
  • all scripts
• --continue-all-on-error acts as shorthand for enabling all script-type options.
• If -- schemachange-no-jinja is present in the sql, skip the jinja parsing

Deployment Behavior

• Deployment loop now:
  • Tallies failed scripts
  • Continues when configured
  • Reports a summary of failures before exiting

Testing

• Deployment continues past a failing script when continue_versioned_on_error is set.
• Failures are surfaced at the end of deployment.
• Session logic:
  • Retries inserts
  • Ensures full error details are persisted

[zroytman]

Oh, I'm waiting for this enhancement so long time! It's great, thank you very much! How to speed it up to be moved to production (master)?

[zroytman]

@sfc-gh-tmathew Hi, who can approve the merge to master and when can it be done? Thanks a lot.

[luisggc]

@zroytman , I don't think It will be reviewed anytime soon, apparently. There are a couple of PRs with no answer for many months. In the meantime I am using:
pip install "git+https://github.com/luisggc/schemachange.git@add-option-to-continue-on-script-error"

[zroytman]

@zroytman , I don't think It will be reviewed anytime soon, apparently. There are a couple of PRs with no answer for many months. In the meantime I am using: pip install "git+https://github.com/luisggc/schemachange.git@add-option-to-continue-on-script-error"

@luisggc do you know who should approve it? Who is the maintainer?

[luisggc]

I think we have a few @sfc-gh-twhite merged a PR today, tough.
Please, help me understand what I can do in the PR to make It better.
I thought perhaps not adding the error msg to the table would make the code simpler.

[zroytman]

I think we have a few @sfc-gh-twhite merged a PR today, tough. Please, help me understand what I can do in the PR to make It better. I thought perhaps not adding the error msg to the table would make the code simpler.

Actually I've reviewed your list of changes and the explanation - it looks great for me. The writing of the error into the db table is not so mandatory for me in case you show the error on the screen during the execution - should be enough for me. But this is a great feature. Thank you so much! Hopefully it will be merged soon.

[sfc-gh-twhite]

Thanks for submitting this PR and for the review. I wanted to acknowledge that we will review this sometime this week. We're currently trying to close out a few other lingering things.

I agree that this is a useful feature and would like @sfc-gh-jhansen to review it as well to determine if it is something we should implement.

[MACKAT05]

I wrote an adapter for the sqlfluff linter #342 that might preclude the need for this option by routing out syntax errors before deploy... also have you looked into dbt... parralelism and sequencing are handled there already...

[zroytman]

Hi ALL. Any news regarding getting this awesome enhancement? Thanks.

[sfc-gh-tmathew]

@zroytman will take a look at this option and make the call by first week of september.

[zroytman]

@zroytman will take a look at this option and make the call by first week of september.

Hi @sfc-gh-tmathew , any updates? Thanks.

[zroytman]

@zroytman will take a look at this option and make the call by first week of september.

Hello @sfc-gh-tmathew , have you had a chance to look at this? Thanks.