Skip to content

Conversation

@iakov-aws
Copy link
Contributor

Issue #, if available:

Description of changes:

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

}
},
"MaxConcurrency": 60,
"ToleratedFailurePercentage": 10,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@esc1144 for discussion

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this will affect all/most modules, what to you think about making the value (i.e. 10%) a parameter that we set in the SF resource declaration in the modules. We would then need to collect a new parameter for it on the main template to pass along to the nested ones. But if a cx ever wanted to adjust it from the 10% default, it would easy to just run a stack update.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there are several general options:

  1. All 100% Green = Everybody is happy
  2. All 100% Red = there is a totall miss in the permissions and we need to fail the full data collection step function
  3. X% are RED - most probably we do not need to fail ALL data collection as some data are collected and we need to run crawler anyway. The data will be partial, sure but there will be at least some of them. The % of failed and other interesting things we need to address in other PR in progress.

Here in the option 3 the number is not that significant. I will update it to 90% so that even 10% of success would count as success.

Again reminder that today we have all green whatever the result of SFs ;)

esc1144
esc1144 previously approved these changes Mar 17, 2025
}
},
"MaxConcurrency": 60,
"ToleratedFailurePercentage": 10,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this will affect all/most modules, what to you think about making the value (i.e. 10%) a parameter that we set in the SF resource declaration in the modules. We would then need to collect a new parameter for it on the main template to pass along to the nested ones. But if a cx ever wanted to adjust it from the 10% default, it would easy to just run a stack update.

@iakov-aws iakov-aws marked this pull request as draft May 11, 2025 18:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants