Make the indexing batch size configurable #20543
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Prerequisites
Description
This PR is a continuation of the work @bielu did in #16546 🚀
It introduces an option to configure the batch size used for indexing, to make things a little more flexible. As per the original PR:
Following an internal discussion, the use of Examine indexing events (
IndexOperationComplete) for progress reporting has been discontinued. At this point they're only used for logging, and it's a volatile solution; any index operation will trigger the event, which potentially leads to false or wrongful progress reporting for index rebuilds.As a final note: #16546 also encompasses a new in-progress tracking for index rebuilds, to replace the runtime cache based implementation. However, this has since been replaced by tracking long-running operations in the database, so this PR does not port over the new in-progress tracking from the original PR.
Testing this PR
Verify that the configurable
BatchSizeactually kicks in - either by attaching a debugger, or by injectingIOptions<IndexingSettings>into some custom code that lets you inspect the currentBatchSizevalue.