Skip to content

Conversation

@kjac
Copy link
Contributor

@kjac kjac commented Oct 17, 2025

Prerequisites

  • I have added steps to test this contribution in the description below

Description

This PR is a continuation of the work @bielu did in #16546 🚀

It introduces an option to configure the batch size used for indexing, to make things a little more flexible. As per the original PR:

(...) current we pulled 10k nodes, which if they have more than 100 properties might cause really slow indexing when we pull more than 1k

Following an internal discussion, the use of Examine indexing events (IndexOperationComplete) for progress reporting has been discontinued. At this point they're only used for logging, and it's a volatile solution; any index operation will trigger the event, which potentially leads to false or wrongful progress reporting for index rebuilds.

As a final note: #16546 also encompasses a new in-progress tracking for index rebuilds, to replace the runtime cache based implementation. However, this has since been replaced by tracking long-running operations in the database, so this PR does not port over the new in-progress tracking from the original PR.

Testing this PR

Verify that the configurable BatchSize actually kicks in - either by attaching a debugger, or by injecting IOptions<IndexingSettings> into some custom code that lets you inspect the current BatchSize value.

Copy link
Contributor

@AndyButland AndyButland left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All looks good, and having verified locally I can see the configured value is being used when populating the indexes.

Just a couple of follow-up tasks that are likely already in mind:

  • Please can you updates the docs here? It should be updated to coincide with the 17.1 release (unless you plan to cherry-pick this in for 17.0, and if so, please amend the label I've added to this PR).
  • Can you check and if appropriate close #19741 please (I think you can now the logging is removed)?

@AndyButland AndyButland merged commit ae2c59b into v17/dev Oct 17, 2025
25 of 26 checks passed
@AndyButland AndyButland deleted the v17/feature/configurable-indexing-batch-size branch October 17, 2025 13:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants