Skip to content

Speed up diffing by taking out logs#8630

Draft
NBKelly wants to merge 1 commit intomtgred:masterfrom
NBKelly:speed-up-diffing-with-logs
Draft

Speed up diffing by taking out logs#8630
NBKelly wants to merge 1 commit intomtgred:masterfrom
NBKelly:speed-up-diffing-with-logs

Conversation

@NBKelly
Copy link
Collaborator

@NBKelly NBKelly commented Mar 18, 2026

Turns out diffing two very long lists that are guaranteed to be identical except for the tail is probably overkill for our needs.

Cuts diff time for log messages from ~3-5ms to ~0.05ms for cases like chat.
Cuts diff time for log messages from ~3-5ms to ~1ms for cases like a game actions updating the log.
Both the above for cases where there's ~1k+ entries in the log (a normal game)

This needs to be stress tested on playtest before I'm confident in it entirely though.

@NBKelly NBKelly mentioned this pull request Mar 21, 2026
:runner-spect-state (when runner-spectators? (strip-for-runner-spect replay-state corp-state runner-state))
:hist-state replay-state})))

(defn- fake-log-diff [old new]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What makes this a "fake" diff? Is that it is too specialized, is it one sided?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm emulating what differ does, but skipping 99.9% of the work (in this extremely specialized case).

We have the following guarantees based on the structure of our product:

  • both elements are vectors
  • the new element will always either be the exact same as the older element, or be the older element with an additional tail
    Which means that a lot of the work that differ does (in this specific case) is expensive and wasteful (indexing and equalizing 1-2k elements in the log for every single server frame).

The differ library we use is here: https://github.com/robinheghan/differ

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But in that situation, does it actually diff its arguments?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants