Skip to content

Optimize root hash computation while loading and saving snapshots #258

@edubart

Description

@edubart

Context

The emulator is starting to be used to create "reader nodes" in production for some dapps where it advances and saves a machine snapshot, then when an inspect comes the saved machine snapshot is loaded and an inspect request is served. The problem is that saving/loading could take minutes (for example when running in a cheap cloud service with just 2 CPU cores and 4GB of ram).

We could optimize all this loading/saving to be faster. The culprit here is root hash computation, most of time is spent there.

Possible solutions

One solution is to save the entire merkle tree state to a file in the snapshot, so not all merkle nodes have to be recomputed after loading. And also save checksum (like sha1sum) of all snapshot files to not lose integrity check, it should be faster to check a sha1sum of a few files rather than a merkle tree hash. Maybe even CRC64 could be enough, I think we don't need a very secure and heavy hash to just check snapshot file integrity.

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type

Projects

Status

PR Available

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions