Tracking: DB Write Optimisation to Static Files

[yongkangc]

Motivation

The goal is to optimise DB Writes to static files

Gravity has metioned to us that:

Block persistence happens asynchronously while non-persisted blocks accumulate in CanonicalInMemoryState.
For Gravity they target sub-second block finalization; when disk writes lag, we cache more blocks in memory until the process OOMs. On a 10M-account dataset we have observed blocks with ~5,000 transactions taking up to five seconds to persist. We need to collapse that gap so write throughput matches block production.

Reference PR: #19008

Goals

• Minimize duplicated DB work (transactions, cursors, allocations) in static-file writers and readers.
• Reuse database transactions, cursors, and shared buffers across the static-file pipeline.
• Trim unnecessary channels and per-block setup overhead so background block persistence keeps up with Gravity latency targets.

Non-Goals

• Changing pruning semantics or CanonicalInMemoryState behaviour.
• Reworking the db or changing it to rocksDB