go/oasis-node/cmd/storage: Add command for offline pruning

[martintomazic]

Closes #6321

Motivation:

When enabling aggressive pruning node may fall behind. Whilst unlikely, the consequence may be the node breaking its commitment, which could results in lost rewards, possibly registering its availability when it is actually not ready, or worst case penalties (don't think this would happen actually).

Solution:

To prevent it, we should offer validators a maintenance command that should be called if pruning is enabled later on, before starting the node. This way freshly started node is guaranteed to be healthy.

As with compaction command, I have started with consensus databases only and suggest to add runtime databases for the follow-up. Moreover, I quickly tested that this command does not prune past the minimum reindexed height, in case of configured runtimes.

Automatically prune on the node startup?

For now this is not the case:

• Just pruning without compacting is not particularly useful. Unless we also automate compaction as part of the startup, the operator is forced to stop the node once the pruning finishes.
  • Bad ux, also possibly wait if the node already registered.
• I don't want to automatically enable forced compaction on the startup as this is inefficient assuming the operators restarts twice in a row, e.g. due to fixing config, possibly triggering corner case observed.
• Given this command is experimental and a bit hackish, I would prefer if we do not automate pruning on the node startup, at least for few months until we know if there is anything unexpected that could happen.

How to test:

Have a node with valid state, doesn't have to be synced. Configure pruning to retain less data and run:

oasis-node storage prune-experimental --config etc/config.yml

Following that trigger compaction:

oasis-node storage compact-experimental --config etc/config.yml

Ensure the node is able to sync correctly and that disk space has been correctly reclaimed. You can also configure a runtime and test this command does not prune past the min reindexed height.

Follow-up:

• docs/Run your node: Add dedicated section about pruning docs#1469
• E2e test?

[netlify]

✅ Deploy Preview for oasisprotocol-oasis-core canceled.

Name	Link
🔨 Latest commit	ce38558
🔍 Latest deploy log	https://app.netlify.com/projects/oasisprotocol-oasis-core/deploys/692977d5c9c0020008899ded

[codecov]

Codecov Report

❌ Patch coverage is 10.75949% with 141 lines in your changes missing coverage. Please review.
✅ Project coverage is 64.06%. Comparing base (d2cc5c8) to head (ce38558).
⚠️ Report is 7 commits behind head on master.

Files with missing lines	Patch %	Lines
go/oasis-node/cmd/storage/storage.go	0.78%	127 Missing ⚠️
go/oasis-node/cmd/common/common.go	0.00%	7 Missing ⚠️
go/consensus/cometbft/db/init.go	64.28%	3 Missing and 2 partials ⚠️
go/consensus/cometbft/full/archive.go	66.66%	1 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #6341      +/-   ##
==========================================
- Coverage   64.92%   64.06%   -0.86%     
==========================================
  Files         698      698              
  Lines       68065    68207     +142     
==========================================
- Hits        44190    43700     -490     
- Misses      18872    19479     +607     
- Partials     5003     5028      +25     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[martintomazic]

Rebased (no changes).

Too see how this would fit in the big picture check the new (pending) pruning section: oasisprotocol/docs#1526