Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
77 changes: 77 additions & 0 deletions docs/node/run-your-node/advanced/pruning.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
# Pruning State

To reduce hardware requirements and improve the overall latency of the node,
operators are encouraged to prune their state (unless they are archivers or
historical nodes).

As will be explained in the following sections, pruning should be configured from
the start, i.e. late changes to the pruning configuration may not be optimal for
the node's health.

## Consensus Pruning

To configure pruning of the consensus data amend your node's configuration with:

```yaml
# ... sections not relevant are omitted ...
consensus:
# ... sections not relevant are omitted ...
prune:
strategy: "keep_n"
num_kept: <n>
```
where `<n>` is the number of blocks and state versions that will be kept.

### Suggested Pruning Configuration

For normal node operation, the minimum requirements is `n=250_000`. However,
it is recommended to set it to `n=400_000`. Assuming blocks are produced every 6
seconds, the latter corresponds to preserving 1 month of the consensus data.

Operators that are not resouce-restrained and want to contribute to the overall
network health are encouraged to set `n=5_000_000` (approximately 1 year of data).

### Late Pruning

Due to the [LSM Tree][] design of the underlying databases, enabling pruning after
your node has been runing for a while, or possibly changing configuration to retain
less data, may may not work as expected.

To change pruning configuration operators must follow this steps:
1. Gracefuly shutdown the node (validators should take [special care][graceful shutdown]).
2. Configure pruning as described in [Consensus Pruning](#consensus-pruning).
3. Run offline pruning (TODO insert link).
Copy link
Contributor Author

@martintomazic martintomazic Nov 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can ad whole Late Pruning section as a follow-up once offline pruning command is merged.

4. Run the [compaction command][].
5. Start the node again.

[LSM tree]: https://en.wikipedia.org/wiki/Log-structured_merge-tree
[graceful shutdown]: ../maintenance/shutting-down-a-node.md
[compaction command]: https://github.com/oasisprotocol/oasis-core/blob/master/docs/oasis-node/cli.md#compact-experimental

## Paratime Pruning


To configure pruning of the paratime data ammend your node's configuration with:

```yaml
# ... sections not relevant are omitted ...
runtime:
# ... sections not relevant are omitted ...
prune:
strategy: "keep_last"
num_kept: <n>
```

where `<n>` is the number of paratime's state versions that will be kept.

### Suggested Pruning Configuration

Paratime state is much larger than the consensus state. As a consequence, as the
number of versions in the state database grows, pruning becomes slower and slower.
For this reason **paratime state pruning must be configured
from the start or not configured at all**.

To configure pruning it is recommended to set `n=250_000`. The maximum value is
`n=400_000`. If you need to preserve more data (e.g. nodes serving historical
Comment on lines +75 to +76
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should preserve enough data so that there is sufficiently old state for 2 latest checkpoints, taking creation time into account as well. I believe 250_000 versions is suitable for the current parameters.

With so little state, creating runtime checkpoint is very fast. If I remember correctly cca 20min and also much less IO intensive than what I believe devops have observed on the Observer nodes (with all historical state)... Consensus checkpoint creation compared to runtime checkpoint creation is instant (seconds).

state) you will have to keep the entire state from the genesis.
Comment on lines +75 to +77
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately if you preserve runtime state, to serve historical queries, you are also forced to preserve all the consensus data, as consensus light block are computed ad-hoc (see). A requirement that should probably be documented somewhere.

We could open an issue, so that consensus light blocks are not computed ad-hoc and are instead stored in the separate DB, with different pruning config. This would allow to run Observer node with much lower hardware requirements, assuming most of the consensus data could be dropped.

Not sure this is a critical priority right now, but issue definitely cannot hurt...

3 changes: 2 additions & 1 deletion sidebarNode.ts
Original file line number Diff line number Diff line change
Expand Up @@ -106,6 +106,7 @@ export const sidebarNode: SidebarsConfig = {
},
items: [
'node/run-your-node/advanced/sync-node-using-state-sync',
'node/run-your-node/advanced/pruning',
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As already mentioned there is nothing "advanced" about pruning, in fact pruning should probably enabled via sane defaults (so possibly no need to even configure it)....

When documentation is reorganized (out of scope of this PR), this could probably be "How to guide"

'node/run-your-node/advanced/copy-state-from-one-node-to-the-other',
'node/run-your-node/advanced/remote-signer',
],
Expand Down Expand Up @@ -165,7 +166,7 @@ export const sidebarNode: SidebarsConfig = {
label: 'Mainnet',
},
{
type: 'doc',
type: 'doc',
id: 'node/reference/upgrade-logs/testnet',
label: 'Testnet',
},
Expand Down