-
Notifications
You must be signed in to change notification settings - Fork 41
docs/node/run-your-node/advanced: Add new pruning section #1526
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,77 @@ | ||
| # Pruning State | ||
|
|
||
| To reduce hardware requirements and improve the overall latency of the node, | ||
| operators are encouraged to prune their state (unless they are archivers or | ||
| historical nodes). | ||
|
|
||
| As will be explained in the following sections, pruning should be configured from | ||
| the start, i.e. late changes to the pruning configuration may not be optimal for | ||
| the node's health. | ||
|
|
||
| ## Consensus Pruning | ||
|
|
||
| To configure pruning of the consensus data amend your node's configuration with: | ||
|
|
||
| ```yaml | ||
| # ... sections not relevant are omitted ... | ||
| consensus: | ||
| # ... sections not relevant are omitted ... | ||
| prune: | ||
| strategy: "keep_n" | ||
| num_kept: <n> | ||
| ``` | ||
| where `<n>` is the number of blocks and state versions that will be kept. | ||
|
|
||
| ### Suggested Pruning Configuration | ||
|
|
||
| For normal node operation, the minimum requirements is `n=250_000`. However, | ||
| it is recommended to set it to `n=400_000`. Assuming blocks are produced every 6 | ||
| seconds, the latter corresponds to preserving 1 month of the consensus data. | ||
|
|
||
| Operators that are not resouce-restrained and want to contribute to the overall | ||
| network health are encouraged to set `n=5_000_000` (approximately 1 year of data). | ||
|
|
||
| ### Late Pruning | ||
|
|
||
| Due to the [LSM Tree][] design of the underlying databases, enabling pruning after | ||
| your node has been runing for a while, or possibly changing configuration to retain | ||
| less data, may may not work as expected. | ||
|
|
||
| To change pruning configuration operators must follow this steps: | ||
| 1. Gracefuly shutdown the node (validators should take [special care][graceful shutdown]). | ||
| 2. Configure pruning as described in [Consensus Pruning](#consensus-pruning). | ||
| 3. Run offline pruning (TODO insert link). | ||
| 4. Run the [compaction command][]. | ||
| 5. Start the node again. | ||
|
|
||
| [LSM tree]: https://en.wikipedia.org/wiki/Log-structured_merge-tree | ||
| [graceful shutdown]: ../maintenance/shutting-down-a-node.md | ||
| [compaction command]: https://github.com/oasisprotocol/oasis-core/blob/master/docs/oasis-node/cli.md#compact-experimental | ||
|
|
||
| ## Paratime Pruning | ||
|
|
||
|
|
||
| To configure pruning of the paratime data ammend your node's configuration with: | ||
|
|
||
| ```yaml | ||
| # ... sections not relevant are omitted ... | ||
| runtime: | ||
| # ... sections not relevant are omitted ... | ||
| prune: | ||
| strategy: "keep_last" | ||
| num_kept: <n> | ||
| ``` | ||
|
|
||
| where `<n>` is the number of paratime's state versions that will be kept. | ||
|
|
||
| ### Suggested Pruning Configuration | ||
|
|
||
| Paratime state is much larger than the consensus state. As a consequence, as the | ||
| number of versions in the state database grows, pruning becomes slower and slower. | ||
| For this reason **paratime state pruning must be configured | ||
| from the start or not configured at all**. | ||
|
|
||
| To configure pruning it is recommended to set `n=250_000`. The maximum value is | ||
| `n=400_000`. If you need to preserve more data (e.g. nodes serving historical | ||
|
Comment on lines
+75
to
+76
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We should preserve enough data so that there is sufficiently old state for 2 latest checkpoints, taking creation time into account as well. I believe With so little state, creating runtime checkpoint is very fast. If I remember correctly cca 20min and also much less IO intensive than what I believe devops have observed on the Observer nodes (with all historical state)... Consensus checkpoint creation compared to runtime checkpoint creation is instant (seconds). |
||
| state) you will have to keep the entire state from the genesis. | ||
|
Comment on lines
+75
to
+77
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Unfortunately if you preserve runtime state, to serve historical queries, you are also forced to preserve all the consensus data, as consensus light block are computed ad-hoc (see). A requirement that should probably be documented somewhere. We could open an issue, so that consensus light blocks are not computed ad-hoc and are instead stored in the separate DB, with different pruning config. This would allow to run Observer node with much lower hardware requirements, assuming most of the consensus data could be dropped. Not sure this is a critical priority right now, but issue definitely cannot hurt... |
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -106,6 +106,7 @@ export const sidebarNode: SidebarsConfig = { | |
| }, | ||
| items: [ | ||
| 'node/run-your-node/advanced/sync-node-using-state-sync', | ||
| 'node/run-your-node/advanced/pruning', | ||
|
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. As already mentioned there is nothing "advanced" about pruning, in fact pruning should probably enabled via sane defaults (so possibly no need to even configure it).... When documentation is reorganized (out of scope of this PR), this could probably be "How to guide" |
||
| 'node/run-your-node/advanced/copy-state-from-one-node-to-the-other', | ||
| 'node/run-your-node/advanced/remote-signer', | ||
| ], | ||
|
|
@@ -165,7 +166,7 @@ export const sidebarNode: SidebarsConfig = { | |
| label: 'Mainnet', | ||
| }, | ||
| { | ||
| type: 'doc', | ||
| type: 'doc', | ||
| id: 'node/reference/upgrade-logs/testnet', | ||
| label: 'Testnet', | ||
| }, | ||
|
|
||
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We can ad whole
Late Pruningsection as a follow-up once offline pruning command is merged.