docs/node/run-your-node/advanced: Add new pruning section #1526
Conversation
✅ Deploy Preview for oasisprotocol-docs ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
| To configure pruning it is recommended to set `n=250_000`. The maximum value is | ||
| `n=400_000`. If you need to preserve more data (e.g. nodes serving historical | ||
| state) you will have to keep the entire state from the genesis. |
There was a problem hiding this comment.
Unfortunately if you preserve runtime state, to serve historical queries, you are also forced to preserve all the consensus data, as consensus light block are computed ad-hoc (see). A requirement that should probably be documented somewhere.
We could open an issue, so that consensus light blocks are not computed ad-hoc and are instead stored in the separate DB, with different pruning config. This would allow to run Observer node with much lower hardware requirements, assuming most of the consensus data could be dropped.
Not sure this is a critical priority right now, but issue definitely cannot hurt...
| }, | ||
| items: [ | ||
| 'node/run-your-node/advanced/sync-node-using-state-sync', | ||
| 'node/run-your-node/advanced/pruning', |
There was a problem hiding this comment.
As already mentioned there is nothing "advanced" about pruning, in fact pruning should probably enabled via sane defaults (so possibly no need to even configure it)....
When documentation is reorganized (out of scope of this PR), this could probably be "How to guide"
martintomazic
force-pushed
the
martin/task/update-pruning-documentation
branch
from
af27d24 to
f4fe4e5
Compare
| To change pruning configuration operators must follow this steps: | ||
| 1. Gracefuly shutdown the node (validators should take [special care][graceful shutdown]). | ||
| 2. Configure pruning as described in [Consensus Pruning](#consensus-pruning). | ||
| 3. Run offline pruning (TODO insert link). |
There was a problem hiding this comment.
We can ad whole Late Pruning section as a follow-up once offline pruning command is merged.
| To configure pruning it is recommended to set `n=250_000`. The maximum value is | ||
| `n=400_000`. If you need to preserve more data (e.g. nodes serving historical |
There was a problem hiding this comment.
We should preserve enough data so that there is sufficiently old state for 2 latest checkpoints, taking creation time into account as well. I believe 250_000 versions is suitable for the current parameters.
With so little state, creating runtime checkpoint is very fast. If I remember correctly cca 20min and also much less IO intensive than what I believe devops have observed on the Observer nodes (with all historical state)... Consensus checkpoint creation compared to runtime checkpoint creation is instant (seconds).
Closes #1469.
Motivation
Today's thread shows pruning has corner cases that are confusing and we should warn about them.
Considerations
Possible follow-up
#1506.