Implement Health Endpoint

[jevonearth]

Implement Health Endpoint

Description

Implement the /health endpoint to monitor the health status of the Tezos RPC node. The endpoint should return a 200 OK status if all the following heuristics pass, and a 500 status if any of them fail:

1. The bootstrapped property from the /chains/<chain_id>/is_bootstrapped endpoint must be true.
2. The synced property from the /chains/<chain_id>/is_bootstrapped endpoint must be synced.
3. The sidecar must observe a new block emitted from the /monitor/heads/<chain_id> endpoint within N + minimal_block_delay seconds.

Tasks

• Create a /health endpoint
  • Define the endpoint route
  • Implement the logic to return 200 OK or 500 status based on heuristics
• Monitor the /chains/<chain_id>/is_bootstrapped endpoint
  • Fetch the bootstrapped property
  • Fetch the synced property
• Observe new blocks from /monitor/heads/<chain_id> within the configured time window
  • Fetch and check new block emission
• Fetch the minimal_block_delay from the /chains/<chain_id>/blocks/head/context/constants endpoint when a new chain is detected
  • Store and update the minimal_block_delay constant
• Monitor for changes in the protocol and update the minimal_block_delay constant from the constants RPC when the protocol changes
• Make the additional-time-window value configurable via a command line flag (default to 10 seconds)
  • Implement command line flag parsing
  • Apply the configurable time window in block observation
• Log errors when they occur
  • Define error logging format
  • Implement error logging
• Optionally log HTTP requests to the /health endpoint (enabled by default)
  • Implement request logging
  • Make logging configurable
• Log all underlying heuristic checks that fail when the health check returns negative
  • Implement detailed logging for failed heuristic checks
• Create unit and integration tests
  • Write tests for each heuristic
  • Write tests for error handling
• Update documentation
  • Update README to reflect the new /health endpoint functionality

Acceptance Criteria

• The /health endpoint returns 200 OK when all heuristics pass.
• The /health endpoint returns 500 when any heuristic fails.
• Errors are logged as specified.
• HTTP requests to the /health endpoint are logged by default.
• Failed heuristic checks are logged in detail when the health check returns negative.
• Tests are implemented and passing.
• Documentation is updated to reflect the new feature.

Notes

• The sidecar should fetch the active chain ID once from the /monitor/active_chains endpoint and assume it does not change.
• Ensure that the health checks are efficient and do not overload the Tezos RPC node.
• Implement logging for error conditions and optionally for all HTTP requests to the /health endpoint.