Merge pull request #4119 from ethereum/dev

Release v1.5.0-beta.2

Author: jtraglia

.circleci/config.yml

@@ -36,13 +36,13 @@ commands:
     steps:
       - restore_cached_venv:
           venv_name: v33-pyspec
-          reqs_checksum: cache-{{ checksum "setup.py" }}-{{ checksum "pyproject.toml" }}
+          reqs_checksum: cache-{{ checksum "setup.py" }}-{{ checksum "pyproject.toml" }}-{{ python3 --version }}
   save_pyspec_cached_venv:
     description: Save a venv into a cache with pyspec keys"
     steps:
       - save_cached_venv:
           venv_name: v33-pyspec
-          reqs_checksum: cache-{{ checksum "setup.py" }}-{{ checksum "pyproject.toml" }}
+          reqs_checksum: cache-{{ checksum "setup.py" }}-{{ checksum "pyproject.toml" }}-{{ python3 --version }}
           venv_path: ./venv
 jobs:
   checkout_specs:

.github/workflows/generate_vectors.yml

@@ -17,6 +17,7 @@ on:
 
 jobs:
   generate-tests:
+    timeout-minutes: 720  # 12 hours
     runs-on: [self-hosted-ghr-custom, size-xl-x64, profile-consensusSpecs]
     steps:
       - name: Checkout repository
@@ -25,12 +26,6 @@ jobs:
           repository: 'ethereum/consensus-specs'
           path: 'consensus-specs'
           ref: ${{ inputs.ref || 'dev' }}
-      - name: Checkout consensus-spec-tests repository
-        uses: actions/checkout@v4
-        with:
-          repository: 'ethereum/consensus-spec-tests'
-          path: 'consensus-spec-tests'
-          fetch-depth: 1
       - name: Setup Python
         uses: actions/setup-python@v5
         with:
@@ -39,7 +34,7 @@ jobs:
       - name: Generate tests
         run: |
           cd consensus-specs
-          make -j 16 gen_all 2>&1 | tee ../consensustestgen.log
+          make -j$(nproc) gen_all 2>&1 | tee ../consensustestgen.log
           cp -r presets/ ../consensus-spec-tests/presets
           cp -r configs/ ../consensus-spec-tests/configs
           find . -type d -empty -delete
@@ -78,4 +73,4 @@ jobs:
         uses: actions/upload-artifact@v4
         with:
           name: consensustestgen.log
-          path: consensustestgen.log
+          path: consensustestgen.log

Makefile

@@ -74,7 +74,7 @@ PYSPEC_DIR = $(TEST_LIBS_DIR)/pyspec
 # Create the pyspec for all phases.
 pyspec: $(VENV) setup.py pyproject.toml
 	@echo "Building eth2spec"
-	@$(PYTHON_VENV) -m uv pip install .[docs,lint,test,generator]
+	@$(PYTHON_VENV) -m uv pip install --reinstall-package=eth2spec .[docs,lint,test,generator]
 	@echo "Building all pyspecs"
 	@$(PYTHON_VENV) setup.py pyspecdev
 

README.md

@@ -6,11 +6,11 @@ To learn more about proof-of-stake and sharding, see the [PoS documentation](htt
 
 This repository hosts the current Ethereum proof-of-stake specifications. Discussions about design rationale and proposed changes can be brought up and discussed as issues. Solidified, agreed-upon changes to the spec can be made through pull requests.
 
-## Specs
+## Specifications
 
 [![GitHub release](https://img.shields.io/github/v/release/ethereum/consensus-specs)](https://github.com/ethereum/consensus-specs/releases/) [![PyPI version](https://badge.fury.io/py/eth2spec.svg)](https://badge.fury.io/py/eth2spec) [![testgen](https://github.com/ethereum/consensus-specs/actions/workflows/generate_vectors.yml/badge.svg?branch=dev&event=schedule)](https://github.com/ethereum/consensus-specs/actions/workflows/generate_vectors.yml)
 
-Core specifications for Ethereum proof-of-stake clients can be found in [specs](./specs). These are divided into features.
+Core specifications for Ethereum proof-of-stake clients can be found in [specs](specs). These are divided into features.
 Features are researched and developed in parallel, and then consolidated into sequential upgrades when ready.
 
 ### Stable Specifications
@@ -30,14 +30,6 @@ Features are researched and developed in parallel, and then consolidated into se
 | 5 | **Electra** | TBD | <ul><li>Core</li><ul><li>[Beacon Chain changes](specs/electra/beacon-chain.md)</li><li>[Electra fork](specs/electra/fork.md)</li></ul><li>Additions</li><ul><li>[Light client sync protocol changes](specs/electra/light-client/sync-protocol.md) ([fork](specs/electra/light-client/fork.md), [networking](specs/electra/light-client/p2p-interface.md))</li><li>[Honest validator guide changes](specs/electra/validator.md)</li><li>[P2P networking](specs/electra/p2p-interface.md)</li></ul></ul> |
 | 6 | **Fulu** | TBD | <ul><li>Core</li><ul><li>[Beacon Chain changes](specs/fulu/beacon-chain.md)</li><li>[Fulu fork](specs/fulu/fork.md)</li><li>[Data availability sampling core](specs/fulu/das-core.md)</li><li>[Polynomial commitments sampling](specs/fulu/polynomial-commitments-sampling.md)</li><li>[Fork choice changes](specs/fulu/fork-choice.md)</li></ul><li>Additions</li><ul><li>[P2P networking](specs/fulu/p2p-interface.md)</li><li>[Peer sampling](specs/fulu/peer-sampling.md)</li></ul></ul> |
 
-### Outdated Specifications
-
-| Code Name or Topic | Specs | Notes |
-| - | - | - |
-| Sharding | <ul><li>Core</li><ul><li>[Beacon Chain changes](specs/_features/sharding/beacon-chain.md)</li></ul><li>Additions</li><ul><li>[P2P networking](specs/_features/sharding/p2p-interface.md)</li></ul></ul> |
-| Custody Game | <ul><li>Core</li><ul><li>[Beacon Chain changes](specs/_features/custody_game/beacon-chain.md)</li></ul><li>Additions</li><ul><li>[Honest validator guide changes](specs/_features/custody_game/validator.md)</li></ul></ul> | Dependent on sharding |
-| Data Availability Sampling | <ul><li>Core</li><ul><li>[Core types and functions](specs/_features/das/das-core.md)</li><li>[Fork choice changes](specs/_features/das/fork-choice.md)</li></ul><li>Additions</li><ul><li>[P2P Networking](specs/_features/das/p2p-interface.md)</li><li>[Sampling process](specs/_features/das/sampling.md)</li></ul></ul> | <ul><li> Dependent on sharding</li><li>[Technical explainer](https://hackmd.io/@HWeNw8hNRimMm2m2GH56Cw/B1YJPGkpD)</li></ul> |
-
 ### Accompanying documents can be found in [specs](specs) and include:
 
 * [SimpleSerialize (SSZ) spec](ssz/simple-serialize.md)
@@ -50,7 +42,7 @@ Additional specifications and standards outside of requisite client functionalit
 
 * [Beacon APIs](https://github.com/ethereum/beacon-apis)
 * [Engine APIs](https://github.com/ethereum/execution-apis/tree/main/src/engine)
-* [Beacon Metrics](https://github.com/ethereum/beacon-metrics/)
+* [Beacon Metrics](https://github.com/ethereum/beacon-metrics)
 
 ## Design goals
 
@@ -81,7 +73,8 @@ Documentation on the different components used during spec writing can be found
 
 Conformance tests built from the executable python spec are available in the [Ethereum Proof-of-Stake Consensus Spec Tests](https://github.com/ethereum/consensus-spec-tests) repo. Compressed tarballs are available in [releases](https://github.com/ethereum/consensus-spec-tests/releases).
 
-## Installation and Usage
+## Installation and usage
+
 The consensus-specs repo can be used by running the tests locally or inside a docker container.
 
 To run the tests locally:

configs/mainnet.yaml

@@ -170,6 +170,8 @@ DATA_COLUMN_SIDECAR_SUBNET_COUNT: 128
 MAX_REQUEST_DATA_COLUMN_SIDECARS: 16384
 SAMPLES_PER_SLOT: 8
 CUSTODY_REQUIREMENT: 4
+VALIDATOR_CUSTODY_REQUIREMENT: 8
+BALANCE_PER_ADDITIONAL_CUSTODY_GROUP: 32000000000
 MAX_BLOBS_PER_BLOCK_FULU: 12
 MIN_EPOCHS_FOR_DATA_COLUMN_SIDECARS_REQUESTS: 4096
 

configs/minimal.yaml

@@ -171,6 +171,8 @@ DATA_COLUMN_SIDECAR_SUBNET_COUNT: 128
 MAX_REQUEST_DATA_COLUMN_SIDECARS: 16384
 SAMPLES_PER_SLOT: 8
 CUSTODY_REQUIREMENT: 4
+VALIDATOR_CUSTODY_REQUIREMENT: 8
+BALANCE_PER_ADDITIONAL_CUSTODY_GROUP: 32000000000
 MAX_BLOBS_PER_BLOCK_FULU: 12
 MIN_EPOCHS_FOR_DATA_COLUMN_SIDECARS_REQUESTS: 4096
 

specs/electra/beacon-chain.md

@@ -830,11 +830,9 @@ def process_registry_updates(state: BeaconState) -> None:
     for index, validator in enumerate(state.validators):
         if is_eligible_for_activation_queue(validator):  # [Modified in Electra:EIP7251]
             validator.activation_eligibility_epoch = current_epoch + 1
-
-        if is_active_validator(validator, current_epoch) and validator.effective_balance <= EJECTION_BALANCE:
+        elif is_active_validator(validator, current_epoch) and validator.effective_balance <= EJECTION_BALANCE:
             initiate_validator_exit(state, ValidatorIndex(index))  # [Modified in Electra:EIP7251]
-
-        if is_eligible_for_activation(state, validator):
+        elif is_eligible_for_activation(state, validator):
             validator.activation_epoch = activation_epoch
 ```
 

specs/electra/validator.md

@@ -153,6 +153,40 @@ def get_eth1_pending_deposit_count(state: BeaconState) -> uint64:
         return uint64(0)
 ```
 
+*Note*: Clients will be able to remove the `Eth1Data` polling mechanism in an uncoordinated fashion once the transition period is finished. The transition period is considered finished when a network reaches the point where `state.eth1_deposit_index == state.deposit_requests_start_index`.
+
+```python
+def get_eth1_vote(state: BeaconState, eth1_chain: Sequence[Eth1Block]) -> Eth1Data:
+    # [New in Electra:EIP6110]
+    if state.eth1_deposit_index == state.deposit_requests_start_index:
+        return state.eth1_data
+
+    period_start = voting_period_start_time(state)
+    # `eth1_chain` abstractly represents all blocks in the eth1 chain sorted by ascending block height
+    votes_to_consider = [
+        get_eth1_data(block) for block in eth1_chain
+        if (
+            is_candidate_block(block, period_start)
+            # Ensure cannot move back to earlier deposit contract states
+            and get_eth1_data(block).deposit_count >= state.eth1_data.deposit_count
+        )
+    ]
+
+    # Valid votes already cast during this period
+    valid_votes = [vote for vote in state.eth1_data_votes if vote in votes_to_consider]
+
+    # Default vote on latest eth1 block data in the period range unless eth1 chain is not live
+    # Non-substantive casting for linter
+    state_eth1_data: Eth1Data = state.eth1_data
+    default_vote = votes_to_consider[len(votes_to_consider) - 1] if any(votes_to_consider) else state_eth1_data
+
+    return max(
+        valid_votes,
+        key=lambda v: (valid_votes.count(v), -valid_votes.index(v)),  # Tiebreak by smallest distance
+        default=default_vote
+    )
+```
+
 #### Execution payload
 
 `prepare_execution_payload` is updated from the Deneb specs.

specs/fulu/das-core.md

@@ -25,6 +25,7 @@
   - [`get_data_column_sidecars`](#get_data_column_sidecars)
 - [Custody](#custody)
   - [Custody requirement](#custody-requirement)
+  - [Validator custody](#validator-custody)
   - [Public, deterministic selection](#public-deterministic-selection)
 - [Custody sampling](#custody-sampling)
 - [Extended data](#extended-data)
@@ -72,6 +73,8 @@ The following values are (non-configurable) constants used throughout the specif
 | `SAMPLES_PER_SLOT` | `8` | Number of `DataColumnSidecar` random samples a node queries per slot |
 | `NUMBER_OF_CUSTODY_GROUPS` | `128` | Number of custody groups available for nodes to custody |
 | `CUSTODY_REQUIREMENT` | `4` | Minimum number of custody groups an honest node custodies and serves samples from |
+| `VALIDATOR_CUSTODY_REQUIREMENT` | `8` | Minimum number of custody groups an honest node with validators attached custodies and serves samples from |
+| `BALANCE_PER_ADDITIONAL_CUSTODY_GROUP` | `Gwei(32 * 10**9)` | Balance increment corresponding to one additional group to custody |
 
 ### Containers
 
@@ -224,12 +227,25 @@ def get_data_column_sidecars(signed_block: SignedBeaconBlock,
 
 ### Custody requirement
 
-Columns are grouped into custody groups. Nodes custodying a custody group MUST custody all the columns in that group.
+Columns are grouped into custody groups. Nodes custodying a custody group MUST custody all the columns in that group. When syncing, a node MUST backfill columns from all of its custody groups.
 
 A node *may* choose to custody and serve more than the minimum honesty requirement. Such a node explicitly advertises a number greater than `CUSTODY_REQUIREMENT` through the peer discovery mechanism, specifically by setting a higher value in the `custody_group_count` field within its ENR. This value can be increased up to `NUMBER_OF_CUSTODY_GROUPS`, indicating a super-full node.
 
 A node stores the custodied columns for the duration of the pruning period and responds to peer requests for samples on those columns.
 
+### Validator custody
+
+A node with validators attached downloads and custodies a higher minimum of custody groups per slot, determined by `get_validators_custody_requirement(state, validator_indices)`. Here, `state` is the current `BeaconState` and `validator_indices` is the list of indices corresponding to validators attached to the node. Any node with at least one validator attached, and with the sum of the balances of all attached validators being `total_node_balance`, downloads and custodies `total_node_balance // BALANCE_PER_ADDITIONAL_CUSTODY_GROUP` custody groups per slot, with a minimum of `VALIDATOR_CUSTODY_REQUIREMENT` and of course a maximum of `NUMBER_OF_CUSTODY_GROUPS`.
+
+```python
+def get_validators_custody_requirement(state: BeaconState, validator_indices: Sequence[ValidatorIndex]) -> uint64:
+    total_node_balance = sum(state.balances[index] for index in validator_indices)
+    count = total_node_balance // BALANCE_PER_ADDITIONAL_CUSTODY_GROUP
+    return min(max(count, VALIDATOR_CUSTODY_REQUIREMENT), NUMBER_OF_CUSTODY_GROUPS)
+```
+
+This higher custody is advertised in the node's Metadata by setting a higher `custody_group_count` and in the node's ENR by setting a higher `cgc`. As with the regular custody requirement, a node with validators *may* still choose to custody, advertise and serve more than this minimum. As with the regular custody requirement, a node MUST backfill columns when syncing. In addition, when the validator custody requirement increases, due to an increase in the total balance of the attached validators, a node MUST backfill columns from the new custody groups. However, a node *may* wait to advertise a higher custody in its Metadata and ENR until backfilling is complete.
+
 ### Public, deterministic selection
 
 The particular columns/groups that a node custodies are selected pseudo-randomly as a function (`get_custody_groups`) of the node-id and custody size -- importantly this function can be run by any party as the inputs are all public.

specs/fulu/fork.md

@@ -130,7 +130,7 @@ def upgrade_to_fulu(pre: electra.BeaconState) -> BeaconState:
         earliest_exit_epoch=pre.earliest_exit_epoch,
         consolidation_balance_to_consume=pre.consolidation_balance_to_consume,
         earliest_consolidation_epoch=pre.earliest_consolidation_epoch,
-        pending_balance_deposits=pre.pending_balance_deposits,
+        pending_deposits=pre.pending_deposits,
         pending_partial_withdrawals=pre.pending_partial_withdrawals,
         pending_consolidations=pre.pending_consolidations,
     )