feat(benchmark): add SLOAD/SSTORE benchmark test with multi-contract support #2256
+753
−0
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
marioevz
reviewed
Oct 3, 2025
CPerezz
commented
Oct 6, 2025
Add test_sload_empty_erc20_balanceof to benchmark SLOAD operations on non-existing storage slots using ERC20 balanceOf() queries. The idea of this benchmark is to exploit within a single or series of N contracts calls to non-existing addresses. On this way, we force clients to resolve as many tree branches as possible.
Add test_sstore_erc20_approve that benchmarks SSTORE operations by calling approve(spender, amount) on pre-deployed ERC20 contracts. Follows the same pattern as the SLOAD benchmark: - Auto-discovers ERC20 contracts from stubs - Splits gas budget evenly across all discovered contracts - Uses counter as both spender address and amount - Forces SSTOREs to allowance mapping storage slots The test measures client performance when writing to many storage slots across multiple contracts, stressing state-handling write operations.
Fixed gas calculation for test_sstore_erc20_approve to ensure accurate gas usage prediction and prevent transaction reverts: Key fixes: - Added memory expansion cost (15 gas per contract) - Corrected G_LOW gas values in comments (5 gas, not 3) - Separated per-contract overhead from per-iteration costs - Improved cost calculation clarity with detailed opcode breakdown Gas calculation (10M gas, 3 contracts): - Intrinsic: 21,000 - Overhead per contract: 38 - Cost per iteration: 20,226 - Calls per contract: 164 - Expected gas used: 9,972,306 (99.72% utilization)
…atios Add test_mixed_sload_sstore to test_multi_opcode.py that combines SLOAD and SSTORE operations with parameterized gas distribution ratios (50-50, 70-30, 90-10). The test stresses clients with mixed read/write workloads by: - Dividing gas budget evenly across all discovered ERC20 contract stubs - Splitting each contract's allocation by the specified percentage ratio - Executing balanceOf (cold SLOAD on empty slots) for the SLOAD portion - Executing approve (SSTORE to new allowance slots) for the SSTORE portion Verified gas calculations for 10M gas budget with 3 contracts (50-50 ratio): - SLOAD operations: ~2,312 gas/iteration → 719 calls per contract - SSTORE operations: ~20,226 gas/iteration → 82 calls per contract - Total operations: 2,403 state operations (2,157 SLOADs + 246 SSTOREs) - Gas usage: 9.98M / 10M (16K buffer, no out-of-gas errors) This benchmark enables testing different read/write ratios to identify client performance characteristics under varying state operation mixes.
…back Address review comments by optimizing loop efficiency: 1. Move function selector MSTORE outside loops (Comment ethereum#2) - BALANCEOF_SELECTOR and APPROVE_SELECTOR now stored once per contract - Saves 3 gas (G_VERY_LOW) per iteration - Total savings: ~6,471 gas for 50-50 ratio with 10M budget and 3 contracts 2. Remove unused return data from CALL operations (Comment ethereum#1) - Changed ret_offset=96/128, ret_size=32 to ret_offset=0, ret_size=0 - Eliminates unnecessary memory expansion - Minor gas savings, cleaner implementation Skipped Comment ethereum#3 (use Op.GAS for addresses): - Would lose determinism (GAS varies per iteration) - Adds complexity for minimal benefit - Counter still needed for loop control Changes applied to: - test_sload_empty_erc20_balanceof - test_sstore_erc20_approve - test_mixed_sload_sstore (both SLOAD and SSTORE loops)
CPerezz
force-pushed
the
feat/bloatnet-sload-sstore-benchmarks
branch
from
e0ae1ee to
552638e
Compare
LouisTsai-Csie
requested changes
Oct 7, 2025
There was a problem hiding this comment.
@CPerezz , I’ve left some suggestions. Please take a look and let me know if they’re unclear or not practical. These changes might not reduce much gas usage, but i wonder if they could help simplify the layout a bit.
I've not yet reviewed test_multi_opcode.py, but i believe it would be quick if we have consensus on the other test cases!
CPerezz
changed the title
…alldata encoding - Move selector MSTORE outside for-loop (saves gas per contract) - Use single counter at MEM[32] instead of duplicate at MEM[0] and MEM[64] - Fix calldata encoding by using args_offset=28 for correct ABI format - Selector now properly positioned at start of calldata
…calldata encoding - Move selector MSTORE outside for-loop (saves gas per contract) - Use single counter at MEM[32] instead of duplicate at MEM[0] - Fix calldata encoding by using args_offset=28 for correct ABI format - Selector now properly positioned at start of calldata
…x calldata encoding - Move selectors MSTORE outside for-loop (saves gas per contract) - Use separate memory regions for balanceOf and approve to avoid conflicts - Fix calldata encoding by using correct args_offset for proper ABI format - Selectors now properly positioned at start of calldata
…stently - Reuse MEM[0] for both selectors (sequential operations, no conflict) - Reuse MEM[32] for both counters (balanceOf then approve) - Reuse MEM[64] and MEM[96] for parameters - Consistent args_offset=28 for both operations (was 28 and 128) - Matches single-opcode test pattern for easier understanding - Reduces memory footprint from 196 bytes to 96 bytes
LouisTsai-Csie
requested changes
Oct 13, 2025
|
Hey, just an FYI. We are about to finalize "The Weld" - moving EEST to EELS. It's best to keep this PR up here in EEST for now. We will then ask you to reopen the PR but in EELS somepoint next week. Hope that okay! More context: https://steel.ethereum.foundation/blog/blog_posts/2025-09-11_weld-announcement/ |
…o single-opcode tests - Add parametrization for num_contracts [1, 5, 10, 20, 100] - Implement stub prefix filtering based on test function name - Add validation to error if insufficient matching stubs - Add SSTORE benchmark architecture documentation - Create README with setup instructions and stubs.json format
… correct gas calculations - Add num_contracts parametrization [1, 5, 10, 20, 100] to multi-opcode test - Implement stub prefix filtering for all benchmarks - Fix gas cost calculations to account for COLD/WARM account access - CALL operations: first call to each contract is COLD (2600), subsequent are WARM (100) - SSTORE operations: add cold storage access cost (2100) for zero-to-non-zero writes - Update gas calculation formulas to solve for calls per contract correctly
LouisTsai-Csie
requested changes
Oct 17, 2025
There was a problem hiding this comment.
@CPerezz Only one comment from my side! But i would leave the final review to @marioevz as you had a discussion about the stub restructuring.
And here i assume the gas usage fits in your scenario, so i do not strictly calculated the gas cost here - we could do it later using this feature: ethereum/execution-specs#1479
… correct gas calculations - Add num_contracts parametrization [1, 5, 10, 20, 100] to multi-opcode test - Implement stub prefix filtering for all benchmarks - Fix gas cost calculations to account for COLD/WARM account access - CALL operations: first call to each contract is COLD (2600), subsequent are WARM (100) - SSTORE operations: add cold storage access cost (2100) for zero-to-non-zero writes - Update gas calculation formulas to solve for calls per contract correctly
LouisTsai-Csie
approved these changes
Oct 23, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
🗒️ Description
Add test_sload_empty_erc20_balanceof to benchmark SLOAD operations on non-existing storage slots using ERC20 balanceOf() queries.
The idea of this benchmark is to exploit within a single or series of N contracts calls to non-existing addresses. On this way, we force clients to resolve as many tree branches as possible.
✅ Checklist
toxchecks to avoid unnecessary CI fails, see also Code Standards and Enabling Pre-commit Checks:uvx --with=tox-uv tox -e lint,typecheck,spellcheck,markdownlinttype(scope):.mkdocs servelocally and verified the auto-generated docs for new tests in the Test Case Reference are correctly formatted.@ported_frommarker.