Skip to content

[Feature] Trainer Algorithms - Configuration System #3189

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 19 commits into
base: gh/vmoens/152/base
Choose a base branch
from
Open

[Feature] Trainer Algorithms - Configuration System #3189

wants to merge 19 commits into from
+401 −1

Conversation

vmoens
Copy link
Collaborator

@vmoens vmoens commented Oct 14, 2025 •
edited
Loading

@vmoens
Update
40c8b63 
[ghstack-poisoned]
@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Oct 14, 2025 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/3189

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

❌ 7 New Failures, 9 Unrelated Failures

As of commit 3092841 with merge base e7ec9c3 (image):

NEW FAILURES - The following jobs have failed:

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

This was referenced Oct 14, 2025
Merged
Merged
Merged
Merged
Merged
Open
Open
Open
Open
@meta-cla meta-cla bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Oct 14, 2025
vmoens added 7 commits October 16, 2025 15:00
@vmoens
Update
74bfb5a 
[ghstack-poisoned]
@vmoens
Update
bfd08f5 
[ghstack-poisoned]
@vmoens
Update
eb85308 
[ghstack-poisoned]
@vmoens
Update
0d1f7ce 
[ghstack-poisoned]
@vmoens
Update
351326c 
[ghstack-poisoned]
@vmoens
Update
bb95ddb 
[ghstack-poisoned]
@vmoens
Update
5fcf578 
[ghstack-poisoned]
@vmoens vmoens mentioned this pull request Oct 18, 2025
Open
vmoens added 4 commits October 18, 2025 19:00
@vmoens
Update
67659d3 
[ghstack-poisoned]
@vmoens
Update
1f9b4c8 
[ghstack-poisoned]
@vmoens
Update
36e3097 
[ghstack-poisoned]
@vmoens
Update
5b3b689 
[ghstack-poisoned]
@vmoens vmoens added the enhancement New feature or request label Oct 20, 2025
@vmoens
Update
ddb96c1 
[ghstack-poisoned]
@vmoens vmoens mentioned this pull request Oct 20, 2025
Merged
@vmoens
Update
920f08a 
[ghstack-poisoned]
@vmoens vmoens mentioned this pull request Oct 20, 2025
Open
@vmoens
Update
e5e892f 
[ghstack-poisoned]
@github-actions GitHub Actions
Copy link

github-actions bot commented Oct 20, 2025 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 154. Improved: $\large\color{#35bf28}14$. Worsened: $\large\color{#d91a1a}8$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_tensor_to_bytestream_speed[pickle] 82.8813μs 81.1874μs 12.3172 KOps/s 12.3188 KOps/s $\color{#d91a1a}-0.01\%$
test_tensor_to_bytestream_speed[torch.save] 0.1389ms 0.1375ms 7.2705 KOps/s 7.1983 KOps/s $\color{#35bf28}+1.00\%$
test_tensor_to_bytestream_speed[untyped_storage] 0.1147s 0.1139s 8.7798 Ops/s 8.7290 Ops/s $\color{#35bf28}+0.58\%$
test_tensor_to_bytestream_speed[numpy] 2.8480μs 2.8433μs 351.7076 KOps/s 359.5939 KOps/s $\color{#d91a1a}-2.19\%$
test_tensor_to_bytestream_speed[safetensors] 44.2554μs 43.9254μs 22.7659 KOps/s 22.5004 KOps/s $\color{#35bf28}+1.18\%$
test_simple 0.5376s 0.5361s 1.8653 Ops/s 1.7786 Ops/s $\color{#35bf28}+4.87\%$
test_transformed 1.2000s 1.1085s 0.9021 Ops/s 0.8957 Ops/s $\color{#35bf28}+0.72\%$
test_serial 1.6174s 1.6131s 0.6199 Ops/s 0.6071 Ops/s $\color{#35bf28}+2.12\%$
test_parallel 1.1318s 1.0765s 0.9289 Ops/s 0.9485 Ops/s $\color{#d91a1a}-2.06\%$
test_step_mdp_speed[True-True-True-True-True] 0.2089ms 44.3835μs 22.5309 KOps/s 22.6676 KOps/s $\color{#d91a1a}-0.60\%$
test_step_mdp_speed[True-True-True-True-False] 45.6600μs 25.2234μs 39.6457 KOps/s 41.5727 KOps/s $\color{#d91a1a}-4.64\%$
test_step_mdp_speed[True-True-True-False-True] 56.2800μs 25.0457μs 39.9270 KOps/s 40.0560 KOps/s $\color{#d91a1a}-0.32\%$
test_step_mdp_speed[True-True-True-False-False] 37.4500μs 13.7532μs 72.7105 KOps/s 71.9309 KOps/s $\color{#35bf28}+1.08\%$
test_step_mdp_speed[True-True-False-True-True] 82.5410μs 47.4578μs 21.0714 KOps/s 21.3813 KOps/s $\color{#d91a1a}-1.45\%$
test_step_mdp_speed[True-True-False-True-False] 64.2510μs 27.8627μs 35.8903 KOps/s 36.7766 KOps/s $\color{#d91a1a}-2.41\%$
test_step_mdp_speed[True-True-False-False-True] 57.2200μs 27.7515μs 36.0341 KOps/s 36.7866 KOps/s $\color{#d91a1a}-2.05\%$
test_step_mdp_speed[True-True-False-False-False] 47.1200μs 16.6237μs 60.1550 KOps/s 60.8426 KOps/s $\color{#d91a1a}-1.13\%$
test_step_mdp_speed[True-False-True-True-True] 79.3210μs 49.7244μs 20.1108 KOps/s 20.2079 KOps/s $\color{#d91a1a}-0.48\%$
test_step_mdp_speed[True-False-True-True-False] 54.5010μs 30.2952μs 33.0086 KOps/s 33.4229 KOps/s $\color{#d91a1a}-1.24\%$
test_step_mdp_speed[True-False-True-False-True] 55.2100μs 27.7772μs 36.0007 KOps/s 36.1516 KOps/s $\color{#d91a1a}-0.42\%$
test_step_mdp_speed[True-False-True-False-False] 55.2910μs 16.5357μs 60.4754 KOps/s 60.7346 KOps/s $\color{#d91a1a}-0.43\%$
test_step_mdp_speed[True-False-False-True-True] 0.1138ms 53.1774μs 18.8050 KOps/s 19.1547 KOps/s $\color{#d91a1a}-1.83\%$
test_step_mdp_speed[True-False-False-True-False] 72.5910μs 32.9681μs 30.3323 KOps/s 30.4510 KOps/s $\color{#d91a1a}-0.39\%$
test_step_mdp_speed[True-False-False-False-True] 53.2700μs 30.3232μs 32.9780 KOps/s 32.6832 KOps/s $\color{#35bf28}+0.90\%$
test_step_mdp_speed[True-False-False-False-False] 47.3900μs 19.3179μs 51.7655 KOps/s 52.2181 KOps/s $\color{#d91a1a}-0.87\%$
test_step_mdp_speed[False-True-True-True-True] 77.7510μs 50.3102μs 19.8767 KOps/s 19.8895 KOps/s $\color{#d91a1a}-0.06\%$
test_step_mdp_speed[False-True-True-True-False] 0.1053ms 30.5321μs 32.7524 KOps/s 33.2820 KOps/s $\color{#d91a1a}-1.59\%$
test_step_mdp_speed[False-True-True-False-True] 2.4802ms 32.3062μs 30.9538 KOps/s 31.5837 KOps/s $\color{#d91a1a}-1.99\%$
test_step_mdp_speed[False-True-True-False-False] 54.2600μs 18.8585μs 53.0265 KOps/s 54.5023 KOps/s $\color{#d91a1a}-2.71\%$
test_step_mdp_speed[False-True-False-True-True] 87.3610μs 52.8742μs 18.9128 KOps/s 19.0200 KOps/s $\color{#d91a1a}-0.56\%$
test_step_mdp_speed[False-True-False-True-False] 58.9900μs 33.4314μs 29.9120 KOps/s 30.4162 KOps/s $\color{#d91a1a}-1.66\%$
test_step_mdp_speed[False-True-False-False-True] 66.1710μs 34.0016μs 29.4104 KOps/s 29.5274 KOps/s $\color{#d91a1a}-0.40\%$
test_step_mdp_speed[False-True-False-False-False] 47.0900μs 20.9955μs 47.6292 KOps/s 47.9859 KOps/s $\color{#d91a1a}-0.74\%$
test_step_mdp_speed[False-False-True-True-True] 95.6110μs 55.6827μs 17.9589 KOps/s 17.8658 KOps/s $\color{#35bf28}+0.52\%$
test_step_mdp_speed[False-False-True-True-False] 63.3010μs 35.9555μs 27.8122 KOps/s 28.1269 KOps/s $\color{#d91a1a}-1.12\%$
test_step_mdp_speed[False-False-True-False-True] 64.1510μs 34.4491μs 29.0283 KOps/s 29.2312 KOps/s $\color{#d91a1a}-0.69\%$
test_step_mdp_speed[False-False-True-False-False] 51.2500μs 21.1300μs 47.3261 KOps/s 47.8185 KOps/s $\color{#d91a1a}-1.03\%$
test_step_mdp_speed[False-False-False-True-True] 83.3410μs 57.0937μs 17.5151 KOps/s 17.4341 KOps/s $\color{#35bf28}+0.46\%$
test_step_mdp_speed[False-False-False-True-False] 67.2410μs 38.5784μs 25.9212 KOps/s 26.2897 KOps/s $\color{#d91a1a}-1.40\%$
test_step_mdp_speed[False-False-False-False-True] 62.3100μs 36.3051μs 27.5443 KOps/s 27.7192 KOps/s $\color{#d91a1a}-0.63\%$
test_step_mdp_speed[False-False-False-False-False] 62.1000μs 23.5762μs 42.4157 KOps/s 43.4400 KOps/s $\color{#d91a1a}-2.36\%$
test_values[generalized_advantage_estimate-True-True] 9.8065ms 9.6455ms 103.6753 Ops/s 106.3752 Ops/s $\color{#d91a1a}-2.54\%$
test_values[vec_generalized_advantage_estimate-True-True] 19.4124ms 17.4171ms 57.4148 Ops/s 57.4042 Ops/s $\color{#35bf28}+0.02\%$
test_values[td0_return_estimate-False-False] 0.2230ms 0.1286ms 7.7768 KOps/s 8.0765 KOps/s $\color{#d91a1a}-3.71\%$
test_values[td1_return_estimate-False-False] 25.8208ms 25.2423ms 39.6160 Ops/s 39.0521 Ops/s $\color{#35bf28}+1.44\%$
test_values[vec_td1_return_estimate-False-False] 18.4003ms 17.4976ms 57.1508 Ops/s 57.2246 Ops/s $\color{#d91a1a}-0.13\%$
test_values[td_lambda_return_estimate-True-False] 39.3268ms 38.3848ms 26.0520 Ops/s 26.1710 Ops/s $\color{#d91a1a}-0.45\%$
test_values[vec_td_lambda_return_estimate-True-False] 17.7026ms 17.4157ms 57.4196 Ops/s 57.3038 Ops/s $\color{#35bf28}+0.20\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 8.1928ms 8.0550ms 124.1461 Ops/s 124.2905 Ops/s $\color{#d91a1a}-0.12\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 1.6965ms 1.4774ms 676.8432 Ops/s 673.5092 Ops/s $\color{#35bf28}+0.50\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.5154ms 0.3999ms 2.5006 KOps/s 2.5395 KOps/s $\color{#d91a1a}-1.53\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 34.4403ms 33.6949ms 29.6781 Ops/s 34.1977 Ops/s $\textbf{\color{#d91a1a}-13.22\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 1.8836ms 1.6888ms 592.1328 Ops/s 583.7344 Ops/s $\color{#35bf28}+1.44\%$
test_dqn_speed[False-None] 2.0603ms 1.4074ms 710.5468 Ops/s 718.3777 Ops/s $\color{#d91a1a}-1.09\%$
test_dqn_speed[False-backward] 1.9991ms 1.9105ms 523.4336 Ops/s 533.4615 Ops/s $\color{#d91a1a}-1.88\%$
test_dqn_speed[True-None] 0.5923ms 0.5117ms 1.9543 KOps/s 1.9756 KOps/s $\color{#d91a1a}-1.08\%$
test_dqn_speed[True-backward] 0.9908ms 0.9482ms 1.0546 KOps/s 1.0308 KOps/s $\color{#35bf28}+2.31\%$
test_dqn_speed[reduce-overhead-None] 0.6336ms 0.5086ms 1.9661 KOps/s 1.9889 KOps/s $\color{#d91a1a}-1.14\%$
test_dqn_speed[reduce-overhead-backward] 0.9721ms 0.9361ms 1.0683 KOps/s 940.4733 Ops/s $\textbf{\color{#35bf28}+13.59\%}$
test_ddpg_speed[False-None] 3.0922ms 2.8274ms 353.6761 Ops/s 354.7590 Ops/s $\color{#d91a1a}-0.31\%$
test_ddpg_speed[False-backward] 4.2885ms 4.0297ms 248.1590 Ops/s 253.1331 Ops/s $\color{#d91a1a}-1.97\%$
test_ddpg_speed[True-None] 1.5301ms 1.3917ms 718.5434 Ops/s 720.8596 Ops/s $\color{#d91a1a}-0.32\%$
test_ddpg_speed[True-backward] 2.4550ms 2.3150ms 431.9590 Ops/s 419.7350 Ops/s $\color{#35bf28}+2.91\%$
test_ddpg_speed[reduce-overhead-None] 1.6019ms 1.3834ms 722.8416 Ops/s 723.3759 Ops/s $\color{#d91a1a}-0.07\%$
test_ddpg_speed[reduce-overhead-backward] 2.4035ms 2.3196ms 431.1106 Ops/s 422.7070 Ops/s $\color{#35bf28}+1.99\%$
test_sac_speed[False-None] 8.4599ms 7.8188ms 127.8972 Ops/s 129.7984 Ops/s $\color{#d91a1a}-1.46\%$
test_sac_speed[False-backward] 11.8351ms 11.0647ms 90.3773 Ops/s 91.9626 Ops/s $\color{#d91a1a}-1.72\%$
test_sac_speed[True-None] 2.3378ms 2.1081ms 474.3657 Ops/s 460.3261 Ops/s $\color{#35bf28}+3.05\%$
test_sac_speed[True-backward] 4.1965ms 4.0187ms 248.8361 Ops/s 222.2756 Ops/s $\textbf{\color{#35bf28}+11.95\%}$
test_sac_speed[reduce-overhead-None] 2.6235ms 2.1139ms 473.0626 Ops/s 468.0929 Ops/s $\color{#35bf28}+1.06\%$
test_sac_speed[reduce-overhead-backward] 4.3521ms 4.0523ms 246.7755 Ops/s 247.2450 Ops/s $\color{#d91a1a}-0.19\%$
test_redq_speed[False-None] 10.5825ms 10.1584ms 98.4404 Ops/s 100.0806 Ops/s $\color{#d91a1a}-1.64\%$
test_redq_speed[False-backward] 23.6128ms 17.9813ms 55.6134 Ops/s 57.1733 Ops/s $\color{#d91a1a}-2.73\%$
test_redq_speed[True-None] 4.5217ms 4.3423ms 230.2948 Ops/s 225.9800 Ops/s $\color{#35bf28}+1.91\%$
test_redq_speed[True-backward] 10.0999ms 9.7807ms 102.2417 Ops/s 103.9141 Ops/s $\color{#d91a1a}-1.61\%$
test_redq_speed[reduce-overhead-None] 4.5865ms 4.2330ms 236.2389 Ops/s 233.3955 Ops/s $\color{#35bf28}+1.22\%$
test_redq_speed[reduce-overhead-backward] 10.0576ms 9.8445ms 101.5795 Ops/s 101.9191 Ops/s $\color{#d91a1a}-0.33\%$
test_redq_deprec_speed[False-None] 11.3473ms 10.8253ms 92.3765 Ops/s 93.9143 Ops/s $\color{#d91a1a}-1.64\%$
test_redq_deprec_speed[False-backward] 16.1155ms 15.5539ms 64.2927 Ops/s 65.4999 Ops/s $\color{#d91a1a}-1.84\%$
test_redq_deprec_speed[True-None] 3.8806ms 3.5718ms 279.9718 Ops/s 269.2682 Ops/s $\color{#35bf28}+3.98\%$
test_redq_deprec_speed[True-backward] 7.5867ms 7.3762ms 135.5703 Ops/s 124.9539 Ops/s $\textbf{\color{#35bf28}+8.50\%}$
test_redq_deprec_speed[reduce-overhead-None] 3.6977ms 3.4955ms 286.0834 Ops/s 271.4511 Ops/s $\textbf{\color{#35bf28}+5.39\%}$
test_redq_deprec_speed[reduce-overhead-backward] 7.6560ms 7.4775ms 133.7343 Ops/s 129.5928 Ops/s $\color{#35bf28}+3.20\%$
test_td3_speed[False-None] 7.8301ms 7.6682ms 130.4084 Ops/s 129.8310 Ops/s $\color{#35bf28}+0.44\%$
test_td3_speed[False-backward] 11.5942ms 10.4865ms 95.3603 Ops/s 95.3170 Ops/s $\color{#35bf28}+0.05\%$
test_td3_speed[True-None] 1.8126ms 1.7899ms 558.6844 Ops/s 552.4305 Ops/s $\color{#35bf28}+1.13\%$
test_td3_speed[True-backward] 3.6949ms 3.5819ms 279.1820 Ops/s 269.2519 Ops/s $\color{#35bf28}+3.69\%$
test_td3_speed[reduce-overhead-None] 1.7995ms 1.7661ms 566.2328 Ops/s 565.1861 Ops/s $\color{#35bf28}+0.19\%$
test_td3_speed[reduce-overhead-backward] 3.7174ms 3.6062ms 277.3006 Ops/s 275.1370 Ops/s $\color{#35bf28}+0.79\%$
test_cql_speed[False-None] 26.3092ms 25.3984ms 39.3726 Ops/s 39.2109 Ops/s $\color{#35bf28}+0.41\%$
test_cql_speed[False-backward] 37.3935ms 34.6023ms 28.8998 Ops/s 28.5916 Ops/s $\color{#35bf28}+1.08\%$
test_cql_speed[True-None] 12.7618ms 12.4622ms 80.2428 Ops/s 81.2108 Ops/s $\color{#d91a1a}-1.19\%$
test_cql_speed[True-backward] 18.6602ms 18.2681ms 54.7404 Ops/s 54.7248 Ops/s $\color{#35bf28}+0.03\%$
test_cql_speed[reduce-overhead-None] 12.7115ms 12.4529ms 80.3027 Ops/s 80.2850 Ops/s $\color{#35bf28}+0.02\%$
test_cql_speed[reduce-overhead-backward] 19.1634ms 18.3787ms 54.4107 Ops/s 54.5116 Ops/s $\color{#d91a1a}-0.19\%$
test_a2c_speed[False-None] 5.5645ms 5.2866ms 189.1573 Ops/s 182.5466 Ops/s $\color{#35bf28}+3.62\%$
test_a2c_speed[False-backward] 11.9723ms 11.6635ms 85.7376 Ops/s 84.5498 Ops/s $\color{#35bf28}+1.40\%$
test_a2c_speed[True-None] 3.7947ms 3.6565ms 273.4834 Ops/s 263.8783 Ops/s $\color{#35bf28}+3.64\%$
test_a2c_speed[True-backward] 8.8290ms 8.6481ms 115.6322 Ops/s 99.6892 Ops/s $\textbf{\color{#35bf28}+15.99\%}$
test_a2c_speed[reduce-overhead-None] 3.8332ms 3.6738ms 272.2014 Ops/s 266.3745 Ops/s $\color{#35bf28}+2.19\%$
test_a2c_speed[reduce-overhead-backward] 8.9149ms 8.7438ms 114.3664 Ops/s 112.8367 Ops/s $\color{#35bf28}+1.36\%$
test_ppo_speed[False-None] 6.3011ms 5.8377ms 171.3016 Ops/s 169.4671 Ops/s $\color{#35bf28}+1.08\%$
test_ppo_speed[False-backward] 12.7442ms 12.3954ms 80.6753 Ops/s 79.4158 Ops/s $\color{#35bf28}+1.59\%$
test_ppo_speed[True-None] 3.7601ms 3.6354ms 275.0694 Ops/s 269.9078 Ops/s $\color{#35bf28}+1.91\%$
test_ppo_speed[True-backward] 8.6135ms 8.3951ms 119.1168 Ops/s 109.5538 Ops/s $\textbf{\color{#35bf28}+8.73\%}$
test_ppo_speed[reduce-overhead-None] 4.0798ms 3.6156ms 276.5763 Ops/s 277.3953 Ops/s $\color{#d91a1a}-0.30\%$
test_ppo_speed[reduce-overhead-backward] 8.8079ms 8.6168ms 116.0521 Ops/s 114.4860 Ops/s $\color{#35bf28}+1.37\%$
test_reinforce_speed[False-None] 4.7310ms 4.5450ms 220.0225 Ops/s 221.3946 Ops/s $\color{#d91a1a}-0.62\%$
test_reinforce_speed[False-backward] 8.2617ms 7.3768ms 135.5596 Ops/s 137.6430 Ops/s $\color{#d91a1a}-1.51\%$
test_reinforce_speed[True-None] 3.0248ms 2.8649ms 349.0483 Ops/s 344.7647 Ops/s $\color{#35bf28}+1.24\%$
test_reinforce_speed[True-backward] 7.9378ms 7.6571ms 130.5980 Ops/s 99.8382 Ops/s $\textbf{\color{#35bf28}+30.81\%}$
test_reinforce_speed[reduce-overhead-None] 2.9938ms 2.8279ms 353.6158 Ops/s 333.3265 Ops/s $\textbf{\color{#35bf28}+6.09\%}$
test_reinforce_speed[reduce-overhead-backward] 8.0870ms 7.8864ms 126.8006 Ops/s 125.6906 Ops/s $\color{#35bf28}+0.88\%$
test_iql_speed[False-None] 25.1803ms 20.0174ms 49.9566 Ops/s 50.7686 Ops/s $\color{#d91a1a}-1.60\%$
test_iql_speed[False-backward] 35.3800ms 30.0149ms 33.3168 Ops/s 33.4755 Ops/s $\color{#d91a1a}-0.47\%$
test_iql_speed[True-None] 8.8193ms 8.4747ms 117.9979 Ops/s 116.7269 Ops/s $\color{#35bf28}+1.09\%$
test_iql_speed[True-backward] 16.9007ms 16.6443ms 60.0806 Ops/s 59.5460 Ops/s $\color{#35bf28}+0.90\%$
test_iql_speed[reduce-overhead-None] 8.7756ms 8.5202ms 117.3682 Ops/s 112.0296 Ops/s $\color{#35bf28}+4.77\%$
test_iql_speed[reduce-overhead-backward] 17.3802ms 17.0992ms 58.4822 Ops/s 57.9420 Ops/s $\color{#35bf28}+0.93\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 7.3829ms 5.9959ms 166.7816 Ops/s 166.1574 Ops/s $\color{#35bf28}+0.38\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.5172ms 0.2922ms 3.4225 KOps/s 2.9668 KOps/s $\textbf{\color{#35bf28}+15.36\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.5505ms 0.3045ms 3.2839 KOps/s 3.2673 KOps/s $\color{#35bf28}+0.51\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.0068ms 5.7795ms 173.0248 Ops/s 174.8269 Ops/s $\color{#d91a1a}-1.03\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 2.0655ms 0.3287ms 3.0425 KOps/s 2.8441 KOps/s $\textbf{\color{#35bf28}+6.97\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.5608ms 0.2788ms 3.5873 KOps/s 3.9331 KOps/s $\textbf{\color{#d91a1a}-8.79\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.5528ms 1.3236ms 755.5201 Ops/s 828.4507 Ops/s $\textbf{\color{#d91a1a}-8.80\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 1.5700ms 1.2442ms 803.7358 Ops/s 896.8704 Ops/s $\textbf{\color{#d91a1a}-10.38\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 9.8982ms 6.0390ms 165.5906 Ops/s 168.4330 Ops/s $\color{#d91a1a}-1.69\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.8312ms 0.4761ms 2.1005 KOps/s 1.9915 KOps/s $\textbf{\color{#35bf28}+5.47\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.7025ms 0.4283ms 2.3350 KOps/s 2.1674 KOps/s $\textbf{\color{#35bf28}+7.73\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 5.9543ms 5.7529ms 173.8253 Ops/s 171.7860 Ops/s $\color{#35bf28}+1.19\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.5603ms 0.2955ms 3.3837 KOps/s 3.2412 KOps/s $\color{#35bf28}+4.39\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.5329ms 0.3150ms 3.1750 KOps/s 3.6788 KOps/s $\textbf{\color{#d91a1a}-13.69\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 5.9440ms 5.7046ms 175.2971 Ops/s 174.5579 Ops/s $\color{#35bf28}+0.42\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.6865s 1.2951ms 772.1660 Ops/s 3.4267 KOps/s $\textbf{\color{#d91a1a}-77.47\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.5860ms 0.3369ms 2.9682 KOps/s 3.0082 KOps/s $\color{#d91a1a}-1.33\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 5.9899ms 5.8888ms 169.8152 Ops/s 168.3809 Ops/s $\color{#35bf28}+0.85\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 2.1914ms 0.4247ms 2.3548 KOps/s 2.2755 KOps/s $\color{#35bf28}+3.48\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.6160ms 0.4016ms 2.4902 KOps/s 2.3767 KOps/s $\color{#35bf28}+4.77\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 6.5398ms 5.0038ms 199.8497 Ops/s 198.0538 Ops/s $\color{#35bf28}+0.91\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 10.1530ms 2.0358ms 491.1977 Ops/s 485.3334 Ops/s $\color{#35bf28}+1.21\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 6.4665ms 1.1453ms 873.1609 Ops/s 820.2048 Ops/s $\textbf{\color{#35bf28}+6.46\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.4835s 14.5554ms 68.7031 Ops/s 193.9113 Ops/s $\textbf{\color{#d91a1a}-64.57\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 5.7200ms 1.9811ms 504.7653 Ops/s 491.3938 Ops/s $\color{#35bf28}+2.72\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 10.1354ms 1.2755ms 784.0239 Ops/s 834.3702 Ops/s $\textbf{\color{#d91a1a}-6.03\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 9.2710ms 5.2405ms 190.8207 Ops/s 56.3092 Ops/s $\textbf{\color{#35bf28}+238.88\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 8.7443ms 2.1873ms 457.1783 Ops/s 474.8562 Ops/s $\color{#d91a1a}-3.72\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 1.9083ms 1.0232ms 977.3658 Ops/s 1.0042 KOps/s $\color{#d91a1a}-2.67\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-True] 34.1621ms 31.9579ms 31.2912 Ops/s 30.8078 Ops/s $\color{#35bf28}+1.57\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-False] 18.3914ms 16.9121ms 59.1294 Ops/s 58.8545 Ops/s $\color{#35bf28}+0.47\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-True] 34.9557ms 32.8260ms 30.4636 Ops/s 30.0997 Ops/s $\color{#35bf28}+1.21\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-False] 18.5616ms 17.1169ms 58.4219 Ops/s 58.3341 Ops/s $\color{#35bf28}+0.15\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True] 36.0886ms 34.5067ms 28.9799 Ops/s 28.6324 Ops/s $\color{#35bf28}+1.21\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False] 19.8392ms 18.4014ms 54.3436 Ops/s 54.7426 Ops/s $\color{#d91a1a}-0.73\%$

vmoens added 4 commits October 19, 2025 20:56
@vmoens
Update
1591d0e 
[ghstack-poisoned]
@vmoens
Update
2c479eb 
[ghstack-poisoned]
@vmoens
Update
1d4c308 
[ghstack-poisoned]
@vmoens
Update
3092841 
[ghstack-poisoned]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. enhancement New feature or request

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@vmoens