[BugFix] Fix collector devices

[vmoens]

Stack from ghstack (oldest at bottom):

• -> [BugFix] Fix collector devices #3223

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/3223

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure, 39 Pending, 4 Unrelated Failures

As of commit 09f9d27 with merge base 3d1748f ():

NEW FAILURE - The following job has failed:

• Push Binary Nightly / build-wheel-unix (linux, ubuntu-22.04, 3.9, cp39-cp39, cpu, cpu) (gh)

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• Push Binary Nightly / build-wheel-unix (macos, macos-latest, 3.9, cp39-cp39, cpu, cpu) (gh) (trunk failure)
  Process completed with exit code 1.
• Push Binary Nightly / build-wheel-windows (3.9, 3.9) (gh) (trunk failure)
  Process completed with exit code 1.
• Push Binary Nightly / test-wheel-unix (linux, ubuntu-22.04, 3.9, cp39-cp39, cpu, cpu) (gh) (trunk failure)
  Process completed with exit code 1.
• Push Binary Nightly / test-wheel-unix (macos, macos-latest, 3.9, cp39-cp39, cpu, cpu) (gh) (trunk failure)
  Process completed with exit code 1.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 154. Improved: $\large\color{#35bf28}18$. Worsened: $\large\color{#d91a1a}11$.

Expand to view detailed results

Name	Max	Mean	Ops	Ops on Repo HEAD	Change
test_tensor_to_bytestream_speed[pickle]	85.3240μs	83.3262μs	12.0010 KOps/s	12.2439 KOps/s	$\color{#d91a1a}-1.98\%$
test_tensor_to_bytestream_speed[torch.save]	0.1466ms	0.1451ms	6.8899 KOps/s	7.1266 KOps/s	$\color{#d91a1a}-3.32\%$
test_tensor_to_bytestream_speed[untyped_storage]	0.1320s	0.1310s	7.6334 Ops/s	8.3534 Ops/s	$\textbf{\color{#d91a1a}-8.62\%}$
test_tensor_to_bytestream_speed[numpy]	2.8832μs	2.8787μs	347.3824 KOps/s	360.7186 KOps/s	$\color{#d91a1a}-3.70\%$
test_tensor_to_bytestream_speed[safetensors]	44.9394μs	44.7050μs	22.3689 KOps/s	24.0653 KOps/s	$\textbf{\color{#d91a1a}-7.05\%}$
test_simple	0.5510s	0.5501s	1.8179 Ops/s	1.7448 Ops/s	$\color{#35bf28}+4.19\%$
test_transformed	1.2316s	1.1368s	0.8797 Ops/s	0.8797 Ops/s	$-0.01\%$
test_serial	1.6761s	1.6747s	0.5971 Ops/s	0.5937 Ops/s	$\color{#35bf28}+0.57\%$
test_parallel	1.1794s	1.0843s	0.9223 Ops/s	0.9438 Ops/s	$\color{#d91a1a}-2.28\%$
test_step_mdp_speed[True-True-True-True-True]	0.2257ms	44.1559μs	22.6470 KOps/s	22.7807 KOps/s	$\color{#d91a1a}-0.59\%$
test_step_mdp_speed[True-True-True-True-False]	0.1083ms	24.9382μs	40.0992 KOps/s	40.2716 KOps/s	$\color{#d91a1a}-0.43\%$
test_step_mdp_speed[True-True-True-False-True]	68.4930μs	24.7927μs	40.3344 KOps/s	40.3568 KOps/s	$\color{#d91a1a}-0.06\%$
test_step_mdp_speed[True-True-True-False-False]	94.9550μs	13.7471μs	72.7427 KOps/s	73.0573 KOps/s	$\color{#d91a1a}-0.43\%$
test_step_mdp_speed[True-True-False-True-True]	0.1382ms	46.9894μs	21.2814 KOps/s	21.2923 KOps/s	$\color{#d91a1a}-0.05\%$
test_step_mdp_speed[True-True-False-True-False]	0.1110ms	27.8681μs	35.8833 KOps/s	36.7080 KOps/s	$\color{#d91a1a}-2.25\%$
test_step_mdp_speed[True-True-False-False-True]	70.6130μs	27.6819μs	36.1247 KOps/s	36.9336 KOps/s	$\color{#d91a1a}-2.19\%$
test_step_mdp_speed[True-True-False-False-False]	98.3450μs	16.6843μs	59.9366 KOps/s	61.1687 KOps/s	$\color{#d91a1a}-2.01\%$
test_step_mdp_speed[True-False-True-True-True]	0.1393ms	50.9189μs	19.6391 KOps/s	20.0053 KOps/s	$\color{#d91a1a}-1.83\%$
test_step_mdp_speed[True-False-True-True-False]	82.5040μs	30.4771μs	32.8115 KOps/s	33.2422 KOps/s	$\color{#d91a1a}-1.30\%$
test_step_mdp_speed[True-False-True-False-True]	0.1209ms	27.8765μs	35.8725 KOps/s	36.8639 KOps/s	$\color{#d91a1a}-2.69\%$
test_step_mdp_speed[True-False-True-False-False]	98.9160μs	16.6510μs	60.0564 KOps/s	61.2211 KOps/s	$\color{#d91a1a}-1.90\%$
test_step_mdp_speed[True-False-False-True-True]	0.1329ms	52.6983μs	18.9760 KOps/s	19.2434 KOps/s	$\color{#d91a1a}-1.39\%$
test_step_mdp_speed[True-False-False-True-False]	0.1183ms	33.3701μs	29.9669 KOps/s	30.6315 KOps/s	$\color{#d91a1a}-2.17\%$
test_step_mdp_speed[True-False-False-False-True]	0.7701ms	30.4523μs	32.8382 KOps/s	33.5426 KOps/s	$\color{#d91a1a}-2.10\%$
test_step_mdp_speed[True-False-False-False-False]	48.9630μs	19.2771μs	51.8750 KOps/s	52.4702 KOps/s	$\color{#d91a1a}-1.13\%$
test_step_mdp_speed[False-True-True-True-True]	0.1286ms	49.8504μs	20.0600 KOps/s	19.9623 KOps/s	$\color{#35bf28}+0.49\%$
test_step_mdp_speed[False-True-True-True-False]	0.1096ms	30.6408μs	32.6362 KOps/s	33.0064 KOps/s	$\color{#d91a1a}-1.12\%$
test_step_mdp_speed[False-True-True-False-True]	2.4673ms	31.8646μs	31.3828 KOps/s	32.2079 KOps/s	$\color{#d91a1a}-2.56\%$
test_step_mdp_speed[False-True-True-False-False]	0.1140ms	18.3195μs	54.5868 KOps/s	55.5107 KOps/s	$\color{#d91a1a}-1.66\%$
test_step_mdp_speed[False-True-False-True-True]	0.1355ms	52.9426μs	18.8884 KOps/s	18.9996 KOps/s	$\color{#d91a1a}-0.59\%$
test_step_mdp_speed[False-True-False-True-False]	0.1180ms	33.1705μs	30.1473 KOps/s	30.1668 KOps/s	$\color{#d91a1a}-0.06\%$
test_step_mdp_speed[False-True-False-False-True]	88.0750μs	34.0869μs	29.3368 KOps/s	29.3452 KOps/s	$\color{#d91a1a}-0.03\%$
test_step_mdp_speed[False-True-False-False-False]	0.1083ms	20.9599μs	47.7101 KOps/s	47.6987 KOps/s	$\color{#35bf28}+0.02\%$
test_step_mdp_speed[False-False-True-True-True]	0.1386ms	54.5378μs	18.3359 KOps/s	18.1980 KOps/s	$\color{#35bf28}+0.76\%$
test_step_mdp_speed[False-False-True-True-False]	84.4450μs	36.2185μs	27.6102 KOps/s	27.7925 KOps/s	$\color{#d91a1a}-0.66\%$
test_step_mdp_speed[False-False-True-False-True]	0.1135ms	34.1730μs	29.2628 KOps/s	29.3226 KOps/s	$\color{#d91a1a}-0.20\%$
test_step_mdp_speed[False-False-True-False-False]	0.1019ms	20.7200μs	48.2626 KOps/s	47.4355 KOps/s	$\color{#35bf28}+1.74\%$
test_step_mdp_speed[False-False-False-True-True]	0.1395ms	57.5525μs	17.3754 KOps/s	17.1577 KOps/s	$\color{#35bf28}+1.27\%$
test_step_mdp_speed[False-False-False-True-False]	75.9740μs	38.5017μs	25.9729 KOps/s	25.9936 KOps/s	$\color{#d91a1a}-0.08\%$
test_step_mdp_speed[False-False-False-False-True]	0.1166ms	36.5006μs	27.3968 KOps/s	27.6726 KOps/s	$\color{#d91a1a}-1.00\%$
test_step_mdp_speed[False-False-False-False-False]	0.1028ms	23.4969μs	42.5589 KOps/s	42.6434 KOps/s	$\color{#d91a1a}-0.20\%$
test_values[generalized_advantage_estimate-True-True]	10.2490ms	10.1245ms	98.7702 Ops/s	101.8649 Ops/s	$\color{#d91a1a}-3.04\%$
test_values[vec_generalized_advantage_estimate-True-True]	20.6842ms	17.8371ms	56.0629 Ops/s	86.7633 Ops/s	$\textbf{\color{#d91a1a}-35.38\%}$
test_values[td0_return_estimate-False-False]	0.2223ms	0.1326ms	7.5435 KOps/s	7.8263 KOps/s	$\color{#d91a1a}-3.61\%$
test_values[td1_return_estimate-False-False]	27.9058ms	27.4433ms	36.4388 Ops/s	37.2835 Ops/s	$\color{#d91a1a}-2.27\%$
test_values[vec_td1_return_estimate-False-False]	18.0711ms	17.8411ms	56.0504 Ops/s	87.8973 Ops/s	$\textbf{\color{#d91a1a}-36.23\%}$
test_values[td_lambda_return_estimate-True-False]	41.7032ms	41.3577ms	24.1793 Ops/s	24.8677 Ops/s	$\color{#d91a1a}-2.77\%$
test_values[vec_td_lambda_return_estimate-True-False]	17.9993ms	17.7403ms	56.3688 Ops/s	88.4157 Ops/s	$\textbf{\color{#d91a1a}-36.25\%}$
test_gae_speed[generalized_advantage_estimate-False-1-512]	8.6675ms	8.5849ms	116.4832 Ops/s	118.1849 Ops/s	$\color{#d91a1a}-1.44\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512]	1.7354ms	1.5227ms	656.7447 Ops/s	676.1419 Ops/s	$\color{#d91a1a}-2.87\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512]	0.4702ms	0.4205ms	2.3779 KOps/s	2.4206 KOps/s	$\color{#d91a1a}-1.76\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512]	34.6857ms	33.9529ms	29.4526 Ops/s	29.7912 Ops/s	$\color{#d91a1a}-1.14\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512]	1.9020ms	1.7416ms	574.1839 Ops/s	567.9031 Ops/s	$\color{#35bf28}+1.11\%$
test_dqn_speed[False-None]	1.5575ms	1.4609ms	684.5053 Ops/s	699.0428 Ops/s	$\color{#d91a1a}-2.08\%$
test_dqn_speed[False-backward]	2.0777ms	1.9687ms	507.9466 Ops/s	516.1502 Ops/s	$\color{#d91a1a}-1.59\%$
test_dqn_speed[True-None]	0.6555ms	0.5253ms	1.9036 KOps/s	1.9207 KOps/s	$\color{#d91a1a}-0.89\%$
test_dqn_speed[True-backward]	1.0699ms	0.9778ms	1.0227 KOps/s	1.0107 KOps/s	$\color{#35bf28}+1.19\%$
test_dqn_speed[reduce-overhead-None]	0.6360ms	0.5180ms	1.9304 KOps/s	2.0099 KOps/s	$\color{#d91a1a}-3.95\%$
test_dqn_speed[reduce-overhead-backward]	1.0170ms	0.9549ms	1.0472 KOps/s	939.7551 Ops/s	$\textbf{\color{#35bf28}+11.44\%}$
test_ddpg_speed[False-None]	3.2927ms	2.9218ms	342.2548 Ops/s	345.3020 Ops/s	$\color{#d91a1a}-0.88\%$
test_ddpg_speed[False-backward]	6.0885ms	4.2465ms	235.4868 Ops/s	241.9573 Ops/s	$\color{#d91a1a}-2.67\%$
test_ddpg_speed[True-None]	1.6931ms	1.3883ms	720.2922 Ops/s	719.0759 Ops/s	$\color{#35bf28}+0.17\%$
test_ddpg_speed[True-backward]	2.4124ms	2.3619ms	423.3915 Ops/s	425.0106 Ops/s	$\color{#d91a1a}-0.38\%$
test_ddpg_speed[reduce-overhead-None]	1.5214ms	1.3821ms	723.5330 Ops/s	738.1863 Ops/s	$\color{#d91a1a}-1.99\%$
test_ddpg_speed[reduce-overhead-backward]	2.5181ms	2.4152ms	414.0433 Ops/s	409.1253 Ops/s	$\color{#35bf28}+1.20\%$
test_sac_speed[False-None]	8.4950ms	7.8746ms	126.9902 Ops/s	128.1505 Ops/s	$\color{#d91a1a}-0.91\%$
test_sac_speed[False-backward]	11.8548ms	11.2036ms	89.2569 Ops/s	90.2368 Ops/s	$\color{#d91a1a}-1.09\%$
test_sac_speed[True-None]	2.4226ms	2.1003ms	476.1193 Ops/s	470.3452 Ops/s	$\color{#35bf28}+1.23\%$
test_sac_speed[True-backward]	4.1236ms	3.9759ms	251.5139 Ops/s	240.7976 Ops/s	$\color{#35bf28}+4.45\%$
test_sac_speed[reduce-overhead-None]	2.2728ms	2.1115ms	473.5914 Ops/s	472.4937 Ops/s	$\color{#35bf28}+0.23\%$
test_sac_speed[reduce-overhead-backward]	4.5542ms	4.0119ms	249.2602 Ops/s	240.0251 Ops/s	$\color{#35bf28}+3.85\%$
test_redq_speed[False-None]	16.7498ms	12.0054ms	83.2956 Ops/s	94.4427 Ops/s	$\textbf{\color{#d91a1a}-11.80\%}$
test_redq_speed[False-backward]	25.2704ms	19.4511ms	51.4110 Ops/s	55.4636 Ops/s	$\textbf{\color{#d91a1a}-7.31\%}$
test_redq_speed[True-None]	4.6324ms	4.3199ms	231.4865 Ops/s	226.8435 Ops/s	$\color{#35bf28}+2.05\%$
test_redq_speed[True-backward]	10.0611ms	9.7386ms	102.6843 Ops/s	103.7008 Ops/s	$\color{#d91a1a}-0.98\%$
test_redq_speed[reduce-overhead-None]	4.4789ms	4.2853ms	233.3569 Ops/s	226.1644 Ops/s	$\color{#35bf28}+3.18\%$
test_redq_speed[reduce-overhead-backward]	10.1063ms	9.8242ms	101.7898 Ops/s	97.0287 Ops/s	$\color{#35bf28}+4.91\%$
test_redq_deprec_speed[False-None]	11.4048ms	11.0070ms	90.8512 Ops/s	88.5244 Ops/s	$\color{#35bf28}+2.63\%$
test_redq_deprec_speed[False-backward]	16.5212ms	15.9848ms	62.5594 Ops/s	61.8482 Ops/s	$\color{#35bf28}+1.15\%$
test_redq_deprec_speed[True-None]	3.7741ms	3.6158ms	276.5610 Ops/s	272.7232 Ops/s	$\color{#35bf28}+1.41\%$
test_redq_deprec_speed[True-backward]	7.9338ms	7.7380ms	129.2327 Ops/s	135.9573 Ops/s	$\color{#d91a1a}-4.95\%$
test_redq_deprec_speed[reduce-overhead-None]	3.7346ms	3.5166ms	284.3663 Ops/s	289.2591 Ops/s	$\color{#d91a1a}-1.69\%$
test_redq_deprec_speed[reduce-overhead-backward]	7.7939ms	7.5553ms	132.3578 Ops/s	123.8662 Ops/s	$\textbf{\color{#35bf28}+6.86\%}$
test_td3_speed[False-None]	8.0370ms	7.9429ms	125.8985 Ops/s	119.6256 Ops/s	$\textbf{\color{#35bf28}+5.24\%}$
test_td3_speed[False-backward]	11.3376ms	10.7705ms	92.8463 Ops/s	91.6497 Ops/s	$\color{#35bf28}+1.31\%$
test_td3_speed[True-None]	1.8907ms	1.7961ms	556.7602 Ops/s	558.3466 Ops/s	$\color{#d91a1a}-0.28\%$
test_td3_speed[True-backward]	3.6480ms	3.5456ms	282.0434 Ops/s	275.8148 Ops/s	$\color{#35bf28}+2.26\%$
test_td3_speed[reduce-overhead-None]	1.8154ms	1.7645ms	566.7274 Ops/s	567.5743 Ops/s	$\color{#d91a1a}-0.15\%$
test_td3_speed[reduce-overhead-backward]	3.6779ms	3.5677ms	280.2952 Ops/s	231.3349 Ops/s	$\textbf{\color{#35bf28}+21.16\%}$
test_cql_speed[False-None]	29.3949ms	26.1152ms	38.2919 Ops/s	37.9438 Ops/s	$\color{#35bf28}+0.92\%$
test_cql_speed[False-backward]	39.7979ms	35.7776ms	27.9505 Ops/s	28.4912 Ops/s	$\color{#d91a1a}-1.90\%$
test_cql_speed[True-None]	13.1597ms	12.3867ms	80.7318 Ops/s	79.0399 Ops/s	$\color{#35bf28}+2.14\%$
test_cql_speed[True-backward]	18.8544ms	18.5099ms	54.0250 Ops/s	57.0409 Ops/s	$\textbf{\color{#d91a1a}-5.29\%}$
test_cql_speed[reduce-overhead-None]	13.3878ms	12.4376ms	80.4016 Ops/s	81.0868 Ops/s	$\color{#d91a1a}-0.85\%$
test_cql_speed[reduce-overhead-backward]	18.9790ms	18.4116ms	54.3137 Ops/s	56.4802 Ops/s	$\color{#d91a1a}-3.84\%$
test_a2c_speed[False-None]	5.5948ms	5.3571ms	186.6668 Ops/s	181.3721 Ops/s	$\color{#35bf28}+2.92\%$
test_a2c_speed[False-backward]	11.9534ms	11.6176ms	86.0762 Ops/s	83.7133 Ops/s	$\color{#35bf28}+2.82\%$
test_a2c_speed[True-None]	3.9487ms	3.6861ms	271.2891 Ops/s	273.3408 Ops/s	$\color{#d91a1a}-0.75\%$
test_a2c_speed[True-backward]	8.6983ms	8.4215ms	118.7444 Ops/s	110.8743 Ops/s	$\textbf{\color{#35bf28}+7.10\%}$
test_a2c_speed[reduce-overhead-None]	3.8452ms	3.6899ms	271.0123 Ops/s	271.2279 Ops/s	$\color{#d91a1a}-0.08\%$
test_a2c_speed[reduce-overhead-backward]	9.2915ms	8.8598ms	112.8695 Ops/s	113.4961 Ops/s	$\color{#d91a1a}-0.55\%$
test_ppo_speed[False-None]	6.1226ms	5.8852ms	169.9190 Ops/s	172.0307 Ops/s	$\color{#d91a1a}-1.23\%$
test_ppo_speed[False-backward]	12.8390ms	12.4492ms	80.3266 Ops/s	79.6677 Ops/s	$\color{#35bf28}+0.83\%$
test_ppo_speed[True-None]	3.8136ms	3.6306ms	275.4370 Ops/s	274.5355 Ops/s	$\color{#35bf28}+0.33\%$
test_ppo_speed[True-backward]	8.8561ms	8.5946ms	116.3528 Ops/s	114.6360 Ops/s	$\color{#35bf28}+1.50\%$
test_ppo_speed[reduce-overhead-None]	3.7930ms	3.6069ms	277.2437 Ops/s	276.0320 Ops/s	$\color{#35bf28}+0.44\%$
test_ppo_speed[reduce-overhead-backward]	8.9457ms	8.6169ms	116.0512 Ops/s	111.2619 Ops/s	$\color{#35bf28}+4.30\%$
test_reinforce_speed[False-None]	4.9346ms	4.6683ms	214.2113 Ops/s	216.2783 Ops/s	$\color{#d91a1a}-0.96\%$
test_reinforce_speed[False-backward]	7.6605ms	7.4929ms	133.4594 Ops/s	135.3106 Ops/s	$\color{#d91a1a}-1.37\%$
test_reinforce_speed[True-None]	3.2178ms	2.8887ms	346.1729 Ops/s	352.0547 Ops/s	$\color{#d91a1a}-1.67\%$
test_reinforce_speed[True-backward]	7.9182ms	7.6453ms	130.7991 Ops/s	128.3034 Ops/s	$\color{#35bf28}+1.95\%$
test_reinforce_speed[reduce-overhead-None]	3.4016ms	2.8498ms	350.9032 Ops/s	339.3677 Ops/s	$\color{#35bf28}+3.40\%$
test_reinforce_speed[reduce-overhead-backward]	8.1963ms	7.9271ms	126.1490 Ops/s	125.7445 Ops/s	$\color{#35bf28}+0.32\%$
test_iql_speed[False-None]	25.8402ms	20.6302ms	48.4727 Ops/s	49.2034 Ops/s	$\color{#d91a1a}-1.49\%$
test_iql_speed[False-backward]	31.1945ms	30.2804ms	33.0247 Ops/s	32.4188 Ops/s	$\color{#35bf28}+1.87\%$
test_iql_speed[True-None]	8.8052ms	8.4178ms	118.7961 Ops/s	118.1617 Ops/s	$\color{#35bf28}+0.54\%$
test_iql_speed[True-backward]	17.0614ms	16.6530ms	60.0492 Ops/s	59.6657 Ops/s	$\color{#35bf28}+0.64\%$
test_iql_speed[reduce-overhead-None]	10.9329ms	8.5551ms	116.8890 Ops/s	116.2048 Ops/s	$\color{#35bf28}+0.59\%$
test_iql_speed[reduce-overhead-backward]	17.3970ms	17.0832ms	58.5371 Ops/s	58.3457 Ops/s	$\color{#35bf28}+0.33\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	7.5194ms	6.0681ms	164.7975 Ops/s	163.0996 Ops/s	$\color{#35bf28}+1.04\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	0.5371ms	0.2753ms	3.6329 KOps/s	3.4102 KOps/s	$\textbf{\color{#35bf28}+6.53\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.5058ms	0.2568ms	3.8942 KOps/s	3.5292 KOps/s	$\textbf{\color{#35bf28}+10.34\%}$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	5.9817ms	5.7550ms	173.7611 Ops/s	175.5386 Ops/s	$\color{#d91a1a}-1.01\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	1.8403ms	0.2907ms	3.4398 KOps/s	2.9407 KOps/s	$\textbf{\color{#35bf28}+16.97\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.5780ms	0.2870ms	3.4846 KOps/s	3.0868 KOps/s	$\textbf{\color{#35bf28}+12.89\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000]	1.5887ms	1.3596ms	735.4857 Ops/s	811.1990 Ops/s	$\textbf{\color{#d91a1a}-9.33\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000]	1.5689ms	1.2492ms	800.5369 Ops/s	856.9760 Ops/s	$\textbf{\color{#d91a1a}-6.59\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	9.7827ms	6.0481ms	165.3417 Ops/s	168.0268 Ops/s	$\color{#d91a1a}-1.60\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	1.2489ms	0.4512ms	2.2165 KOps/s	2.0616 KOps/s	$\textbf{\color{#35bf28}+7.52\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.6644ms	0.4355ms	2.2965 KOps/s	2.1574 KOps/s	$\textbf{\color{#35bf28}+6.45\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	5.9630ms	5.7735ms	173.2060 Ops/s	171.1369 Ops/s	$\color{#35bf28}+1.21\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	2.1169ms	0.3229ms	3.0972 KOps/s	829.9205 Ops/s	$\textbf{\color{#35bf28}+273.20\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.5061ms	0.2867ms	3.4882 KOps/s	3.0175 KOps/s	$\textbf{\color{#35bf28}+15.60\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	5.9908ms	5.7212ms	174.7892 Ops/s	174.3405 Ops/s	$\color{#35bf28}+0.26\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	0.8524ms	0.2893ms	3.4565 KOps/s	2.9484 KOps/s	$\textbf{\color{#35bf28}+17.23\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.4472ms	0.2566ms	3.8971 KOps/s	3.0613 KOps/s	$\textbf{\color{#35bf28}+27.30\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	5.9949ms	5.9120ms	169.1474 Ops/s	168.9618 Ops/s	$\color{#35bf28}+0.11\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	1.1342ms	0.4274ms	2.3396 KOps/s	2.0602 KOps/s	$\textbf{\color{#35bf28}+13.56\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.6876ms	0.4497ms	2.2238 KOps/s	2.1251 KOps/s	$\color{#35bf28}+4.64\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400]	6.6761ms	5.1491ms	194.2082 Ops/s	193.7195 Ops/s	$\color{#35bf28}+0.25\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400]	7.2644ms	2.3034ms	434.1476 Ops/s	439.4814 Ops/s	$\color{#d91a1a}-1.21\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400]	8.2468ms	1.2008ms	832.7857 Ops/s	846.6129 Ops/s	$\color{#d91a1a}-1.63\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400]	0.5060s	15.0788ms	66.3181 Ops/s	55.6439 Ops/s	$\textbf{\color{#35bf28}+19.18\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400]	9.7486ms	1.9294ms	518.3039 Ops/s	482.9148 Ops/s	$\textbf{\color{#35bf28}+7.33\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400]	7.6309ms	1.1722ms	853.0927 Ops/s	909.3062 Ops/s	$\textbf{\color{#d91a1a}-6.18\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400]	7.0063ms	5.2783ms	189.4554 Ops/s	184.1003 Ops/s	$\color{#35bf28}+2.91\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400]	9.2696ms	2.1912ms	456.3757 Ops/s	461.1175 Ops/s	$\color{#d91a1a}-1.03\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400]	7.1738ms	1.3165ms	759.5693 Ops/s	743.2301 Ops/s	$\color{#35bf28}+2.20\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-True]	35.7102ms	32.6233ms	30.6530 Ops/s	30.3447 Ops/s	$\color{#35bf28}+1.02\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-False]	19.6986ms	17.6631ms	56.6151 Ops/s	57.0024 Ops/s	$\color{#d91a1a}-0.68\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-True]	35.6771ms	33.8645ms	29.5294 Ops/s	29.3792 Ops/s	$\color{#35bf28}+0.51\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-False]	19.8024ms	18.3609ms	54.4636 Ops/s	56.3503 Ops/s	$\color{#d91a1a}-3.35\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True]	36.9972ms	35.4354ms	28.2204 Ops/s	28.0113 Ops/s	$\color{#35bf28}+0.75\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False]	20.1747ms	19.1885ms	52.1146 Ops/s	52.2510 Ops/s	$\color{#d91a1a}-0.26\%$