Skip to content

Commit b742815

Browse files
committed
cluster_link/model: change default max partition bytes
Prior to this commit the default max partition bytes was 1MiB. The default min fetch bytes was 5MiB. And the max wait ms was 500ms. In cases where only one partition on the shard was being produced to these defaults limited our consumer throughput to 2MiB/s. This is since we'd only be able to read 1MiB every 500ms from the one partition with new data. Hence this commit sets max partition bytes to equal min fetch bytes to prevent this issue.
1 parent b822ebe commit b742815

File tree

5 files changed

+6
-6
lines changed

5 files changed

+6
-6
lines changed

proto/redpanda/core/admin/v2/shadow_link.proto

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -290,7 +290,7 @@ message ShadowLinkClientOptions {
290290
int32 effective_fetch_max_bytes = 17
291291
[(google.api.field_behavior) = OUTPUT_ONLY];
292292
// Fetch partition max bytes.
293-
// If 0 is provided, defaults to 1 MiB
293+
// If 0 is provided, defaults to 5 MiB
294294
int32 fetch_partition_max_bytes = 18;
295295
// The effective fetch partition max bytes
296296
int32 effective_fetch_partition_max_bytes = 19

src/v/cluster_link/model/types.h

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -276,7 +276,7 @@ struct connection_config
276276
// fetch response.
277277
std::optional<int32_t> fetch_partition_max_bytes;
278278
// Default maximum number of bytes to fetch per partition
279-
static constexpr auto default_fetch_partition_max_bytes = 1_MiB;
279+
static constexpr auto default_fetch_partition_max_bytes = 5_MiB;
280280
// Default maximum number of bytes to fetch (20MiB)
281281
static constexpr auto fetch_max_bytes_default = 20 * 1024 * 1024;
282282

tests/rptest/clients/admin/proto/redpanda/core/admin/v2/cluster_pb2.pyi

Lines changed: 1 addition & 1 deletion
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

tests/rptest/clients/admin/proto/redpanda/core/admin/v2/shadow_link_pb2.pyi

Lines changed: 1 addition & 1 deletion
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

tests/rptest/tests/cluster_linking_e2e_test.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -270,9 +270,9 @@ def test_create_default_link(self):
270270
f"Expected 0, got {client_options.fetch_partition_max_bytes}"
271271
)
272272
assert client_options.effective_fetch_partition_max_bytes == (
273-
1 * 1024 * 1024
273+
5 * 1024 * 1024
274274
), (
275-
f"Expected {1 * 1024 * 1024}, got {client_options.effective_fetch_partition_max_bytes}"
275+
f"Expected {5 * 1024 * 1024}, got {client_options.effective_fetch_partition_max_bytes}"
276276
)
277277

278278
topic_metadata_config = shadow_link.configurations.topic_metadata_sync_options

0 commit comments

Comments
 (0)