Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
107 commits
Select commit Hold shift + click to select a range
cc8e403
Starting to work on export part sanity test.
Oct 17, 2025
e1f2086
Fixing export_events.
Oct 17, 2025
6c6adaa
Updates.
Oct 17, 2025
ebaf3fc
Updating sanity test.
Oct 17, 2025
9ce37c0
Export part sanity test
Oct 19, 2025
84333c8
Helpers tables querying common function
Oct 19, 2025
71828e6
Clean up export_part sanity test
Oct 19, 2025
dd5e27f
invalid part export test
Oct 19, 2025
67cff9c
Duplicate exports test
Oct 20, 2025
cf928d0
Updates.
Oct 20, 2025
f066974
Clean up test setup, remove some debugging notes, implement unique s3…
Oct 20, 2025
1f9e43d
Clean up notes
Oct 20, 2025
33d85e2
Initial categorization
Oct 22, 2025
3ef4fff
Add tests, clusters broken right now
Oct 23, 2025
03597fd
clean tables.py
Oct 24, 2025
52d7aa6
add arguments to create.py functions
Oct 24, 2025
4d1a025
steps.py rewrite
Oct 24, 2025
98d1539
configurable number of partitions and parts in create.py
Oct 25, 2025
b6e28f8
clean steps, sanity almost working
Oct 25, 2025
ccb4cf7
fix collapsing merge tree
Selfeer Oct 27, 2025
79ded58
fix other engines
Selfeer Oct 27, 2025
7471a8f
update graphite rollup example in s3
Selfeer Oct 27, 2025
ff2fc53
basic clusters tests
Oct 27, 2025
e559672
Move engine tests, move default_columns function
Oct 27, 2025
9ad790d
Sanity fix
Oct 27, 2025
5312d75
black everything
Oct 27, 2025
a7daf8b
fixes
Oct 27, 2025
fcfb511
sanity, add mismatched_columns test
Oct 27, 2025
c1b61d9
invalid_part_name rewrite, plus black
Oct 27, 2025
3950d7d
working tests
Oct 27, 2025
14b535c
update requirements
Selfeer Oct 28, 2025
3f60ce7
push requirements
Selfeer Oct 28, 2025
2a00de5
Initial datatypes tests, duplicate test, cleanup
Oct 28, 2025
60d2767
Basic concurrency test, move getting nodes to queries helper
Oct 28, 2025
10c5447
Broken setting test
Oct 29, 2025
c57b40d
Partition key types tests
Oct 29, 2025
44b4a26
Add requirements
Oct 29, 2025
cf7cb68
Export to same table error test
Oct 29, 2025
34256fa
Add req
Oct 29, 2025
bba8e90
Local table export error test
Oct 29, 2025
555dd39
Update reqs, no partition type test
Oct 29, 2025
a984005
Refactor reqs
Oct 30, 2025
bb44c1e
push usage of netem to s3
Selfeer Oct 30, 2025
e1444da
Wide compact parts, fixes
Oct 30, 2025
ed16c9f
Add reqs
Oct 30, 2025
61b6f3e
Export setting test, black
Oct 30, 2025
65baf31
Rename files, prep for network tests
Oct 30, 2025
343e373
Network jigglin
Oct 30, 2025
f52afb4
Different partition key error test
Oct 31, 2025
f5acef1
Reuse source matches destination step
Oct 31, 2025
f4b428d
System monitoring time woohoo
Oct 31, 2025
0c14e38
Datatypes change
Nov 3, 2025
9b8b6ec
Uncomment tests
Nov 3, 2025
a956d91
Starting to link export part tests with requirements.
Nov 3, 2025
6ec3d9c
Expanding engines tables combo test.
Nov 3, 2025
1d04e45
Verbose query output
Nov 3, 2025
313f214
Merge branch 'export_part' of https://github.com/Altinity/clickhouse-…
Nov 3, 2025
8acaaf6
Large export
Nov 3, 2025
ef6dc93
Trying to restart minio
Nov 3, 2025
9f41cfe
Disks, volumes, storage_policies
Nov 4, 2025
ce9b259
minio kill, export, minio start
Nov 4, 2025
e46c111
all working tests
Nov 4, 2025
af9bc02
Starting to work on export partition.xy
Nov 5, 2025
c5a3e01
changes
Nov 5, 2025
eb888bb
Merge branch 'export_part' of https://github.com/Altinity/clickhouse-…
Nov 5, 2025
1bb65dd
Update steps.py
vzakaznikov Nov 5, 2025
e313814
Update cluster.py
vzakaznikov Nov 5, 2025
0a69b76
clickhouse node and minio network interruptions, safe TERM and unsafe…
Nov 6, 2025
1fc939c
Working suite
Nov 6, 2025
ee0f4ec
change step from And to By
Selfeer Nov 6, 2025
8c02325
Merge branch 'main' into export_part
Selfeer Nov 6, 2025
5497bd7
Requirements update
Nov 6, 2025
8903b7c
small fix
Selfeer Nov 6, 2025
a6a6c72
Merge branch 'export_part' of https://github.com/Altinity/clickhouse-…
Nov 6, 2025
615bd03
Merge branch 'export_part' of https://github.com/Altinity/clickhouse-…
Nov 6, 2025
0fd10d2
get steps to True again
Selfeer Nov 6, 2025
6cc638f
remove apt for now, need to move to Dockerfile
Selfeer Nov 6, 2025
2415797
Merge remote-tracking branch 'origin/export_part' into export_part
Selfeer Nov 6, 2025
9d3676f
sort the output
Selfeer Nov 6, 2025
776b6a8
change the static table name to getuid
Selfeer Nov 6, 2025
b21319c
fix temporary tables
Selfeer Nov 6, 2025
0c53815
Combinatorics cleanup and parallelization
Nov 6, 2025
ac45d0d
remove storage.xml, move the logic to dynamically generate this file …
Selfeer Nov 6, 2025
94603a0
Drop column query
Nov 6, 2025
f69380b
Merge branch 'export_part' of https://github.com/Altinity/clickhouse-…
Nov 6, 2025
68e5e7c
Clean up
Nov 6, 2025
cd192a5
black
Nov 6, 2025
0122844
Basic system logs
Nov 6, 2025
d6ee332
Clean up
Nov 6, 2025
b1076cb
black
Nov 6, 2025
58a3827
bring back cluster list for minio
Selfeer Nov 7, 2025
a8c07be
Move minio steps
Nov 7, 2025
b15049b
Concurrent export verification via part_log
Nov 7, 2025
af58375
System.exports logging
Nov 8, 2025
35c13d7
Requirements update
Nov 8, 2025
f90515e
Revert commented out regression code
Nov 8, 2025
af6c72b
Req update
Nov 8, 2025
971f29a
add requirements
Selfeer Nov 10, 2025
105fb4f
update requirements
Selfeer Nov 10, 2025
23759ab
upd
Selfeer Nov 10, 2025
a921386
small update for steps
Selfeer Nov 10, 2025
57178dd
add specification
Selfeer Nov 10, 2025
d0881f5
fix sanity
Selfeer Nov 10, 2025
df4570b
fix sanity 2
Selfeer Nov 10, 2025
e23ed61
Network rate limit for consistency in system exports
Nov 10, 2025
98acba4
update skips
Selfeer Nov 11, 2025
dc18bdb
Merge branch 'main' into export_part
Selfeer Nov 11, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions alter/table/replace_partition/common.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,26 @@
from helpers.tables import create_table_partitioned_by_column, create_table, Column


@TestStep(Given)
def create_partitions_for_collapsing_merge_tree(
self,
table_name,
number_of_values=3,
number_of_partitions=5,
number_of_parts=1,
node=None,
):
"""Insert random UInt64 values into a column and create multiple partitions based on the value of number_of_partitions."""
if node is None:
node = self.context.node

with By("Inserting random values into a column with uint64 datatype"):
for i in range(1, number_of_partitions + 1):
for parts in range(1, number_of_parts + 1):
node.query(
f"INSERT INTO {table_name} (p, i) SELECT {random.choice([-1, 1])}, rand64() FROM numbers({number_of_values})"
)

@TestStep(Given)
def create_partitions_with_random_uint64(
self,
Expand Down
2 changes: 1 addition & 1 deletion engines/tests/replacing_merge_tree/replacing_merge_tree.py
Original file line number Diff line number Diff line change
Expand Up @@ -533,7 +533,7 @@ def incorrect_data_insert_with_disabled_optimize_on_insert(self, node=None):
):
node.query(f"SELECT * FROM {name} FORMAT JSONEachRow;")

with And("I optimize table"):
with By("optimizing a table"):
node.query(
f"OPTIMIZE TABLE {name} FINAL;",
message="DB::Exception:",
Expand Down
10 changes: 10 additions & 0 deletions helpers/cluster.py
Original file line number Diff line number Diff line change
Expand Up @@ -1012,6 +1012,7 @@ def query(
ignore_exception=False,
step=By,
settings=None,
inline_settings=None,
retry_count=5,
messages_to_retry=None,
retry_delay=5,
Expand All @@ -1035,6 +1036,7 @@ def query(
:param no_check: disable exitcode and message checks, default: False
:param step: wrapping step class, default: By
:param settings: list of settings to be used for the query in the form [(name, value),...], default: None
:param inline_settings: list of inline settings to be used for the query in the form [(name, value),...], default: None
:param retry_count: number of retries, default: 5
:param messages_to_retry: list of messages in the query output for
which retry should be triggered, default: MESSAGES_TO_RETRY
Expand All @@ -1057,6 +1059,7 @@ def query(
retry_count = max(0, int(retry_count))
retry_delay = max(0, float(retry_delay))
settings = list(settings or [])
inline_settings = list(inline_settings or [])
query_settings = list(settings)

if raise_on_exception:
Expand All @@ -1077,6 +1080,13 @@ def query(
if query_id is not None:
query_settings += [("query_id", f"{query_id}")]

if inline_settings:
sql = (
"; ".join([f"SET {name} = {value}" for name, value in inline_settings])
+ "; "
+ sql
)

client = "clickhouse client -n"
if secure:
client += (
Expand Down
Loading
Loading