Skip to content

s3 tiered storage policies - default disk tried to restore into object disk #1121

@Slach

Description

@Slach
 <storage_configuration>
        <disks>
            <miniocached>
                <type>s3</type>
                <endpoint>http://minio:8000/data/s3</endpoint>
                <use_environment_credentials>0</use_environment_credentials>
                <access_key_id>minioadmin</access_key_id>
                <secret_access_key>minioadmin</secret_access_key>
            </miniocached>
            <cache>
                <type>cache</type>
                <disk>miniocached</disk>
                <path>./disks/minio_cache/</path>
                <max_size>50Gi</max_size> <!-- 50GB local cache to cache remote data -->
            </cache>
        </disks>
        <policies>
            <miniotiered>
                <volumes>
                    <default>
                        <disk>default</disk>
                        <max_data_part_size_bytes>50000000000</max_data_part_size_bytes> <!-- only for parts less than 50GB after they moved to minio during merges -->
                    </default>
                    <minio_cache>
                        <disk>cache</disk> <!-- sandwich cache plus miniocached -->
                        <!-- prefer_not_to_merge>true</prefer_not_to_merge>
                        <perform_ttl_move_on_insert>false</perform_ttl_move_on_insert-->
                    </minio_cache>
                </volumes>
            </miniotiered>
        </policies>
    </storage_configuration>
clickhouse-backup create my_backup
clickhouse-backup restore my_backup

during restore wrong destination disk choosen

 > Link /Users/MManj3/backup/all_partition_backup/shadow/default/test_s3/default/all_1_1_0/checksums.txt -> disks/miniocached/store/a3d/a3da913a-700c-4ba0-b3d0-a01fd00d51ff/detached/all_1_1_0/checksums.txt
> Link /Users/MManj3/backup/all_partition_backup/shadow/default/test_s3/default/all_1_1_0/columns.txt -> disks/miniocached/store/a3d/a3da913a-700c-4ba0-b3d0-a01fd00d51ff/detached/all_1_1_0/columns.txt

and

2025-03-24 15:56:49.135 INF pkg/clickhouse/clickhouse.go:1122 > ALTER TABLE `default`.`test_s3` ATTACH PART 'all_1_1_0'
2025-03-24 15:56:49.269 INF pkg/clickhouse/clickhouse.go:335 > clickhouse connection closed
2025-03-24 15:56:49.269 FTL cmd/clickhouse-backup/main.go:742 > error="one of restoreDataRegular go-routine return error: can't attach data parts for table 'default.test_s3': code: 27, message: Cannot parse input: expected '\\n' before: 'gu\x13Ź\\f\\\\\x03dv\x1cO�\x02��\"\\0\\0\\0\x10\\0\\0\\0(�/� \x10�\\0\\0>\v\\0\\0\\0\\0\\0\\0>\v\\0\\0\\0\\0\\0\\0'"
➜  ~ 
SELECT d.path AS path, any(d.name) AS name,
any(lower(if(d.type='ObjectStorage',d.object_storage_type,d.type))) AS type,
min(d.free_space) AS free_space, 
groupUniqArray(s.policy_name) AS storage_policies 
FROM system.disks AS d  LEFT JOIN (SELECT policy_name, arrayJoin(disks) AS disk FROM system.storage_policies) AS s 
ON s.disk = d.name 
GROUP BY d.path

return following results

Image

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions