Skip to content

Conversation

@sorenhansendk
Copy link
Contributor

@sorenhansendk sorenhansendk commented Sep 8, 2025

The MinIO SDK is designed to talk to any S3-compatible object storage, and the project actively maintains interoperability across providers. This directly addresses the signature/header edge cases we’ve hit with S3-compatible endpoints (e.g., checksum and Accept-Encoding interactions)

Why

  • Better cross-provider compatibility: MinIO’s Go client targets the S3 API surface used by “S3-compatible” stores, which reduces surprises when talking to non-AWS endpoints.
  • Avoids recent checksum/signing pitfalls: AWS SDK v2 introduced default response/integrity checksum behaviors that can interact poorly with some S3-compatible services, contributing to SignatureDoesNotMatch failures. Using minio-go sidesteps those defaults and focuses on portable S3 semantics.
  • GCS S3 interoperability: We rely on HMAC/S3-style access against Google Cloud Storage; their interoperability path is documented, and minio-go is frequently used with such endpoints

What changes

  • Implementation detail only: all object operations (HEAD/GET/PUT/COPY) are handled by minio-go
  • No changes to caller-facing config/flags (bucket/prefix/path-style/endpoint)

Closes: #3749 (minio-go works with GCS via HMAC)
Closes: #5804 (minio-go works with OpenStack S3)
Closes: #4487 (minio-go works with Alibaba Cloud)

@github-actions github-actions bot added area/dependencies Pull requests that update a dependency file area/remotecache labels Sep 8, 2025
@sorenhansendk sorenhansendk force-pushed the use-minio-for-s3 branch 4 times, most recently from f1ccd17 to 528b206 Compare September 8, 2025 08:58
@tonistiigi
Copy link
Member

cc @bpaquet

@sorenhansendk sorenhansendk changed the title Refactor S3 cache backend to use minio-go client remotecache: Refactor S3 cache backend to use minio-go client Sep 8, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/dependencies Pull requests that update a dependency file area/remotecache

Projects

None yet

2 participants