[Bug]: Permission denied reading redpanda.yaml stopping container from proper start

[KenxinKun]

Testcontainers version

0.37.0

Using the latest Testcontainers version?

Yes

Host OS

Mac

Host arch

ARM

Go version

go1.24.2 darwin/arm64

Docker version

Client:
 Version:           28.0.1
 API version:       1.48
 Go version:        go1.23.6
 Git commit:        068a01e
 Built:             Wed Feb 26 10:38:16 2025
 OS/Arch:           darwin/arm64
 Context:           desktop-linux

Server: Docker Desktop 4.39.0 (184744)
 Engine:
  Version:          28.0.1
  API version:      1.48 (minimum version 1.24)
  Go version:       go1.23.6
  Git commit:       bbd0a17
  Built:            Wed Feb 26 10:40:57 2025
  OS/Arch:          linux/arm64
  Experimental:     false
 containerd:
  Version:          1.7.25
  GitCommit:        bcc810d6b9066471b0b6fa75f557a15a1cbf31bb
 runc:
  Version:          1.2.4
  GitCommit:        v1.2.4-0-g6c52b3f
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

Docker info

Client:
 Version:    28.0.1
 Context:    desktop-linux
 Debug Mode: false
 Plugins:
  ai: Docker AI Agent - Ask Gordon (Docker Inc.)
    Version:  v0.9.4
    Path:     /Users/borja/.docker/cli-plugins/docker-ai
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.21.1-desktop.2
    Path:     /Users/borja/.docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.33.1-desktop.1
    Path:     /Users/borja/.docker/cli-plugins/docker-compose
  debug: Get a shell into any image or container (Docker Inc.)
    Version:  0.0.38
    Path:     /Users/borja/.docker/cli-plugins/docker-debug
  desktop: Docker Desktop commands (Beta) (Docker Inc.)
    Version:  v0.1.5
    Path:     /Users/borja/.docker/cli-plugins/docker-desktop
  dev: Docker Dev Environments (Docker Inc.)
    Version:  v0.1.2
    Path:     /Users/borja/.docker/cli-plugins/docker-dev
  extension: Manages Docker extensions (Docker Inc.)
    Version:  v0.2.27
    Path:     /Users/borja/.docker/cli-plugins/docker-extension
  feedback: Provide feedback, right in your terminal! (Docker Inc.)
    Version:  v1.0.5
    Path:     /Users/borja/.docker/cli-plugins/docker-feedback
  init: Creates Docker-related starter files for your project (Docker Inc.)
    Version:  v1.4.0
    Path:     /Users/borja/.docker/cli-plugins/docker-init
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.)
    Version:  0.6.0
    Path:     /Users/borja/.docker/cli-plugins/docker-sbom
  scout: Docker Scout (Docker Inc.)
    Version:  v1.16.3
    Path:     /Users/borja/.docker/cli-plugins/docker-scout

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 2
 Server Version: 28.0.1
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 CDI spec directories:
  /etc/cdi
  /var/run/cdi
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: bcc810d6b9066471b0b6fa75f557a15a1cbf31bb
 runc version: v1.2.4-0-g6c52b3f
 init version: de40ad0
 Security Options:
  seccomp
   Profile: unconfined
  cgroupns
 Kernel Version: 6.10.14-linuxkit
 Operating System: Docker Desktop
 OSType: linux
 Architecture: aarch64
 CPUs: 8
 Total Memory: 7.751GiB
 Name: docker-desktop
 ID: 88bc5eb8-5e29-42ee-8e31-e23464759f95
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Labels:
  com.docker.desktop.address=unix:///Users/borja/Library/Containers/com.docker.docker/Data/docker-cli.sock
 Experimental: false
 Insecure Registries:
  hubproxy.docker.internal:5555
  ::1/128
  127.0.0.0/8
 Live Restore Enabled: false

WARNING: daemon is not using the default seccomp profile

What happened?

When starting a redpanda container using v0.37.0 it fails with /etc/redpanda/redpanda.yaml: Permission denied.
This works correctly up to v0.36.0.

Relevant log output

`grep: /etc/redpanda/redpanda.yaml: Permission denied`

Additional information

Simple reproduction code:

package service

import (
	"context"
	"log"
	"testing"

	"github.com/testcontainers/testcontainers-go"
	"github.com/testcontainers/testcontainers-go/modules/redpanda"
)

func TestRedpanda(t *testing.T) {
	ctx := context.Background()

	redpandaContainer, err := redpanda.Run(ctx,
		"docker.redpanda.com/redpandadata/redpanda:latest",
	)
	defer func() {
		if err := testcontainers.TerminateContainer(redpandaContainer); err != nil {
			log.Printf("failed to terminate container: %s", err)
		}
	}()
	if err != nil {
		log.Printf("failed to start container: %s", err)
		return
	}
}

[KenxinKun]

The file in question is owned by root:root but the exec command by default goes to the user redpanda:

Is it possible that the user is incorrectly set @mdelapenya ?

[mdelapenya]

In v0.36.0, the module already used root:root as User: https://github.com/testcontainers/testcontainers-go/blob/v0.36.0/modules/redpanda/redpanda.go#L71, so not sure about this 🤔

Can you run your own code with the redpanda image that is used in the tests? I guess the "bug" could come from the latest tag

[mdelapenya]

I've run all the tests using latest and they pass... weird 🤔

[KenxinKun]

@mdelapenya after some more digging I have some code that can reproduce (I tried with the same image in the tests but that's not the issue). The code to reproduce:

package experiment

import (
	"context"
	"testing"

	"github.com/docker/docker/api/types/container"
	"github.com/stretchr/testify/require"
	"github.com/testcontainers/testcontainers-go"
	"github.com/testcontainers/testcontainers-go/modules/redpanda"
)

func TestRedpanda(t *testing.T) {
	t.Parallel()

	ctx := context.Background()

	redpandaContainer, err := redpanda.Run(ctx,
		"docker.redpanda.com/redpandadata/redpanda:latest",
		testcontainers.WithConfigModifier(func(config *container.Config) {
			// Setting this config modifier overrides the existing one which applies the user
			// In my own code, I was just trying to add some labels here
		}),
	)
	require.NoError(t, err)

	defer func() {
		require.NoError(t, testcontainers.TerminateContainer(redpandaContainer))
	}()
}

The cause seems to be that I was using testcontainers.WithConfigModifier, which I was doing just to add some labels. This seems to replace the original configuration from the screenshot above in the main code.

Would you say this is expected behaviour? My intention wasn't to replace the entire config modifier originally, just add to it...

[mdelapenya]

Ok then, the config modifier option overrides the existing one, it's not appending. We have a similar discussion when adding WithCmd and WithCmdArgs: should the options override, or append/merge?

In any case, you have now a testcontainers.WithLabels option, which is adding the labels to the existing map.

I think we are victims here of the semantics in the names, as mentioned above: should the options override, or append/merge? It depends of the field that we are modifying. For CMD and ENTRYPOINT seemed easy to identify two different use cases. For maps and slices fields (Env, Labels, LifecycleHooks...) it seems easy to "append" values. For the rest, we can implicitly think that the user wanted to override.

Probably a bug in the documentation, which is not explaining it correctly. cc/ @stevenh

[KenxinKun]

Oh at least for my use case I can definitely use the testcontainers.WithLabels option to get around this :) Thanks for the pointer!

[stevenh]

Ok then, the config modifier option overrides the existing one, it's not appending. We have a similar discussion when adding WithCmd and WithCmdArgs: should the options override, or append/merge?

In any case, you have now a testcontainers.WithLabels option, which is adding the labels to the existing map.

I think we are victims here of the semantics in the names, as mentioned above: should the options override, or append/merge? It depends of the field that we are modifying. For CMD and ENTRYPOINT seemed easy to identify two different use cases. For maps and slices fields (Env, Labels, LifecycleHooks...) it seems easy to "append" values. For the rest, we can implicitly think that the user wanted to override.

Probably a bug in the documentation, which is not explaining it correctly. cc/ @stevenh

Yer I think we need to consider ensuring it's very obvious with naming. We have the same issue with WithWaitStrategy I just flagged here

[KenxinKun]

So just for completeness I fixed my issue by just using the suggested WithLabels. So it's up to you if you want to close or not this issue and or make those documentation clarifications 😮

[mdelapenya]

We are the following name convention for options:

• WithAdditionalXXX -> add to existing
• WithXXX -> overrides current value.

There are exceptions to this, like any option affecting maps and slices: they always append.

We probably need to make it explicit in the docs, so will close this one once updated.

[mdelapenya]

https://youtube.com/watch?v=5uM6z7RnReE&si=ItMgdN0DRx3xg4Y2 good watch to recall we are doing a not-that-bad job with the options, also pointing out that the docs must be crystal-clear about additivity or replaceability in the opts behavior, which aligns with the discussion here.