Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
87 changes: 87 additions & 0 deletions docs/features/files.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
# Files and Mounts

Rust Testcontainers lets you seed container filesystems before startup, collect artifacts produced inside containers, and bind host paths at runtime. The APIs deliver smooth ergonomics while staying idiomatic to Rust.

## Copying Files Into Containers (Before Startup)

Use `ImageExt::with_copy_to` to stage files or directories before the container starts. Content can come from raw bytes or host paths:

```rust
// Example: copying inline bytes and directories into a container
use testcontainers::{GenericImage, WaitFor};

let project_assets = std::path::Path::new("tests/fixtures/assets");
let image = GenericImage::new("alpine", "latest")
.with_wait_for(WaitFor::seconds(1))
.with_copy_to("/opt/app/config.yaml", br#"mode = "test""#.to_vec())
.with_copy_to("/opt/app/assets", project_assets);
```

Everything is packed into a TAR archive, preserving nested directories. The helper accepts either `Vec<u8>` or any path-like value implementing `CopyDataSource`.
Note: file permissions and symbolic links follow Docker’s default TAR handling.

## Copying Files From Containers (After Execution)

Use `copy_file_from` to pull data produced inside the container:

```rust
// Example: copying a file from a running container to the host
use tempfile::tempdir;
use testcontainers::{GenericImage, WaitFor};

#[tokio::test]
async fn copy_example() -> anyhow::Result<()> {
let container = GenericImage::new("alpine", "latest")
.with_cmd(["sh", "-c", "echo '42' > /tmp/result.txt && sleep 10"])
.with_wait_for(WaitFor::seconds(1))
.start()
.await?;

let destination = tempdir()?.path().join("result.txt");
container
.copy_file_from("/tmp/result.txt", destination.as_path())
.await?;
assert_eq!(tokio::fs::read_to_string(&destination).await?, "42\n");
Ok(())
}
```

- `copy_file_from` streams file contents into any destination implementing `CopyFileFromContainer` (for example `&Path` or `&mut Vec<u8>`). When the requested path is not a regular file you’ll receive a `CopyFromContainerError`.
- Targets like `Vec<u8>` and filesystem paths overwrite existing data: vectors are cleared before writing, and files are truncated or recreated if they already exist.
- To capture the contents in memory:
```rust
let mut bytes = Vec::new();
container.copy_file_from("/tmp/result.txt", &mut bytes).await?;
```

The blocking `Container` type provides the same `copy_file_from` API.

## Using Mounts for Writable Workspaces

When a bind or tmpfs mount fits better than copy semantics, use the `Mount` helpers:

```rust
// Example: mounting a host directory for read/write access
use std::path::Path;
use testcontainers::core::{mounts::Mount, AccessMode, MountType};

let host_data = Path::new("/var/tmp/integration-data");
let mount = Mount::bind(host_data, "/workspace")
.with_mode(AccessMode::ReadWrite)
.with_type(MountType::Bind);

let image = GenericImage::new("python", "3.13")
.with_mount(mount)
.with_cmd(["python", "/workspace/run.py"]);
```

Bind mounts share host state directly. Tmpfs mounts create ephemeral in-memory storage useful for scratch data or caches.

## Selecting an Approach

- **Copy before startup** — for deterministic inputs.
- **Copy from containers** — to capture build artifacts, logs, or test fixtures produced during a run.
- **Use mounts** — when containers need to read/write large amounts of data efficiently without re-tarring.

Mixing these tools keeps tests hermetic (isolated and reproducible) while letting you inspect outputs locally.
Document each choice in code so teammates know whether data is ephemeral (`tmpfs`), seeded once (`with_copy_to`), or captured for later assertions (`copy_file_from`).
1 change: 1 addition & 0 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@ nav:
- features/configuration.md
- features/wait_strategies.md
- features/exec_commands.md
- features/files.md
- features/networking.md
- features/building_images.md
- features/docker_compose.md
Expand Down
5 changes: 4 additions & 1 deletion testcontainers/src/core.rs
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,10 @@ pub use self::{
buildable::BuildableImage,
},
containers::*,
copy::{CopyDataSource, CopyToContainer, CopyToContainerCollection, CopyToContainerError},
copy::{
CopyDataSource, CopyFileFromContainer, CopyFromContainerError, CopyToContainer,
CopyToContainerCollection, CopyToContainerError,
},
healthcheck::Healthcheck,
image::{ContainerState, ExecCommand, Image, ImageExt},
mounts::{AccessMode, Mount, MountTmpfsOptions, MountType},
Expand Down
91 changes: 78 additions & 13 deletions testcontainers/src/core/client.rs
Original file line number Diff line number Diff line change
@@ -1,9 +1,4 @@
use std::{
collections::HashMap,
io::{self},
str::FromStr,
sync::Arc,
};
use std::{collections::HashMap, io, str::FromStr, sync::Arc};

use bollard::{
auth::DockerCredentials,
Expand All @@ -17,21 +12,30 @@ use bollard::{
},
query_parameters::{
BuildImageOptionsBuilder, BuilderVersion, CreateContainerOptions,
CreateImageOptionsBuilder, InspectContainerOptions, InspectContainerOptionsBuilder,
InspectNetworkOptions, InspectNetworkOptionsBuilder, ListContainersOptionsBuilder,
ListNetworksOptions, LogsOptionsBuilder, RemoveContainerOptionsBuilder,
StartContainerOptions, StopContainerOptionsBuilder, UploadToContainerOptionsBuilder,
CreateImageOptionsBuilder, DownloadFromContainerOptionsBuilder, InspectContainerOptions,
InspectContainerOptionsBuilder, InspectNetworkOptions, InspectNetworkOptionsBuilder,
ListContainersOptionsBuilder, ListNetworksOptions, LogsOptionsBuilder,
RemoveContainerOptionsBuilder, StartContainerOptions, StopContainerOptionsBuilder,
UploadToContainerOptionsBuilder,
},
Docker,
};
use ferroid::{base32::Base32UlidExt, id::ULID};
use futures::{StreamExt, TryStreamExt};
use tokio::sync::{Mutex, OnceCell};
use futures::{pin_mut, StreamExt, TryStreamExt};
use tokio::{
io::AsyncRead,
sync::{Mutex, OnceCell},
};
use tokio_tar::{Archive as AsyncTarArchive, EntryType};
use tokio_util::io::StreamReader;
use url::Url;

use crate::core::{
client::exec::ExecResult,
copy::{CopyToContainer, CopyToContainerCollection, CopyToContainerError},
copy::{
CopyFileFromContainer, CopyFromContainerError, CopyToContainer, CopyToContainerCollection,
CopyToContainerError,
},
env::{self, ConfigurationError},
logs::{
stream::{LogStream, RawLogStream},
Expand Down Expand Up @@ -127,6 +131,8 @@ pub enum ClientError {
UploadToContainerError(BollardError),
#[error("failed to prepare data for copy-to-container: {0}")]
CopyToContainerError(CopyToContainerError),
#[error("failed to handle data copied from container: {0}")]
CopyFromContainerError(CopyFromContainerError),
}

/// The internal client.
Expand Down Expand Up @@ -404,6 +410,65 @@ impl Client {
.map_err(ClientError::UploadToContainerError)
}

pub(crate) async fn copy_file_from_container<T>(
&self,
container_id: impl AsRef<str>,
container_path: impl AsRef<str>,
target: T,
) -> Result<T::Output, ClientError>
where
T: CopyFileFromContainer,
{
let container_id = container_id.as_ref();
let options = DownloadFromContainerOptionsBuilder::new()
.path(container_path.as_ref())
.build();

let stream = self
.bollard
.download_from_container(container_id, Some(options))
.map_err(|err| io::Error::new(io::ErrorKind::Other, err));
let reader = StreamReader::new(stream);
Self::extract_file_entry(reader, target)
.await
.map_err(ClientError::CopyFromContainerError)
}

async fn extract_file_entry<R, T>(
reader: R,
target: T,
) -> Result<T::Output, CopyFromContainerError>
where
R: AsyncRead + Unpin,
T: CopyFileFromContainer,
{
let mut archive = AsyncTarArchive::new(reader);
let entries = archive.entries().map_err(CopyFromContainerError::Io)?;

pin_mut!(entries);

while let Some(entry) = entries
.try_next()
.await
.map_err(CopyFromContainerError::Io)?
{
match entry.header().entry_type() {
EntryType::GNULongName
| EntryType::GNULongLink
| EntryType::XGlobalHeader
| EntryType::XHeader
| EntryType::GNUSparse => continue, // skip metadata entries
EntryType::Directory => return Err(CopyFromContainerError::IsDirectory),
EntryType::Regular | EntryType::Continuous => {
return target.copy_from_reader(entry).await
}
et @ _ => return Err(CopyFromContainerError::UnsupportedEntry(et)),
}
}

Err(CopyFromContainerError::EmptyArchive)
}

pub(crate) async fn container_is_running(
&self,
container_id: &str,
Expand Down
24 changes: 23 additions & 1 deletion testcontainers/src/core/containers/async_container.rs
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,10 @@ use tokio_stream::StreamExt;
#[cfg(feature = "host-port-exposure")]
use super::host::HostPortExposure;
use crate::{
core::{async_drop, client::Client, env, error::Result, network::Network, ContainerState},
core::{
async_drop, client::Client, copy::CopyFileFromContainer, env, error::Result,
network::Network, ContainerState,
},
ContainerRequest, Image,
};

Expand Down Expand Up @@ -179,6 +182,25 @@ where
Ok(exit_code)
}

/// Copies a single file from the container into an arbitrary target implementing [`CopyFileFromContainer`].
///
/// # Behavior
/// - Regular files are streamed directly into the target (e.g. `PathBuf`, `Vec<u8>`).
/// - Additional archive entries (metadata or other files) are skipped after the first regular file.
/// - If `container_path` resolves to a directory, an error is returned and no data is written.
/// - Symlink handling follows Docker's `GET /containers/{id}/archive` endpoint behavior without extra processing.
pub async fn copy_file_from<T>(
&self,
container_path: impl Into<String>,
target: T,
) -> Result<T::Output>
where
T: CopyFileFromContainer,
{
let container_path = container_path.into();
self.raw.copy_file_from(container_path, target).await
}

/// Removes the container.
pub async fn rm(mut self) -> Result<()> {
log::debug!("Deleting docker container {}", self.id());
Expand Down
16 changes: 16 additions & 0 deletions testcontainers/src/core/containers/async_container/raw.rs
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ use tokio::io::{AsyncBufRead, AsyncReadExt};
use super::{exec, Client};
use crate::{
core::{
copy::CopyFileFromContainer,
error::{ContainerMissingInfo, ExecError, Result},
ports::Ports,
wait::WaitStrategy,
Expand Down Expand Up @@ -38,6 +39,21 @@ impl RawContainer {
self.docker_client.ports(&self.id).await.map_err(Into::into)
}

pub(crate) async fn copy_file_from<T>(
&self,
container_path: impl Into<String>,
target: T,
) -> Result<T::Output>
where
T: CopyFileFromContainer,
{
let container_path = container_path.into();
self.docker_client
.copy_file_from_container(self.id(), &container_path, target)
.await
.map_err(TestcontainersError::from)
}

/// Returns the mapped host port for an internal port of this docker container, on the host's
/// IPv4 interfaces.
///
Expand Down
24 changes: 23 additions & 1 deletion testcontainers/src/core/containers/sync_container.rs
Original file line number Diff line number Diff line change
@@ -1,7 +1,9 @@
use std::{fmt, io::BufRead, net::IpAddr, sync::Arc};

use crate::{
core::{env, error::Result, ports::Ports, ContainerPort, ExecCommand},
core::{
copy::CopyFileFromContainer, env, error::Result, ports::Ports, ContainerPort, ExecCommand,
},
ContainerAsync, Image,
};

Expand Down Expand Up @@ -130,6 +132,26 @@ where
})
}

/// Copies a single file from the container into an arbitrary target implementing [`CopyFileFromContainer`].
///
/// # Behavior
/// - Regular files are streamed directly into the target (e.g. `PathBuf`, `Vec<u8>`).
/// - Additional archive entries (metadata or other files) are skipped after the first regular file.
/// - If `container_path` resolves to a directory, an error is returned and no data is written.
/// - Symlink handling follows Docker's `GET /containers/{id}/archive` endpoint behavior without extra processing.
pub fn copy_file_from<T>(
&self,
container_path: impl Into<String>,
target: T,
) -> Result<T::Output>
where
T: CopyFileFromContainer,
{
let container_path = container_path.into();
self.rt()
.block_on(self.async_impl().copy_file_from(container_path, target))
}

/// Stops the container (not the same with `pause`) using the default 10 second timeout.
pub fn stop(&self) -> Result<()> {
self.rt().block_on(self.async_impl().stop())
Expand Down
Loading
Loading