bzfs-1.13.0 – subsecond ZFS snapshot replication frequency at fleet scale

[whoschek]

Quick heads‑up that bzfs 1.13.0 is out. bzfs is a simple, reliable CLI to replicate ZFS snapshots (zfs send/receive) locally or over SSH, plus an optional bzfs_jobrunner wrapper to run periodic snapshot/replication/prune jobs across multiple hosts or large fleets.

What's new in 1.13.0 (highlights)

• Faster over SSH: connections are now reused across zpools and on startup, reducing latency — you’ll notice it most with many small sends or lots of datasets or when replicating every second, or even more frequently.
• Starts sending sooner: bzfs now estimates send size in parallel so streaming begins with less upfront delay.
• More resilient connects: retries SSH before giving up; useful for brief hiccups or busy hosts.
• Cleaner UX: avoids repeated “Broken pipe” noise if you abort a pipeline early; normalized exit codes.
• Smarter snapshot caching: better hashing and shorter cache file paths for speed and clarity.
• Jobrunner safety: fixed an option‑leak across subjobs; multi‑target runs are more predictable.
• Security hardening: stricter file permission validation.
• Platform updates: nightly tests include Python 3.14; dropped support for Python 3.8 (EOL) and legacy Solaris.

Why it matters

• Lower latency per replication round, especially with lots of small changes.
• Fewer spurious errors and clearer logs during day‑to‑day ops.
• Safer, more predictable periodic workflows with bzfs_jobrunner.

Upgrade

• pip: pip install -U bzfs
• Compatibility: Python ≥ 3.9

Quick start (local and SSH)

• Local: bzfs pool/src/ds pool/backup/ds
• Pull from remote: bzfs user@host:pool/src/ds pool/backup/ds
• First time transfers everything; subsequent runs are incremental from the latest common snapshot. Add --dryrun to see what would happen without changing anything.

Docs and links

• Project
• README: see usage, options, and examples
• README_bzfs_jobrunner: multi‑host periodic jobs and fleet configs
• Changelog

Tips

• For periodic jobs, take snapshots and replicate on a schedule (e.g., hourly and daily), and prune old snapshots on both source and destination.
• Start with --dryrun and a non‑critical dataset to validate filters and retention before enabling deletes.

Feedback

• Bugs, ideas, and PRs welcome. If you hit issues, sharing logs (with sensitive bits redacted), your command line, and rough dataset scale helps a lot.

Happy replicating!