Unify Btrfs send parent finding algorithm for snbk #1080

jamesljlster · 2025-12-25T09:29:27Z

The pull request implements the tree-based Btrfs send parent finding mechanism previously mentioned in #1072.
The new implementation unifies the algorithm used for determining send parents in both snapshot transferring and restoring.

If the PR is accepted, technically, there will be no algorithmic difference between snapshot transfer and restoration.
This will help unify the underlying code for snapshot transferring and restoring, and I would like to contribute it in a follow-up PR.

This is a relatively large change. Please let me know if anything needs adjustment or clarification.

Details about this PR

New Proposed `TreeView` Module

Types

TreeView:

Constructs a reusable tree from a list of ProxyNode objects and provides the following features:
- Finds a valid Btrfs send parent with the shortest walking distance from a given node (or node UUID).
- Renders the tree structure in Mermaid format for visualization and debugging.
TreeView::ProxyNode

Base class for tree nodes.
It provides metadata about Btrfs subvolumes on the sender side and determines whether a node is valid to be used as a send parent.
TreeView::VirtualNode

Internal class representing Btrfs subvolumes that are not managed by Snapper (i.e., no corresponding TheBigThing exists for a UUID). Virtual nodes are always considered invalid as send parents.
TreeView::SearchResult

Wraps a search result, including the selected parent node and its walking distance.
TreeView::ParentType

Enumeration describing how a parent node is associated:
- DIRECT_PARENT: a direct Btrfs parent UUID relationship exists
- IMPLICIT_PARENT: the parent is manually assigned without an explicit Btrfs relationship

Technical Details

TreeView stores nodes using shared pointers, and each node holds an iterator to its associated TheBigThing.
The node validity is retrieved dynamically, making the tree reusable across multiple searches.

A virtual root node without an actual UUID is created to connect all discovered subtrees, ensuring that no nodes remain orphaned. This is a special usage of TreeView::VirtualNode.

Breadth-first search strategy is used to find the nearest valid node as the send parent.
Parent nodes are prioritized over siblings. Among siblings, nodes with higher snapshot numbers (i.e., more recent snapshots) are preferred.
Even if the starting node itself is a valid send parent, it is explicitly prohibited from being selected as its own send parent.

Some basic documentation and comments have been added to the code.
Please let me know if the documentation is insufficient. I will refine it accordingly.

Demo

The following demo was made from an openSUSE Tumbleweed virtual machine.

Snapshots:

   # │ Type   │ Pre # │ Date                            │ User │ Used Space │ Cleanup  │ Description                                     │ Userdata
─────┼────────┼───────┼─────────────────────────────────┼──────┼────────────┼──────────┼─────────────────────────────────────────────────┼──────────────
  0  │ single │       │                                 │ root │            │          │ current                                         │
  1  │ single │       │ Fri 19 Dec 2025 11:40:50 PM CST │ root │   1.67 MiB │          │ first root filesystem                           │
  2  │ single │       │ Sat 20 Dec 2025 12:02:38 AM CST │ root │ 132.07 MiB │          │ after installation                              │ important=yes
  9  │ pre    │       │ Sat 20 Dec 2025 12:33:55 AM CST │ root │  80.00 KiB │          │ yast firewall: allow ssh                        │
 10  │ post   │     9 │ Sat 20 Dec 2025 12:34:23 AM CST │ root │  16.00 KiB │          │                                                 │
 11  │ pre    │       │ Sat 20 Dec 2025 12:35:00 AM CST │ root │  16.00 KiB │          │ yast services-manager: enable sshd              │
 12  │ post   │    11 │ Sat 20 Dec 2025 12:36:02 AM CST │ root │  16.00 KiB │          │                                                 │
 17  │ pre    │       │ Sat 20 Dec 2025 12:39:06 AM CST │ root │  16.00 KiB │          │ modify snapper root config                      │
 18  │ post   │    17 │ Sat 20 Dec 2025 12:42:22 AM CST │ root │  64.00 KiB │          │                                                 │
 23  │ single │       │ Sat 20 Dec 2025 01:00:07 AM CST │ root │  48.00 KiB │ timeline │ timeline                                        │
 46  │ single │       │ Sun 21 Dec 2025 12:00:07 AM CST │ root │  80.00 KiB │ timeline │ timeline                                        │
 70  │ single │       │ Mon 22 Dec 2025 12:00:07 AM CST │ root │   3.03 MiB │ timeline │ timeline                                        │
 80  │ single │       │ Mon 22 Dec 2025 09:08:05 AM CST │ root │   1.69 MiB │          │ created snapper config for home                 │
 93  │ pre    │       │ Mon 22 Dec 2025 03:40:32 PM CST │ root │   3.59 MiB │ number   │ zypp(ruby.ruby3.4)                              │ important=yes
 94  │ post   │    93 │ Mon 22 Dec 2025 03:41:58 PM CST │ root │  55.31 MiB │ number   │                                                 │ important=yes
100  │ single │       │ Mon 22 Dec 2025 03:45:14 PM CST │ root │  32.00 KiB │          │ [virtualization] installed virtualization tools │
104  │ single │       │ Mon 22 Dec 2025 03:48:22 PM CST │ root │  64.00 KiB │ number   │ rollback backup of #91                          │ important=yes
105  │ single │       │ Mon 22 Dec 2025 03:48:22 PM CST │ root │   1.26 MiB │          │ writable copy of #80                            │
110  │ single │       │ Mon 22 Dec 2025 03:54:47 PM CST │ root │   1.10 MiB │          │ [default] head                                  │
112  │ single │       │ Mon 22 Dec 2025 04:13:13 PM CST │ root │   9.18 MiB │          │ [dev-snapper] head                              │
113  │ pre    │       │ Mon 22 Dec 2025 04:17:05 PM CST │ root │ 452.00 KiB │          │ [dev-snapper] install snapper development tools │
118  │ post   │   113 │ Mon 22 Dec 2025 04:19:23 PM CST │ root │   5.91 MiB │          │                                                 │
121  │ pre    │       │ Mon 22 Dec 2025 04:36:09 PM CST │ root │  96.00 KiB │          │ [dev-snapper] install missing dependencies      │
122  │ pre    │       │ Mon 22 Dec 2025 04:42:00 PM CST │ root │  16.00 KiB │ number   │ zypp(zypper)                                    │ important=no
123  │ post   │   122 │ Mon 22 Dec 2025 04:42:03 PM CST │ root │   1.77 MiB │ number   │                                                 │ important=no
124  │ post   │   121 │ Mon 22 Dec 2025 04:44:32 PM CST │ root │  80.00 KiB │          │                                                 │
132  │ single │       │ Tue 23 Dec 2025 12:00:18 AM CST │ root │ 228.00 KiB │ timeline │ timeline                                        │
149* │ single │       │ Tue 23 Dec 2025 04:13:19 PM CST │ root │   2.59 MiB │          │ [dev-snapper/snbk] head                         │
154  │ pre    │       │ Tue 23 Dec 2025 04:23:04 PM CST │ root │   1.88 MiB │ number   │ yast partitioner                                │
155  │ post   │   154 │ Tue 23 Dec 2025 04:24:53 PM CST │ root │  16.00 KiB │ number   │                                                 │
156  │ pre    │       │ Tue 23 Dec 2025 04:25:02 PM CST │ root │  16.00 KiB │ number   │ yast partitioner                                │
157  │ post   │   156 │ Tue 23 Dec 2025 04:25:47 PM CST │ root │  48.00 KiB │ number   │                                                 │
158  │ single │       │ Tue 23 Dec 2025 04:28:47 PM CST │ root │  80.00 KiB │          │ [dev-snapper/snbk] checkpoint                   │
166  │ single │       │ Wed 24 Dec 2025 12:00:00 AM CST │ root │ 416.00 KiB │ timeline │ timeline                                        │
170  │ single │       │ Wed 24 Dec 2025 04:00:00 AM CST │ root │  80.00 KiB │ timeline │ timeline                                        │
171  │ single │       │ Wed 24 Dec 2025 05:00:00 AM CST │ root │  80.00 KiB │ timeline │ timeline                                        │
172  │ single │       │ Wed 24 Dec 2025 06:00:00 AM CST │ root │  80.00 KiB │ timeline │ timeline                                        │
173  │ single │       │ Wed 24 Dec 2025 07:00:00 AM CST │ root │  80.00 KiB │ timeline │ timeline                                        │
174  │ single │       │ Wed 24 Dec 2025 08:00:00 AM CST │ root │  80.00 KiB │ timeline │ timeline                                        │
175  │ single │       │ Wed 24 Dec 2025 09:00:00 AM CST │ root │  80.00 KiB │ timeline │ timeline                                        │
176  │ single │       │ Wed 24 Dec 2025 10:00:00 AM CST │ root │  80.00 KiB │ timeline │ timeline                                        │
177  │ single │       │ Wed 24 Dec 2025 11:00:00 AM CST │ root │  80.00 KiB │ timeline │ timeline                                        │
178  │ single │       │ Wed 24 Dec 2025 12:00:00 PM CST │ root │  80.00 KiB │ timeline │ timeline                                        │
179  │ single │       │ Wed 24 Dec 2025 01:00:00 PM CST │ root │  80.00 KiB │ timeline │ timeline                                        │
180  │ single │       │ Wed 24 Dec 2025 02:00:00 PM CST │ root │  80.00 KiB │ timeline │ timeline                                        │

Backups:

  # │ Date                            │ Source State │ Target State
────┼─────────────────────────────────┼──────────────┼─────────────
  1 │ Fri 19 Dec 2025 11:40:50 PM CST │ read-write   │
  2 │ Sat 20 Dec 2025 12:02:38 AM CST │ read-only    │ valid
  9 │ Sat 20 Dec 2025 12:33:55 AM CST │ read-only    │
 10 │ Sat 20 Dec 2025 12:34:23 AM CST │ read-only    │
 11 │ Sat 20 Dec 2025 12:35:00 AM CST │ read-only    │
 12 │ Sat 20 Dec 2025 12:36:02 AM CST │ read-only    │
 17 │ Sat 20 Dec 2025 12:39:06 AM CST │ read-only    │
 18 │ Sat 20 Dec 2025 12:42:22 AM CST │ read-only    │
 23 │ Sat 20 Dec 2025 01:00:07 AM CST │ read-only    │
 46 │ Sun 21 Dec 2025 12:00:07 AM CST │ read-only    │
 70 │ Mon 22 Dec 2025 12:00:07 AM CST │ read-only    │
 80 │ Mon 22 Dec 2025 09:08:05 AM CST │ read-only    │
 93 │ Mon 22 Dec 2025 03:40:32 PM CST │ read-only    │ valid
 94 │ Mon 22 Dec 2025 03:41:58 PM CST │ read-only    │
100 │ Mon 22 Dec 2025 03:45:14 PM CST │ read-only    │
104 │ Mon 22 Dec 2025 03:48:22 PM CST │ read-only    │
105 │ Mon 22 Dec 2025 03:48:22 PM CST │ read-write   │
110 │ Mon 22 Dec 2025 03:54:47 PM CST │ read-write   │
112 │ Mon 22 Dec 2025 04:13:13 PM CST │ read-write   │
113 │ Mon 22 Dec 2025 04:17:05 PM CST │ read-only    │
118 │ Mon 22 Dec 2025 04:19:23 PM CST │ read-only    │
121 │ Mon 22 Dec 2025 04:36:09 PM CST │ read-only    │
122 │ Mon 22 Dec 2025 04:42:00 PM CST │ read-only    │
123 │ Mon 22 Dec 2025 04:42:03 PM CST │ read-only    │
124 │ Mon 22 Dec 2025 04:44:32 PM CST │ read-only    │
132 │ Tue 23 Dec 2025 12:00:18 AM CST │ read-only    │ valid
149 │ Tue 23 Dec 2025 04:13:19 PM CST │ read-write   │
154 │ Tue 23 Dec 2025 04:23:04 PM CST │ read-only    │
155 │ Tue 23 Dec 2025 04:24:53 PM CST │ read-only    │ valid
156 │ Tue 23 Dec 2025 04:25:02 PM CST │ read-only    │
157 │ Tue 23 Dec 2025 04:25:47 PM CST │ read-only    │
158 │ Tue 23 Dec 2025 04:28:47 PM CST │ read-only    │
166 │ Wed 24 Dec 2025 12:00:00 AM CST │ read-only    │
170 │ Wed 24 Dec 2025 04:00:00 AM CST │ read-only    │
171 │ Wed 24 Dec 2025 05:00:00 AM CST │ read-only    │
172 │ Wed 24 Dec 2025 06:00:00 AM CST │ read-only    │
173 │ Wed 24 Dec 2025 07:00:00 AM CST │ read-only    │
174 │ Wed 24 Dec 2025 08:00:00 AM CST │ read-only    │
175 │ Wed 24 Dec 2025 09:00:00 AM CST │ read-only    │
176 │ Wed 24 Dec 2025 10:00:00 AM CST │ read-only    │
177 │ Wed 24 Dec 2025 11:00:00 AM CST │ read-only    │
178 │ Wed 24 Dec 2025 12:00:00 PM CST │ read-only    │
179 │ Wed 24 Dec 2025 01:00:00 PM CST │ read-only    │
180 │ Wed 24 Dec 2025 02:00:00 PM CST │ read-only    │

The constructed tree:

graph LR
    0x39736570[0x39736570 - virtual]
    0x39736570 -.-> 0x397455d0
    0x397455d0[0x397455d0 - virtual]
    0x397455d0 --> 104
    104[104]
    0x397455d0 --> 100
    100[100]
    0x397455d0 --> 94
    94[94]
    0x397455d0 --> 93
    93[93 - valid]
    0x39736570 -.-> 1
    1[1]
    1 --> 80
    80[80]
    80 --> 105
    105[105]
    105 --> 110
    110[110]
    110 --> 112
    112[112]
    112 --> 149
    149[149]
    149 --> 180
    180[180]
    149 --> 179
    179[179]
    149 --> 178
    178[178]
    149 --> 177
    177[177]
    149 --> 176
    176[176]
    149 --> 175
    175[175]
    149 --> 174
    174[174]
    149 --> 173
    173[173]
    149 --> 172
    172[172]
    149 --> 171
    171[171]
    149 --> 170
    170[170]
    149 --> 166
    166[166]
    149 --> 158
    158[158]
    149 --> 157
    157[157]
    149 --> 156
    156[156]
    149 --> 155
    155[155 - valid]
    149 --> 154
    154[154]
    112 --> 132
    132[132 - valid]
    112 --> 124
    124[124]
    112 --> 123
    123[123]
    112 --> 122
    122[122]
    112 --> 121
    121[121]
    112 --> 118
    118[118]
    112 --> 113
    113[113]
    1 --> 70
    70[70]
    1 --> 46
    46[46]
    1 --> 23
    23[23]
    1 --> 18
    18[18]
    1 --> 17
    17[17]
    1 --> 12
    12[12]
    1 --> 11
    11[11]
    1 --> 10
    10[10]
    1 --> 9
    9[9]
    1 --> 2
    2[2 - valid]

Resolved send parents for each snapshot:

Parent of 2: 93 (distance: 4)
Parent of 9: 2 (distance: 2)
Parent of 10: 2 (distance: 2)
Parent of 11: 2 (distance: 2)
Parent of 12: 2 (distance: 2)
Parent of 17: 2 (distance: 2)
Parent of 18: 2 (distance: 2)
Parent of 23: 2 (distance: 2)
Parent of 46: 2 (distance: 2)
Parent of 70: 2 (distance: 2)
Parent of 80: 2 (distance: 2)
Parent of 93: 2 (distance: 4)
Parent of 94: 93 (distance: 2)
Parent of 100: 93 (distance: 2)
Parent of 104: 93 (distance: 2)
Parent of 113: 132 (distance: 2)
Parent of 118: 132 (distance: 2)
Parent of 121: 132 (distance: 2)
Parent of 122: 132 (distance: 2)
Parent of 123: 132 (distance: 2)
Parent of 124: 132 (distance: 2)
Parent of 132: 155 (distance: 3)
Parent of 154: 155 (distance: 2)
Parent of 155: 132 (distance: 3)
Parent of 156: 155 (distance: 2)
Parent of 157: 155 (distance: 2)
Parent of 158: 155 (distance: 2)
Parent of 166: 155 (distance: 2)
Parent of 170: 155 (distance: 2)
Parent of 171: 155 (distance: 2)
Parent of 172: 155 (distance: 2)
Parent of 173: 155 (distance: 2)
Parent of 174: 155 (distance: 2)
Parent of 175: 155 (distance: 2)
Parent of 176: 155 (distance: 2)
Parent of 177: 155 (distance: 2)
Parent of 178: 155 (distance: 2)
Parent of 179: 155 (distance: 2)
Parent of 180: 155 (distance: 2)

Improvements

After re-transferring snapshots from the source to the target using the new algorithm, disk usage is significantly reduced for the selected snapshots:

Changes to `TheBigThings` module

The following types were added:

~~TheBigThings::BaseNode~~ anonymous::BaseNode

Base class for tree nodes.
It provides a constructor interface using an iterator to TheBigThing, and determines node validity
(i.e., whether the snapshot exists on both sender and receiver sides).
~~TheBigThings::SourceNode~~ anonymous::SourceNode

Used when transferring snapshots from the source to the target.
~~TheBigThings::TargetNode~~ anonymous::TargetNode

Used when restoring snapshots from the target to the source.

Two member variables were added:

source_tree:

TreeView with ~~TheBigThings::SourceNode~~ anonymous::SourceNode.
Used to find the Btrfs send parent when transferring snapshots.
target_tree:

TreeView with ~~TheBigThings::TargetNode~~ anonymous::TargetNode.
Used to find the Btrfs send parent when restoring snapshots.

The following functions were replaced by TreeView::find_nearest_valid_node:

TheBigThings::find_send_parent
TheBigThings::find_restore_parent

jamesljlster · 2025-12-26T14:46:31Z

I just realized that the ProxyNode derivatives can be moved into an anonymous namespace. They are only used internally and don't need to be exposed in the header file. The latest commit (b619946) was added for this purpose.

jamesljlster · 2026-01-05T06:56:39Z

While investigating possible Clang-Format configurations, I realized that my indentation for access modifiers was incorrect. The latest commit 761b7a3 fixed it. Sorry!

jamesljlster added 5 commits December 25, 2025 12:22

Added a TreeView module for finding Btrfs send parent

dbb6e5d

Added default constructor for TreeView

1ba3983

Unified Btrfs send parent finding algorithm

ee5c07d

Added copyright header for client/snbk/TreeView.cc

4fd1715

Moved ProxyNode derivatives to anonymous namespace

b619946

jamesljlster added 2 commits December 26, 2025 23:02

Adopted boost::none to return empty search result

4433435

Fixed indentation for access modifier

761b7a3

aschnell added the enhancement label Jan 5, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Unify Btrfs send parent finding algorithm for snbk #1080

Unify Btrfs send parent finding algorithm for snbk #1080

jamesljlster commented Dec 25, 2025 •

edited

Loading

Uh oh!

jamesljlster commented Dec 26, 2025

Uh oh!

jamesljlster commented Jan 5, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Unify Btrfs send parent finding algorithm for snbk #1080

Are you sure you want to change the base?

Unify Btrfs send parent finding algorithm for snbk #1080

Conversation

jamesljlster commented Dec 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Details about this PR

New Proposed TreeView Module

Types

Technical Details

Demo

Improvements

Changes to TheBigThings module

Uh oh!

jamesljlster commented Dec 26, 2025

Uh oh!

jamesljlster commented Jan 5, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

jamesljlster commented Dec 25, 2025 •

edited

Loading

New Proposed `TreeView` Module

Changes to `TheBigThings` module