Rule ordering (--exclude and --include) in Rsync

[buhtz]

This is a meta issue collection issues, ideas and discussions regarding the ordering of rsyncs include and exclude rules.

• Include file inside excluded path will include all files in excluded path #561
• Can't include specific files from excluded wildcard match (rsync: ordering of --exclude/--include) - Workaround exist #1420

There is also a PR

• fix: Sort rsync include/exclude options according to specificity (#561, #1420) #1895

Relevant code locations

backintime/common/snapshots.py

Lines 1797 to 1800 in 88d19d4

	# TODO: fix bug #561:
	# after rsync_exclude we need to explicite include files inside excluded
	# folders, recursive exclude folder-content again and finally add the
	# rest from rsync_include2

[DerekVeit]

If you will re-open #1895, I will address those concerns that you brought up and keep on it until it's committed.

[buhtz]

If you will re-open #1895, I will address those concerns that you brought up and keep on it until it's committed.

OK, I do see your motivation and energy. 😃 🚀

I was thinking about what do I need to make it mergeable.

My primary concern is that your proposed solution, independent from its concrete implementation, will have negative impact on some users. Therefore, I was looking for use cases, usually in the unit tests. But the test code is to complex. Tests don't need to be efficient, pythonic or somehow beautiful. Write everything explicit. Don't care about code duplication or number of lines. One test method need to be understandable by itself without jumping to any other method.

Of course it make sense to write system tests using real rsync. But I also need real unit tests without any dependencies, testing one "unit" in the meaning of behavior. "Behavior" is not restricted to one method only but in practice often breaks down to that.

An improved debug output would help also. Currently the final rsync command is printed out as one line, which makes it hard to read. The output should be somehow more verbose and structured. One include/exclude file/dir entry per line. It must also be clear if these entry is an entry by the user, a default exclude entry or one of the entries BIT add itself (e.g. --exclude=/). This would improve my understanding.

EDIT: And what about his config thing? getattr(self.config, "SELECTIONS_MODE", "")? I don't see a modification in config.py or in the GUI. I am also not convinced that it is a good idea to make this behavior configurable. It is IMHO to complex and difficult for the targeted BIT users (low-skilled desktop end-users). Maybe an expert option? What would be the default (behavior)?

What else?

I'll reopen the PR. But I encourage you to break it down into several PRs.

About the timeline: Currently there is no rush. I'll accept such a feature/fix only in the 2nd release from now or later. Currently I am preparing the release candidate for 1.6.0 which I am assuming is quit unstable and full of bugs. It will (hopefully) be released end of February before the Debian Import Freeze of the upcoming Ubuntu LTS.

Thank you for sticking with that topic. 🤟

Christian