Feature Request: Add support for Rclone as a remote storage backend

[liblaf]

Summary

This is a feature request to add support for Rclone as a remote storage backend in DVC. Rclone is a powerful command-line program to manage files on over 70 cloud storage providers. Integrating Rclone would vastly expand the number of supported storage backends for DVC and potentially offer performance benefits.

Motivation

• Expanded Storage Support: Rclone supports a massive number of cloud storage providers, many of which are not natively supported by DVC. This would give users more options for their remote storage, including popular services like Backblaze B2, Dropbox, and many others.
• Performance: Rclone is highly optimized for cloud storage transfers. It uses multi-threaded downloads and uploads, and its performance is often superior to other tools. This could lead to faster dvc push and dvc pull operations.
• Simplified Configuration: Many users already use Rclone to manage their cloud storage. Allowing DVC to use the existing Rclone configuration would simplify the setup and reduce the need to configure credentials separately for DVC.

Proposed Implementation

A promising approach for implementation would be to use the rclone-filesystem library, which provides an fsspec-compatible interface for Rclone. Since DVC already uses fsspec for its existing remote backends, this library could allow for a relatively seamless integration.

Another option could be to use a library like rclone-python, which is a Python wrapper for the Rclone command-line tool.

Context

This feature has been requested before in issue #1801, which was closed due to being stale. This indicates that there is an interest in this feature from the community.

Current Workarounds

It is currently possible to use Rclone with DVC by mounting a remote as a local directory using rclone mount and then using a DVC local remote. However, this approach is not as convenient as a native integration.