Suppport Memory Storage Backend for Peers (Bypassing Disk)

[gaius-qi]

Description

This feature only needs to be implemented on the Rust Client.

Introduce a new memory storage type for Dragonfly Peers. This allows users to configure a specific capacity for an in-memory cache.

Key Requirements:

1. Configuration: Add a configuration option to select memory as the storage type and specify its maximum capacity.
2. LRU Cache(Memory): When a Peer starts with the memory storage type enabled, it should automatically create cache storage with the configured capacity.
3. Garbage Collection (GC): Implement a GC mechanism specifically for the memory storage. This GC process should evict data based on two strategies: * Time-To-Live (TTL): Remove data that has exceeded its defined lifespan.
   • Capacity: Remove data (e.g., using an LRU or similar policy) when the storage utilization approaches its configured capacity limit.

Refer to https://github.com/dragonflyoss/client/blob/main/dragonfly-client-config/src/dfdaemon.rs#L1000.

4. Download Integration: Modify all download entry points and processing logic within the Peer. When a download request specifies or defaults to the memory storage type, the downloaded data must be written to and read from the cache storage instead of the traditional disk-based storage.

Benefit:

Using memory storage bypasses disk I/O, enabling significantly faster read and write operations for cached files, improving overall P2P distribution performance, especially for latency-sensitive workloads.

Link

Client Repo: Client Repo: https://github.com/dragonflyoss/client

[FlySky-z]

Hi @gaius-qi , I’m interested in contributing to this issue. If it’s still open, could you kindly assign it to me? Thank you!

[gaius-qi]

@fu220