Skip to content

Suppport Memory Storage Backend for Peers (Bypassing Disk) #4027

@gaius-qi

Description

@gaius-qi

Description

This feature only needs to be implemented on the Rust Client.

Introduce a new memory storage type for Dragonfly Peers. This allows users to configure a specific capacity for an in-memory cache.

Key Requirements:

  1. Configuration: Add a configuration option to select memory as the storage type and specify its maximum capacity.
  2. LRU Cache(Memory): When a Peer starts with the memory storage type enabled, it should automatically create cache storage with the configured capacity.
  3. Garbage Collection (GC): Implement a GC mechanism specifically for the memory storage. This GC process should evict data based on two strategies: * Time-To-Live (TTL): Remove data that has exceeded its defined lifespan.
    • Capacity: Remove data (e.g., using an LRU or similar policy) when the storage utilization approaches its configured capacity limit.

Refer to https://github.com/dragonflyoss/client/blob/main/dragonfly-client-config/src/dfdaemon.rs#L1000.

  1. Download Integration: Modify all download entry points and processing logic within the Peer. When a download request specifies or defaults to the memory storage type, the downloaded data must be written to and read from the cache storage instead of the traditional disk-based storage.

Benefit:

Using memory storage bypasses disk I/O, enabling significantly faster read and write operations for cached files, improving overall P2P distribution performance, especially for latency-sensitive workloads.

Link

Client Repo: Client Repo: https://github.com/dragonflyoss/client

Metadata

Metadata

Assignees

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions