Skip to content

feat: 新的下载器#2461

Open
whitecat346 wants to merge 8 commits intoPCL-Community:devfrom
whitecat346:feat/ndl
Open

feat: 新的下载器#2461
whitecat346 wants to merge 8 commits intoPCL-Community:devfrom
whitecat346:feat/ndl

Conversation

@whitecat346
Copy link
Member

普普通通的下载器。
因为过于久远,我忘了要求,如果哪里不好提个review。
使用方法见PCL.Core.Test.Downloader.RealFileDownloadTests.DownloadFile_ShouldCompleteSuccessfully()

@whitecat346 whitecat346 self-assigned this Feb 21, 2026
@pcl-ce-automation pcl-ce-automation bot added 🛠️ 等待审查 Pull Request 已完善,等待维护者或负责人进行代码审查 size: XXL PR 大小评估:巨型 labels Feb 21, 2026
@ruattd
Copy link
Contributor

ruattd commented Feb 21, 2026

建议你同步一下上游,现在上游的目录结构有比较大的改变

# Conflicts:
#	PCL.Core/IO/Download/IDlConnection.cs
#	PCL.Core/IO/Download/IDlResourceMapping.cs
#	PCL.Core/IO/Download/IDlWriter.cs
#	PCL.Core/IO/Download/NDlFactory.cs
#	PCL.Core/Net/NDownload/IDlConnection.cs
#	PCL.Core/Net/NDownload/IDlResourceMapping.cs
#	PCL.Core/Net/NDownload/IDlWriter.cs
#	PCL.Core/Net/NDownload/Interfaces/IDlConnection.cs
#	PCL.Core/Net/NDownload/Interfaces/IDlResourceMapping.cs
#	PCL.Core/Net/NDownload/Interfaces/IDlWriter.cs
@whitecat346
Copy link
Member Author

建议你同步一下上游,现在上游的目录结构有比较大的改变

done

@ruattd ruattd self-requested a review February 21, 2026 11:52
Copy link
Member

@SALTWOOD SALTWOOD left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

一般

Copy link
Contributor

@ruattd ruattd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

存在几个问题:

  1. 应该存在一个全局共用的线程数限制
  2. 应当支持处理多个文件和多次提交,并复用已有的 DownloadClient 实例,一个下载任务很可能包含一大批文件且可能先后多次提交,它们要与已存在的所有任务共享全局线程数限制,在有必要的时候需要排队机制
  3. HttpClient 应使用传入 getter delegate 的方式获取,每次下载均创建一个 HttpClient 会导致大量小文件下载时初始化时长远大于实际数据传输时长,极其影响性能
  4. 需要支持持久化 mirror 数据,因为一个 mirror 的质量在一个用户那里大概率是一直相同的,每次都收集数据并动态决策本身就是很耗时的操作了(因此也引出了新的问题:需要考虑更复杂更全面的 mirror 评估算法)
  5. 测试明显像 AI 写的,代码的 AI 参与度比较存疑,这点需要一个明确解释

@whitecat346
Copy link
Member Author

  1. 测试明显像 AI 写的,代码的 AI 参与度比较存疑,这点需要一个明确解释

测试懒得写,直接丢给AI让他写了(

并复用已有的 DownloadClient 实例

这个PR只是一个底层的部分,我计划把这个作为上层部分来实现(DownloadManager)

需要考虑更复杂更全面的 mirror 评估算法

我实在想不出来,请大神来co-auther

@ruattd ruattd requested a review from tangge233 February 24, 2026 10:12
Copy link
Contributor

@ruattd ruattd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

建议实现一个测试模仿 MC 客户端安装的密集小文件下载,试试性能,如果你不知道下什么就直接硬编码一个最新版的所有资源文件就好了

另,如果你对 req changes 有什么疑问请提出来,而不是自己闷头写一堆不符合需求的代码,我觉得你对我上个 req 的内容就是没懂或者疑问很多

@whitecat346
Copy link
Member Author

whitecat346 commented Feb 24, 2026

我觉得你对我上个 req 的内容就是没懂或者疑问很多

yeah, i pretty sure i didnt understand what you mean at your last commit

thk for your suggestion. im goona make some change

@DotnetInstall
Copy link

粉猪打算做个mirror算法优化

Introduce a runtime mirror selection system and improve metadata probing and download logic. Key changes:

- Add Network/MirrorSelector.cs: new MirrorState, FailureType, and MirrorSelector implementing multi-factor scoring (latency, bandwidth, stability, success rate, recency), exploration (UCB/ε-greedy), greylisting, and revive/backoff logic.
- Enhance Network/MetadataProber.cs: add bandwidth probe, TTFB measurement, caching (ConcurrentDictionary), consensus grouping by ETag+size, improved scoring helpers, ProbeAsync now accepts a CancellationToken, and public cache invalidation helpers.
- Update Core/Models.cs: MirrorInfo now includes EstimatedBandwidthBps and clearer comments.
- Update DownloadClient.cs: wire MetadataProber and MirrorSelector into the worker loop, improved chunk handling (cancellation scopes, speed monitoring, bandwidth reporting), mirror failure reporting, mirror switch notifications, and helper methods (_ReturnChunk, _NotifyMirrorSwitch). Adjust concurrency handling and progress reporting.
- Tests: expand MetadataProberTests with more scenarios and add new MirrorSelectorTests covering selection, failures, greylisting, EMA updates, exploration, thread-safety and Wilson score checks. MockHttpMessageHandler improved to support configurable responses.

These changes add smarter multi-mirror probing and adaptive selection to increase robustness and performance when downloading from multiple mirrors.
@whitecat346
Copy link
Member Author

粉猪打算做个mirror算法优化

no problem

@lhx077

This comment was marked as outdated.

@lhx077 lhx077 requested review from copytiao and ruattd and removed request for copytiao, ruattd and tangge233 February 27, 2026 10:27
@lhx077 lhx077 requested a review from a team February 28, 2026 05:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size: XXL PR 大小评估:巨型 🛠️ 等待审查 Pull Request 已完善,等待维护者或负责人进行代码审查

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants