Multi-threaded snapshot download

[Wauplin]

This PR makes snapshot_download download files in parallel all the time. Previously we were not doing it if HF_XET_HIGH_PERFORMANCE=1 was set as environment variable. Goal was to avoid bloating the CPU if Xet starts N threads for each of the X files being downloaded in parallel. We had a similar logic for hf_transfer. However after some discussions on slack (private) and following this gist, it turns out that hf_xet is doing things much more carefully than hf_transfer by reusing the same pool of connections no matter the number of parallel calls. This means that even if snapshot_download starts N threads, they will all share the same Xet resources without freezing the user's machine.

Hence this PR.

---

Quick benchmark (not a scientific one^^):

HF_XET_HIGH_PERFORMANCE=1 hf download Agent-Ark/Toucan-1.5M

On main: 7'20

On this PR: 3'50

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[Wauplin]

Bypassing approval to avoid conflicts. Happy to open a follow-up PR in case we need to update something.