Support both huggingface_hub v0.x and v1.x

[Wauplin]

Related to huggingface/huggingface_hub#3340 huggingface/transformers#40889

With the upcoming huggingface_hub v1.0 release, we don't need many changes in diffusers. That's why it will be possible to support both 0.x and 1.x versions -and let users decide which one suits them better-. The main difference is the use of httpx instead of requests which requires to catch different errors compared to before.

In this PR I did not work on moving diffusers completely out of requests in favor of httpx. It would still be good to do it in ~2 months once the transformers v5 + hfh v1 will have been released. Doing that will definitely harmonize the HTTP backend but will require to remove hfh v0.x support (hence why it's best to wait until 1.0 is properly released).

---

Note that in this PR the CI will still be triggered on huggingface_hub v.0.x (and should pass). I opened #12384 in parallel that has more changes to trigger on v1.0 (requires some tricky stuff because of release candidates).

Plan is to review/merge this PR and close #12384 afterwards.

[sayakpaul]

Thanks! LGTM! It would be good to also have a note on why the move towards httpx from requests. I know you elaborated on this beautifully in Slack. But in the interest of openness, I think it'd be good have that documented somewhere public if that makes sense.

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[Wauplin]

It would be good to also have a note on why the move towards httpx from requests. I know you elaborated on this beautifully in Slack. But in the interest of openness, I think it'd be good have that documented somewhere public if that makes sense.

Good idea! I added a section about that in our upcoming migration guide: huggingface/huggingface_hub@481eb49

[Wauplin]

As mentioned in DM, a lot of CI tests are failing but those are also failing on main. I realized that quite some CI workflows rely on transformers 's main branch (with pip uninstall transformers -y && python -m uv pip install -U transformers@git+https://github.com/huggingface/transformers.git --no-deps ). They are now all failing because transformers 's main relies on a prerelease version of huggingface_hub meaning the uv pip install commands need a the --allow-prerelease flag.

So failing tests are not related to the PR (but to the current state of transformers' repo). Options are either to pin transformers to a commit hash or add --allow-prerelease` (might have some side effects). In any case, not out of scope for this PR.

[Wauplin]

As discussed in DMs, let's merge this! 😃