hf-xet Integration on Windows Causes Ctrl+C Hang and Shows Confusing Installation Warnings

[hksbindra]

Describe the bug

**Main Issue: ** hf-xet Prevents Graceful Shutdown of FastAPI/Uvicorn on Windows

**Description: ** When huggingface-hub uses the hf-xet backend for downloads, a FastAPI application served by Uvicorn on Windows becomes unresponsive to Ctrl+C (SIGINT). The process hangs indefinitely and must be forcibly terminated. This issue appears to stem from a low-level incompatibility between hf-xet's I/O or threading model and the IocpProactor event loop used by asyncio on Windows.

A secondary issue - Confusing "package is not installed" Warning with Poetry
Description: When huggingface-hub is installed via Poetry without specifying the [hf-xet] extra in pyproject.toml, a confusing situation arises. The poetry.lock file correctly resolves and includes hf-xet as a transitive dependency. However, at runtime, huggingface-hub fails to detect it and issues the following warning: Xet Storage is enabled for this repo, but the 'hf_xet' package is not installed. Falling back to regular HTTP download. For better performance, install the package with: pip install huggingface_hub[hf_xet]orpip install hf_xet
This is misleading because the package is present in the virtual environment managed by Poetry. It seems the mechanism huggingface-hub uses to detect the presence of hf-xet may not be fully compatible with how Poetry handles dependencies that are not explicitly requested as "extras."

Reproduction

Steps to Reproduce:

Create a pyproject.toml with the following dependencies:

toml
[tool.poetry.dependencies]
python = ">=3.10, <4.0"
fastapi = "*"
uvicorn = {extras = ["standard"], version = "*"}
huggingface-hub = {extras = ["hf-xet"], version = "*"}

Create a minimal FastAPI application (main.py):
`from fastapi import FastAPI
from huggingface_hub import hf_hub_download

app = FastAPI()

@app.get("/download")
def download_file():
# Use a repo known to use Xet Storage
file_path = hf_hub_download(
repo_id="mixedbread-ai/mxbai-embed-large-v1",
filename="onnx/model_fp16.onnx",
repo_type="model",
)
return {"message": "Download complete", "path": file_path}
`
Run the application from a PowerShell terminal: poetry run python main.py
Trigger the download by accessing the /download endpoint.
After the download completes, press Ctrl+C in the terminal where the server is running.
Expected Behavior: The Uvicorn server should catch the SIGINT signal, log a "Shutting down" message, and exit gracefully.

Actual Behavior: The process does not respond to Ctrl+C. It hangs and must be killed via Task Manager or by closing the terminal window.

• Workaround: The issue is resolved by uninstalling the hf-xet package, I did it by removing the [hf-xet] extra from my pyproject.toml and then deleting and re-creating the poetry venv.

Logs

System info

OS: Windows 11
Python: 3.11 (via Poetry)
huggingface-hub version: 0.35.3
fastapi version: 0.118.2
uvicorn version: 0.37.0 (with [standard] extras)

[Wauplin]

cc @assafvayner @bpronan on the hf-xet part