Remove any hf-transfer related code

docs/source/en/guides/cli.md

@@ -680,39 +680,36 @@ The `hf env` command prints details about your machine setup. This is useful whe
 
 Copy-and-paste the text below in your GitHub issue.
 
-- huggingface_hub version: 0.19.0.dev0
-- Platform: Linux-6.2.0-36-generic-x86_64-with-glibc2.35
-- Python version: 3.10.12
+- huggingface_hub version: 1.0.0.rc6
+- Platform: Linux-6.8.0-85-generic-x86_64-with-glibc2.35
+- Python version: 3.11.14
 - Running in iPython ?: No
 - Running in notebook ?: No
 - Running in Google Colab ?: No
+- Running in Google Colab Enterprise ?: No
 - Token path ?: /home/wauplin/.cache/huggingface/token
 - Has saved token ?: True
 - Who am I ?: Wauplin
 - Configured git credential helpers: store
-- FastAI: N/A
-- Torch: 1.12.1
-- Jinja2: 3.1.2
-- Graphviz: 0.20.1
-- Pydot: 1.4.2
-- Pillow: 9.2.0
-- hf_transfer: 0.1.3
-- gradio: 4.0.2
-- tensorboard: 2.6
-- numpy: 1.23.2
-- pydantic: 2.4.2
-- aiohttp: 3.8.4
+- Installation method: unknown
+- Torch: N/A
+- httpx: 0.28.1
+- hf_xet: 1.1.10
+- gradio: 5.41.1
+- tensorboard: N/A
+- pydantic: 2.11.7
 - ENDPOINT: https://huggingface.co
 - HF_HUB_CACHE: /home/wauplin/.cache/huggingface/hub
 - HF_ASSETS_CACHE: /home/wauplin/.cache/huggingface/assets
 - HF_TOKEN_PATH: /home/wauplin/.cache/huggingface/token
+- HF_STORED_TOKENS_PATH: /home/wauplin/.cache/huggingface/stored_tokens
 - HF_HUB_OFFLINE: False
 - HF_HUB_DISABLE_TELEMETRY: False
 - HF_HUB_DISABLE_PROGRESS_BARS: None
 - HF_HUB_DISABLE_SYMLINKS_WARNING: False
 - HF_HUB_DISABLE_EXPERIMENTAL_WARNING: False
 - HF_HUB_DISABLE_IMPLICIT_TOKEN: False
-- HF_HUB_ENABLE_HF_TRANSFER: False
+- HF_HUB_DISABLE_XET: False
 - HF_HUB_ETAG_TIMEOUT: 10
 - HF_HUB_DOWNLOAD_TIMEOUT: 10
 ```

docs/source/en/guides/download.md

@@ -243,13 +243,6 @@ Finally, you can also make a dry-run programmatically by passing `dry_run=True`
 
 ## Faster downloads
 
-There are two options to speed up downloads. Both involve installing a Python package written in Rust.
-
-* `hf_xet` is newer and uses the Xet storage backend for upload/download. Xet storage is the [default for all new Hub users and organizations](https://huggingface.co/changelog/xet-default-for-new-users), and is in the process of being rolled out to all users. If you don't have access, join the [waitlist](https://huggingface.co/join/xet) to make Xet the default for all your repositories!
-* `hf_transfer` is a power-tool to download and upload to our LFS storage backend (note: this is less future-proof than Xet). It is thoroughly tested and has been in production for a long time, but it has some limitations. 
-
-### hf_xet
-
 Take advantage of faster downloads through `hf_xet`, the Python binding to the [`xet-core`](https://github.com/huggingface/xet-core) library that enables 
 chunk-based deduplication for faster downloads and uploads. `hf_xet` integrates seamlessly with `huggingface_hub`, but uses the Rust `xet-core` library and Xet storage instead of LFS.
 
@@ -263,23 +256,6 @@ pip install -U "huggingface_hub"
 
 As of `huggingface_hub` 0.32.0, this will also install `hf_xet`.
 
-Note: `hf_xet` will only be utilized when the files being downloaded are being stored with Xet Storage.
-
 All other `huggingface_hub` APIs will continue to work without any modification. To learn more about the benefits of Xet storage and `hf_xet`, refer to this [section](https://huggingface.co/docs/hub/storage-backends).
 
-### hf_transfer
-
-If you are running on a machine with high bandwidth,
-you can increase your download speed with [`hf_transfer`](https://github.com/huggingface/hf_transfer),
-a Rust-based library developed to speed up file transfers with the Hub.
-To enable it:
-
-1. Specify the `hf_transfer` extra when installing `huggingface_hub`
-   (e.g. `pip install huggingface_hub[hf_transfer]`).
-2. Set `HF_HUB_ENABLE_HF_TRANSFER=1` as an environment variable.
-
-> [!WARNING]
-> `hf_transfer` is a power user tool!
-> It is tested and production-ready,
-> but it lacks user-friendly features like advanced error handling or proxies.
-> For more details, please take a look at this [section](https://huggingface.co/docs/huggingface_hub/hf_transfer).
+Note: `hf_transfer` was formerly used with the LFS storage backend and is now deprecated; use hf_xet instead.

docs/source/en/guides/upload.md

@@ -155,16 +155,7 @@ Check out our [Repository limitations and recommendations](https://huggingface.c
 
 - **Start small**: We recommend starting with a small amount of data to test your upload script. It's easier to iterate on a script when failing takes only a little time.
 - **Expect failures**: Streaming large amounts of data is challenging. You don't know what can happen, but it's always best to consider that something will fail at least once -no matter if it's due to your machine, your connection, or our servers. For example, if you plan to upload a large number of files, it's best to keep track locally of which files you already uploaded before uploading the next batch. You are ensured that an LFS file that is already committed will never be re-uploaded twice but checking it client-side can still save some time. This is what [`upload_large_folder`] does for you.
-- **Use `hf_xet`**: this leverages the new storage backend for Hub, is written in Rust, and is being rolled out to users right now. In order to upload using `hf_xet` your repo must be enabled to use the Xet storage backend. It is being rolled out now, so join the [waitlist](https://huggingface.co/join/xet) to get onboarded soon!
-- **Use `hf_transfer`**: this is a Rust-based [library](https://github.com/huggingface/hf_transfer) meant to speed up uploads on machines with very high bandwidth (uploads LFS files). To use `hf_transfer`:
-    1. Specify the `hf_transfer` extra when installing `huggingface_hub`
-       (i.e., `pip install huggingface_hub[hf_transfer]`).
-    2. Set `HF_HUB_ENABLE_HF_TRANSFER=1` as an environment variable.
-
-> [!WARNING]
-> `hf_transfer` is a power user tool for uploading LFS files! It is tested and production-ready, but it is less future-proof and lacks user-friendly features like advanced error handling or proxies. For more details, please take a look at this [section](https://huggingface.co/docs/huggingface_hub/hf_transfer).
->
-> Note that `hf_xet` and `hf_transfer` tools are mutually exclusive. The former is used to upload files to Xet-enabled repos while the later uploads LFS files to regular repos.
+- **Use `hf_xet`**: this leverages the new storage backend for Hub, is written in Rust, and is now available for everyone to use. In fact, `hf_xet` is already enabled by default when using `huggingface_hub`! For maximum performances, set [`HF_XET_HIGH_PERFORMANCE=1`](../package_reference/environment_variables.md#hf_xet_high_performance) as environment variable. Be aware that by enabled high performance, the tool will try to bloat all the available bandwidth and CPU cores.
 
 ## Advanced features
 
@@ -175,9 +166,6 @@ However, `huggingface_hub` has more advanced features to make things easier. Let
 
 Take advantage of faster uploads through `hf_xet`, the Python binding to the [`xet-core`](https://github.com/huggingface/xet-core) library that enables chunk-based deduplication for faster uploads and downloads. `hf_xet` integrates seamlessly with `huggingface_hub`, but uses the Rust `xet-core` library and Xet storage instead of LFS. 
 
-> [!WARNING]
-> As of May 23rd, 2025, Xet-enabled repositories [are the default for all new Hugging Face Hub users and organizations](https://huggingface.co/changelog/xet-default-for-new-users). If your user or organization was created before then, you may need Xet enabled on your repo for `hf_xet` to actually upload to the Xet backend. Join the [waitlist](https://huggingface.co/join/xet) to make Xet the default for all your repositories. Also, note that while `hf_xet` works with in-memory bytes or bytearray data, support for BinaryIO streams is still pending.
-
 `hf_xet` uses the Xet storage system, which breaks files down into immutable chunks, storing collections of these chunks (called blocks or xorbs) remotely and retrieving them to reassemble the file when requested. When uploading, after confirming the user is authorized to write to this repo, `hf_xet` will scan the files, breaking them down into their chunks and collecting those chunks into xorbs (and deduplicating across known chunks), and then will be upload these xorbs to the Xet content-addressable service (CAS), which will verify the integrity of the xorbs, register the xorb metadata along with the LFS SHA256 hash (to support lookup/download), and write the xorbs to remote storage.
 
 To enable it, simply install the latest version of `huggingface_hub`:

docs/source/en/package_reference/environment_variables.md

@@ -71,10 +71,6 @@ Defaults to `"warning"`.
 
 For more details, see [logging reference](../package_reference/utilities#huggingface_hub.utils.logging.get_verbosity).
 
-### HF_HUB_LOCAL_DIR_AUTO_SYMLINK_THRESHOLD
-
-This environment variable has been deprecated and is now ignored by `huggingface_hub`. Downloading files to the local dir does not rely on symlinks anymore.
-
 ### HF_HUB_ETAG_TIMEOUT
 
 Integer value to define the number of seconds to wait for server response when fetching the latest metadata from a repo before downloading a file. If the request times out, `huggingface_hub` will default to the locally cached files. Setting a lower value speeds up the workflow for machines with a slow connection that have already cached files. A higher value guarantees the metadata call to succeed in more cases. Default to 10s.
@@ -177,31 +173,18 @@ Set to disable using `hf-xet`, even if it is available in your Python environmen
 
 ### HF_HUB_ENABLE_HF_TRANSFER
 
-Set to `True` for faster uploads and downloads from the Hub using `hf_transfer`.
-
-By default, `huggingface_hub` uses the Python-based `httpx.get` and `httpx.post` functions.
-Although these are reliable and versatile,
-they may not be the most efficient choice for machines with high bandwidth.
-[`hf_transfer`](https://github.com/huggingface/hf_transfer) is a Rust-based package developed to
-maximize the bandwidth used by dividing large files into smaller parts
-and transferring them simultaneously using multiple threads.
-This approach can potentially double the transfer speed.
-To use `hf_transfer`:
+> [!WARNING]
+> This is a deprecated environment variable.
+> Now that the Hugging Face Hub is fully powered by the Xet backend storage, all file transfers are going through the `hf-xet` binary package. It provides efficient file transfers through a chunk-based deduplication strategy, and a seamless integration with `huggingface_hub`.
+> This means `hf_transfer` can't be used anymore. If you are interested in high performances, check out the [`HF_XET_HIGH_PERFORMANCE` section](#hf_xet_high_performance)
 
-1. Specify the `hf_transfer` extra when installing `huggingface_hub`
-   (e.g. `pip install huggingface_hub[hf_transfer]`).
-2. Set `HF_HUB_ENABLE_HF_TRANSFER=1` as an environment variable.
-
-Please note that using `hf_transfer` comes with certain limitations. Since it is not purely Python-based, debugging errors may be challenging. Additionally, `hf_transfer` lacks several user-friendly features such as resumable downloads and proxies. These omissions are intentional to maintain the simplicity and speed of the Rust logic. Consequently, `hf_transfer` is not enabled by default in `huggingface_hub`.
+### HF_XET_HIGH_PERFORMANCE
 
-> [!TIP]
-> `hf_xet` is an alternative to `hf_transfer`. It provides efficient file transfers through a chunk-based deduplication strategy, custom Xet storage (replacing Git LFS), and a seamless integration with `huggingface_hub`. 
->
-> [Read more about the package](https://huggingface.co/docs/hub/storage-backends) and enable with `pip install "huggingface_hub[hf_xet]"`.
+Set `hf-xet` to operate with increased settings to maximize network and disk resources on the machine. Enabling high performance mode will try to saturate the network bandwidth of this machine and utilize all CPU cores for parallel upload/download activity.
 
-### HF_XET_HIGH_PERFORMANCE
+Consider this analogous to the legacy `HF_HUB_ENABLE_HF_TRANSFER=1` environment variable but applied to `hf-xet`.
 
-Set `hf-xet` to operate with increased settings to maximize network and disk resources on the machine. Enabling high performance mode will try to saturate the network bandwidth of this machine and utilize all CPU cores for parallel upload/download activity. Consider this analogous to setting `HF_HUB_ENABLE_HF_TRANSFER=True` when uploading / downloading using `hf-xet` to the Xet storage backend.
+To learn more about the benefits of Xet storage and `hf_xet`, refer to this [section](https://huggingface.co/docs/hub/storage-backends).
 
 ### HF_XET_RECONSTRUCT_WRITE_SEQUENTIALLY
 

docs/source/ko/package_reference/environment_variables.md

@@ -57,10 +57,6 @@ Hub에 인증하기 위한 사용자 액세스 토큰을 구성합니다. 이 
 
 더 자세한 정보를 알아보고 싶다면, [logging reference](../package_reference/utilities#huggingface_hub.utils.logging.get_verbosity)를 살펴보세요.
 
-### HF_HUB_LOCAL_DIR_AUTO_SYMLINK_THRESHOLD[[hfhublocaldirautosymlinkthreshold]]
-
-이 환경 변수는 더 이상 사용되지 않으며 이제 `huggingface_hub`에서 무시됩니다. 로컬 디렉터리로 파일을 다운로드할 때 더 이상 심볼릭 링크에 의존하지 않습니다.
-
 ### HF_HUB_ETAG_TIMEOUT[[hfhubetagtimeout]]
 
 파일을 다운로드하기 전에 리포지토리에서 최신 메타데이터를 가져올 때 서버 응답을 기다리는 시간(초)을 정의하는 정수 값입니다. 요청 시간이 초과되면 `huggingface_hub`는 기본적으로 로컬에 캐시된 파일을 사용합니다. 값을 낮게 설정하면 이미 파일을 캐시한 연결 속도가 느린 컴퓨터의 워크플로 속도가 빨라집니다. 값이 클수록 더 많은 경우에서 메타데이터 호출이 성공할 수 있습니다. 기본값은 10초입니다.
@@ -128,11 +124,11 @@ Hub에서 `hf_transfer`를 사용하여 더 빠르게 업로드 및 다운로드
 
 Hugging Face 생태계의 모든 환경 변수를 표준화하기 위해 일부 변수는 사용되지 않는 것으로 표시되었습니다. 해당 변수는 여전히 작동하지만 더 이상 대체한 변수보다 우선하지 않습니다. 다음 표에는 사용되지 않는 변수와 해당 대체 변수가 간략하게 설명되어 있습니다:
 
-| 사용되지 않는 변수 | 대체 변수 |
-| --- | --- |
-| `HUGGINGFACE_HUB_CACHE` | `HF_HUB_CACHE` |
-| `HUGGINGFACE_ASSETS_CACHE` | `HF_ASSETS_CACHE` |
-| `HUGGING_FACE_HUB_TOKEN` | `HF_TOKEN` |
+| 사용되지 않는 변수          | 대체 변수          |
+| --------------------------- | ------------------ |
+| `HUGGINGFACE_HUB_CACHE`     | `HF_HUB_CACHE`     |
+| `HUGGINGFACE_ASSETS_CACHE`  | `HF_ASSETS_CACHE`  |
+| `HUGGING_FACE_HUB_TOKEN`    | `HF_TOKEN`         |
 | `HUGGINGFACE_HUB_VERBOSITY` | `HF_HUB_VERBOSITY` |
 
 ## 외부 도구[[from-external-tools]]

setup.py

@@ -47,16 +47,13 @@ def get_version() -> str:
     "torch",
     "safetensors[torch]",
 ]
-extras["hf_transfer"] = [
-    "hf_transfer>=0.1.4",  # Pin for progress bars
-]
 extras["fastai"] = [
     "toml",
     "fastai>=2.4",
     "fastcore>=1.3.27",
 ]
 
-extras["hf_xet"] = ["hf-xet>=1.1.2,<2.0.0"]
+extras["hf_xet"] = ["hf-xet>=1.1.3,<2.0.0"]
 
 extras["mcp"] = [
     "mcp>=1.8.0",

src/huggingface_hub/_commit_api.py

@@ -503,11 +503,7 @@ def _wrapped_lfs_upload(batch_action) -> None:
         except Exception as exc:
             raise RuntimeError(f"Error while uploading '{operation.path_in_repo}' to the Hub.") from exc
 
-    if constants.HF_HUB_ENABLE_HF_TRANSFER:
-        logger.debug(f"Uploading {len(filtered_actions)} LFS files to the Hub using `hf_transfer`.")
-        for action in hf_tqdm(filtered_actions, name="huggingface_hub.lfs_upload"):
-            _wrapped_lfs_upload(action)
-    elif len(filtered_actions) == 1:
+    if len(filtered_actions) == 1:
         logger.debug("Uploading 1 LFS file to the Hub")
         _wrapped_lfs_upload(filtered_actions[0])
     else:

src/huggingface_hub/_snapshot_download.py

@@ -403,20 +403,14 @@ def _inner_hf_hub_download(repo_file: str) -> None:
             )
         )
 
-    if constants.HF_HUB_ENABLE_HF_TRANSFER and not dry_run:
-        # when using hf_transfer we don't want extra parallelism
-        # from the one hf_transfer provides
-        for file in filtered_repo_files:
-            _inner_hf_hub_download(file)
-    else:
-        thread_map(
-            _inner_hf_hub_download,
-            filtered_repo_files,
-            desc=tqdm_desc,
-            max_workers=max_workers,
-            # User can use its own tqdm class or the default one from `huggingface_hub.utils`
-            tqdm_class=tqdm_class or hf_tqdm,
-        )
+    thread_map(
+        _inner_hf_hub_download,
+        filtered_repo_files,
+        desc=tqdm_desc,
+        max_workers=max_workers,
+        # User can use its own tqdm class or the default one from `huggingface_hub.utils`
+        tqdm_class=tqdm_class or hf_tqdm,
+    )
 
     if dry_run:
         assert all(isinstance(r, DryRunFileInfo) for r in results)

src/huggingface_hub/_upload_large_folder.py

@@ -27,7 +27,6 @@
 from typing import TYPE_CHECKING, Any, Optional, Union
 from urllib.parse import quote
 
-from . import constants
 from ._commit_api import CommitOperationAdd, UploadInfo, _fetch_upload_modes
 from ._local_folder import LocalUploadFileMetadata, LocalUploadFilePaths, get_local_upload_paths, read_upload_metadata
 from .constants import DEFAULT_REVISION, REPO_TYPES
@@ -199,16 +198,7 @@ def upload_large_folder_internal(
     logger.info(f"Repo created: {repo_url}")
     repo_id = repo_url.repo_id
     # 2.1 Check if xet is enabled to set batch file upload size
-    is_xet_enabled = (
-        is_xet_available()
-        and api.repo_info(
-            repo_id=repo_id,
-            repo_type=repo_type,
-            revision=revision,
-            expand="xetEnabled",
-        ).xet_enabled
-    )
-    upload_batch_size = UPLOAD_BATCH_SIZE_XET if is_xet_enabled else UPLOAD_BATCH_SIZE_LFS
+    upload_batch_size = UPLOAD_BATCH_SIZE_XET if is_xet_available() else UPLOAD_BATCH_SIZE_LFS
 
     # 3. List files to upload
     filtered_paths_list = filter_repo_objects(
@@ -559,10 +549,7 @@ def _determine_next_job(status: LargeUploadStatus) -> Optional[tuple[WorkerJob,
             return (WorkerJob.GET_UPLOAD_MODE, _get_n(status.queue_get_upload_mode, MAX_NB_FILES_FETCH_UPLOAD_MODE))
 
         # 7. Preupload LFS file if at least `status.upload_batch_size` files
-        #    Skip if hf_transfer is enabled and there is already a worker preuploading LFS
-        elif status.queue_preupload_lfs.qsize() >= status.upload_batch_size and (
-            status.nb_workers_preupload_lfs == 0 or not constants.HF_HUB_ENABLE_HF_TRANSFER
-        ):
+        elif status.queue_preupload_lfs.qsize() >= status.upload_batch_size:
             status.nb_workers_preupload_lfs += 1
             logger.debug("Job: preupload LFS")
             return (WorkerJob.PREUPLOAD_LFS, _get_n(status.queue_preupload_lfs, status.upload_batch_size))

src/huggingface_hub/cli/repo.py

@@ -147,12 +147,6 @@ def repo_settings(
             help="Whether the repository should be private.",
         ),
     ] = None,
-    xet_enabled: Annotated[
-        Optional[bool],
-        typer.Option(
-            help=" Whether the repository should be enabled for Xet Storage.",
-        ),
-    ] = None,
     token: TokenOpt = None,
     repo_type: RepoTypeOpt = RepoType.model,
 ) -> None:
@@ -161,7 +155,6 @@ def repo_settings(
         repo_id=repo_id,
         gated=(gated.value if gated else None),  # type: ignore [arg-type]
         private=private,
-        xet_enabled=xet_enabled,
         repo_type=repo_type.value,
     )
     print(f"Successfully updated the settings of {ANSI.bold(repo_id)} on the Hub.")