Add cuvs-bench-elastic: HTTP backend for Elasticsearch GPU vector search#1907
Draft
afourniernv wants to merge 8 commits intorapidsai:mainfrom
Draft
Add cuvs-bench-elastic: HTTP backend for Elasticsearch GPU vector search#1907afourniernv wants to merge 8 commits intorapidsai:mainfrom
afourniernv wants to merge 8 commits intorapidsai:mainfrom
Conversation
cjnolet
added
improvement
jnke2016
reviewed
Mar 11, 2026
| ep.load()() | ||
| except ImportError as e: | ||
| if "elasticsearch" in str(e).lower() or "elasticsearch" in str(e): | ||
| raise ImportError( |
Contributor
There was a problem hiding this comment.
Nice. This is what I discussed with @cjnolet pertaining to lazy import and used Milvus as an example
class MilvusBackend(BenchmarkBackend):
def __init__(self, config: Dict[str, Any]):
super().__init__(config)
try:
from pymilvus import connections, Collection
except ImportError:
raise ImportError(
"pymilvus is required for MilvusBackend. "
"Install with: pip install pymilvus"
)
connections.connect(host=config["host"], port=config["port"])
Introduce cuvs-bench-elastic as an optional plugin for cuvs-bench that provides an Elasticsearch backend. The backend communicates with Elasticsearch via HTTP and supports HNSW indexing with optional GPU acceleration when using the Elasticsearch GPU image (cuVS-accelerated vector search). - Add cuvs_bench_elastic package with backend and config loader entry points - Extend cuvs_bench registry and search spaces for pluggable backends - Add elastic and integration optional dependencies to cuvs-bench - Add modularization tests and integration test scaffolding (disabled until CI has ES GPU image, cuVS libs, and GPU runner) Signed-off-by: Alex Fournier <afournier@nvidia.com>
Use single-doc format (_index, _id, vector_field) instead of two-part NDJSON (index action + source) so ES accepts the bulk request. Signed-off-by: Alex Fournier <afournier@nvidia.com>
Expose ELASTIC constant and convenience wrappers for build-only, search-only, or full benchmark runs. Signed-off-by: Alex Fournier <afournier@nvidia.com>
afourniernv
force-pushed
the
fea-1856-cuvs-lucene-backend
branch
from
ec0b3e3 to
7b25521
Compare
- Document run_build, run_search, run_benchmark convenience API - Document ELASTIC constant and orchestrator usage - Add username/password support in config loader (converts to basic_auth) Signed-off-by: Alex Fournier <afournier@nvidia.com>
cjnolet
reviewed
Mar 25, 2026
|
|
||
|
|
||
| @pytest.fixture(scope="module") | ||
| def elasticsearch_container(): |
Member
There was a problem hiding this comment.
Why a container? We should just be able to assume an elasticsearch cluster is ready to accept requests, right? cuvs-bench doesn't need to be self-contained, just be able to send the proper requests to an existing elasticsearch cluster. Or am I missing a big detail here?
cjnolet
reviewed
Mar 25, 2026
| class ElasticBackend(BenchmarkBackend): | ||
| """Elasticsearch GPU backend for vector benchmarking.""" | ||
|
|
||
| def __init__(self, config: Dict[str, Any]): |
Member
There was a problem hiding this comment.
Oh I see- is the container just for the integration tests?
cjnolet
reviewed
Apr 9, 2026
| @@ -0,0 +1,50 @@ | |||
| # SPDX-FileCopyrightText: Copyright (c) 2024-2026, NVIDIA CORPORATION. | |||
| # SPDX-License-Identifier: Apache-2.0 | |||
|
|
|||
Member
There was a problem hiding this comment.
We can't have different packages deployed for every backend. This significantly adds to the maintenance burden as every new package now needs to be versioned, deloyed, and audited for dependency trails. Let's consolidate this into the existing cuvs-bench package. All dependencies on anything elasticsearch or cuvs-lucene should be soft dependendencies in Python (that is, they should test if they can import the package and if they can't, throw a warning and fail gracefully).
- New `cuvs_bench_elastic` package with HTTP backend for Elasticsearch GPU vector search (HNSW, int8_hnsw, int4_hnsw, bbq_hnsw index types) - Supports `pip install cuvs-bench[elastic]` without a separate PyPI publish: `cuvs_bench` bundles the plugin via setuptools packages.find - Plugin registers via entry points (`cuvs_bench.backends` / `cuvs_bench.config_loaders`) — no changes to core cuvs-bench required - `ElasticConfigLoader` reads shared `datasets.yaml` from cuvs_bench and `elastic.yaml` from the plugin config; supports sweep and tune modes - `build()` checks index existence before file validation so `force=False` returns immediately without requiring the base file on disk - Removed testcontainers-based integration tests; added unit tests for pre-flight failure, force=False skip, dry-run, helper functions - `elasticsearch` client is an optional dep (`cuvs-bench[elastic]` extra)
afourniernv
force-pushed
the
fea-1856-cuvs-lucene-backend
branch
from
dca9c6f to
087289d
Compare
The separate cuvs_bench_elastic package required bundling via packages.find and complicated the build. Instead, keep the backend inside cuvs_bench and use entry points pointing back into the same package so the backend only registers when elasticsearch is installed. - git mv backend.py to cuvs_bench/backends/elasticsearch.py - git mv elastic.yaml to cuvs_bench/config/algos/ - Fix imports to relative paths - Fix _get_elastic_config_path() to use ../config from backends/ - Update pyproject.toml: entry points -> cuvs_bench.backends.elasticsearch:register - Remove packages.find (no longer needed) - Remove cuvs_bench_elastic/ package entirely DX unchanged: pip install cuvs-bench[elastic] One package, one publish pipeline. Signed-off-by: Alex Fournier <afournier@nvidia.com>
…ch.py Restores the high-level API that was previously in cuvs_bench_elastic/__init__.py so existing demo scripts continue to work after the module consolidation. Signed-off-by: Alex Fournier <afournier@nvidia.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Introduce cuvs-bench-elastic as an optional plugin for cuvs-bench that provides an Elasticsearch backend. The backend communicates with Elasticsearch via HTTP and supports HNSW indexing with optional GPU acceleration when using the Elasticsearch GPU image (cuVS-accelerated vector search).