rank_eval-9.3-python.md

endpoint	rank_eval
lang	python
es_version	9.3
client	elasticsearch==9.3.0

Elasticsearch 9.3 rank_eval endpoint (Python example)

Use client.rank_eval() to measure how well your search queries rank relevant documents. Provide rated documents and a metric to evaluate:

response = client.rank_eval(
    index="products",
    requests=[
        {
            "id": "electronics_query",
            "request": {
                "query": {"match": {"category": "electronics"}}
            },
            "ratings": [
                {"_index": "products", "_id": "prod-2", "rating": 3},
                {"_index": "products", "_id": "prod-4", "rating": 2},
                {"_index": "products", "_id": "prod-1", "rating": 0},
            ],
        }
    ],
    metric={"precision": {"k": 10, "relevant_rating_threshold": 1}},
)

print(f"Quality score: {response['metric_score']:.3f}")

Multiple queries

Evaluate several queries at once to get an aggregate quality score and per-query breakdowns:

response = client.rank_eval(
    index="products",
    requests=[
        {
            "id": "electronics",
            "request": {"query": {"match": {"category": "electronics"}}},
            "ratings": [
                {"_index": "products", "_id": "prod-2", "rating": 3},
                {"_index": "products", "_id": "prod-4", "rating": 2},
            ],
        },
        {
            "id": "affordable",
            "request": {"query": {"range": {"price": {"lte": 200}}}},
            "ratings": [
                {"_index": "products", "_id": "prod-4", "rating": 3},
                {"_index": "products", "_id": "prod-2", "rating": 1},
            ],
        },
    ],
    metric={"precision": {"k": 10, "relevant_rating_threshold": 1}},
)

for query_id, detail in response["details"].items():
    print(f"{query_id}: {detail['metric_score']:.3f}")

Mean reciprocal rank

Use mean_reciprocal_rank to measure how quickly the first relevant result appears:

response = client.rank_eval(
    index="products",
    requests=[
        {
            "id": "electronics_query",
            "request": {"query": {"match": {"category": "electronics"}}},
            "ratings": [
                {"_index": "products", "_id": "prod-2", "rating": 3},
                {"_index": "products", "_id": "prod-4", "rating": 2},
            ],
        }
    ],
    metric={"mean_reciprocal_rank": {"k": 10, "relevant_rating_threshold": 1}},
)

print(f"MRR: {response['metric_score']:.3f}")