quaterion.eval.pair.retrieval_reciprocal_rank module

class RetrievalReciprocalRank(distance_metric_name: ~quaterion.distances.Distance = Distance.COSINE, reduce_func: ~typing.Callable | None = <built-in method mean of type object>)[source]

Bases: PairMetric

Calculates retrieval reciprocal rank for pair based datasets

Calculates the reciprocal of the rank at which the first relevant document was retrieved.

  • distance_metric_name – name of a distance metric to calculate distance or similarity matrices. Available names could be found in Distance.

  • reduce_func – function to reduce calculated metric. E.g. torch.mean, torch.max and others. functools.partial might be useful if you want to capture some custom arguments.


Response on a query returned 10 documents, 3 of them are relevant. Assume positions of relevant documents are [2, 5, 9]. Then retrieval reciprocal rank being calculated as 1/2 = 0.5.

raw_compute(distance_matrix: Tensor, labels: Tensor)[source]

Compute retrieval precision

  • distance_matrix – matrix with distances between embeddings. Assumed that distance from embedding to itself is meaningless. (e.g. equal to max element of matrix + 1)

  • labels – labels to compute metric. Assumed that label from object to itself has been made meaningless. (E.g. was set to 0)


torch.Tensor - computed metric

retrieval_reciprocal_rank(distance_matrix: Tensor, labels: Tensor)[source]

Calculates retrieval reciprocal rank given distance matrix and labels

  • distance_matrix – distance matrix having max possible distance value on a diagonal

  • labels – labels matrix having False or 0. on a diagonal


torch.Tensor – retrieval reciprocal rank


Learn more about Qdrant vector search project and ecosystem

Discover Qdrant

Similarity Learning

Explore practical problem solving with Similarity Learning

Learn Similarity Learning


Find people dealing with similar problems and get answers to your questions

Join Community