quaterion.eval.pair.retrieval_precision module

class RetrievalPrecision(k=1, distance_metric_name: ~quaterion.distances.Distance = Distance.COSINE, reduce_func: ~typing.Callable | None = <built-in method mean of type object>)[source]

Bases: PairMetric

Calculates retrieval precision@k for pair based datasets

  • k – number of documents among which to search a relevant one

  • distance_metric_name – name of a distance metric to calculate distance or similarity matrices. Available names could be found in Distance.

  • reduce_func – function to reduce calculated metric. E.g. torch.mean, torch.max and others. functools.partial might be useful if you want to capture some custom arguments.


Assume k is 4. Then only 4 documents being retrieved as a query response. Only 2 of them are relevant and score will be 2/4 = 0.5.


If k is greater than overall amount of relevant documents, then precision@k will always have score < 1.

raw_compute(distance_matrix: Tensor, labels: Tensor)[source]

Compute retrieval precision

  • distance_matrix – matrix with distances between embeddings. Assumed that distance from embedding to itself is meaningless. (e.g. equal to max element of matrix + 1)

  • labels – labels to compute metric. Assumed that label from object to itself has been made meaningless. (E.g. was set to 0)


torch.Tensor - computed metric

retrieval_precision(distance_matrix: Tensor, labels: Tensor, k: int)[source]

Calculates retrieval precision@k given distance matrix, labels and k

  • distance_matrix – distance matrix having max possible distance value on a diagonal

  • labels – labels matrix having False or 0. on a diagonal

  • k – number of documents to retrieve


torch.Tensor – retrieval precision@k for each row in tensor


Learn more about Qdrant vector search project and ecosystem

Discover Qdrant

Similarity Learning

Explore practical problem solving with Similarity Learning

Learn Similarity Learning


Find people dealing with similar problems and get answers to your questions

Join Community