quaterion.eval.evaluator module

class Evaluator(metrics: BaseMetric | Dict[str, BaseMetric], sampler: BaseSampler)[source]

Bases: object

Calculate metrics on the whole datasets

Calculates metric on the whole dataset or on sampled part of it. Evaluation might be time and memory consuming operation.

  • metrics – dictionary of metrics instances for calculation

  • sampler – sampler selects embeddings and labels to perform partial evaluation

evaluate(dataset: Sized | Iterable | Dataset, model: SimilarityModel) Dict[str, Tensor][source]

Compute metrics on a dataset

  • dataset – Sized object, like list, tuple,, etc. to compute metrics

  • model – SimilarityModel instance to perform objects encoding


Dict[str, torch.Tensor] - dict of computed metrics


Learn more about Qdrant vector search project and ecosystem

Discover Qdrant

Similarity Learning

Explore practical problem solving with Similarity Learning

Learn Similarity Learning


Find people dealing with similar problems and get answers to your questions

Join Community