Shortcuts

quaterion.eval.evaluator module

class Evaluator(metrics: BaseMetric | Dict[str, BaseMetric], sampler: BaseSampler)[source]

Bases: object

Calculate metrics on the whole datasets

Calculates metric on the whole dataset or on sampled part of it. Evaluation might be time and memory consuming operation.

Parameters:
  • metrics – dictionary of metrics instances for calculation

  • sampler – sampler selects embeddings and labels to perform partial evaluation

evaluate(dataset: Sized | Iterable | Dataset, model: SimilarityModel) Dict[str, Tensor][source]

Compute metrics on a dataset

Parameters:
  • dataset – Sized object, like list, tuple, torch.utils.data.Dataset, etc. to compute metrics

  • model – SimilarityModel instance to perform objects encoding

Returns:

Dict[str, torch.Tensor] - dict of computed metrics

Qdrant

Learn more about Qdrant vector search project and ecosystem

Discover Qdrant

Similarity Learning

Explore practical problem solving with Similarity Learning

Learn Similarity Learning

Community

Find people dealing with similar problems and get answers to your questions

Join Community