quaterion.eval.base_metric module¶
- class BaseMetric(distance_metric_name: Distance = Distance.COSINE)[source]¶
Bases:
object
Base class for evaluation metrics
Provides a default implementation for distance matrix calculation.
- Parameters:
distance_metric_name – name of a distance metric to calculate distance or similarity matrices. Available names could be found in
Distance
.
- compute(*args, **kwargs) Tensor [source]¶
Compute metric value
- Parameters:
args –
metric. (kwargs - contain embeddings and targets required to compute) –
- Returns:
torch.Tensor - computed metric
- precompute(embeddings: Tensor, **targets) Tuple[Tensor, Tensor] [source]¶
Prepares data for computation
Compute distance matrix and final labels based on groups.
- Parameters:
embeddings – embeddings to compute metric value
targets – objects to compute final labels
- Returns:
torch.Tensor, torch.Tensor - labels and distance matrix
- static prepare_labels(**targets) Tensor [source]¶
Compute metric labels
- Parameters:
**targets – objects to compute final labels. **targets in PairMetric consists of labels, pairs and subgroups, in GroupMetric - of groups.
- Returns:
targets – torch.Tensor - labels to be used during metric computation
- raw_compute(distance_matrix: Tensor, labels: Tensor) Tensor [source]¶
Perform metric computation on ready distance_matrix and labels
This method does not make any data and labels preparation. It is assumed that distance_matrix has already been calculated, required changes such masking distance from an element to itself have already been applied and corresponding labels have been prepared.
- Parameters:
distance_matrix – distance matrix ready to metric computation
labels – labels ready to metric computation with the same shape as distance_matrix. For PairMetric values are taken from SimilarityPairSample.score, for GroupMetric the possible values are in {0, 1}.
- Returns:
torch.Tensor - calculated metric value