quaterion.eval.pair.pair_metric module¶

class PairMetric(distance_metric_name: Distance = Distance.COSINE)[source]¶

Bases: BaseMetric

Base class for metrics computation for pair based data

Parameters:

distance_metric_name – name of a distance metric to calculate distance or similarity matrices. Available names could be found in Distance.

Provides default implementations for distance and interaction matrices calculation. Accumulates embeddings and labels in an accumulator.

compute(embeddings: Tensor, labels: Tensor, pairs: LongTensor, subgroups: Tensor)[source]¶

Compute metric value

Parameters:

• embeddings – embeddings to calculate metrics on

• labels – labels to distinguish similar and dissimilar objects.

• pairs – indices to determine objects of one pair

• subgroups – subgroups numbers to determine which samples can be considered negative

Returns:

torch.Tensor - computed metric

evaluate() → Tensor[source]¶

Perform metric computation with accumulated state

static prepare_labels(labels: Tensor, pairs: LongTensor, subgroups: Tensor) → Tensor[source]¶

Compute metric labels based on samples labels and pairs

Parameters:

• labels – labels to distinguish similar and dissimilar objects

• pairs – indices to determine objects belong to the same pair

• subgroups – indices to determine negative examples. Currently, they are not used for labels computation.

Returns:

target – torch.Tensor - labels to be used during metric computation

raw_compute(distance_matrix: Tensor, labels: Tensor) → Tensor[source]¶

Perform metric computation on ready distance_matrix and labels

This method does not make any data and labels preparation. It is assumed that distance_matrix has already been calculated, required changes such masking distance from an element to itself have already been applied and corresponding labels have been prepared.

Parameters:

• distance_matrix – distance matrix ready to metric computation

• labels – labels ready to metric computation with the same shape as distance_matrix. Values are taken from SimilarityPairSample.score.

Returns:

torch.Tensor - calculated metric value

reset()[source]¶

Reset accumulated state

Reset embeddings, labels, pairs, subgroups, etc.

update(embeddings: Tensor, labels: Tensor, pairs: LongTensor, subgroups: Tensor, device=None)[source]¶

Process and accumulate batch

Parameters:

• embeddings – embeddings to accumulate

• labels – labels to distinguish similar and dissimilar objects.

• pairs – indices to determine objects of one pair

• subgroups – subgroups numbers to determine which samples can be considered negative

• device – device to store calculated embeddings and targets on.