quaterion.dataset.train_collator module

class TrainCollator(pre_collate_fn: Callable, encoder_collates: Dict[str, CollateFnType], meta_extractors: Dict[str, Callable[[List[Any]], List[dict]]])[source]

Bases: object

Functional object, that aggregates all required information for performing collate on train batches.


Should be serializable for sending among worker processes.

  • pre_collate_fn – function to split origin batch into ids, features and labels. Ids are means to keep track of repeatable usage of the same elements. Features are commonly encoders input. Labels usually allow distinguishing positive and negative samples.

  • encoder_collates – mapping of encoder name to its collate function

pre_encoder_collate(features: List[Any], ids: Optional[List[int]] = None, encoder_name: Optional[str] = None)[source]

Default implementation of per-encoder batch preparation, might be overridden

process_meta(meta: Dict[str, List]) Any[source]


Learn more about Qdrant vector search project and ecosystem

Discover Qdrant

Similarity Learning

Explore practical problem solving with Similarity Learning

Learn Similarity Learning


Find people dealing with similar problems and get answers to your questions

Join Community