FlagAutoModel#

class FlagEmbedding.inference.FlagAutoModel[source]#

Automatically choose the appropriate class to load the embedding model.

Methods#

classmethod FlagAutoModel.from_finetuned(model_name_or_path: str, model_class: str | EmbedderModelClass | None = None, normalize_embeddings: bool = True, use_fp16: bool = True, query_instruction_for_retrieval: str | None = None, devices: str | List[str] | None = None, pooling_method: str | None = None, trust_remote_code: bool | None = None, query_instruction_format: str | None = None, **kwargs)[source]#

Load a finetuned model according to the provided vars.

Parameters:

  • model_name_or_path (str) – If it’s a path to a local model, it loads the model from the path. Otherwise tries to download and load a model from HuggingFace Hub with the name.

  • model_class (Optional[Union[str, EmbedderModelClass]], optional) – The embedder class to use. Defaults to None.

  • normalize_embeddings (bool, optional) – If True, the output embedding will be a Numpy array. Otherwise, it will be a Torch Tensor. Defaults to True.

  • use_fp16 (bool, optional) – If true, use half-precision floating-point to speed up computation with a slight performance degradation. Defaults to True.

  • query_instruction_for_retrieval (Optional[str], optional) – Query instruction for retrieval tasks, which will be used with query_instruction_format. Defaults to None.

  • devices (Optional[Union[str, List[str]]], optional) – Devices to use for model inference. Defaults to None.

  • pooling_method (Optional[str], optional) – Pooling method to get embedding vector from the last hidden state. Defaults to None.

  • trust_remote_code (Optional[bool], optional) – trust_remote_code for HF datasets or models. Defaults to None.

  • query_instruction_format (Optional[str], optional) – The template for query_instruction_for_retrieval. Defaults to None.

Raises:

ValueError –

Returns:

The model class to load model, which is child class of AbsEmbedder.

Return type:

AbsEmbedder