Sentence Transformer models consist of a sequence of `Modules <../package_reference/sentence_transformer/modules.html>`_ or `Custom Modules <usage/custom_models.html#advanced-custom-modules>`_, allowing for a lot of flexibility. If you want to further finetune a SentenceTransformer model (e.g. it has a `modules.json file <https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2/blob/main/modules.json>`_), then you don't have to worry about which modules are used::

    from sentence_transformers import SentenceTransformer

    model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")

But if instead you want to train from another checkpoint, or from scratch, then these are the most common architectures you can use:

.. tab:: Transformers

    Most Sentence Transformer models use the :class:`~sentence_transformers.base.modules.Transformer` and :class:`~sentence_transformers.sentence_transformer.modules.Pooling` modules. The former loads a pretrained transformer model (e.g. `BERT <https://huggingface.co/google-bert/bert-base-uncased>`_, `RoBERTa <https://huggingface.co/FacebookAI/roberta-base>`_, `DistilBERT <https://huggingface.co/distilbert/distilbert-base-uncased>`_, `ModernBERT <https://huggingface.co/answerdotai/ModernBERT-base>`_, etc.) and the latter pools the output of the transformer to produce a single vector representation for each input sentence.

    .. raw:: html

        <div class="sidebar">
            <p class="sidebar-title">Documentation</p>
            <ul class="simple">
                <li><a class="reference internal" href="../package_reference/base/modules.html#sentence_transformers.base.modules.transformer.Transformer"><code class="xref py py-class docutils literal notranslate"><span class="pre">sentence_transformers.base.modules.Transformer</span></code></a></li>
                <li><a class="reference internal" href="../package_reference/sentence_transformer/modules.html#sentence_transformers.sentence_transformer.modules.pooling.Pooling"><code class="xref py py-class docutils literal notranslate"><span class="pre">sentence_transformers.sentence_transformer.modules.Pooling</span></code></a></li>
            </ul>
        </div>

    ::

        from sentence_transformers import SentenceTransformer
        from sentence_transformers.sentence_transformer.modules import Transformer, Pooling

        transformer = Transformer("google-bert/bert-base-uncased")
        pooling = Pooling(transformer.get_embedding_dimension(), pooling_mode="mean")

        model = SentenceTransformer(modules=[transformer, pooling])
    
    This is the default option in Sentence Transformers, so it's easier to use the shortcut:

    ::

        from sentence_transformers import SentenceTransformer

        model = SentenceTransformer("google-bert/bert-base-uncased")

    .. tip::

        The strongest base models are often "encoder models", i.e. models that are trained to produce a meaningful token embedding for inputs. You can find strong candidates here:

        - `fill-mask models <https://huggingface.co/models?pipeline_tag=fill-mask>`_ - trained for token embeddings
        - `sentence similarity models <https://huggingface.co/models?pipeline_tag=sentence-similarity>`_ - trained for text embeddings
        - `feature-extraction models <https://huggingface.co/models?pipeline_tag=feature-extraction>`_ - trained for text embeddings

        Consider looking for base models that are designed on your language and/or domain of interest. For example, `FacebookAI/xlm-roberta-base <https://huggingface.co/FacebookAI/xlm-roberta-base>`_ will work better than `google-bert/bert-base-uncased <https://huggingface.co/google-bert/bert-base-uncased>`_ for Turkish.

.. tab:: Static

    Static Embedding models (`blogpost <https://huggingface.co/blog/static-embeddings>`_) use the :class:`~sentence_transformers.sentence_transformer.modules.StaticEmbedding` module, and are encoder models that don't use slow transformers or attention mechanisms. For these models, computing embeddings is simply: given the input token, return the pre-computed token embedding. These models are orders of magnitude faster, but cannot capture complex semantics as token embeddings are computed separate from the context.

    .. raw:: html

        <div class="sidebar">
            <p class="sidebar-title">Documentation</p>
            <ul class="simple">
                <li><a class="reference external" href="https://huggingface.co/blog/static-embeddings">Static Embedding Models</a></li>
                <li><a class="reference internal" href="../package_reference/sentence_transformer/modules.html#sentence_transformers.sentence_transformer.modules.static_embedding.StaticEmbedding"><code class="xref py py-class docutils literal notranslate"><span class="pre">sentence_transformers.sentence_transformer.modules.StaticEmbedding</span></code></a></li>
                <li><a class="reference internal" href="../package_reference/sentence_transformer/modules.html#sentence_transformers.sentence_transformer.modules.static_embedding.StaticEmbedding.from_model2vec"><code class="xref py py-meth docutils literal notranslate"><span class="pre">sentence_transformers.sentence_transformer.modules.StaticEmbedding.from_model2vec</span></code></a></li>
                <li><a class="reference internal" href="../package_reference/sentence_transformer/modules.html#sentence_transformers.sentence_transformer.modules.static_embedding.StaticEmbedding.from_distillation"><code class="xref py py-meth docutils literal notranslate"><span class="pre">sentence_transformers.sentence_transformer.modules.StaticEmbedding.from_distillation</span></code></a></li>
            </ul>
        </div>

    ::

        from sentence_transformers import SentenceTransformer
        from sentence_transformers.sentence_transformer.modules import StaticEmbedding
        from tokenizers import Tokenizer

        # Load any Tokenizer from Hugging Face
        tokenizer = Tokenizer.from_pretrained("google-bert/bert-base-uncased")
        # The `embedding_dim` is the dimensionality (size) of the token embeddings
        static_embedding = StaticEmbedding(tokenizer, embedding_dim=512)

        model = SentenceTransformer(modules=[static_embedding])

.. tab:: Multimodal / Vision-Language

    .. tip::

        Multimodal models require additional dependencies. Install them with e.g. ``pip install -U "sentence-transformers[image]"`` for image support. See `Installation <../installation.html>`_ for all options.

    You can also train multimodal Sentence Transformer models from Vision-Language Model (VLM) checkpoints. The :class:`~sentence_transformers.base.modules.Transformer` module automatically detects supported modalities (text, image, audio, video) from the model's processor. You can load an already-trained multimodal embedding model:

    .. raw:: html

        <div class="sidebar">
            <p class="sidebar-title">Documentation</p>
            <ul class="simple">
                <li><a class="reference internal" href="../package_reference/base/modules.html#sentence_transformers.base.modules.transformer.Transformer"><code class="xref py py-class docutils literal notranslate"><span class="pre">sentence_transformers.base.modules.Transformer</span></code></a></li>
                <li><a class="reference internal" href="../package_reference/base/modules.html#sentence_transformers.base.modules.transformer.Transformer.modalities"><code class="xref py py-attr docutils literal notranslate"><span class="pre">Transformer.modalities</span></code></a></li>
                <li><a class="reference internal" href="../package_reference/sentence_transformer/model.html#sentence_transformers.sentence_transformer.model.SentenceTransformer.supports"><code class="xref py py-meth docutils literal notranslate"><span class="pre">SentenceTransformer.supports()</span></code></a></li>
                <li><a class="reference external" href="training/examples.html#multimodal">Multimodal Training Examples</a></li>
            </ul>
        </div>

    ::

        from sentence_transformers import SentenceTransformer

        model = SentenceTransformer(
            "Qwen/Qwen3-VL-Embedding-2B",
            model_kwargs={"attn_implementation": "flash_attention_2", "torch_dtype": "bfloat16"},
            processor_kwargs={"min_pixels": 28 * 28, "max_pixels": 600 * 600},
            revision="refs/pr/23",
        )

    Or initialize from a fresh VLM checkpoint that hasn't been trained for embeddings yet::

        from sentence_transformers import SentenceTransformer

        model = SentenceTransformer("Qwen/Qwen3-VL-2B")

    In both cases, the :class:`~sentence_transformers.base.modules.Transformer` module inspects the processor to determine which modalities are available, and :class:`~sentence_transformers.sentence_transformer.modules.Pooling` is added automatically if needed. You can verify the supported modalities::

        print(model.modalities)
        # ['text', 'image', 'video', 'message']

        print(model.supports("image"))
        # True

    The ``"message"`` modality means the model can handle mixed-modality inputs (e.g. an image and text together) via the chat template format. Multimodal training data can include text strings, PIL images, image file paths or URLs, audio arrays, or multimodal dicts like ``{"image": <PIL.Image>, "text": "describe this"}``. See the `multimodal training examples <training/examples.html#multimodal>`_ for full training scripts.

.. tab:: Multimodal via Router

    You can also build multimodal models by composing separate encoders for different modalities using the :class:`~sentence_transformers.base.modules.Router` module. Unlike the VLM approach above (which uses a single backbone that natively handles multiple modalities), the Router approach lets you combine *any* existing encoders. For example, a text encoder and an image encoder, and route inputs to the appropriate encoder based on the detected modality.

    .. raw:: html

        <div class="sidebar">
            <p class="sidebar-title">Documentation</p>
            <ul class="simple">
                <li><a class="reference internal" href="../package_reference/base/modules.html#sentence_transformers.base.modules.Router"><code class="xref py py-class docutils literal notranslate"><span class="pre">sentence_transformers.base.modules.Router</span></code></a></li>
                <li><a class="reference internal" href="../package_reference/base/modules.html#sentence_transformers.base.modules.Transformer"><code class="xref py py-class docutils literal notranslate"><span class="pre">sentence_transformers.base.modules.Transformer</span></code></a></li>
                <li><a class="reference internal" href="../package_reference/sentence_transformer/modules.html#sentence_transformers.sentence_transformer.modules.Pooling"><code class="xref py py-class docutils literal notranslate"><span class="pre">sentence_transformers.sentence_transformer.modules.Pooling</span></code></a></li>
                <li><a class="reference internal" href="../package_reference/base/modules.html#sentence_transformers.base.modules.Dense"><code class="xref py py-class docutils literal notranslate"><span class="pre">sentence_transformers.base.modules.Dense</span></code></a></li>
            </ul>
        </div>

    ::

        from sentence_transformers import SentenceTransformer
        from sentence_transformers.sentence_transformer.modules import Dense, Pooling, Router, Transformer

        # Create separate encoders for different modalities
        text_encoder = Transformer("sentence-transformers/all-MiniLM-L6-v2")
        text_pooling = Pooling(text_encoder.get_embedding_dimension(), pooling_mode="mean")
        # Project text embeddings to match image encoder dimension
        text_projection = Dense(text_encoder.get_embedding_dimension(), 768)

        # SigLIP outputs pooled embeddings directly, so no separate Pooling module is needed
        image_encoder = Transformer("google/siglip2-base-patch16-224")

        # Route inputs based on modality
        router = Router(
            sub_modules={
                "text": [text_encoder, text_pooling, text_projection],
                "image": [image_encoder],
            },
        )

        model = SentenceTransformer(modules=[router])

    The Router automatically infers the modality from the input data. Text strings are routed to the text encoder, and image URLs (or PIL images) are routed to the image encoder::

        text_embeddings = model.encode(["A photo of a cat", "A pollinator on a flower"])
        image_embeddings = model.encode([
            "https://huggingface.co/datasets/huggingface/cats-image/resolve/main/cats_image.jpeg",
            "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/bee.jpg",
        ])
        print(text_embeddings.shape, image_embeddings.shape)
        # (2, 768) (2, 768)

        similarity = model.similarity(text_embeddings, image_embeddings)
        print(similarity)
        # tensor([[ 0.0028, -0.0144],
        #         [-0.0233, -0.0355]])

    You can also combine Router-based multimodality with task-based routing (e.g. different encoders for queries vs. documents) using ``route_mappings``. See the :class:`~sentence_transformers.base.modules.Router` documentation for advanced routing scenarios.

    .. warning::

        Since Router-based multimodal models use separate encoders per modality, their embedding spaces are initially unaligned. Training is required to align the spaces for meaningful cross-modal similarity. Consider using a :class:`~sentence_transformers.base.modules.Dense` projection layer to map embeddings from different encoders into a shared space, as shown above.

The :class:`SentenceTransformerTrainer` trains and evaluates using :class:`datasets.Dataset` (one dataset) or :class:`datasets.DatasetDict` instances (multiple datasets, see also `Multi-dataset training <#multi-dataset-training>`_).

.. tab:: Data on 🤗 Hugging Face Hub

    If you want to load data from the `Hugging Face Datasets <https://huggingface.co/datasets>`_, then you should use :func:`datasets.load_dataset`:

    .. raw:: html

        <div class="sidebar">
            <p class="sidebar-title">Documentation</p>
            <ul class="simple">
                <li><a class="reference external" href="https://huggingface.co/docs/datasets/main/en/loading#hugging-face-hub">Datasets, Loading from the Hugging Face Hub</a></li>
                <li><a class="reference external" href="https://huggingface.co/docs/datasets/main/en/package_reference/loading_methods#datasets.load_dataset"><code class="xref py py-func docutils literal notranslate"><span class="pre">datasets.load_dataset()</span></code></a></li>
                <li><a class="reference external" href="https://huggingface.co/datasets/sentence-transformers/all-nli">sentence-transformers/all-nli</a></li>
            </ul>
        </div>

    ::

        from datasets import load_dataset

        train_dataset = load_dataset("sentence-transformers/all-nli", "pair-class", split="train")
        eval_dataset = load_dataset("sentence-transformers/all-nli", "pair-class", split="dev")

        print(train_dataset)
        """
        Dataset({
            features: ['premise', 'hypothesis', 'label'],
            num_rows: 942069
        })
        """

    Some datasets (including `sentence-transformers/all-nli <https://huggingface.co/datasets/sentence-transformers/all-nli>`_) require you to provide a "subset" alongside the dataset name. ``sentence-transformers/all-nli`` has 4 subsets, each with different data formats: `pair <https://huggingface.co/datasets/sentence-transformers/all-nli/viewer/pair>`_, `pair-class <https://huggingface.co/datasets/sentence-transformers/all-nli/viewer/pair-class>`_, `pair-score <https://huggingface.co/datasets/sentence-transformers/all-nli/viewer/pair-score>`_, `triplet <https://huggingface.co/datasets/sentence-transformers/all-nli/viewer/triplet>`_.

    .. note::

        Many Hugging Face datasets that work out of the box with Sentence Transformers have been tagged with ``sentence-transformers``, allowing you to easily find them by browsing to `https://huggingface.co/datasets?other=sentence-transformers <https://huggingface.co/datasets?other=sentence-transformers>`_. We strongly recommend that you browse these datasets to find training datasets that might be useful for your tasks.

.. tab:: Local Data (CSV, JSON, Parquet, Arrow, SQL)

    If you have local data in common file-formats, then you can load this data easily using :func:`datasets.load_dataset`:

    .. raw:: html

        <div class="sidebar">
            <p class="sidebar-title">Documentation</p>
            <ul class="simple">
                <li><a class="reference external" href="https://huggingface.co/docs/datasets/main/en/loading#local-and-remote-files">Datasets, Loading local files</a></li>
                <li><a class="reference external" href="https://huggingface.co/docs/datasets/main/en/package_reference/loading_methods#datasets.load_dataset" title="(in datasets vmain)"><code class="xref py py-func docutils literal notranslate"><span class="pre">datasets.load_dataset()</span></code></a></li>
            </ul>
        </div>

    ::

        from datasets import load_dataset

        dataset = load_dataset("csv", data_files="my_file.csv")
    
    or::

        from datasets import load_dataset

        dataset = load_dataset("json", data_files="my_file.json")

.. tab:: Local Data that requires pre-processing

    If you have local data that requires some extra pre-processing, my recommendation is to initialize your dataset using :meth:`datasets.Dataset.from_dict` and a dictionary of lists, like so:

    .. raw:: html

        <div class="sidebar">
            <p class="sidebar-title">Documentation</p>
            <ul class="simple">
                <li><a class="reference external" href="https://huggingface.co/docs/datasets/main/en/package_reference/main_classes#datasets.Dataset.from_dict" title="(in datasets vmain)"><code class="xref py py-meth docutils literal notranslate"><span class="pre">datasets.Dataset.from_dict()</span></code></a></li>
            </ul>
        </div>

    ::

        from datasets import Dataset

        anchors = []
        positives = []
        # Open a file, do preprocessing, filtering, cleaning, etc.
        # and append to the lists

        dataset = Dataset.from_dict({
            "anchor": anchors,
            "positive": positives,
        })

    Each key from the dictionary will become a column in the resulting dataset.

It is important that your dataset format matches your loss function (or that you choose a loss function that matches your dataset format). Verifying whether a dataset format works with a loss function involves two steps:

1. If your loss function requires a *Label* according to the `Loss Overview <loss_overview.html>`_ table, then your dataset must have a **column named "label", "labels", "score" or "scores"**. This column is automatically taken as the label.
2. All columns not named "label", "labels", "score" or "scores" are considered *Inputs* according to the `Loss Overview <loss_overview.html>`_ table. The number of remaining columns must match the number of valid inputs for your chosen loss. The names of these columns are **irrelevant**, only the **order matters**. 

For example, given a dataset with columns ``["text1", "text2", "label"]`` where the "label" column has float similarity score between 0 and 1, we can use it with :class:`~sentence_transformers.sentence_transformer.losses.CoSENTLoss`, :class:`~sentence_transformers.sentence_transformer.losses.AnglELoss`, and :class:`~sentence_transformers.sentence_transformer.losses.CosineSimilarityLoss` because it:

1. has a "label" column as is required for these loss functions.
2. has 2 non-label columns, exactly the amount required by these loss functions.

Be sure to re-order your dataset columns with :meth:`Dataset.select_columns <datasets.Dataset.select_columns>` if your columns are not ordered correctly. For example, if your dataset has ``["good_answer", "bad_answer", "question"]`` as columns, then this dataset can technically be used with a loss that requires (anchor, positive, negative) triplets, but the ``good_answer`` column will be taken as the anchor, ``bad_answer`` as the positive, and ``question`` as the negative.

Additionally, if your dataset has extraneous columns (e.g. sample_id, metadata, source, type), you should remove these with :meth:`Dataset.remove_columns <datasets.Dataset.remove_columns>` as they will be used as inputs otherwise. You can also use :meth:`Dataset.select_columns <datasets.Dataset.select_columns>` to keep only the desired columns.

.. tip::

   Multimodal models require additional dependencies. Install them with e.g. ``pip install -U "sentence-transformers[image]"`` for image support. See `Installation <../installation.html>`_ for all options.

SentenceTransformer datasets are not limited to text columns. When using a multimodal model (e.g. a vision-language model), input columns can also contain images (as PIL images, file paths, or URLs), audio, video, or multimodal dictionaries combining multiple modalities. The dataset format rules from `Dataset Format <#dataset-format>`_ still apply: non-label columns are treated as inputs, and a ``"label"`` column provides the target score or class.

For example, a dataset for document screenshot retrieval might have a text ``"query"`` column and an ``"image"`` column containing PIL images::

    from datasets import load_dataset

    dataset = load_dataset("tomaarsen/llamaindex-vdr-en-train-preprocessed", "train", split="train")
    """
    Dataset({
        features: ['query', 'image', 'negative_0', 'negative_1', 'negative_2', 'negative_3'],
        num_rows: 10000
    })
    """

The data collator automatically handles multimodal preprocessing via the model's ``preprocess`` method, so no manual tokenization or image processing is needed. See `Training Examples > Multimodal <../../examples/sentence_transformer/training/multimodal/README.html>`_ for complete training scripts.

Most loss functions can be initialized with just the :class:`~sentence_transformers.sentence_transformer.model.SentenceTransformer` that you're training, alongside some optional parameters, e.g.:

.. sidebar:: Documentation

    - :class:`sentence_transformers.sentence_transformer.losses.CoSENTLoss`
    - `Losses API Reference <../package_reference/sentence_transformer/losses.html>`_
    - `Loss Overview <loss_overview.html>`_

::

    from datasets import load_dataset
    from sentence_transformers import SentenceTransformer
    from sentence_transformers.sentence_transformer.losses import CoSENTLoss

    # Load a model to train/finetune
    model = SentenceTransformer("FacebookAI/xlm-roberta-base")

    # Initialize the CoSENTLoss
    # This loss requires pairs of text and a float similarity score as a label
    loss = CoSENTLoss(model)

    # Load an example training dataset that works with our loss function:
    train_dataset = load_dataset("sentence-transformers/all-nli", "pair-score", split="train")
    """
    Dataset({
        features: ['sentence1', 'sentence2', 'score'],
        num_rows: 942069
    })
    """

The :class:`~sentence_transformers.sentence_transformer.training_args.SentenceTransformerTrainingArguments` class can be used to specify parameters for influencing training performance as well as defining the tracking/debugging parameters. Although it is optional, it is heavily recommended to experiment with the various useful arguments.

Here is an example of how :class:`~sentence_transformers.sentence_transformer.training_args.SentenceTransformerTrainingArguments` can be initialized:

args = SentenceTransformerTrainingArguments(
    # Required parameter:
    output_dir="models/mpnet-base-all-nli-triplet",
    # Optional training parameters:
    num_train_epochs=1,
    per_device_train_batch_size=16,
    per_device_eval_batch_size=16,
    learning_rate=2e-5,
    warmup_steps=0.1,
    fp16=True,  # Set to False if you get an error that your GPU can't run on FP16
    bf16=False,  # Set to True if you have a GPU that supports BF16
    batch_sampler=BatchSamplers.NO_DUPLICATES,  # losses that use "in-batch negatives" benefit from no duplicates
    # Optional tracking/debugging parameters:
    eval_strategy="steps",
    eval_steps=100,
    save_strategy="steps",
    save_steps=100,
    save_total_limit=2,
    logging_steps=100,
    run_name="mpnet-base-all-nli-triplet",  # Will be used in W&B if `wandb` is installed
)

You can provide the :class:`~sentence_transformers.sentence_transformer.trainer.SentenceTransformerTrainer` with an ``eval_dataset`` to get the evaluation loss during training, but it may be useful to get more concrete metrics during training, too. For this, you can use evaluators to assess the model's performance with useful metrics before, during, or after training. You can use both an ``eval_dataset`` and an evaluator, one or the other, or neither. They evaluate based on the ``eval_strategy`` and ``eval_steps`` `Training Arguments <#training-arguments>`_.

Here are the implemented Evaluators that come with Sentence Transformers:

=============================================================================================  ===========================================================================================================================
Evaluator                                                                                      Required Data
=============================================================================================  ===========================================================================================================================
:class:`~sentence_transformers.sentence_transformer.evaluation.BinaryClassificationEvaluator`  Pairs with class labels.
:class:`~sentence_transformers.sentence_transformer.evaluation.EmbeddingSimilarityEvaluator`   Pairs with similarity scores.
:class:`~sentence_transformers.sentence_transformer.evaluation.InformationRetrievalEvaluator`  Queries (qid => question), Corpus (cid => document), and relevant documents (qid => set[cid]).
:class:`~sentence_transformers.sentence_transformer.evaluation.NanoBEIREvaluator`              No data required.
:class:`~sentence_transformers.sentence_transformer.evaluation.MSEEvaluator`                   Source sentences to embed with a teacher model and target sentences to embed with the student model. Can be the same texts.
:class:`~sentence_transformers.sentence_transformer.evaluation.ParaphraseMiningEvaluator`      Mapping of IDs to sentences & pairs with IDs of duplicate sentences.
:class:`~sentence_transformers.sentence_transformer.evaluation.RerankingEvaluator`             List of ``{'query': '...', 'positive': [...], 'negative': [...]}`` dictionaries.
:class:`~sentence_transformers.sentence_transformer.evaluation.TranslationEvaluator`           Pairs of sentences in two separate languages.
:class:`~sentence_transformers.sentence_transformer.evaluation.TripletEvaluator`               (anchor, positive, negative) pairs.
=============================================================================================  ===========================================================================================================================

Additionally, :class:`~sentence_transformers.sentence_transformer.evaluation.SequentialEvaluator` should be used to combine multiple evaluators into one Evaluator that can be passed to the :class:`~sentence_transformers.sentence_transformer.trainer.SentenceTransformerTrainer`.

Sometimes you don't have the required evaluation data to prepare one of these evaluators on your own, but you still want to track how well the model performs on some common benchmarks. In that case, you can use these evaluators with data from Hugging Face.

.. tab:: EmbeddingSimilarityEvaluator with STSb

    .. raw:: html

        <div class="sidebar">
            <p class="sidebar-title">Documentation</p>
            <ul class="simple">
                <li><a class="reference external" href="https://huggingface.co/datasets/sentence-transformers/stsb">sentence-transformers/stsb</a></li>
                <li><a class="reference internal" href="../package_reference/sentence_transformer/evaluation.html#sentence_transformers.sentence_transformer.evaluation.embedding_similarity.EmbeddingSimilarityEvaluator" title="sentence_transformers.sentence_transformer.evaluation.embedding_similarity.EmbeddingSimilarityEvaluator"><code class="xref py py-class docutils literal notranslate"><span class="pre">sentence_transformers.sentence_transformer.evaluation.EmbeddingSimilarityEvaluator</span></code></a></li>
                <li><a class="reference internal" href="../package_reference/util/similarity.html#sentence_transformers.util.similarity.SimilarityFunction" title="sentence_transformers.util.similarity.SimilarityFunction"><code class="xref py py-class docutils literal notranslate"><span class="pre">sentence_transformers.util.similarity.SimilarityFunction</span></code></a></li>
            </ul>
        </div>

    ::

        from datasets import load_dataset
        from sentence_transformers.sentence_transformer.evaluation import EmbeddingSimilarityEvaluator, SimilarityFunction

        # Load the STSB dataset (https://huggingface.co/datasets/sentence-transformers/stsb)
        eval_dataset = load_dataset("sentence-transformers/stsb", split="validation")

        # Initialize the evaluator
        dev_evaluator = EmbeddingSimilarityEvaluator(
            sentences1=eval_dataset["sentence1"],
            sentences2=eval_dataset["sentence2"],
            scores=eval_dataset["score"],
            main_similarity=SimilarityFunction.COSINE,
            name="sts-dev",
        )
        # You can run evaluation like so:
        # results = dev_evaluator(model)

.. tab:: TripletEvaluator with AllNLI

    .. raw:: html

        <div class="sidebar">
            <p class="sidebar-title">Documentation</p>
            <ul class="simple">
                <li><a class="reference external" href="https://huggingface.co/datasets/sentence-transformers/all-nli">sentence-transformers/all-nli</a></li>
                <li><a class="reference internal" href="../package_reference/sentence_transformer/evaluation.html#sentence_transformers.sentence_transformer.evaluation.triplet.TripletEvaluator" title="sentence_transformers.sentence_transformer.evaluation.triplet.TripletEvaluator"><code class="xref py py-class docutils literal notranslate"><span class="pre">sentence_transformers.sentence_transformer.evaluation.TripletEvaluator</span></code></a></li>
                <li><a class="reference internal" href="../package_reference/util/similarity.html#sentence_transformers.util.similarity.SimilarityFunction" title="sentence_transformers.util.similarity.SimilarityFunction"><code class="xref py py-class docutils literal notranslate"><span class="pre">sentence_transformers.util.similarity.SimilarityFunction</span></code></a></li>
            </ul>
        </div>

    ::

        from datasets import load_dataset
        from sentence_transformers.sentence_transformer.evaluation import TripletEvaluator, SimilarityFunction

        # Load triplets from the AllNLI dataset (https://huggingface.co/datasets/sentence-transformers/all-nli)
        max_samples = 1000
        eval_dataset = load_dataset("sentence-transformers/all-nli", "triplet", split=f"dev[:{max_samples}]")

        # Initialize the evaluator
        dev_evaluator = TripletEvaluator(
            anchors=eval_dataset["anchor"],
            positives=eval_dataset["positive"],
            negatives=eval_dataset["negative"],
            main_similarity_function=SimilarityFunction.COSINE,
            name="all-nli-dev",
        )
        # You can run evaluation like so:
        # results = dev_evaluator(model)

.. tab:: NanoBEIREvaluator

    .. raw:: html

        <div class="sidebar">
            <p class="sidebar-title">Documentation</p>
            <ul class="simple">
                <li><a class="reference internal" href="../package_reference/sentence_transformer/evaluation.html#sentence_transformers.sentence_transformer.evaluation.nano_beir.NanoBEIREvaluator" title="sentence_transformers.sentence_transformer.evaluation.nano_beir.NanoBEIREvaluator"><code class="xref py py-class docutils literal notranslate"><span class="pre">sentence_transformers.sentence_transformer.evaluation.NanoBEIREvaluator</span></code></a></li>
            </ul>
        </div>

    ::

        from sentence_transformers.sentence_transformer.evaluation import NanoBEIREvaluator

        # Initialize the evaluator. Unlike most other evaluators, this one loads the relevant datasets
        # directly from Hugging Face, so there's no mandatory arguments
        dev_evaluator = NanoBEIREvaluator()
        # You can run evaluation like so:
        # results = dev_evaluator(model)

.. tip::

    When evaluating frequently during training with a small ``eval_steps``, consider using a tiny ``eval_dataset`` to minimize evaluation overhead. If you're concerned about the evaluation set size, a 90-1-9 train-eval-test split can provide a balance, reserving a reasonably sized test set for final evaluations. After training, you can assess your model's performance using ``trainer.evaluate(test_dataset)`` for test loss or initialize a testing evaluator with ``test_evaluator(model)`` for detailed test metrics.

    If you evaluate after training, but before saving the model, your automatically generated model card will still include the test results.

.. warning::

    When using `Distributed Training <training/distributed.html>`_, the evaluator only runs on the first device, unlike the training and evaluation datasets, which are shared across all devices. 

The :class:`~sentence_transformers.sentence_transformer.trainer.SentenceTransformerTrainer` is where all previous components come together. We only have to specify the trainer with the model, training arguments (optional), training dataset, evaluation dataset (optional), loss function, evaluator (optional) and we can start training. Let's have a look at a script where all of these components come together:

.. sidebar:: Documentation

    #. :class:`~sentence_transformers.sentence_transformer.model.SentenceTransformer`
    #. :class:`~sentence_transformers.sentence_transformer.model_card.SentenceTransformerModelCardData`
    #. :func:`~datasets.load_dataset`
    #. :class:`~sentence_transformers.sentence_transformer.losses.MultipleNegativesRankingLoss`
    #. :class:`~sentence_transformers.sentence_transformer.training_args.SentenceTransformerTrainingArguments`
    #. :class:`~sentence_transformers.sentence_transformer.evaluation.TripletEvaluator`
    #. :class:`~sentence_transformers.sentence_transformer.trainer.SentenceTransformerTrainer`
    #. :class:`SentenceTransformer.save_pretrained <sentence_transformers.sentence_transformer.model.SentenceTransformer.save_pretrained>`
    #. :class:`SentenceTransformer.push_to_hub <sentence_transformers.sentence_transformer.model.SentenceTransformer.push_to_hub>`

    - `Training Examples <training/examples.html>`_

::

    from datasets import load_dataset
    from sentence_transformers import (
        SentenceTransformer,
        SentenceTransformerTrainer,
        SentenceTransformerTrainingArguments,
        SentenceTransformerModelCardData,
    )
    from sentence_transformers.sentence_transformer.losses import MultipleNegativesRankingLoss
    from sentence_transformers.sentence_transformer.training_args import BatchSamplers
    from sentence_transformers.sentence_transformer.evaluation import TripletEvaluator

    # 1. Load a model to finetune with 2. (Optional) model card data
    model = SentenceTransformer(
        "microsoft/mpnet-base",
        model_card_data=SentenceTransformerModelCardData(
            language="en",
            license="apache-2.0",
            model_name="MPNet base trained on AllNLI triplets",
        )
    )

    # 3. Load a dataset to finetune on
    dataset = load_dataset("sentence-transformers/all-nli", "triplet")
    train_dataset = dataset["train"].select(range(100_000))
    eval_dataset = dataset["dev"]
    test_dataset = dataset["test"]

    # 4. Define a loss function
    loss = MultipleNegativesRankingLoss(model)

    # 5. (Optional) Specify training arguments
    args = SentenceTransformerTrainingArguments(
        # Required parameter:
        output_dir="models/mpnet-base-all-nli-triplet",
        # Optional training parameters:
        num_train_epochs=1,
        per_device_train_batch_size=16,
        per_device_eval_batch_size=16,
        learning_rate=2e-5,
        warmup_steps=0.1,
        fp16=True,  # Set to False if you get an error that your GPU can't run on FP16
        bf16=False,  # Set to True if you have a GPU that supports BF16
        batch_sampler=BatchSamplers.NO_DUPLICATES,  # MultipleNegativesRankingLoss benefits from no duplicate samples in a batch
        # Optional tracking/debugging parameters:
        eval_strategy="steps",
        eval_steps=100,
        save_strategy="steps",
        save_steps=100,
        save_total_limit=2,
        logging_steps=100,
        run_name="mpnet-base-all-nli-triplet",  # Will be used in W&B if `wandb` is installed
    )

    # 6. (Optional) Create an evaluator & evaluate the base model
    dev_evaluator = TripletEvaluator(
        anchors=eval_dataset["anchor"],
        positives=eval_dataset["positive"],
        negatives=eval_dataset["negative"],
        name="all-nli-dev",
    )
    dev_evaluator(model)

    # 7. Create a trainer & train
    trainer = SentenceTransformerTrainer(
        model=model,
        args=args,
        train_dataset=train_dataset,
        eval_dataset=eval_dataset,
        loss=loss,
        evaluator=dev_evaluator,
    )
    trainer.train()

    # (Optional) Evaluate the trained model on the test set
    test_evaluator = TripletEvaluator(
        anchors=test_dataset["anchor"],
        positives=test_dataset["positive"],
        negatives=test_dataset["negative"],
        name="all-nli-test",
    )
    test_evaluator(model)

    # 8. Save the trained model
    model.save_pretrained("models/mpnet-base-all-nli-triplet/final")
    
    # 9. (Optional) Push it to the Hugging Face Hub
    model.push_to_hub("mpnet-base-all-nli-triplet")

This Sentence Transformers trainer integrates support for various :class:`transformers.TrainerCallback` subclasses, such as:

- :class:`~transformers.integrations.WandbCallback` to automatically log training metrics to W&B if ``wandb`` is installed
- :class:`~transformers.integrations.TensorBoardCallback` to log training metrics to TensorBoard if ``tensorboard`` is accessible.
- :class:`~transformers.integrations.CodeCarbonCallback` to track the carbon emissions of your model during training if ``codecarbon`` is installed.

    - Note: These carbon emissions will be included in your automatically generated model card.

See the Transformers `Callbacks <https://huggingface.co/docs/transformers/main/en/main_classes/callback>`_
documentation for more information on the integrated callbacks and how to write your own callbacks.

The top performing models are trained using many datasets at once. Normally, this is rather tricky, as each dataset has a different format. However, :class:`sentence_transformers.sentence_transformer.trainer.SentenceTransformerTrainer` can train with multiple datasets without having to convert each dataset to the same format. It can even apply different loss functions to each of the datasets. The steps to train with multiple datasets are:

- Use a dictionary of :class:`~datasets.Dataset` instances (or a :class:`~datasets.DatasetDict`) as the ``train_dataset``  (and optionally also ``eval_dataset``).
- (Optional) Use a dictionary of loss functions mapping dataset names to losses. Only required if you wish to use different loss function for different datasets.

Each training/evaluation batch will only contain samples from one of the datasets. The order in which batches are samples from the multiple datasets is defined by the :class:`~sentence_transformers.sentence_transformer.training_args.MultiDatasetBatchSamplers` enum, which can be passed to the :class:`~sentence_transformers.sentence_transformer.training_args.SentenceTransformerTrainingArguments` via ``multi_dataset_batch_sampler``. Valid options are:

- ``MultiDatasetBatchSamplers.ROUND_ROBIN``: Round-robin sampling from each dataset until one is exhausted. With this strategy, it’s likely that not all samples from each dataset are used, but each dataset is sampled from equally.
- ``MultiDatasetBatchSamplers.PROPORTIONAL`` (default): Sample from each dataset in proportion to its size. With this strategy, all samples from each dataset are used and larger datasets are sampled from more frequently.

This multi-task training has been shown to be very effective, e.g. `Huang et al. <https://huggingface.co/papers/2405.06932>`_ employed :class:`~sentence_transformers.sentence_transformer.losses.MultipleNegativesRankingLoss`, :class:`~sentence_transformers.sentence_transformer.losses.CoSENTLoss`, and a variation on :class:`~sentence_transformers.sentence_transformer.losses.MultipleNegativesRankingLoss` without in-batch negatives and only hard negatives to reach state-of-the-art performance on Chinese. They even applied :class:`~sentence_transformers.sentence_transformer.losses.MatryoshkaLoss` to allow the model to produce `Matryoshka Embeddings <../../examples/sentence_transformer/training/matryoshka/README.html>`_.

Training on multiple datasets looks like this:

.. sidebar:: Documentation

    - :func:`datasets.load_dataset`
    - :class:`~sentence_transformers.sentence_transformer.model.SentenceTransformer`
    - :class:`~sentence_transformers.sentence_transformer.trainer.SentenceTransformerTrainer`
    - :class:`~sentence_transformers.sentence_transformer.losses.CoSENTLoss`
    - :class:`~sentence_transformers.sentence_transformer.losses.MultipleNegativesRankingLoss`
    - :class:`~sentence_transformers.sentence_transformer.losses.SoftmaxLoss`
    - `sentence-transformers/all-nli <https://huggingface.co/datasets/sentence-transformers/all-nli>`_
    - `sentence-transformers/stsb <https://huggingface.co/datasets/sentence-transformers/stsb>`_
    - `sentence-transformers/quora-duplicates <https://huggingface.co/datasets/sentence-transformers/quora-duplicates>`_
    - `sentence-transformers/natural-questions <https://huggingface.co/datasets/sentence-transformers/natural-questions>`_

    **Training Examples:**

    - `Quora Duplicate Questions > Multi-task learning <https://github.com/huggingface/sentence-transformers/blob/main/examples/sentence_transformer/training/quora_duplicate_questions/training_multi_task_learning.py>`_
    - `AllNLI + STSb > Multi-task learning <https://github.com/huggingface/sentence-transformers/blob/main/examples/sentence_transformer/training/other/training_multi_task.py>`_

::

    from datasets import load_dataset
    from sentence_transformers import SentenceTransformer, SentenceTransformerTrainer
    from sentence_transformers.sentence_transformer.losses import CoSENTLoss, MultipleNegativesRankingLoss, SoftmaxLoss

    # 1. Load a model to finetune
    model = SentenceTransformer("google-bert/bert-base-uncased")

    # 2. Load several Datasets to train with
    # (anchor, positive)
    all_nli_pair_train = load_dataset("sentence-transformers/all-nli", "pair", split="train[:10000]")
    # (premise, hypothesis) + label
    all_nli_pair_class_train = load_dataset("sentence-transformers/all-nli", "pair-class", split="train[:10000]")
    # (sentence1, sentence2) + score
    all_nli_pair_score_train = load_dataset("sentence-transformers/all-nli", "pair-score", split="train[:10000]")
    # (anchor, positive, negative)
    all_nli_triplet_train = load_dataset("sentence-transformers/all-nli", "triplet", split="train[:10000]")
    # (sentence1, sentence2) + score
    stsb_pair_score_train = load_dataset("sentence-transformers/stsb", split="train[:10000]")
    # (anchor, positive)
    quora_pair_train = load_dataset("sentence-transformers/quora-duplicates", "pair", split="train[:10000]")
    # (query, answer)
    natural_questions_train = load_dataset("sentence-transformers/natural-questions", split="train[:10000]")

    # We can combine all datasets into a dictionary with dataset names to datasets
    train_dataset = {
        "all-nli-pair": all_nli_pair_train,
        "all-nli-pair-class": all_nli_pair_class_train,
        "all-nli-pair-score": all_nli_pair_score_train,
        "all-nli-triplet": all_nli_triplet_train,
        "stsb": stsb_pair_score_train,
        "quora": quora_pair_train,
        "natural-questions": natural_questions_train,
    }

    # 3. Load several Datasets to evaluate with
    # (anchor, positive, negative)
    all_nli_triplet_dev = load_dataset("sentence-transformers/all-nli", "triplet", split="dev")
    # (sentence1, sentence2, score)
    stsb_pair_score_dev = load_dataset("sentence-transformers/stsb", split="validation")
    # (anchor, positive)
    quora_pair_dev = load_dataset("sentence-transformers/quora-duplicates", "pair", split="train[10000:11000]")
    # (query, answer)
    natural_questions_dev = load_dataset("sentence-transformers/natural-questions", split="train[10000:11000]")

    # We can use a dictionary for the evaluation dataset too, but we don't have to. We could also just use
    # no evaluation dataset, or one dataset.
    eval_dataset = {
        "all-nli-triplet": all_nli_triplet_dev,
        "stsb": stsb_pair_score_dev,
        "quora": quora_pair_dev,
        "natural-questions": natural_questions_dev,
    }

    # 4. Load several loss functions to train with
    # (anchor, positive), (anchor, positive, negative)
    mnrl_loss = MultipleNegativesRankingLoss(model)
    # (sentence_A, sentence_B) + class
    softmax_loss = SoftmaxLoss(model, model.get_embedding_dimension(), 3)
    # (sentence_A, sentence_B) + score
    cosent_loss = CoSENTLoss(model)

    # Create a mapping with dataset names to loss functions, so the trainer knows which loss to apply where.
    # Note that you can also just use one loss if all of your training/evaluation datasets use the same loss
    losses = {
        "all-nli-pair": mnrl_loss,
        "all-nli-pair-class": softmax_loss,
        "all-nli-pair-score": cosent_loss,
        "all-nli-triplet": mnrl_loss,
        "stsb": cosent_loss,
        "quora": mnrl_loss,
        "natural-questions": mnrl_loss,
    }

    # 5. Define a simple trainer, although it's recommended to use one with args & evaluators
    trainer = SentenceTransformerTrainer(
        model=model,
        train_dataset=train_dataset,
        eval_dataset=eval_dataset,
        loss=losses,
    )
    trainer.train()

    # 6. save the trained model and optionally push it to the Hugging Face Hub
    model.save_pretrained("bert-base-all-nli-stsb-quora-nq")
    model.push_to_hub("bert-base-all-nli-stsb-quora-nq")

Prior to the Sentence Transformers v3.0 release, models would be trained with the :meth:`SentenceTransformer.fit() <sentence_transformers.sentence_transformer.model.SentenceTransformer.fit>` method and a :class:`~torch.utils.data.DataLoader` of :class:`~sentence_transformers.sentence_transformer.readers.InputExample`, which looked something like this::

    from sentence_transformers import SentenceTransformer, InputExample, losses
    from torch.utils.data import DataLoader

    # Define the model. Either from scratch of by loading a pre-trained model
    model = SentenceTransformer("distilbert/distilbert-base-uncased")

    # Define your train examples. You need more than just two examples...
    train_examples = [
        InputExample(texts=["My first sentence", "My second sentence"], label=0.8),
        InputExample(texts=["Another pair", "Unrelated sentence"], label=0.3),
    ]

    # Define your train dataset, the dataloader and the train loss
    train_dataloader = DataLoader(train_examples, shuffle=True, batch_size=16)
    train_loss = losses.CosineSimilarityLoss(model)

    # Tune the model
    model.fit(train_objectives=[(train_dataloader, train_loss)], epochs=1, warmup_steps=100)

Since the v3.0 release, using :meth:`SentenceTransformer.fit() <sentence_transformers.sentence_transformer.model.SentenceTransformer.fit>` is still possible, but it will initialize a :class:`~sentence_transformers.sentence_transformer.trainer.SentenceTransformerTrainer` behind the scenes. It is recommended to use the Trainer directly, as you will have more control via the :class:`~sentence_transformers.sentence_transformer.training_args.SentenceTransformerTrainingArguments`, but existing training scripts relying on :meth:`SentenceTransformer.fit() <sentence_transformers.sentence_transformer.model.SentenceTransformer.fit>` should still work.

In case there are issues with the updated :meth:`SentenceTransformer.fit() <sentence_transformers.sentence_transformer.model.SentenceTransformer.fit>`, you can also get exactly the old behaviour by calling :meth:`SentenceTransformer.old_fit() <sentence_transformers.sentence_transformer.model.SentenceTransformer.old_fit>` instead, but this method is planned to be deprecated fully in the future.

Training :class:`~sentence_transformers.sentence_transformer.model.SentenceTransformer` models is very similar as training :class:`~sentence_transformers.cross_encoder.model.CrossEncoder` models, with some key differences:

- For :class:`~sentence_transformers.cross_encoder.model.CrossEncoder` training, you can use (variably sized) lists of texts in a column. In :class:`~sentence_transformers.sentence_transformer.model.SentenceTransformer` training, you **cannot** use lists of inputs (e.g. texts) in a column of the training/evaluation dataset(s). In short: training with a variable number of negatives is not supported.

See the `Cross Encoder > Training Overview <../cross_encoder/training_overview.html>`_ documentation for more details on training :class:`~sentence_transformers.cross_encoder.model.CrossEncoder` models.