Skip to content

Commit

Permalink
Update training overview docs based on the blogpost reviews
Browse files Browse the repository at this point in the history
  • Loading branch information
tomaarsen committed May 28, 2024
1 parent 946a97d commit 85890d5
Show file tree
Hide file tree
Showing 2 changed files with 9 additions and 8 deletions.
15 changes: 8 additions & 7 deletions docs/sentence_transformer/training_overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -128,14 +128,14 @@ The :class:`SentenceTransformerTrainer` trains and evaluates using :class:`datas
from datasets import Dataset
sentence1_list = []
sentence2_list = []
anchors = []
positives = []
# Open a file, do preprocessing, filtering, cleaning, etc.
# and append to the lists
dataset = Dataset.from_dict({
"sentence1": sentence1_list,
"sentence2": sentence2_list,
"anchor": anchors,
"positive": positives,
})
Each key from the dictionary will become a column in the resulting dataset.
Expand Down Expand Up @@ -276,9 +276,10 @@ args = SentenceTransformerTrainingArguments(

## Evaluator

```eval_rst
Several evaluators exist that can help with evaluation before, during, and after training:
You can provide the [`SentenceTransformerTrainer`](https://sbert.net/docs/package_reference/sentence_transformer/SentenceTransformer.html#sentence_transformers.SentenceTransformer) with an `eval_dataset` to get the evaluation loss during training, but it may be useful to get more concrete metrics during training, too. For this, you can use evaluators to assess the model's performance with useful metrics before, during, or after training. You can both an `eval_dataset` and an evaluator, one or the other, or neither. They evaluate based on the `eval_strategy` and `eval_steps` [Training Arguments](#training-arguments).

Here are the implemented Evaluators that come with Sentence Tranformers:
```eval_rst
======================================================================== ===========================================================================================================================
Evaluator Required Data
======================================================================== ===========================================================================================================================
Expand All @@ -292,7 +293,7 @@ Evaluator Requir
:class:`~sentence_transformers.evaluation.TripletEvaluator` (anchor, positive, negative) pairs.
======================================================================== ===========================================================================================================================
Additionally, :class:`~sentence_transformers.evaluation.SequentialEvaluator` should be used to combine multiple evaluators into one Evaluator that can be passed to the :class:`~sentence_transformers.trainer.SentenceTransformerTrainer`. When the evaluator is run depends on the ``eval_strategy`` and ``eval_steps`` `Training Arguments <#training-arguments>`_.
Additionally, :class:`~sentence_transformers.evaluation.SequentialEvaluator` should be used to combine multiple evaluators into one Evaluator that can be passed to the :class:`~sentence_transformers.trainer.SentenceTransformerTrainer`.
Sometimes you don't have the required evaluation data to prepare one of these evaluators on your own, but you still want to track how well the model performs on some common benchmarks. In that case, you can use these evaluators with data from Hugging Face.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -95,7 +95,7 @@ def __init__(
the slower the training will be. It's recommended to set it as high as your GPU memory allows. The default
value is 32.
show_progress_bar: If True, a progress bar for the mini-batches is shown during training. The default is False.
References:
- Efficient Natural Language Response Suggestion for Smart Reply, Section 4.4: https://arxiv.org/pdf/1705.00652.pdf
- Scaling Deep Contrastive Learning Batch Size under Memory Limited Setup: https://arxiv.org/pdf/2101.06983.pdf
Expand Down

0 comments on commit 85890d5

Please sign in to comment.