Skip to content

Commit 85890d5

Browse files
committed
Update training overview docs based on the blogpost reviews
1 parent 946a97d commit 85890d5

File tree

2 files changed

+9
-8
lines changed

2 files changed

+9
-8
lines changed

docs/sentence_transformer/training_overview.md

Lines changed: 8 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -128,14 +128,14 @@ The :class:`SentenceTransformerTrainer` trains and evaluates using :class:`datas
128128
129129
from datasets import Dataset
130130
131-
sentence1_list = []
132-
sentence2_list = []
131+
anchors = []
132+
positives = []
133133
# Open a file, do preprocessing, filtering, cleaning, etc.
134134
# and append to the lists
135135
136136
dataset = Dataset.from_dict({
137-
"sentence1": sentence1_list,
138-
"sentence2": sentence2_list,
137+
"anchor": anchors,
138+
"positive": positives,
139139
})
140140
141141
Each key from the dictionary will become a column in the resulting dataset.
@@ -276,9 +276,10 @@ args = SentenceTransformerTrainingArguments(
276276

277277
## Evaluator
278278

279-
```eval_rst
280-
Several evaluators exist that can help with evaluation before, during, and after training:
279+
You can provide the [`SentenceTransformerTrainer`](https://sbert.net/docs/package_reference/sentence_transformer/SentenceTransformer.html#sentence_transformers.SentenceTransformer) with an `eval_dataset` to get the evaluation loss during training, but it may be useful to get more concrete metrics during training, too. For this, you can use evaluators to assess the model's performance with useful metrics before, during, or after training. You can both an `eval_dataset` and an evaluator, one or the other, or neither. They evaluate based on the `eval_strategy` and `eval_steps` [Training Arguments](#training-arguments).
281280

281+
Here are the implemented Evaluators that come with Sentence Tranformers:
282+
```eval_rst
282283
======================================================================== ===========================================================================================================================
283284
Evaluator Required Data
284285
======================================================================== ===========================================================================================================================
@@ -292,7 +293,7 @@ Evaluator Requir
292293
:class:`~sentence_transformers.evaluation.TripletEvaluator` (anchor, positive, negative) pairs.
293294
======================================================================== ===========================================================================================================================
294295
295-
Additionally, :class:`~sentence_transformers.evaluation.SequentialEvaluator` should be used to combine multiple evaluators into one Evaluator that can be passed to the :class:`~sentence_transformers.trainer.SentenceTransformerTrainer`. When the evaluator is run depends on the ``eval_strategy`` and ``eval_steps`` `Training Arguments <#training-arguments>`_.
296+
Additionally, :class:`~sentence_transformers.evaluation.SequentialEvaluator` should be used to combine multiple evaluators into one Evaluator that can be passed to the :class:`~sentence_transformers.trainer.SentenceTransformerTrainer`.
296297
297298
Sometimes you don't have the required evaluation data to prepare one of these evaluators on your own, but you still want to track how well the model performs on some common benchmarks. In that case, you can use these evaluators with data from Hugging Face.
298299

sentence_transformers/losses/CachedMultipleNegativesRankingLoss.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -95,7 +95,7 @@ def __init__(
9595
the slower the training will be. It's recommended to set it as high as your GPU memory allows. The default
9696
value is 32.
9797
show_progress_bar: If True, a progress bar for the mini-batches is shown during training. The default is False.
98-
98+
9999
References:
100100
- Efficient Natural Language Response Suggestion for Smart Reply, Section 4.4: https://arxiv.org/pdf/1705.00652.pdf
101101
- Scaling Deep Contrastive Learning Batch Size under Memory Limited Setup: https://arxiv.org/pdf/2101.06983.pdf

0 commit comments

Comments
 (0)