You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This framework provides an easy method to compute dense vector representations for **sentences**, **paragraphs**, and **images**. The models are based on transformer networks like BERT / RoBERTa / XLM-RoBERTa etc. and achieve state-of-the-art performance in various tasks. Text is embedded in vector space such that similar text are closer and can efficiently be found using cosine similarity.
22
18
23
-
We provide an increasing number of **[state-of-the-art pretrained models](https://www.sbert.net/docs/pretrained_models.html)** for more than 100 languages, fine-tuned for various use-cases.
19
+
We provide an increasing number of **[state-of-the-art pretrained models](https://www.sbert.net/docs/sentence_transformer/pretrained_models.html)** for more than 100 languages, fine-tuned for various use-cases.
24
20
25
-
Further, this framework allows an easy **[fine-tuning of custom embeddings models](https://www.sbert.net/docs/training/overview.html)**, to achieve maximal performance on your specific task.
21
+
Further, this framework allows an easy **[fine-tuning of custom embeddings models](https://www.sbert.net/docs/sentence_transformer/training_overview.html)**, to achieve maximal performance on your specific task.
26
22
27
23
For the **full documentation**, see **[www.SBERT.net](https://www.sbert.net)**.
28
24
29
-
The following publications are integrated in this framework:
30
-
31
-
-[Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://arxiv.org/abs/1908.10084) (EMNLP 2019)
32
-
-[Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillation](https://arxiv.org/abs/2004.09813) (EMNLP 2020)
33
-
-[Augmented SBERT: Data Augmentation Method for Improving Bi-Encoders for Pairwise Sentence Scoring Tasks](https://arxiv.org/abs/2010.08240) (NAACL 2021)
34
-
-[The Curse of Dense Low-Dimensional Information Retrieval for Large Index Sizes](https://arxiv.org/abs/2012.14210) (arXiv 2020)
35
-
-[TSDAE: Using Transformer-based Sequential Denoising Auto-Encoder for Unsupervised Sentence Embedding Learning](https://arxiv.org/abs/2104.06979) (arXiv 2021)
36
-
-[BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models](https://arxiv.org/abs/2104.08663) (arXiv 2021)
We recommend **Python 3.8** or higher, **[PyTorch 1.11.0](https://pytorch.org/get-started/locally/)** or higher and **[transformers v4.32.0](https://github.com/huggingface/transformers)** or higher. The code does **not** work with Python 2.7.
27
+
We recommend **Python 3.8+**, **[PyTorch 1.11.0+](https://pytorch.org/get-started/locally/)**, and **[transformers v4.34.0+](https://github.com/huggingface/transformers)**.
42
28
43
29
**Install with pip**
44
30
45
-
Install the *sentence-transformers* with `pip`:
46
-
47
31
```
48
32
pip install -U sentence-transformers
49
33
```
50
34
51
35
**Install with conda**
52
36
53
-
You can install the *sentence-transformers* with `conda`:
@@ -73,8 +55,6 @@ If you want to use a GPU / CUDA, you must install PyTorch with the matching CUDA
73
55
74
56
See [Quickstart](https://www.sbert.net/docs/quickstart.html) in our documenation.
75
57
76
-
[This example](https://github.com/UKPLab/sentence-transformers/tree/master/examples/applications/computing-embeddings/computing_embeddings.py) shows you how to use an already trained Sentence Transformer model to embed sentences for another task.
77
-
78
58
First download a pretrained model.
79
59
80
60
````python
@@ -87,58 +67,52 @@ Then provide some sentences to the model.
87
67
88
68
````python
89
69
sentences = [
90
-
"This framework generates embeddings for each input sentence",
91
-
"Sentences are passed as a list of string.",
92
-
"The quick brown fox jumps over the lazy dog.",
70
+
"The weather is lovely today.",
71
+
"It's so sunny outside!",
72
+
"He drove to the stadium.",
93
73
]
94
-
sentence_embeddings = model.encode(sentences)
74
+
embeddings = model.encode(sentences)
75
+
print(embeddings.shape)
76
+
# => (3, 384)
95
77
````
96
78
97
-
And that's it already. We now have a list of numpy arrays with the embeddings.
79
+
And that's already it. We now have a numpy arrays with the embeddings, one for each text. We can use these to compute similarities.
98
80
99
81
````python
100
-
for sentence, embedding inzip(sentences, sentence_embeddings):
We provide a large list of [Pretrained Models](https://www.sbert.net/docs/pretrained_models.html) for more than 100 languages. Some models are general purpose models, while others produce embeddings for specific use cases. Pre-trained models can be loaded by just passing the model name: `SentenceTransformer('model_name')`.
109
-
110
-
[» Full list of pretrained models](https://www.sbert.net/docs/pretrained_models.html)
91
+
We provide a large list of [Pretrained Models](https://www.sbert.net/docs/sentence_transformer/pretrained_models.html) for more than 100 languages. Some models are general purpose models, while others produce embeddings for specific use cases. Pre-trained models can be loaded by just passing the model name: `SentenceTransformer('model_name')`.
111
92
112
93
## Training
113
94
114
95
This framework allows you to fine-tune your own sentence embedding methods, so that you get task-specific sentence embeddings. You have various options to choose from in order to get perfect sentence embeddings for your specific task.
115
96
116
-
See [Training Overview](https://www.sbert.net/docs/training/overview.html) for an introduction how to train your own embedding models. We provide [various examples](https://github.com/UKPLab/sentence-transformers/tree/master/examples/training) how to train models on various datasets.
97
+
See [Training Overview](https://www.sbert.net/docs/sentence_transformer/training_overview.html) for an introduction how to train your own embedding models. We provide [various examples](https://github.com/UKPLab/sentence-transformers/tree/master/examples/training) how to train models on various datasets.
117
98
118
99
Some highlights are:
119
100
- Support of various transformer networks including BERT, RoBERTa, XLM-R, DistilBERT, Electra, BART, ...
120
101
- Multi-Lingual and multi-task learning
121
102
- Evaluation during training to find optimal model
122
-
-[20+ loss-functions](https://www.sbert.net/docs/package_reference/losses.html) allowing to tune models specifically for semantic search, paraphrase mining, semantic similarity comparison, clustering, triplet loss, contrastive loss.
123
-
124
-
## Performance
125
-
126
-
Our models are evaluated extensively on 15+ datasets including challening domains like Tweets, Reddit, emails. They achieve by far the **best performance** from all available sentence embedding methods. Further, we provide several **smaller models** that are **optimized for speed**.
127
-
128
-
[» Full list of pretrained models](https://www.sbert.net/docs/pretrained_models.html)
103
+
-[20+ loss-functions](https://www.sbert.net/docs/package_reference/sentence_transformer/losses.html) allowing to tune models specifically for semantic search, paraphrase mining, semantic similarity comparison, clustering, triplet loss, contrastive loss, etc.
@@ -193,7 +167,7 @@ If you use one of the multilingual models, feel free to cite our publication [Ma
193
167
194
168
Please have a look at [Publications](https://www.sbert.net/docs/publications.html) for our different publications that are integrated into SentenceTransformers.
0 commit comments