Skip to content

Commit f7b8bae

Browse files
maziyarpanahimaziyarpanahi
committed
Release 4.2.0 on Conda [skip test]
1 parent 79f3a8e commit f7b8bae

File tree

3 files changed

+21
-16
lines changed

3 files changed

+21
-16
lines changed

‎conda/README.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -35,13 +35,13 @@ conda config --set anaconda_upload no
3535
Build `spark-nlp` from the latest PyPI tar:
3636

3737
```bash
38-
conda build . --python=3.6 && conda build . --python=3.7 && conda build . --python=3.8
38+
conda build . --python=3.7 && conda build . --python=3.8
3939
```
4040

4141
Example of uploading Conda package to Anaconda Cloud:
4242

4343
```bash
44-
anaconda upload /anaconda3/conda-bld/noarch/spark-nlp-version-py36_0.tar.bz2
44+
anaconda upload /anaconda3/conda-bld/noarch/spark-nlp-version-py37_0.tar.bz2
4545
```
4646

4747
## Install

‎conda/meta.yaml‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,15 +1,15 @@
11
package:
22
name: "spark-nlp"
3-
version: 4.1.0
3+
version: 4.2.0
44

55
app:
66
entry: spark-nlp
77
summary: Natural Language Understanding Library for Apache Spark.
88

99
source:
10-
fn: spark-nlp-4.1.0.tar.gz
11-
url: https://files.pythonhosted.org/packages/4f/3d/b00ee9426bea493e4c754f59ecd219fe8f04e664991b1be7b570c3714cdb/spark-nlp-4.1.0.tar.gz
12-
sha256: 5afc3a7e9e8381fd2f8a5d6a47bcf2fdd4c6e247b82cfab411b9aee0e9a6f69d
10+
fn: spark-nlp-4.2.0.tar.gz
11+
url: https://files.pythonhosted.org/packages/fd/99/9345ba8d7303ac45402ad18c71c604b7bccc1f113409f306ba46bbec4612/spark-nlp-4.2.0.tar.gz
12+
sha256: abe49534f4961486e404f5219987e0cc3e507ca19bc582b26da6ba9233a253b5
1313
build:
1414
noarch: generic
1515
number: 0

‎docs/en/transformers.md‎

Lines changed: 15 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ seotitle: Spark NLP
55
title: Transformers
66
permalink: /docs/en/transformers
77
key: docs-transformers
8-
modify_date: "2022-08-23"
8+
modify_date: "2022-09-27"
99
use_language_switcher: "Python-Scala-Java"
1010
show_nav: true
1111
sidebar:
@@ -27,7 +27,7 @@ sidebar:
2727

2828
### Overview
2929

30-
We have extended support for `HuggingFace` 🤗 and `TF Hub` exported models since `3.1.0` to equivalent Spark NLP 🚀 annotators. Starting this release, you can easily use the `saved_model` feature in HuggingFace within a few lines of codes and import any `BERT`, `DistilBERT`, `CamemBERT`, `RoBERTa`, `DeBERTa`, `XLM-RoBERTa`, `Longformer`, `BertForTokenClassification`, `DistilBertForTokenClassification`, `AlbertForTokenClassification`, `RoBertaForTokenClassification`, `DeBertaForTokenClassification`, `XlmRoBertaForTokenClassification`, `XlnetForTokenClassification`, `LongformerForTokenClassification`, `BertForSequenceClassification`, `DistilBertForSequenceClassification`, `AlbertForSequenceClassification`, `RoBertaForSequenceClassification`, `DeBertaForSequenceClassification`, `XlmRoBertaForSequenceClassification`, `XlnetForSequenceClassification`, `LongformerForSequenceClassification`, `AlbertForQuestionAnswering`, `BertForQuestionAnswering`, `DeBertaForQuestionAnswering`, `DistilBertForQuestionAnswering`, `LongformerForQuestionAnswering`, `RoBertaForQuestionAnswering`, and `XlmRoBertaForQuestionAnswering` models to Spark NLP. We will work on the remaining annotators and extend this support to the rest with each release 😊
30+
We have extended support for `HuggingFace` 🤗 and `TF Hub` exported models since `3.1.0` to equivalent Spark NLP 🚀 annotators. Starting this release, you can easily use the `saved_model` feature in HuggingFace within a few lines of codes and import any `BERT`, `DistilBERT`, `CamemBERT`, `RoBERTa`, `DeBERTa`, `XLM-RoBERTa`, `Longformer`, `BertForTokenClassification`, `DistilBertForTokenClassification`, `AlbertForTokenClassification`, `RoBertaForTokenClassification`, `DeBertaForTokenClassification`, `XlmRoBertaForTokenClassification`, `XlnetForTokenClassification`, `LongformerForTokenClassification`, `CamemBertForTokenClassification`, `BertForSequenceClassification`, `DistilBertForSequenceClassification`, `AlbertForSequenceClassification`, `RoBertaForSequenceClassification`, `DeBertaForSequenceClassification`, `XlmRoBertaForSequenceClassification`, `XlnetForSequenceClassification`, `LongformerForSequenceClassification`, `AlbertForQuestionAnswering`, `BertForQuestionAnswering`, `DeBertaForQuestionAnswering`, `DistilBertForQuestionAnswering`, `LongformerForQuestionAnswering`, `RoBertaForQuestionAnswering`, `XlmRoBertaForQuestionAnswering`, `TapasForQuestionAnswering`, and `Vision Transformers (ViT)` models to Spark NLP. We will work on the remaining annotators and extend this support to the rest with each release 😊
3131

3232
### Compatibility
3333

@@ -64,6 +64,7 @@ DeBertaForTokenClassification | | ✅ | [TFDebertaV2ForTokenClassification](
6464
XlmRoBertaForTokenClassification | | ✅ | [TFXLMRobertaForTokenClassification](https://huggingface.co/docs/transformers/model_doc/xlmroberta#transformers.TFXLMRobertaForTokenClassification)
6565
XlnetForTokenClassification | | ✅ | [TFXLNetForTokenClassificationet](https://huggingface.co/docs/transformers/model_doc/xlnet#transformers.TFXLNetForTokenClassificationet)
6666
LongformerForTokenClassification | | ✅ | [TFLongformerForTokenClassification](https://huggingface.co/docs/transformers/model_doc/longformer#transformers.TFLongformerForTokenClassification)
67+
CamemBertForTokenClassification | | ✅ | [TFCamemBertForTokenClassification](https://huggingface.co/docs/transformers/model_doc/camembert#transformers.TFCamembertForTokenClassification)
6768
BertForSequenceClassification | | ✅ | [TFBertForSequenceClassification](https://huggingface.co/docs/transformers/model_doc/bert#transformers.TFBertForSequenceClassification)
6869
DistilBertForSequenceClassification | | ✅ | [TFDistilBertForSequenceClassification](https://huggingface.co/docs/transformers/model_doc/distilbert#transformers.TFDistilBertForSequenceClassification)
6970
AlbertForSequenceClassification | | ✅ | [TFAlbertForSequenceClassification](https://huggingface.co/docs/transformers/model_doc/albert#transformers.TFAlbertForSequenceClassification)
@@ -72,17 +73,19 @@ DeBertaForSequenceClassification | | ✅ | [TFDebertaV2ForSequenceClassifica
7273
XlmRoBertaForSequenceClassification | | ✅ | [TFXLMRobertaForSequenceClassification](https://huggingface.co/docs/transformers/model_doc/xlmroberta#transformers.TFXLMRobertaForSequenceClassification)
7374
XlnetForSequenceClassification | | ✅ | [TFXLNetForSequenceClassification](https://huggingface.co/docs/transformers/model_doc/xlnet#transformers.TFXLNetForSequenceClassification)
7475
LongformerForSequenceClassification | | ✅ | [TFLongformerForSequenceClassification](https://huggingface.co/docs/transformers/model_doc/longformer#transformers.TFLongformerForSequenceClassification)
75-
AlbertForQuestionAnswering | |✅ | [TFAlbertForQuestionAnswering](https://huggingface.co/docs/transformers/model_doc/albert#transformers.TFAlbertForQuestionAnswering)
76-
BertForQuestionAnswering | |✅ | [TFBertForQuestionAnswering](https://huggingface.co/docs/transformers/model_doc/bert#transformers.TFBertForQuestionAnswering)
77-
DeBertaForQuestionAnswering | |✅ | [TFDebertaV2ForQuestionAnswering](https://huggingface.co/docs/transformers/model_doc/deberta-v2#transformers.TFDebertaV2ForQuestionAnswering)
78-
DistilBertForQuestionAnswering | |✅ | [TFDistilBertForQuestionAnswering](https://huggingface.co/docs/transformers/model_doc/distilbert#transformers.TFDistilBertForQuestionAnswering)
79-
LongformerForQuestionAnswering | |✅ | [TFLongformerForQuestionAnswering](https://huggingface.co/docs/transformers/model_doc/longformer#transformers.TFLongformerForQuestionAnswering)
80-
RoBertaForQuestionAnswering | |✅ | [TFRobertaForQuestionAnswering](https://huggingface.co/docs/transformers/model_doc/roberta#transformers.TFRobertaForQuestionAnswering)
81-
XlmRoBertaForQuestionAnswering | |✅ | [TFXLMRobertaForQuestionAnswering](https://huggingface.co/docs/transformers/model_doc/xlm-roberta#transformers.TFXLMRobertaForQuestionAnswering)
76+
| AlbertForQuestionAnswering | |✅ | [TFAlbertForQuestionAnswering](https://huggingface.co/docs/transformers/model_doc/albert#transformers.TFAlbertForQuestionAnswering)
77+
| BertForQuestionAnswering | |✅ | [TFBertForQuestionAnswering](https://huggingface.co/docs/transformers/model_doc/bert#transformers.TFBertForQuestionAnswering)
78+
| DeBertaForQuestionAnswering | |✅ | [TFDebertaV2ForQuestionAnswering](https://huggingface.co/docs/transformers/model_doc/deberta-v2#transformers.TFDebertaV2ForQuestionAnswering)
79+
| DistilBertForQuestionAnswering | |✅ | [TFDistilBertForQuestionAnswering](https://huggingface.co/docs/transformers/model_doc/distilbert#transformers.TFDistilBertForQuestionAnswering)
80+
| LongformerForQuestionAnswering | |✅ | [TFLongformerForQuestionAnswering](https://huggingface.co/docs/transformers/model_doc/longformer#transformers.TFLongformerForQuestionAnswering)
81+
| RoBertaForQuestionAnswering | |✅ | [TFRobertaForQuestionAnswering](https://huggingface.co/docs/transformers/model_doc/roberta#transformers.TFRobertaForQuestionAnswering)
82+
| XlmRoBertaForQuestionAnswering | |✅ | [TFXLMRobertaForQuestionAnswering](https://huggingface.co/docs/transformers/model_doc/xlm-roberta#transformers.TFXLMRobertaForQuestionAnswering)
83+
| TapasForQuestionAnswering | | ❎ | [TFTapasForQuestionAnswering](https://huggingface.co/docs/transformers/model_doc/tapas#transformers.TFTapasForQuestionAnswering)
84+
ViTForImageClassification | ❌ | ✅ | [TFViTForImageClassification](https://huggingface.co/docs/transformers/model_doc/vit#transformers.TFViTForImageClassification)
85+
Automatic Speech Recognition (Wav2Vec2ForCTC)| | ❎ | [TFWav2Vec2ForCTC](https://huggingface.co/docs/transformers/model_doc/wav2vec2#transformers.TFWav2Vec2ForCTC)
8286
T5Transformer | | ❌ |
8387
MarianTransformer| | ❌ |
8488
OpenAI GPT2| | ❌ |
85-
Vision Transformer (ViT)| ❎ | ❎ | [TFViTForImageClassification](https://huggingface.co/docs/transformers/model_doc/vit#transformers.TFViTForImageClassification)
8689

8790
### Example Notebooks
8891

@@ -105,6 +108,7 @@ DistilBertForTokenClassification|[HuggingFace in Spark NLP - DistilBertForTokenC
105108
AlbertForTokenClassification|[HuggingFace in Spark NLP - AlbertForTokenClassification](https://github.com/JohnSnowLabs/spark-nlp-workshop/blob/master/jupyter/transformers/HuggingFace%20in%20Spark%20NLP%20-%20AlbertForTokenClassification.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/JohnSnowLabs/spark-nlp-workshop/blob/master/jupyter/transformers/HuggingFace%20in%20Spark%20NLP%20-%20AlbertForTokenClassification.ipynb)
106109
RoBertaForTokenClassification|[HuggingFace in Spark NLP - RoBertaForTokenClassification](https://github.com/JohnSnowLabs/spark-nlp-workshop/blob/master/jupyter/transformers/HuggingFace%20in%20Spark%20NLP%20-%20RoBertaForTokenClassification.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/JohnSnowLabs/spark-nlp-workshop/blob/master/jupyter/transformers/HuggingFace%20in%20Spark%20NLP%20-%20RoBertaForTokenClassification.ipynb)
107110
XlmRoBertaForTokenClassification|[HuggingFace in Spark NLP - XlmRoBertaForTokenClassification](https://github.com/JohnSnowLabs/spark-nlp-workshop/blob/master/jupyter/transformers/HuggingFace%20in%20Spark%20NLP%20-%20XlmRoBertaForTokenClassification.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/JohnSnowLabs/spark-nlp-workshop/blob/master/jupyter/transformers/HuggingFace%20in%20Spark%20NLP%20-%20XlmRoBertaForTokenClassification.ipynb)
111+
CamemBertForTokenClassification|[HuggingFace in Spark NLP - CamemBertForTokenClassification](https://github.com/JohnSnowLabs/spark-nlp-workshop/blob/master/jupyter/transformers/HuggingFace%20in%20Spark%20NLP%20-%20CamemBertForTokenClassification.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/JohnSnowLabs/spark-nlp-workshop/blob/master/jupyter/transformers/HuggingFace%20in%20Spark%20NLP%20-%20CamemBertForTokenClassification.ipynb)
108112
BertForSequenceClassification |[HuggingFace in Spark NLP - BertForSequenceClassification](https://github.com/JohnSnowLabs/spark-nlp-workshop/blob/master/jupyter/transformers/HuggingFace%20in%20Spark%20NLP%20-%20BertForSequenceClassification.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/JohnSnowLabs/spark-nlp-workshop/blob/master/jupyter/transformers/HuggingFace%20in%20Spark%20NLP%20-%20BertForSequenceClassification.ipynb)
109113
DistilBertForSequenceClassification |[HuggingFace in Spark NLP - DistilBertForSequenceClassification](https://github.com/JohnSnowLabs/spark-nlp-workshop/blob/master/jupyter/transformers/HuggingFace%20in%20Spark%20NLP%20-%20DistilBertForSequenceClassification.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/JohnSnowLabs/spark-nlp-workshop/blob/master/jupyter/transformers/HuggingFace%20in%20Spark%20NLP%20-%20DistilBertForSequenceClassification.ipynb)
110114
AlbertForSequenceClassification |[HuggingFace in Spark NLP - AlbertForSequenceClassification](https://github.com/JohnSnowLabs/spark-nlp-workshop/blob/master/jupyter/transformers/HuggingFace%20in%20Spark%20NLP%20-%20BertForSequenceClassification.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/JohnSnowLabs/spark-nlp-workshop/blob/master/jupyter/transformers/HuggingFace%20in%20Spark%20NLP%20-%20AlbertForSequenceClassification.ipynb)
@@ -119,6 +123,7 @@ DistilBertForQuestionAnswering|[HuggingFace in Spark NLP - DistilBertForQuestion
119123
LongformerForQuestionAnswering|[HuggingFace in Spark NLP - LongformerForQuestionAnswering](https://github.com/JohnSnowLabs/spark-nlp-workshop/blob/master/jupyter/transformers/HuggingFace%20in%20Spark%20NLP%20-%20LongformerForQuestionAnswering.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/JohnSnowLabs/spark-nlp-workshop/blob/master/jupyter/transformers/HuggingFace%20in%20Spark%20NLP%20-%20LongformerForQuestionAnswering.ipynb)
120124
RoBertaForQuestionAnswering|[HuggingFace in Spark NLP - RoBertaForQuestionAnswering](https://github.com/JohnSnowLabs/spark-nlp-workshop/blob/master/jupyter/transformers/HuggingFace%20in%20Spark%20NLP%20-%20RoBertaForQuestionAnswering.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/JohnSnowLabs/spark-nlp-workshop/blob/master/jupyter/transformers/HuggingFace%20in%20Spark%20NLP%20-%20RoBertaForQuestionAnswering.ipynb)
121125
XlmRobertaForQuestionAnswering|[HuggingFace in Spark NLP - XlmRobertaForQuestionAnswering](https://github.com/JohnSnowLabs/spark-nlp-workshop/blob/master/jupyter/transformers/HuggingFace%20in%20Spark%20NLP%20-%20XlmRobertaForQuestionAnswering.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/JohnSnowLabs/spark-nlp-workshop/blob/master/jupyter/transformers/HuggingFace%20in%20Spark%20NLP%20-%20XlmRobertaForQuestionAnswering.ipynb)
126+
ViTForImageClassification|[HuggingFace in Spark NLP - ViTForImageClassification](https://github.com/JohnSnowLabs/spark-nlp-workshop/blob/master/jupyter/transformers/HuggingFace%20in%20Spark%20NLP%20-%20ViTForImageClassification.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/JohnSnowLabs/spark-nlp-workshop/blob/master/jupyter/transformers/HuggingFace%20in%20Spark%20NLP%20-%20ViTForImageClassification.ipynb)
122127

123128
#### TF Hub to Spark NLP
124129

0 commit comments

Comments
 (0)