Skip to content

Commit 3be5630

Browse files
authored
Merge pull request #529 from JohnSnowLabs/206-release-candidate
Release candidate 2.0.6
2 parents e0a1645 + 8914afa commit 3be5630

File tree

12 files changed

+60
-46
lines changed

12 files changed

+60
-46
lines changed

CHANGELOG

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,16 @@
1+
========
2+
2.0.6
3+
========
4+
---------------
5+
Overview
6+
---------------
7+
Following the 2.0.5 (read notes below), this release fixes a bug when disabling contrib param in NerDLApproach on non-windows OS
8+
9+
---------------
10+
Bugfixes
11+
---------------
12+
* Fixed NerDLApproach failing when training with setUseContrib(false)
13+
114
========
215
2.0.5
316
========

README.md

Lines changed: 14 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -40,7 +40,7 @@ Take a look at our official Spark NLP page: [http://nlp.johnsnowlabs.com/](http:
4040

4141
## Apache Spark Support
4242

43-
Spark NLP *2.0.5* has been built on top of Apache Spark 2.4.3
43+
Spark NLP *2.0.6* has been built on top of Apache Spark 2.4.3
4444

4545
Note that Spark is not retrocompatible with Spark 2.3.x, so models and environments might not work.
4646

@@ -65,18 +65,18 @@ This library has been uploaded to the [spark-packages repository](https://spark-
6565

6666
Benefit of spark-packages is that makes it available for both Scala-Java and Python
6767

68-
To use the most recent version just add the `--packages JohnSnowLabs:spark-nlp:2.0.5` to you spark command
68+
To use the most recent version just add the `--packages JohnSnowLabs:spark-nlp:2.0.6` to you spark command
6969

7070
```sh
71-
spark-shell --packages JohnSnowLabs:spark-nlp:2.0.5
71+
spark-shell --packages JohnSnowLabs:spark-nlp:2.0.6
7272
```
7373

7474
```sh
75-
pyspark --packages JohnSnowLabs:spark-nlp:2.0.5
75+
pyspark --packages JohnSnowLabs:spark-nlp:2.0.6
7676
```
7777

7878
```sh
79-
spark-submit --packages JohnSnowLabs:spark-nlp:2.0.5
79+
spark-submit --packages JohnSnowLabs:spark-nlp:2.0.6
8080
```
8181

8282
This can also be used to create a SparkSession manually by using the `spark.jars.packages` option in both Python and Scala
@@ -144,7 +144,7 @@ Our package is deployed to maven central. In order to add this package as a depe
144144
<dependency>
145145
<groupId>com.johnsnowlabs.nlp</groupId>
146146
<artifactId>spark-nlp_2.11</artifactId>
147-
<version>2.0.5</version>
147+
<version>2.0.6</version>
148148
</dependency>
149149
```
150150

@@ -155,22 +155,22 @@ and
155155
<dependency>
156156
<groupId>com.johnsnowlabs.nlp</groupId>
157157
<artifactId>spark-nlp-ocr_2.11</artifactId>
158-
<version>2.0.5</version>
158+
<version>2.0.6</version>
159159
</dependency>
160160
```
161161

162162
### SBT
163163

164164
```sbtshell
165165
// https://mvnrepository.com/artifact/com.johnsnowlabs.nlp/spark-nlp
166-
libraryDependencies += "com.johnsnowlabs.nlp" %% "spark-nlp" % "2.0.5"
166+
libraryDependencies += "com.johnsnowlabs.nlp" %% "spark-nlp" % "2.0.6"
167167
```
168168

169169
and
170170

171171
```sbtshell
172172
// https://mvnrepository.com/artifact/com.johnsnowlabs.nlp/spark-nlp-ocr
173-
libraryDependencies += "com.johnsnowlabs.nlp" %% "spark-nlp-ocr" % "2.0.5"
173+
libraryDependencies += "com.johnsnowlabs.nlp" %% "spark-nlp-ocr" % "2.0.6"
174174
```
175175

176176
Maven Central: [https://mvnrepository.com/artifact/com.johnsnowlabs.nlp](https://mvnrepository.com/artifact/com.johnsnowlabs.nlp)
@@ -184,7 +184,7 @@ Maven Central: [https://mvnrepository.com/artifact/com.johnsnowlabs.nlp](https:/
184184
If you installed pyspark through pip, you can install `spark-nlp` through pip as well.
185185

186186
```bash
187-
pip install spark-nlp==2.0.5
187+
pip install spark-nlp==2.0.6
188188
```
189189

190190
PyPI [spark-nlp package](https://pypi.org/project/spark-nlp/)
@@ -207,7 +207,7 @@ spark = SparkSession.builder \
207207
.master("local[4]")\
208208
.config("spark.driver.memory","4G")\
209209
.config("spark.driver.maxResultSize", "2G") \
210-
.config("spark.jars.packages", "JohnSnowLabs:spark-nlp:2.0.5")\
210+
.config("spark.jars.packages", "JohnSnowLabs:spark-nlp:2.0.6")\
211211
.config("spark.kryoserializer.buffer.max", "500m")\
212212
.getOrCreate()
213213
```
@@ -221,7 +221,7 @@ Use either one of the following options
221221
* Add the following Maven Coordinates to the interpreter's library list
222222

223223
```bash
224-
com.johnsnowlabs.nlp:spark-nlp_2.11:2.0.5
224+
com.johnsnowlabs.nlp:spark-nlp_2.11:2.0.6
225225
```
226226

227227
* Add path to pre-built jar from [here](#pre-compiled-spark-nlp-and-spark-nlp-ocr) in the interpreter's library list making sure the jar is available to driver path
@@ -231,7 +231,7 @@ com.johnsnowlabs.nlp:spark-nlp_2.11:2.0.5
231231
Apart from previous step, install python module through pip
232232

233233
```bash
234-
pip install spark-nlp==2.0.5
234+
pip install spark-nlp==2.0.6
235235
```
236236

237237
Or you can install `spark-nlp` from inside Zeppelin by using Conda:
@@ -256,7 +256,7 @@ export PYSPARK_PYTHON=python3
256256
export PYSPARK_DRIVER_PYTHON=jupyter
257257
export PYSPARK_DRIVER_PYTHON_OPTS=notebook
258258

259-
pyspark --packages JohnSnowLabs:spark-nlp:2.0.5
259+
pyspark --packages JohnSnowLabs:spark-nlp:2.0.6
260260
```
261261

262262
Alternatively, you can mix in using `--jars` option for pyspark + `pip install spark-nlp`

build.sbt

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@ if(is_gpu.equals("false")){
1616

1717
organization:= "com.johnsnowlabs.nlp"
1818

19-
version := "2.0.5"
19+
version := "2.0.6"
2020

2121
scalaVersion in ThisBuild := scalaVer
2222

@@ -178,7 +178,7 @@ assemblyMergeStrategy in assembly := {
178178
lazy val ocr = (project in file("ocr"))
179179
.settings(
180180
name := "spark-nlp-ocr",
181-
version := "2.0.5",
181+
version := "2.0.6",
182182

183183
test in assembly := {},
184184

docs/Gemfile.lock

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -211,7 +211,7 @@ GEM
211211
sawyer (~> 0.8.0, >= 0.5.3)
212212
pathutil (0.16.2)
213213
forwardable-extended (~> 2.6)
214-
public_suffix (2.0.5)
214+
public_suffix (2.0.6)
215215
rb-fsevent (0.10.3)
216216
rb-inotify (0.10.0)
217217
ffi (~> 1.0)

docs/_layouts/landing.html

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -49,22 +49,22 @@ <h1>{{ _section.title }}</h1>
4949
<div class="cell cell--12 cell--lg-12" style="text-align: left; background-color: #2d2d2d; padding: 10px">
5050
{% highlight bash %}
5151
# Install Spark NLP from PyPI
52-
$ pip install spark-nlp==2.0.5
52+
$ pip install spark-nlp==2.0.6
5353

5454
# Install Spark NLP from Anacodna/Conda
5555
$ conda install -c johnsnowlabs spark-nlp
5656

5757
# Load Spark NLP with Spark Shell
58-
$ spark-shell --packages JohnSnowLabs:spark-nlp:2.0.5
58+
$ spark-shell --packages JohnSnowLabs:spark-nlp:2.0.6
5959

6060
# Load Spark NLP with PySpark
61-
$ pyspark --packages JohnSnowLabs:spark-nlp:2.0.5
61+
$ pyspark --packages JohnSnowLabs:spark-nlp:2.0.6
6262

6363
# Load Spark NLP with Spark Submit
64-
$ spark-submit --packages JohnSnowLabs:spark-nlp:2.0.5
64+
$ spark-submit --packages JohnSnowLabs:spark-nlp:2.0.6
6565

6666
# Load Spark NLP as external JAR after comiling and bulding Spark NLP by `sbt assembly`
67-
$ spark-shell --jar spark-nlp-assembly-2.0.5
67+
$ spark-shell --jar spark-nlp-assembly-2.0.6
6868
{% endhighlight %}
6969
</div>
7070
</div>

docs/en/install.md

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@ modify_date: "2019-05-16"
1313
If you installed pyspark through pip, you can install `spark-nlp` through pip as well.
1414

1515
```bash
16-
pip install spark-nlp==2.0.5
16+
pip install spark-nlp==2.0.6
1717
```
1818

1919
PyPI [spark-nlp package](https://pypi.org/project/spark-nlp/)
@@ -36,7 +36,7 @@ spark = SparkSession.builder \
3636
.master("local[*]")\
3737
.config("spark.driver.memory","8G")\
3838
.config("spark.driver.maxResultSize", "2G") \
39-
.config("spark.jars.packages", "JohnSnowLabs:spark-nlp:2.0.5")\
39+
.config("spark.jars.packages", "JohnSnowLabs:spark-nlp:2.0.6")\
4040
.config("spark.kryoserializer.buffer.max", "500m")\
4141
.getOrCreate()
4242
```
@@ -97,7 +97,7 @@ Our package is deployed to maven central. In order to add this package as a depe
9797
<dependency>
9898
<groupId>com.johnsnowlabs.nlp</groupId>
9999
<artifactId>spark-nlp_2.11</artifactId>
100-
<version>2.0.5</version>
100+
<version>2.0.6</version>
101101
</dependency>
102102
```
103103

@@ -108,22 +108,22 @@ and
108108
<dependency>
109109
<groupId>com.johnsnowlabs.nlp</groupId>
110110
<artifactId>spark-nlp-ocr_2.11</artifactId>
111-
<version>2.0.5</version>
111+
<version>2.0.6</version>
112112
</dependency>
113113
```
114114

115115
### SBT
116116

117117
```sbtshell
118118
// https://mvnrepository.com/artifact/com.johnsnowlabs.nlp/spark-nlp
119-
libraryDependencies += "com.johnsnowlabs.nlp" %% "spark-nlp" % "2.0.5"
119+
libraryDependencies += "com.johnsnowlabs.nlp" %% "spark-nlp" % "2.0.6"
120120
```
121121

122122
and
123123

124124
```sbtshell
125125
// https://mvnrepository.com/artifact/com.johnsnowlabs.nlp/spark-nlp-ocr
126-
libraryDependencies += "com.johnsnowlabs.nlp" %% "spark-nlp-ocr" % "2.0.5"
126+
libraryDependencies += "com.johnsnowlabs.nlp" %% "spark-nlp-ocr" % "2.0.6"
127127
```
128128

129129
Maven Central: [https://mvnrepository.com/artifact/com.johnsnowlabs.nlp](https://mvnrepository.com/artifact/com.johnsnowlabs.nlp)
@@ -151,7 +151,7 @@ Note: You can import these notebooks by using their URLs.
151151
4- From the Source drop-down menu, select **Maven Coordinate:**
152152
![Databricks](https://databricks.com/wp-content/uploads/2015/07/select-maven-1024x711.png)
153153

154-
5- Now, all available **Spark Packages** are at your fingertips! Just search for **JohnSnowLabs:spark-nlp:version** where **version** stands for the library version such as: `1.8.4` or `2.0.5`
154+
5- Now, all available **Spark Packages** are at your fingertips! Just search for **JohnSnowLabs:spark-nlp:version** where **version** stands for the library version such as: `1.8.4` or `2.0.6`
155155
![Databricks](https://databricks.com/wp-content/uploads/2015/07/browser-1024x548.png)
156156

157157
6- Select **spark-nlp** package and we are good to go!

docs/en/quickstart.md

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -29,17 +29,17 @@ Spark NLP is built on top of **Apache Spark 2.4.0** and such is the **only** sup
2929
To start using the library, execute any of the following lines depending on your desired use case:
3030

3131
```bash
32-
spark-shell --packages JohnSnowLabs:spark-nlp:2.0.5
33-
pyspark --packages JohnSnowLabs:spark-nlp:2.0.5
34-
spark-submit --packages JohnSnowLabs:spark-nlp:2.0.5
32+
spark-shell --packages JohnSnowLabs:spark-nlp:2.0.6
33+
pyspark --packages JohnSnowLabs:spark-nlp:2.0.6
34+
spark-submit --packages JohnSnowLabs:spark-nlp:2.0.6
3535
```
3636

3737
### **Straight forward Python on jupyter notebook**
3838

3939
Use pip to install (after you pip installed numpy and pyspark)
4040

4141
```bash
42-
pip install spark-nlp==2.0.5
42+
pip install spark-nlp==2.0.6
4343
jupyter notebook
4444
```
4545

@@ -60,7 +60,7 @@ spark = SparkSession.builder \
6060
.appName('OCR Eval') \
6161
.config("spark.driver.memory", "6g") \
6262
.config("spark.executor.memory", "6g") \
63-
.config("spark.jars.packages", "JohnSnowLabs:spark-nlp:2.0.5") \
63+
.config("spark.jars.packages", "JohnSnowLabs:spark-nlp:2.0.6") \
6464
.getOrCreate()
6565
```
6666

@@ -69,13 +69,13 @@ spark = SparkSession.builder \
6969
Add the following maven coordinates in the dependency configuration page:
7070

7171
```bash
72-
com.johnsnowlabs.nlp:spark-nlp_2.11:2.0.5
72+
com.johnsnowlabs.nlp:spark-nlp_2.11:2.0.6
7373
```
7474

7575
For Python in **Apache Zeppelin** you may need to setup _**SPARK_SUBMIT_OPTIONS**_ utilizing --packages instruction shown above like this
7676

7777
```bash
78-
export SPARK_SUBMIT_OPTIONS="--packages JohnSnowLabs:spark-nlp:2.0.5"
78+
export SPARK_SUBMIT_OPTIONS="--packages JohnSnowLabs:spark-nlp:2.0.6"
7979
```
8080

8181
### **Python Jupyter Notebook with PySpark**
@@ -85,7 +85,7 @@ export SPARK_HOME=/path/to/your/spark/folder
8585
export PYSPARK_DRIVER_PYTHON=jupyter
8686
export PYSPARK_DRIVER_PYTHON_OPTS=notebook
8787

88-
pyspark --packages JohnSnowLabs:spark-nlp:2.0.5
88+
pyspark --packages JohnSnowLabs:spark-nlp:2.0.6
8989
```
9090

9191
### S3 based standalone cluster (No Hadoop)
@@ -297,7 +297,7 @@ lightPipeline.annotate("Hello world, please annotate my text")
297297
Spark NLP OCR Module is not included within Spark NLP. It is not an annotator and not an extension to Spark ML. You can include it with the following coordinates for Maven:
298298

299299
```bash
300-
com.johnsnowlabs.nlp:spark-nlp-ocr_2.11:2.0.5
300+
com.johnsnowlabs.nlp:spark-nlp-ocr_2.11:2.0.6
301301
```
302302

303303
### Creating Spark datasets from PDF (To be used with Spark NLP)

python/setup.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -40,7 +40,7 @@
4040
# For a discussion on single-sourcing the version across setup.py and the
4141
# project code, see
4242
# https://packaging.python.org/en/latest/single_source_version.html
43-
version='2.0.5', # Required
43+
version='2.0.6', # Required
4444

4545
# This is a one-line description or tagline of what your project does. This
4646
# corresponds to the "Summary" metadata field:

python/sparknlp/__init__.py

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -40,14 +40,14 @@ def start(include_ocr=False):
4040

4141
if include_ocr:
4242
builder \
43-
.config("spark.jars.packages", "JohnSnowLabs:spark-nlp:2.0.5,com.johnsnowlabs.nlp:spark-nlp-ocr_2.11:2.0.5,javax.media.jai:com.springsource.javax.media.jai.core:1.1.3") \
43+
.config("spark.jars.packages", "JohnSnowLabs:spark-nlp:2.0.6,com.johnsnowlabs.nlp:spark-nlp-ocr_2.11:2.0.6,javax.media.jai:com.springsource.javax.media.jai.core:1.1.3") \
4444
.config("spark.jars.repositories", "http://repo.spring.io/plugins-release")
4545

4646
else:
47-
builder.config("spark.jars.packages", "JohnSnowLabs:spark-nlp:2.0.5") \
47+
builder.config("spark.jars.packages", "JohnSnowLabs:spark-nlp:2.0.6") \
4848

4949
return builder.getOrCreate()
5050

5151

5252
def version():
53-
print('2.0.5')
53+
print('2.0.6')

src/main/scala/com/johnsnowlabs/nlp/SparkNLP.scala

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -13,18 +13,18 @@ object SparkNLP {
1313

1414
if (includeOcr) {
1515
build
16-
.config("spark.jars.packages", "JohnSnowLabs:spark-nlp:2.0.5,com.johnsnowlabs.nlp:spark-nlp-ocr_2.11:2.0.5,javax.media.jai:com.springsource.javax.media.jai.core:1.1.3")
16+
.config("spark.jars.packages", "JohnSnowLabs:spark-nlp:2.0.6,com.johnsnowlabs.nlp:spark-nlp-ocr_2.11:2.0.6,javax.media.jai:com.springsource.javax.media.jai.core:1.1.3")
1717
.config("spark.jars.repositories", "http://repo.spring.io/plugins-release")
1818
} else {
1919
build
20-
.config("spark.jars.packages", "JohnSnowLabs:spark-nlp:2.0.5")
20+
.config("spark.jars.packages", "JohnSnowLabs:spark-nlp:2.0.6")
2121
}
2222

2323
build.getOrCreate()
2424
}
2525

2626
def version(): Unit = {
27-
println("2.0.5")
27+
println("2.0.6")
2828
}
2929

3030
}

0 commit comments

Comments
 (0)