Skip to content

Spark NLP 4.2.8: Patch release
Compare
Choose a tag to compare
@maziyarpanahi maziyarpanahi released this 24 Jan 18:22
· 955 commits to master since this release
4.2.8
04b5ac3

📢 Overview

Spark NLP 4.2.8 🚀 comes with some important bug fixes and improvements. As a result, we highly recommend to update to this latest version if you are using Spark NLP 4.2.x.

As always, we would like to thank our community for their feedback, questions, and feature requests. 🎉

⭐ 🐛 Bug Fixes & Improvements

  • Fix the issue with optional keys (labels) in metadata when using XXXForSequenceClassitication annotators. This fixes Some(neg) -> 0.13602075 as neg -> 0.13602075 to be in harmony with all the other classifiers. #13396

before 4.2.8:

+-----------------------------------------------------------------------------------------------+
|label                                                                                          |
+-----------------------------------------------------------------------------------------------+
|[{category, 0, 87, pos, {sentence -> 0, Some(neg) -> 0.13602075, Some(pos) -> 0.8639792}, []}] |
|[{category, 0, 47, neg, {sentence -> 0, Some(neg) -> 0.7505674, Some(pos) -> 0.24943262}, []}] |
|[{category, 0, 17, pos, {sentence -> 0, Some(neg) -> 0.31065974, Some(pos) -> 0.6893403}, []}] |
|[{category, 0, 71, neg, {sentence -> 0, Some(neg) -> 0.5079189, Some(pos) -> 0.4920811}, []}]  |
+-----------------------------------------------------------------------------------------------+

after 4.2.8:

+-----------------------------------------------------------------------------------+
|label                                                                              |
+-----------------------------------------------------------------------------------+
|[{category, 0, 87, pos, {sentence -> 0, neg -> 0.13602075, pos -> 0.8639792}, []}] |
|[{category, 0, 47, neg, {sentence -> 0, neg -> 0.7505674, pos -> 0.24943262}, []}] |
|[{category, 0, 17, pos, {sentence -> 0, neg -> 0.31065974, pos -> 0.6893403}, []}] |
|[{category, 0, 71, neg, {sentence -> 0, neg -> 0.5079189, pos -> 0.4920811}, []}]  |
+-----------------------------------------------------------------------------------+
  • Introducing a config to skip LightPipeline validation for inputCols on the Python side for projects depending on Spark NLP. This toggle should only be used for specific annotators that do not follow the convention of predefined inputAnnotatorTypes and outputAnnotatorType #13402

📖 Documentation

Installation

Python

#PyPI

pip install spark-nlp==4.2.8

Spark Packages

spark-nlp on Apache Spark 3.0.x, 3.1.x, 3.2.x, and 3.3.x (Scala 2.12):

spark-shell --packages com.johnsnowlabs.nlp:spark-nlp_2.12:4.2.8

pyspark --packages com.johnsnowlabs.nlp:spark-nlp_2.12:4.2.8

GPU

spark-shell --packages com.johnsnowlabs.nlp:spark-nlp-gpu_2.12:4.2.8

pyspark --packages com.johnsnowlabs.nlp:spark-nlp-gpu_2.12:4.2.8

M1

spark-shell --packages com.johnsnowlabs.nlp:spark-nlp-m1_2.12:4.2.8

pyspark --packages com.johnsnowlabs.nlp:spark-nlp-m1_2.12:4.2.8

AArch64

spark-shell --packages com.johnsnowlabs.nlp:spark-nlp-aarch64_2.12:4.2.8

pyspark --packages com.johnsnowlabs.nlp:spark-nlp-aarch64_2.12:4.2.8

Maven

spark-nlp on Apache Spark 3.0.x, 3.1.x, 3.2.x, and 3.3.x:

<dependency>
    <groupId>com.johnsnowlabs.nlp</groupId>
    <artifactId>spark-nlp_2.12</artifactId>
    <version>4.2.8</version>
</dependency>

spark-nlp-gpu:

<dependency>
    <groupId>com.johnsnowlabs.nlp</groupId>
    <artifactId>spark-nlp-gpu_2.12</artifactId>
    <version>4.2.8</version>
</dependency>

spark-nlp-m1:

<dependency>
    <groupId>com.johnsnowlabs.nlp</groupId>
    <artifactId>spark-nlp-m1_2.12</artifactId>
    <version>4.2.8</version>
</dependency>

spark-nlp-aarch64:

<!-- https://mvnrepository.com/artifact/com.johnsnowlabs.nlp/spark-nlp-aarch64 -->
<dependency>
    <groupId>com.johnsnowlabs.nlp</groupId>
    <artifactId>spark-nlp-aarch64_2.12</artifactId>
    <version>4.2.8</version>
</dependency>

FAT JARs

What's Changed

Full Changelog: 4.2.7...4.2.8

Contributors

pabla, albertoandreottiATgmail, and 9 other contributors
Assets 2
Loading
0 Join discussion