Skip to content

Commit 74be84e

Browse files
committed
Update version to 6.3.0 and supporting docs for release.
1 parent e9e92a3 commit 74be84e

File tree

3 files changed

+15
-6
lines changed

3 files changed

+15
-6
lines changed

CHANGELOG

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,15 @@
1616

1717
# spark-redshift Changelog
1818

19+
## 6.3.0 (2024-07-03)
20+
- Validates connector tests pass with Spark releases 3.4.3 and 3.5.1 [Beaux Sharifi]
21+
- Adds support for three-part table names to allow connector to query Redshift data sharing tables (#153) [Prashant Singh, Beaux Sharifi]
22+
- Corrects mapping of Spark ShortType to use Redshift SMALLINT instead of INTEGER to better match expected data size (#152) [Akira Ajisaka, Ruei Huang]
23+
- Adds pushdown of toprettystring() function to support df.show() operations used by Spark 3.5 [Beaux Sharifi]
24+
- Corrects pushdown of inner joins with no join condition to result in a cross join instead of an invalid inner join [Beaux Sharifi]
25+
- Corrects pushdown when casting null boolean values to strings to result in the value null instead of the string "null". [Beaux Sharifi]
26+
- Upgrades Redshift JDBC driver to the latest available version 2.1.0.29. [Beaux Sharifi]
27+
1928
## 6.2.0 (2024-01-12)
2029
- Validates support for Spark 3.3.4 and Spark 3.4.2
2130
- Upgrades Redshift JDBC driver to version 2.1.0.24

README.md

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -103,7 +103,7 @@ You may use this library in your applications with the following dependency info
103103
spark-submit \
104104
--deploy-mode cluster \
105105
--master yarn \
106-
--packages com.amazon.redshift:redshift-jdbc42:2.1.0.29,org.apache.spark:spark-avro_2.12:3.5.1,io.github.spark-redshift-community:spark-redshift_2.12:6.2.0-spark_3.5 \
106+
--packages com.amazon.redshift:redshift-jdbc42:2.1.0.29,org.apache.spark:spark-avro_2.12:3.5.1,io.github.spark-redshift-community:spark-redshift_2.12:6.3.0-spark_3.5 \
107107
my_script.py
108108
```
109109

@@ -114,14 +114,14 @@ You may use this library in your applications with the following dependency info
114114
<dependency>
115115
<groupId>io.github.spark-redshift-community</groupId>
116116
<artifactId>spark-redshift_2.12</artifactId>
117-
<version>6.2.0-spark_3.5</version>
117+
<version>6.3.0-spark_3.5</version>
118118
</dependency>
119119
```
120120

121121
- **In SBT**:
122122

123123
```SBT
124-
libraryDependencies += "io.github.spark-redshift-community" %% "spark-redshift_2.12" % "6.2.0-spark_3.5"
124+
libraryDependencies += "io.github.spark-redshift-community" %% "spark-redshift_2.12" % "6.3.0-spark_3.5"
125125
```
126126

127127

@@ -816,7 +816,7 @@ for more information.</p>
816816
<td>""</td>
817817
<td>
818818
An identifier to include in the query group set when running queries with the connector. Should be 100 or fewer characters and all characters must be valid unicodeIdentifierParts. Characters in excess of 100 will be trimmed.
819-
When running a query with the connector a json formatted string will be set as the query group (for example `{"spark-redshift-connector":{"svc":"","ver":"6.2.0-spark_3.5","op":"Read","lbl":"","tid":""}}`).
819+
When running a query with the connector a json formatted string will be set as the query group (for example `{"spark-redshift-connector":{"svc":"","ver":"6.3.0-spark_3.5","op":"Read","lbl":"","tid":""}}`).
820820
This option will be substituted for the value of the `lbl` key.
821821
</td>
822822
</tr></table>
@@ -924,7 +924,7 @@ SET spark.datasource.redshift.community.autopushdown.lazyMode=false
924924
### trace_id
925925
A new tracing identifier field that is added to the existing `label` parameter. When set, the provided string value will be used as part of label. Otherwise, it will default to the Spark application identifier. For example:
926926
927-
`{"spark-redshift-connector":{"svc":"","ver":"6.2.0-spark_3.5","op":"Read","lbl":"","tid":"..."}}`)
927+
`{"spark-redshift-connector":{"svc":"","ver":"6.3.0-spark_3.5","op":"Read","lbl":"","tid":"..."}}`)
928928
929929
To set the value, run the following command:
930930
```sparksql

version.sbt

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,4 +14,4 @@
1414
* limitations under the License.
1515
*/
1616

17-
ThisBuild / version := "6.2.0"
17+
ThisBuild / version := "6.3.0"

0 commit comments

Comments
 (0)