Skip to content

Commit

Permalink
Merge pull request #18 from sproutsocial/upstream_fetch
Browse files Browse the repository at this point in the history
[DFZ-973] Upstream fetch
  • Loading branch information
ravisanwal authored Aug 6, 2024
2 parents 7f7a58c + 83baa97 commit 1e60f57
Show file tree
Hide file tree
Showing 107 changed files with 7,295 additions and 4,920 deletions.
139 changes: 121 additions & 18 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,22 +1,127 @@

<!---
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
-->
# HBASE Changelog

## Release hbase-connectors-1.0.1 - Unreleased (as of 2023-10-18)



### NEW FEATURES:

| JIRA | Summary | Priority | Component |
|:---- |:---- | :--- |:---- |
| [HBASE-28152](https://issues.apache.org/jira/browse/HBASE-28152) | Replace scala.util.parsing.json with org.json4s.jackson which used in Spark too | Major | spark |
| [HBASE-28137](https://issues.apache.org/jira/browse/HBASE-28137) | Add scala-parser-combinators dependency to connectors for Spark 3.4 | Major | spark |
| [HBASE-27397](https://issues.apache.org/jira/browse/HBASE-27397) | Spark-hbase support for 'startWith' predicate. | Minor | hbase-connectors |


### IMPROVEMENTS:

| JIRA | Summary | Priority | Component |
|:---- |:---- | :--- |:---- |
| [HBASE-27625](https://issues.apache.org/jira/browse/HBASE-27625) | Bump commons-lang3 to 3.12.0, surefire to 3.0.0 and dependency manage reload4j to 1.2.25 to support jdk11 | Major | hbase-connectors, spark |
| [HBASE-27705](https://issues.apache.org/jira/browse/HBASE-27705) | Respect SparkContext hadoop configuration | Major | spark |
| [HBASE-27639](https://issues.apache.org/jira/browse/HBASE-27639) | Support hbase-connectors compilation with HBase 2.5.3, Hadoop 3.2.4 and Spark 3.2.3 | Major | hbase-connectors, spark |
| [HBASE-26534](https://issues.apache.org/jira/browse/HBASE-26534) | Update dependencies in hbase-connectors: HBase version to 2.4.8, and make Hadoop 3 and Spark 3 defaults | Minor | hadoop3, hbase-connectors, spark |
| [HBASE-25684](https://issues.apache.org/jira/browse/HBASE-25684) | Dependency manage log4j in hbase-connectors | Minor | hbase-connectors |
| [HBASE-25326](https://issues.apache.org/jira/browse/HBASE-25326) | Allow hbase-connector to be used with Apache Spark 3.0 | Minor | . |
| [HBASE-23606](https://issues.apache.org/jira/browse/HBASE-23606) | Remove external deprecations in hbase-connectors | Minor | . |
| [HBASE-24230](https://issues.apache.org/jira/browse/HBASE-24230) | Support user-defined version timestamp when bulk load data | Minor | hbase-connectors |
| [HBASE-23608](https://issues.apache.org/jira/browse/HBASE-23608) | Remove redundant groupId from spark module in hbase-connectors | Trivial | . |
| [HBASE-23592](https://issues.apache.org/jira/browse/HBASE-23592) | Refactor tests in hbase-kafka-proxy in hbase-connectors | Trivial | . |
| [HBASE-24110](https://issues.apache.org/jira/browse/HBASE-24110) | Move to Apache parent POM version 23 for connectors | Minor | hbase-connectors |
| [HBASE-23603](https://issues.apache.org/jira/browse/HBASE-23603) | Update Apache POM to version 21 for hbase-connectors | Trivial | . |
| [HBASE-23607](https://issues.apache.org/jira/browse/HBASE-23607) | Update Maven plugins in hbase-connectors | Minor | . |
| [HBASE-23579](https://issues.apache.org/jira/browse/HBASE-23579) | Fix Checkstyle errors in hbase-connectors | Minor | . |
| [HBASE-23586](https://issues.apache.org/jira/browse/HBASE-23586) | Use StandardCharsets instead of String in TestQualifierMatching in hbase-connectors | Trivial | . |
| [HBASE-23580](https://issues.apache.org/jira/browse/HBASE-23580) | Refactor TestRouteRules in hbase-connectors | Trivial | . |
| [HBASE-23576](https://issues.apache.org/jira/browse/HBASE-23576) | Bump Checkstyle from 8.11 to 8.18 in hbase-connectors | Minor | . |
| [HBASE-22817](https://issues.apache.org/jira/browse/HBASE-22817) | Use hbase-shaded dependencies in hbase-spark | Major | hbase-connectors |
| [HBASE-23075](https://issues.apache.org/jira/browse/HBASE-23075) | Upgrade jackson to version 2.9.10 due to CVE-2019-16335 and CVE-2019-14540 | Major | dependencies, hbase-connectors, REST, security |


### BUG FIXES:

| JIRA | Summary | Priority | Component |
|:---- |:---- | :--- |:---- |
| [HBASE-26863](https://issues.apache.org/jira/browse/HBASE-26863) | Rowkey pushdown does not work with complex conditions | Major | hbase-connectors |
| [HBASE-27488](https://issues.apache.org/jira/browse/HBASE-27488) | [hbase-connectors] Duplicate result when searching HBase by Spark | Major | hbase-connectors |
| [HBASE-27656](https://issues.apache.org/jira/browse/HBASE-27656) | Make sure the close method of the SmartSonnection is called | Major | spark |
| [HBASE-27176](https://issues.apache.org/jira/browse/HBASE-27176) | [hbase-connectors] Fail to build hbase-connectors because of checkstyle error | Major | build, hbase-connectors |
| [HBASE-27801](https://issues.apache.org/jira/browse/HBASE-27801) | Remove redundant avro.version property from Kafka connector | Minor | hbase-connectors, kafka |
| [HBASE-27630](https://issues.apache.org/jira/browse/HBASE-27630) | hbase-spark bulkload stage directory limited to hdfs only | Major | spark |
| [HBASE-27624](https://issues.apache.org/jira/browse/HBASE-27624) | Cannot Specify Namespace via the hbase.table Option in Spark Connector | Major | hbase-connectors, spark |
| [HBASE-22338](https://issues.apache.org/jira/browse/HBASE-22338) | LICENSE file only contains Apache 2.0 | Critical | hbase-connectors |
| [HBASE-26211](https://issues.apache.org/jira/browse/HBASE-26211) | [hbase-connectors] Pushdown filters in Spark do not work correctly with long types | Major | hbase-connectors |
| [HBASE-25236](https://issues.apache.org/jira/browse/HBASE-25236) | [hbase-connectors] Run package phase on spark modules | Major | hbase-connectors |
| [HBASE-24276](https://issues.apache.org/jira/browse/HBASE-24276) | hbase spark connector doesn't support writing to table not in default namespace | Major | hbase-connectors, spark |
| [HBASE-24088](https://issues.apache.org/jira/browse/HBASE-24088) | Solve the ambiguous reference for scala 2.12 | Minor | hbase-connectors |
| [HBASE-23295](https://issues.apache.org/jira/browse/HBASE-23295) | hbase-connectors HBaseContext should use most recent delegation token | Major | hbase-connectors |
| [HBASE-23351](https://issues.apache.org/jira/browse/HBASE-23351) | Updating hbase version to 2.2.2 | Major | hbase-connectors |
| [HBASE-23348](https://issues.apache.org/jira/browse/HBASE-23348) | Spark's createTable method throws an exception while the table is being split | Major | hbase-connectors |
| [HBASE-23346](https://issues.apache.org/jira/browse/HBASE-23346) | Import ReturnCode in SparkSQLPushDownFilter | Major | hbase-connectors |
| [HBASE-23327](https://issues.apache.org/jira/browse/HBASE-23327) | Add missing maven functions for hb\_maven | Critical | hbase-connectors |
| [HBASE-23059](https://issues.apache.org/jira/browse/HBASE-23059) | Run mvn install for root in precommit | Major | hbase-connectors |
| [HBASE-22711](https://issues.apache.org/jira/browse/HBASE-22711) | Spark connector doesn't use the given mapping when inserting data | Major | hbase-connectors |
| [HBASE-22674](https://issues.apache.org/jira/browse/HBASE-22674) | precommit docker image installs JRE over JDK (multiple repos) | Critical | build, hbase-connectors |
| [HBASE-22336](https://issues.apache.org/jira/browse/HBASE-22336) | Add CHANGELOG, README and RELEASENOTES to binary tarball | Critical | hbase-connectors |
| [HBASE-22329](https://issues.apache.org/jira/browse/HBASE-22329) | Fix for warning The parameter forkMode is deprecated since version in hbase-spark-it | Minor | hbase-connectors |
| [HBASE-22320](https://issues.apache.org/jira/browse/HBASE-22320) | hbase-connectors personality skips non-scaladoc tests | Critical | . |
| [HBASE-22319](https://issues.apache.org/jira/browse/HBASE-22319) | Fix for warning The assembly descriptor contains a filesystem-root relative reference | Minor | hbase-connectors |


### SUB-TASKS:

| JIRA | Summary | Priority | Component |
|:---- |:---- | :--- |:---- |
| [HBASE-28054](https://issues.apache.org/jira/browse/HBASE-28054) | [hbase-connectors] Add spotless in hbase-connectors pre commit check | Major | build, community, hbase-connectors, jenkins |
| [HBASE-28006](https://issues.apache.org/jira/browse/HBASE-28006) | [hbase-connectors] Run spotless:apply on code base | Major | build, hbase-connectors |
| [HBASE-27178](https://issues.apache.org/jira/browse/HBASE-27178) | [hbase-connectors] Add spotless plugin to format code (including scala code) | Major | build, hbase-connectors |
| [HBASE-25136](https://issues.apache.org/jira/browse/HBASE-25136) | Migrate HBase-Connectors-PreCommit jenkins job from Hadoop to hbase | Major | hbase-connectors, jenkins |


### OTHER:

| JIRA | Summary | Priority | Component |
|:---- |:---- | :--- |:---- |
| [HBASE-27114](https://issues.apache.org/jira/browse/HBASE-27114) | Upgrade scalatest maven plugin for thread-safety | Major | build, spark |
| [HBASE-27883](https://issues.apache.org/jira/browse/HBASE-27883) | [hbase-connectors] Use log4j2 instead of log4j for logging | Blocker | hbase-connectors |
| [HBASE-27679](https://issues.apache.org/jira/browse/HBASE-27679) | Bump junit to 4.13.2 in hbase-connectors | Major | hbase-connectors |
| [HBASE-27680](https://issues.apache.org/jira/browse/HBASE-27680) | Bump hbase, hbase-thirdparty, hadoop and spark for hbase-connectors | Major | hbase-connectors |
| [HBASE-27678](https://issues.apache.org/jira/browse/HBASE-27678) | Update checkstyle in hbase-connectors | Major | hbase-connectors |
| [HBASE-27285](https://issues.apache.org/jira/browse/HBASE-27285) | Fix sonar report paths | Minor | hbase-connectors |
| [HBASE-27272](https://issues.apache.org/jira/browse/HBASE-27272) | Enable code coverage reporting to SonarQube in hbase-connectors | Minor | hbase-connectors |
| [HBASE-26664](https://issues.apache.org/jira/browse/HBASE-26664) | HBASE-26664 hbase-connector upgrades extra-enforcer-rules to 1.5.1 | Major | hbase-connectors |
| [HBASE-26334](https://issues.apache.org/jira/browse/HBASE-26334) | Upgrade commons-io to 2.11.0 in hbase-connectors | Major | hbase-connectors |
| [HBASE-26314](https://issues.apache.org/jira/browse/HBASE-26314) | Upgrade commons-io to 2.8.0 in hbase-connectors | Major | hbase-connectors |
| [HBASE-25579](https://issues.apache.org/jira/browse/HBASE-25579) | HBase Connectors pom should include nexus staging repo management | Major | community, hbase-connectors |
| [HBASE-25479](https://issues.apache.org/jira/browse/HBASE-25479) | [connectors] Purge use of VisibleForTesting | Major | hbase-connectors |
| [HBASE-25388](https://issues.apache.org/jira/browse/HBASE-25388) | Replacing Producer implementation with an extension of MockProducer on testing side in hbase-connectors | Major | hbase-connectors |
| [HBASE-24883](https://issues.apache.org/jira/browse/HBASE-24883) | Migrate hbase-connectors testing to ci-hadoop | Major | build, hbase-connectors |
| [HBASE-24261](https://issues.apache.org/jira/browse/HBASE-24261) | Redo all of our github notification integrations on new ASF infra feature | Major | community, hbase-connectors |
| [HBASE-23565](https://issues.apache.org/jira/browse/HBASE-23565) | Execute tests in hbase-connectors precommit | Critical | hbase-connectors |
| [HBASE-23032](https://issues.apache.org/jira/browse/HBASE-23032) | Upgrade to Curator 4.2.0 | Major | . |
| [HBASE-22599](https://issues.apache.org/jira/browse/HBASE-22599) | Let hbase-connectors compile against HBase 2.2.0 | Major | hbase-connectors |
| [HBASE-22698](https://issues.apache.org/jira/browse/HBASE-22698) | [hbase-connectors] Add license header to README.md | Major | hbase-connectors |
| [HBASE-22636](https://issues.apache.org/jira/browse/HBASE-22636) | hbase spark module README is in txt format. | Trivial | hbase-connectors |
| [HBASE-22318](https://issues.apache.org/jira/browse/HBASE-22318) | Fix for warning The POM for org.glassfish:javax.el:jar is missing | Minor | hbase-connectors |

# HBase Changelog

## Release connector-1.0.0 - Unreleased (as of 2019-04-26)

Expand Down Expand Up @@ -121,5 +226,3 @@ limitations under the License.
| [HBASE-22210](https://issues.apache.org/jira/browse/HBASE-22210) | Fix hbase-connectors-assembly to include every jar | Major | hbase-connectors |
| [HBASE-22266](https://issues.apache.org/jira/browse/HBASE-22266) | Add yetus personality to connectors to avoid scaladoc issues | Major | hbase-connectors |
| [HBASE-22257](https://issues.apache.org/jira/browse/HBASE-22257) | Remove json4s and jackson dependency from hbase spark connector | Major | hbase-connectors |


2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ limitations under the License.

# hbase-connectors

Connectors for [Apache HBase&trade;](https://hbase.apache.org)
Connectors for [Apache HBase&trade;](https://hbase.apache.org)

* [Kafka Proxy](https://github.com/apache/hbase-connectors/tree/master/kafka)
* [Spark](https://github.com/apache/hbase-connectors/tree/master/spark)
73 changes: 57 additions & 16 deletions RELEASENOTES.md
Original file line number Diff line number Diff line change
@@ -1,20 +1,62 @@
# RELEASENOTES
<!---
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
-->
# HBASE hbase-connectors-1.0.1 Release Notes

These release notes cover new developer and user-facing incompatibilities, important issues, features, and major improvements.


---

* [HBASE-26534](https://issues.apache.org/jira/browse/HBASE-26534) | *Minor* | **Update dependencies in hbase-connectors: HBase version to 2.4.8, and make Hadoop 3 and Spark 3 defaults**

HBASE-26534 upgrades hbase-thirdparty to 4.0.1, hbase to 2.4.9, spark to 3.1.2, and hadoop to 3.2.0. Also It builds with spark3 with scala-2.12 and hadoop3 profile as default option.


---

* [HBASE-26334](https://issues.apache.org/jira/browse/HBASE-26334) | *Major* | **Upgrade commons-io to 2.11.0 in hbase-connectors**

Upgraded commons-io to 2.11.0.


---

* [HBASE-26314](https://issues.apache.org/jira/browse/HBASE-26314) | *Major* | **Upgrade commons-io to 2.8.0 in hbase-connectors**

Upgraded commons-io to 2.8.0.


---

* [HBASE-23576](https://issues.apache.org/jira/browse/HBASE-23576) | *Minor* | **Bump Checkstyle from 8.11 to 8.18 in hbase-connectors**

Bumped the Checkstyle version from 8.11 to 8.18


---

* [HBASE-22817](https://issues.apache.org/jira/browse/HBASE-22817) | *Major* | **Use hbase-shaded dependencies in hbase-spark**

<!-- markdown -->
The HBase connector for working with Apache Spark now works with the shaded client artifacts provided by the Apache HBase project and avoids adding additional third party dependencies to the classpath.



# HBase connector-1.0.0 Release Notes

Expand Down Expand Up @@ -150,4 +192,3 @@ New features in hbase-spark:
\* support for Dataframe writes,
\* avro support,
\* catalog can be defined in json.

54 changes: 54 additions & 0 deletions dev-support/.scalafmt.conf
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#

# Template based off apache spark: https://github.com/apache/spark/blob/master/dev/.scalafmt.conf
# Align settings
align = none
align.openParenDefnSite = false
align.openParenCallSite = false
align.tokens = []

# Rewrites
rewrite.rules = [Imports]

# Imports
rewrite.imports.sort = scalastyle
rewrite.imports.groups = [
[".*"],
["org.apache.hbase.thirdparty\\..*"],
["org.apache.hadoop.hbase.shaded\\..*"]
]
rewrite.imports.contiguousGroups = no
importSelectors = "singleLine"

# Newlines
newlines.beforeCurlyLambdaParams = multiline
newlines.afterCurlyLambdaParams = squash
danglingParentheses.preset = false
optIn.configStyleArguments = false

# Scaladoc
docstrings.style = Asterisk
# See https://github.com/scalameta/scalafmt/issues/1387
docstrings.wrap = no

# Max column
maxColumn = 100

# Version
runner.dialect = scala212
version = 3.7.12
Loading

0 comments on commit 1e60f57

Please sign in to comment.