Skip to content

Commit

Permalink
Add GraalVM Reachability Metadata and corresponding nativeTest for Cl…
Browse files Browse the repository at this point in the history
…ickHouse integration over http
  • Loading branch information
linghengqian committed May 30, 2024
1 parent aa5ea90 commit e854409
Show file tree
Hide file tree
Showing 28 changed files with 816 additions and 180 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -258,6 +258,36 @@ Caused by: java.io.UnsupportedEncodingException: Codepage Cp1252 is not supporte
</project>
```

6. 当需要通过 ShardingSphere JDBC 使用 ClickHouse 方言时,
用户需要手动引入相关的可选模块和 classifier 为 `http` 的 ClickHouse JDBC 驱动。
原则上,ShardingSphere 的 GraalVM Native Image 集成不希望使用 classifier 为 `all` 的 `com.clickhouse:clickhouse-jdbc`,
因为 Uber Jar 会导致采集重复的 GraalVM Reachability Metadata。
可能的配置例子如下,
```xml
<project>
<dependencies>
<dependency>
<groupId>org.apache.shardingsphere</groupId>
<artifactId>shardingsphere-jdbc</artifactId>
<version>${shardingsphere.version}</version>
</dependency>
<dependency>
<groupId>org.apache.shardingsphere</groupId>
<artifactId>shardingsphere-parser-sql-clickhouse</artifactId>
<version>${shardingsphere.version}</version>
</dependency>
<dependency>
<groupId>com.clickhouse</groupId>
<artifactId>clickhouse-jdbc</artifactId>
<version>0.6.0-patch5</version>
<classifier>http</classifier>
</dependency>
</dependencies>
</project>
```

ClickHouse 不支持 ShardingSphere 集成级别的本地事务,XA 事务和 Seata AT 模式事务,更多讨论位于 https://github.com/ClickHouse/clickhouse-docs/issues/2300 。

## 贡献 GraalVM Reachability Metadata

ShardingSphere 对在 GraalVM Native Image 下的可用性的验证,是通过 GraalVM Native Build Tools 的 Maven Plugin 子项目来完成的。
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -269,6 +269,36 @@ Possible configuration examples are as follows,
</project>
```

6. When using the ClickHouse dialect through ShardingSphere JDBC,
users need to manually introduce the relevant optional modules and the ClickHouse JDBC driver with the classifier `http`.
In principle, ShardingSphere's GraalVM Native Image integration does not want to use `com.clickhouse:clickhouse-jdbc` with classifier `all`,
because Uber Jar will cause the collection of duplicate GraalVM Reachability Metadata.
Possible configuration examples are as follows,
```xml
<project>
<dependencies>
<dependency>
<groupId>org.apache.shardingsphere</groupId>
<artifactId>shardingsphere-jdbc</artifactId>
<version>${shardingsphere.version}</version>
</dependency>
<dependency>
<groupId>org.apache.shardingsphere</groupId>
<artifactId>shardingsphere-parser-sql-clickhouse</artifactId>
<version>${shardingsphere.version}</version>
</dependency>
<dependency>
<groupId>com.clickhouse</groupId>
<artifactId>clickhouse-jdbc</artifactId>
<version>0.6.0-patch5</version>
<classifier>http</classifier>
</dependency>
</dependencies>
</project>
```
ClickHouse does not support local transactions, XA transactions, and Seata AT mode transactions at the ShardingSphere integration level.
More discussion is at https://github.com/ClickHouse/clickhouse-docs/issues/2300 .

## Contribute GraalVM Reachability Metadata

The verification of ShardingSphere's availability under GraalVM Native Image is completed through the Maven Plugin subproject
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,6 @@ ShardingSphere 默认情况下仅包含核心 SPI 的实现,在 Git Source 存
- `org.apache.shardingsphere:shardingsphere-sql-parser-opengauss`, SQL 解析的 OpenGauss 方言实现
- `org.apache.shardingsphere:shardingsphere-sql-parser-oracle`, SQL 解析的 Oracle 方言解析实现
- `org.apache.shardingsphere:shardingsphere-sql-parser-sqlserver`, SQL 解析的 SQL Server 方言实现
- `org.apache.shardingsphere:shardingsphere-sql-parser-clickhouse`, SQL 解析的 ClickHouse 方言实现
- `org.apache.shardingsphere:shardingsphere-sql-parser-doris`, SQL 解析的 Doris 方言实现
- `org.apache.shardingsphere:shardingsphere-sql-parser-hive`, SQL 解析的 Hive 方言实现
- `org.apache.shardingsphere:shardingsphere-sql-parser-presto`, SQL 解析的 Presto 方言实现
Expand All @@ -44,8 +43,6 @@ ShardingSphere 默认情况下仅包含核心 SPI 的实现,在 Git Source 存

下以 `groupId:artifactId` 的表现形式列出所有的可选插件。

- 单机模式配置信息持久化定义
- `org.apache.shardingsphere:shardingsphere-standalone-mode-repository-jdbc-mysql`,基于 MySQL 的持久化
- 集群模式配置信息持久化定义
- `org.apache.shardingsphere:shardingsphere-cluster-mode-repository-zookeeper`,基于 Zookeeper 的持久化实现
- `org.apache.shardingsphere:shardingsphere-cluster-mode-repository-etcd`,基于 Etcd 的持久化实现
Expand All @@ -55,5 +52,7 @@ ShardingSphere 默认情况下仅包含核心 SPI 的实现,在 Git Source 存
- `org.apache.shardingsphere:shardingsphere-infra-expr-espresso`,基于 GraalVM Truffle 的 Espresso 实现的使用 Groovy 语法的行表达式
- 数据库类型识别
- `org.apache.shardingsphere:shardingsphere-infra-database-testcontainers`, 对 `testcontainers-java``JDBC support` 的 jdbcURL 的识别适配
- SQL 解析
- `org.apache.shardingsphere:shardingsphere-sql-parser-clickhouse`, SQL 解析的 ClickHouse 方言实现

除了以上可选插件外,ShardingSphere 社区开发者还贡献了大量的插件实现,可以在 [ShardingSphere Plugin](https://github.com/apache/shardingsphere-plugin) 仓库中查看插件的使用说明,ShardingSphere Plugin 仓库中的插件会和 ShardingSphere 保持相同的发布节奏,可以在 https://central.sonatype.com/ 进行检索,并安装到 ShardingSphere 中。
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,6 @@ All the built-in plugins for ShardingSphere-JDBC are listed below in the form of
- `org.apache.shardingsphere:shardingsphere-sql-parser-opengauss`, OpenGauss dialect implementation of SQL parsing
- `org.apache.shardingsphere:shardingsphere-sql-parser-oracle`, Oracle dialect implementation of SQL parsing
- `org.apache.shardingsphere:shardingsphere-sql-parser-sqlserver`, SQL Server dialect implementation of SQL parsing
- `org.apache.shardingsphere:shardingsphere-sql-parser-clickhouse`, ClickHouse dialect implementation of SQL parsing
- `org.apache.shardingsphere:shardingsphere-sql-parser-doris`, Doris dialect implementation of SQL parsing
- `org.apache.shardingsphere:shardingsphere-sql-parser-hive`, Hive dialect implementation of SQL parsing
- `org.apache.shardingsphere:shardingsphere-sql-parser-presto`, Presto dialect implementation of SQL parsing
Expand All @@ -44,8 +43,6 @@ If ShardingSphere-JDBC needs to use optional plugins, you need to download the J

All optional plugins are listed below in the form of `groupId:artifactId`.

- Standalone mode configuration information persistence definition
- `org.apache.shardingsphere:shardingsphere-standalone-mode-repository-jdbc-mysql`, MySQL based persistence
- Cluster mode configuration information persistence definition
- `org.apache.shardingsphere:shardingsphere-cluster-mode-repository-zookeeper`, Zookeeper based persistence
- `org.apache.shardingsphere:shardingsphere-cluster-mode-repository-etcd`, Etcd based persistence
Expand All @@ -55,5 +52,7 @@ All optional plugins are listed below in the form of `groupId:artifactId`.
- `org.apache.shardingsphere:shardingsphere-infra-expr-espresso`,Row Value Expressions that uses the Groovy syntax based on GraalVM Truffle's Espresso implementation
- Database type identification
- `org.apache.shardingsphere:shardingsphere-infra-database-testcontainers`, Adaptation of jdbcURL for `JDBC support` of `testcontainers-java`
- SQL parsing
- `org.apache.shardingsphere:shardingsphere-sql-parser-clickhouse`, ClickHouse dialect implementation of SQL parsing

In addition to the above optional plugins, ShardingSphere community developers have contributed a number of plugin implementations. These plugins can be found in [ShardingSphere Plugins] (https://github.com/apache/shardingsphere-plugin) repository. Plugins in ShardingSphere Plugin repository would remain the same release plan with ShardingSphere, they can be retrieved at https://central.sonatype.com/, and install into ShardingSphere.
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package org.apache.shardingsphere.infra.database.clickhouse.connector;

import org.apache.shardingsphere.infra.database.core.connector.ConnectionProperties;
import org.apache.shardingsphere.infra.database.core.connector.ConnectionPropertiesParser;
import org.apache.shardingsphere.infra.database.core.connector.StandardConnectionProperties;
import org.apache.shardingsphere.infra.database.core.connector.url.JdbcUrl;
import org.apache.shardingsphere.infra.database.core.connector.url.StandardJdbcUrlParser;

import java.util.Properties;

/**
* Connection properties parser of ClickHouse.
*/
public final class ClickHouseConnectionPropertiesParser implements ConnectionPropertiesParser {

private static final int DEFAULT_PORT = 8123;

@Override
public ConnectionProperties parse(final String url, final String username, final String catalog) {
JdbcUrl jdbcUrl = new StandardJdbcUrlParser().parse(url);
return new StandardConnectionProperties(jdbcUrl.getHostname(), jdbcUrl.getPort(DEFAULT_PORT), jdbcUrl.getDatabase(), null, jdbcUrl.getQueryProperties(), new Properties());
}

@Override
public String getDatabaseType() {
return "ClickHouse";
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package org.apache.shardingsphere.infra.database.clickhouse.metadata.database;

import org.apache.shardingsphere.infra.database.core.metadata.database.DialectDatabaseMetaData;
import org.apache.shardingsphere.infra.database.core.metadata.database.enums.NullsOrderType;
import org.apache.shardingsphere.infra.database.core.metadata.database.enums.QuoteCharacter;

/**
* Database meta data of ClickHouse.
*/
public final class ClickHouseDatabaseMetaData implements DialectDatabaseMetaData {

@Override
public QuoteCharacter getQuoteCharacter() {
return QuoteCharacter.QUOTE;
}

@Override
public NullsOrderType getDefaultNullsOrderType() {
return NullsOrderType.FIRST;
}

@Override
public String getDatabaseType() {
return "ClickHouse";
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -18,15 +18,12 @@
package org.apache.shardingsphere.infra.database.clickhouse.type;

import org.apache.shardingsphere.infra.database.core.type.DatabaseType;
import org.apache.shardingsphere.infra.spi.type.typed.TypedSPILoader;

import java.util.Arrays;
import java.util.Collection;
import java.util.Optional;

/**
* Database type of ClickHouse.
* ClickHouse currently uses MySQL dialect parsing.
*/
public final class ClickHouseDatabaseType implements DatabaseType {

Expand All @@ -35,11 +32,6 @@ public Collection<String> getJdbcUrlPrefixes() {
return Arrays.asList("jdbc:ch:", "jdbc:clickhouse:");
}

@Override
public Optional<DatabaseType> getTrunkDatabaseType() {
return Optional.of(TypedSPILoader.getService(DatabaseType.class, "MySQL"));
}

@Override
public String getType() {
return "ClickHouse";
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#

org.apache.shardingsphere.infra.database.clickhouse.connector.ClickHouseConnectionPropertiesParser
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#

org.apache.shardingsphere.infra.database.clickhouse.metadata.database.ClickHouseDatabaseMetaData
Original file line number Diff line number Diff line change
Expand Up @@ -20,18 +20,23 @@
import org.apache.shardingsphere.infra.database.core.type.DatabaseType;
import org.apache.shardingsphere.infra.spi.type.typed.TypedSPILoader;

import java.util.Arrays;
import java.util.Collection;
import java.util.Collections;
import java.util.Optional;

/**
* Database type of ClickHouse in testcontainers-java.
*/
public final class TcClickHouseDatabaseType implements TestcontainersDatabaseType {

/**
* TODO See the JavaDoc for `org.apache.shardingsphere.test.natived.jdbc.commons.testcontainers.ClickHouseProvider`.
*
* @return prefixes of JDBC URL
*/
@Override
public Collection<String> getJdbcUrlPrefixes() {
return Collections.singleton("jdbc:tc:clickhouse:");
return Arrays.asList("jdbc:tc:clickhouse:", "jdbc:tc:shardingsphere0clickhouse:");
}

@Override
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@
import org.apache.shardingsphere.infra.spi.type.typed.TypedSPILoader;
import org.junit.jupiter.api.Test;

import java.util.Arrays;
import java.util.Collections;

import static org.hamcrest.CoreMatchers.is;
Expand All @@ -30,7 +31,7 @@ class TestcontainersDatabaseTypeTest {

@Test
void assertGetJdbcUrlPrefixes() {
assertThat(TypedSPILoader.getService(DatabaseType.class, "TC-ClickHouse").getJdbcUrlPrefixes(), is(Collections.singleton("jdbc:tc:clickhouse:")));
assertThat(TypedSPILoader.getService(DatabaseType.class, "TC-ClickHouse").getJdbcUrlPrefixes(), is(Arrays.asList("jdbc:tc:clickhouse:", "jdbc:tc:shardingsphere0clickhouse:")));
assertThat(TypedSPILoader.getService(DatabaseType.class, "TC-MariaDB").getJdbcUrlPrefixes(), is(Collections.singleton("jdbc:tc:mariadb:")));
assertThat(TypedSPILoader.getService(DatabaseType.class, "TC-MySQL").getJdbcUrlPrefixes(), is(Collections.singleton("jdbc:tc:mysql:")));
assertThat(TypedSPILoader.getService(DatabaseType.class, "TC-Oracle").getJdbcUrlPrefixes(), is(Collections.singleton("jdbc:tc:oracle:")));
Expand Down
Loading

0 comments on commit e854409

Please sign in to comment.