Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updates the minimum GraalVM CE version for compiling ShardingSphere's Native Image products to JDK 21.0.2 #29764

Merged
merged 1 commit into from
Jan 18, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/graalvm.yml
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ jobs:
fail-fast: false
matrix:
os: [ 'ubuntu-latest' ]
java-version: [ '17.0.9' ]
java-version: [ '21.0.2' ]
steps:
- uses: actions/checkout@v3
- uses: graalvm/setup-graalvm@v1
Expand Down
5 changes: 3 additions & 2 deletions .github/workflows/nightly-build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -107,7 +107,8 @@ jobs:

build-proxy-native-image:
if: github.repository == 'apache/shardingsphere'
name: Build GraalVM Native Image
name: Build Proxy Native Image
needs: global-environment
timeout-minutes: 90
permissions:
contents: read
Expand All @@ -119,7 +120,7 @@ jobs:
ref: ${{ inputs.commit-id }}
- uses: graalvm/setup-graalvm@v1
with:
java-version: '17.0.9'
java-version: '21.0.2'
distribution: 'graalvm-community'
github-token: ${{ secrets.GITHUB_TOKEN }}
- uses: actions/cache@v3
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/nightly-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -115,7 +115,7 @@ jobs:
- uses: actions/checkout@v3
- uses: graalvm/setup-graalvm@v1
with:
java-version: '17.0.9'
java-version: '21.0.2'
distribution: 'graalvm-community'
github-token: ${{ secrets.GITHUB_TOKEN }}
- uses: actions/cache@v3
Expand Down
2 changes: 1 addition & 1 deletion distribution/proxy-native/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@
<profile>
<id>release.native</id>
<properties>
<java.version>17</java.version>
<java.version>21</java.version>
</properties>
<build>
<finalName>apache-shardingsphere-${project.version}</finalName>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,7 @@ CE 的 `native-image` 命令行工具的长篇大论的 shell 命令。
ShardingSphere JDBC 要求在如下或更高版本的 `GraalVM CE` 完成构建 GraalVM Native Image。使用者可通过 SDKMAN! 快速切换 JDK。这同理
适用于 `Oracle GraalVM`, `Liberica Native Image Kit` 和 `Mandrel` 等 `GraalVM CE` 的下游发行版。

- GraalVM CE 23.0.2 For JDK 17.0.9,对应于 SDKMAN! 的 `17.0.9-graalce`
- GraalVM CE 23.1.1 For JDK 21.0.1,对应于 SDKMAN! 的 `21.0.1-graalce`
- GraalVM CE 23.1.2 For JDK 21.0.2,对应于 SDKMAN! 的 `21.0.2-graalce`

### Maven 生态

Expand Down Expand Up @@ -241,8 +240,8 @@ ShardingSphere 定义了 `nativeTestInShardingSphere` 的 Maven Profile 用于
sudo apt install unzip zip curl sed -y
curl -s "https://get.sdkman.io" | bash
source "$HOME/.sdkman/bin/sdkman-init.sh"
sdk install java 17.0.9-graalce
sdk use java 17.0.9-graalce
sdk install java 21.0.1-graalce
sdk use java 21.0.1-graalce
sudo apt-get install build-essential libz-dev zlib1g-dev -y

git clone [email protected]:apache/shardingsphere.git
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,7 @@ ShardingSphere JDBC requires GraalVM Native Image to be built with GraalVM CE as
JDK through `SDKMAN!`. Same reason applicable to downstream distributions of `GraalVM CE` such as `Oracle GraalVM`, `Liberica Native Image Kit`
and `Mandrel`.

- GraalVM CE 23.0.2 For JDK 17.0.9, corresponding to `17.0.9-graalce` of SDKMAN!
- GraalVM CE 23.1.1 For JDK 21.0.1, corresponding to `21.0.1-graalce` of SDKMAN!
- GraalVM CE 23.1.2 For JDK 21.0.2, corresponding to `21.0.2-graalce` of SDKMAN!

### Maven Ecology

Expand Down Expand Up @@ -251,8 +250,8 @@ You must install Docker Engine to execute `testcontainers-java` related unit tes
sudo apt install unzip zip curl sed -y
curl -s "https://get.sdkman.io" | bash
source "$HOME/.sdkman/bin/sdkman-init.sh"
sdk install java 17.0.9-graalce
sdk use java 17.0.9-graalce
sdk install java 21.0.1-graalce
sdk use java 21.0.1-graalce
sudo apt-get install build-essential libz-dev zlib1g-dev -y

git clone [email protected]:apache/shardingsphere.git
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -30,11 +30,11 @@ services:

## 前提条件

1. 根据 https://www.graalvm.org/downloads/ 要求安装和配置 JDK 17 对应的 `GraalVM Community Edition`
1. 根据 https://www.graalvm.org/downloads/ 要求安装和配置 JDK 21 对应的 `GraalVM Community Edition`
或 `GraalVM Community Edition` 的下游发行版。若使用 `SDKMAN!`,

```shell
sdk install java 17.0.9-graalce
sdk install java 21.0.2-graalce
```

2. 根据 https://www.graalvm.org/jdk17/reference-manual/native-image/#prerequisites 的要求安装本地工具链。
Expand Down Expand Up @@ -82,7 +82,7 @@ sdk install java 17.0.9-graalce
- 通过命令行构建 GraalVM Native Image。

```bash
./mvnw -am -pl distribution/proxy-native -B -T1C -Prelease.native -DskipTests clean package
./mvnw -am -pl distribution/proxy-native -T1C -Prelease.native -DskipTests clean package
```

3. 通过命令行启动 Native Image, 需要带上 4 个参数。
Expand All @@ -97,7 +97,7 @@ sdk install java 17.0.9-graalce
4. 如果需要构建 Docker Image, 在添加存在 SPI 实现的依赖或第三方依赖后, 在命令行执行如下命令。

```shell
./mvnw -am -pl distribution/proxy-native -B -T1C -Prelease.native,docker.native -DskipTests clean package
./mvnw -am -pl distribution/proxy-native -T1C -Prelease.native,docker.native -DskipTests clean package
```

- 假设存在包含`server.yaml` 的 `conf` 文件夹为 `./custom/conf`,可通过如下的 `docker-compose.yml` 文件启动 GraalVM Native
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -34,10 +34,10 @@ services:
## Premise

1. Install and configure `GraalVM Community Edition` or a downstream distribution of `GraalVM Community Edition` for
JDK 17 according to https://www.graalvm.org/downloads/. If `SDKMAN!` is used,
JDK 21 according to https://www.graalvm.org/downloads/. If `SDKMAN!` is used,

```shell
sdk install java 17.0.9-graalce
sdk install java 21.0.2-graalce
```

2. Install the local toolchain as required by https://www.graalvm.org/jdk17/reference-manual/native-image/#prerequisites.
Expand All @@ -59,7 +59,7 @@ sdk install java 17.0.9-graalce
Image.

```bash
./mvnw -am -pl distribution/proxy-native -B -T1C -Prelease.native -DskipTests clean package
./mvnw -am -pl distribution/proxy-native -T1C -Prelease.native -DskipTests clean package
```

- Scenario 2: It is necessary to use a JAR that has an SPI implementation or a third-party dependent JAR of a LICENSE
Expand Down Expand Up @@ -87,7 +87,7 @@ sdk install java 17.0.9-graalce
- Build GraalVM Native Image via command line.

```bash
./mvnw -am -pl distribution/proxy-native -B -T1C -Prelease.native -DskipTests clean package
./mvnw -am -pl distribution/proxy-native -T1C -Prelease.native -DskipTests clean package
```

3. To start Native Image through the command line, you need to bring 4 parameters. The first parameter is the `Port`
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,7 @@ public static JDBCRepositorySQL load(final String type) {
}

/**
* Under the GraalVM Native Image corresponding to GraalVM CE 23.0.2 For JDK 17.0.9, although there is
* Under the GraalVM Native Image, although there is
* `com.oracle.svm.core.jdk.resources.NativeImageResourceFileSystemProvider`, the corresponding
* `com.oracle.svm.core.jdk.resources.NativeImageResourceFileSystem` does not autoload. This is mainly to align the
* behavior of `ZipFileSystemProvider`, so ShardingSphere need to manually open and close the FileSystem
Expand Down
4 changes: 2 additions & 2 deletions pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -996,7 +996,7 @@
<profile>
<id>generateMetadata</id>
<properties>
<java.version>17</java.version>
<java.version>21</java.version>
</properties>
<build>
<pluginManagement>
Expand Down Expand Up @@ -1060,7 +1060,7 @@
<profile>
<id>nativeTestInShardingSphere</id>
<properties>
<java.version>17</java.version>
<java.version>21</java.version>
</properties>
<build>
<pluginManagement>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@

/**
* The background for this class comes from <a href="https://github.com/oracle/graal/issues/7682">oracle/graal#7682</a>
* and <a href="https://github.com/oracle/graal/blob/vm-ce-23.0.2/docs/reference-manual/native-image/Resources.md">Accessing Resources in Native Image</a>.
* and <a href="https://github.com/oracle/graal/blob/vm-ce-23.1.2/docs/reference-manual/native-image/Resources.md">Accessing Resources in Native Image</a>.
* GraalVM Native Image has special features in its handling of file systems.
* This means we are better off reading the file via `java.io.InputStream` instead of `java.net.URL` to avoid extra code
* processing.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,13 @@ class EtcdTest {

private TestShardingService testShardingService;

/**
* TODO On low-performance devices in Github Actions, `INSERT` related SQLs may throw a table not found error under nativeTest.
* So that we need to wait for a period of time after executing `CREATE TABLE` related SQLs before executing `INSERT` related SQLs.
* This may mean that the implementation of {@link org.apache.shardingsphere.mode.repository.cluster.etcd.EtcdRepository} needs optimization.
*
* @see org.apache.shardingsphere.mode.repository.cluster.etcd.EtcdRepository
*/
@SuppressWarnings("resource")
@Test
@EnabledInNativeImage
Expand All @@ -82,6 +89,10 @@ void assertShardingInLocalTransactions() throws SQLException, IOException {
DataSource dataSource = YamlShardingSphereDataSourceFactory.createDataSource(FileTestUtils.readFromFileURLString("test-native/yaml/mode/cluster/etcd.yaml"));
testShardingService = new TestShardingService(dataSource);
initEnvironment();
Awaitility.await().atMost(Duration.ofSeconds(30L)).ignoreExceptions().until(() -> {
dataSource.getConnection().close();
return true;
});
testShardingService.processSuccess();
testShardingService.cleanEnvironment();
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ rules:
shardingColumn: user_id
shardingAlgorithmName: inline
shardingAlgorithms:
# `groovy.lang.Closure` related classes are not available on GraalVM CE 23.0.2 For JDK 17.0.9,
# `groovy.lang.Closure` related classes are not available on GraalVM Native Image.
# This CLASS_BASE algorithm class is designed to emulate INLINE's `ds_${user_id % 2}`.
# See https://github.com/oracle/graal/issues/5522 .
inline:
Expand Down