Skip to content

Commit

Permalink
Refactor ShardingSpherePreparedStatement
Browse files Browse the repository at this point in the history
  • Loading branch information
terrymanu committed May 26, 2024
2 parents e59fadd + ced58c7 commit 07a8fca
Show file tree
Hide file tree
Showing 6 changed files with 26 additions and 97 deletions.
10 changes: 5 additions & 5 deletions docs/document/content/test-manual/integration-test/_index.cn.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ weight = 1
- Embed 环境由测试框架自动搭建嵌入式 MySQL,适用于 ShardingSphere-JDBC 的本地环境测试。

当前默认采用 Docker 环境,使用 Testcontainer 创建运行时环境并执行测试用例。
未来将采用 Embed 环境的 ShardingSphere-JDBC + MySQL,替换 Native 执行测试用例的默认环境类型。
未来将采用 Embed 环境的 ShardingSphere-JDBC + MySQL,替换 Native 执行测试用例的默认环境类型。

数据库类型目前支持 MySQL、PostgreSQL、SQLServer 和 Oracle,并且可以支持使用 ShardingSphere-JDBC 或是使用 ShardingSphere-Proxy 执行测试用例。

Expand Down Expand Up @@ -64,7 +64,7 @@ SQL 用例在 `resources/cases/${SQL-TYPE}/${SQL-TYPE}-integration-test-cases.xm
<!-- ... more assertions -->
<assertion parameters="${value_3}:${type_3}, ${value_4}:${type_4}" expected-data-file="${dataset_file_2}.xml" />
</test-case>

<!-- ... more test cases -->
</integration-test-cases>
```
Expand Down Expand Up @@ -181,7 +181,7 @@ it.cluster.databases=H2,MySQL,Oracle,SQLServer,PostgreSQL
```

##### 远程调试通过镜像启动的 Proxy
E2E 测试的 Proxy 镜像默认开启了 3308 端口用于远程调试容器中的实例。
E2E 测试的 Proxy 镜像默认开启了 3308 端口用于远程调试容器中的实例。
使用 IDEA 等 IDE 工具可以通过如下方式连接并 debug 容器中的 Proxy 代码:

IDEA -> Run -> Edit Configurations -> Add New Configuration -> Remote JVM Debug
Expand All @@ -197,8 +197,8 @@ IDEA -> Run -> Edit Configurations -> Add New Configuration -> Remote JVM Debug
##### 远程调试通过 Testcontainer 启动的 Proxy
> 注意:如果通过 Testcontainer 启动 Proxy 容器,由于 Testcontainer 启动前 3308 端口还没有暴露出来,无法通过 `远程调试通过镜像启动的 Proxy` 方式进行 debug。
可以通过如下方式 debug Testcontainer 启动的 Proxy 容器:
- 在 Testcontainer 的相关启动类后打一个断点,例如 sql 测试中 E2EIT#setUp() -> `containerComposer.start();` 后面的一行打断点,此时相关容器一定已经启动。
- 通过快捷键 Alt + F8,进入断点调试模式,查看 containerComposer 下的 Proxy 对象 3308 映射的端口(Testcontainer 对外映射端口是随机的)。例如本次通过该表达式:`((ShardingSphereProxyClusterContainer)((java.util.LinkedList)((ITContainers)((ClusterContainerComposer)containerComposer).containers).dockerContainers).getLast()).getMappedPort(3308)` 获取到映射的对外随机端口为 51837。(或者通过命令 `docker ps` 查看)
- 在 Testcontainer 的相关启动类后打一个断点,例如 sql 测试中 E2EContainerComposer -> `containerComposer.start();` 后面的一行打断点,此时相关容器一定已经启动。
- 通过快捷键 Alt + F8,进入断点调试模式,通过命令 `docker ps` 查看 containerComposer 下的 Proxy 对象 3308 映射的端口(Testcontainer 对外映射端口是随机的)。
- 参考 `远程调试通过镜像启动的 Proxy` 中的方式,将 Remote JVM Debug 配置中的 Port 设置为上一步中获取到的端口,例如 51837。

编辑好上面的信息后,在 IDEA 中 Run -> Run -> e2e-debug -> debug 即可启动 IDEA 的远程 debug。
Expand Down
18 changes: 9 additions & 9 deletions docs/document/content/test-manual/integration-test/_index.en.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ In the future, ShardingSphere-JDBC + MySQL of the Embed environment will be adop

Database types currently support MySQL, PostgreSQL, SQLServer, and Oracle, and test cases can be executed using ShardingSphere-JDBC or ShardingSphere-Proxy.

Scenarios are used to test the supporting rules of ShardingSphere. Currently, data sharding and read/write splitting and other related scenarios are supported, and the combination of scenarios will be improved continuously in the future.
Scenarios are used to test the supporting rules of ShardingSphere. Currently, data sharding and read/write splitting and other related scenarios are supported, and the combination of scenarios will be improved continuously in the future.

### Test engine

Expand All @@ -43,7 +43,7 @@ Each SQL generates a test report in the combination of `database type * access p
- Database types: H2, MySQL, PostgreSQL, SQLServer, and Oracle;
- Access port types: ShardingSphere-JDBC and ShardingSphere-Proxy;
- SQL execution modes: Statement and PreparedStatement;

- JDBC execution modes: execute and executeQuery/executeUpdate;
- Scenarios: database shards, table shards, read/write splitting and sharding + read/write splitting

Expand All @@ -66,12 +66,12 @@ The case file format is as follows:
<!-- ... more assertions -->
<assertion parameters="${value_3}:${type_3}, ${value_4}:${type_4}" expected-data-file="${dataset_file_2}.xml" />
</test-case>

<!-- ... more test cases -->
</integration-test-cases>
```

The lookup rule of `expected-data-file`is as follows:
The lookup rule of `expected-data-file`is as follows:
1. Find the file `dataset\${SCENARIO_NAME}\${DATABASE_TYPE}\${dataset_file}.xml` in the same level directory;
2. Find the file `dataset\${SCENARIO_NAME}\${dataset_file}.xml` in the same level directory;
3. Find the file `dataset\${dataset_file}.xml` in the same level directory;
Expand Down Expand Up @@ -123,7 +123,7 @@ Directory: `src/test/resources/docker/${SCENARIO-TYPE}`

#### Configure the running environment of the test engine

Control the test engine by configuring `src/test/resources/env/engine-env.properties`.
Control the test engine by configuring `src/test/resources/env/engine-env.properties`.

All attribute values can be dynamically injected via Maven command line `-D`.

Expand All @@ -135,7 +135,7 @@ it.scenarios=db,tbl,dbtbl_with_replica_query,replica_query
# Whether to run additional test cases
it.run.additional.cases=false

# Configure the environment type. Only one value is supported. Optional value: docker or null. The default value: null.
# Configure the environment type. Only one value is supported. Optional value: docker or null. The default value: null.
it.cluster.env.type=${it.env}
# Access port types to be tested. Multiple values can be separated by commas. Optional value: jdbc, proxy. The default value: jdbc
it.cluster.adapters=jdbc
Expand Down Expand Up @@ -197,13 +197,13 @@ After editing the above information, run Run -> Run -> e2e-debug in IDEA to star
##### Remote debug Proxy started by Testcontainer
> Note: If the Proxy container is started by Testcontainer, because the 3308 port is not exposed before Testcontainer starts, it cannot be debugged by the `Remote debug Proxy started by docker image` method.
Debug Testcontainer started Proxy container by the following method:
- Set a breakpoint in the relevant startup class of Testcontainer, for example, after the line `containerComposer.start();` in BaseE2EIT#setUp() in the suite test, at this time, the relevant containers must have been started.
- Access breakpoint debugging mode through shortcut key Alt + F8, and view the port mapped by the 3308 mapping of the Proxy object under the containerComposer (the external mapping port of Testcontainer is random). For example, the expression `((ShardingSphereProxyClusterContainer)((java.util.LinkedList)((ITContainers)((ClusterContainerComposer)containerComposer).containers).dockerContainers).getLast()).getMappedPort(3308)` get the mapped random port 51837.(or get mapped port by `docker ps`)
- Set a breakpoint in the relevant startup class of Testcontainer, for example, after the line `containerComposer.start();` in E2EContainerComposer in the suite test, at this time, the relevant containers must have been started.
- Access breakpoint debugging mode through shortcut key Alt + F8, and get mapped port by `docker ps` for the 3308 mapping of the Proxy object under the containerComposer (the external mapping port of Testcontainer is random).
- See the `Remote debug Proxy started by docker image` method, set the Name, Host, Port, and use the port got in previous step, e.g. 51837.

After editing the above information, run Run -> Run -> e2e-debug -> debug in IDEA to start the remote debug of IDEA.

#### Notice

1. To test Oracle, add an Oracle driver dependency to pom.xml.
2. In order to ensure the integrity and legibility of the test data, 10 database shards and 10 table shards are used in the sharding of the integration testing, which takes a long time to run the test cases completely.
2. In order to ensure the integrity and legibility of the test data, 10 database shards and 10 table shards are used in the sharding of the integration testing, which takes a long time to run the test cases completely.

This file was deleted.

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -122,7 +122,17 @@ public List<String> getChildrenKeys(final String key) {

@Override
public boolean isExisted(final String key) {
return !Strings.isNullOrEmpty(query(key));
try (
Connection connection = dataSource.getConnection();
PreparedStatement preparedStatement = connection.prepareStatement(repositorySQL.getSelectByKeySQL())) {
preparedStatement.setString(1, key);
try (ResultSet resultSet = preparedStatement.executeQuery()) {
return resultSet.next();
}
} catch (final SQLException ex) {
log.error("Check existence of {} data by key: {} failed", getType(), key, ex);
}
return Boolean.FALSE;
}

@Override
Expand All @@ -138,8 +148,7 @@ public void persist(final String key, final String value) {
// Create key level directory recursively.
for (int i = 0; i < paths.length - 1; i++) {
String tempKey = tempPrefix + SEPARATOR + paths[i];
String tempKeyVal = query(tempKey);
if (Strings.isNullOrEmpty(tempKeyVal)) {
if (!isExisted(tempKey)) {
insert(tempKey, "", parent);
}
tempPrefix = tempKey;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -147,7 +147,6 @@ void assertPersistWithUpdateForSimpleKeys() throws SQLException {
when(mockJdbcConnection.prepareStatement(repositorySQL.getUpdateSQL())).thenReturn(mockPreparedStatementForPersist);
when(mockPreparedStatement.executeQuery()).thenReturn(mockResultSet);
when(mockResultSet.next()).thenReturn(true);
when(mockResultSet.getString("value")).thenReturn("oldValue");
repository.persist(key, value);
verify(mockPreparedStatement).setString(1, key);
verify(mockPreparedStatementForPersist).setString(eq(1), anyString());
Expand Down Expand Up @@ -199,7 +198,6 @@ void assertPersistFailureDuringUpdate() throws SQLException {
when(mockJdbcConnection.prepareStatement(repositorySQL.getSelectByKeySQL())).thenReturn(mockPreparedStatement);
when(mockPreparedStatement.executeQuery()).thenReturn(mockResultSet);
when(mockResultSet.next()).thenReturn(true);
when(mockResultSet.getString("value")).thenReturn("oldValue");
when(mockJdbcConnection.prepareStatement(repositorySQL.getUpdateSQL())).thenReturn(mockPreparedStatement);
repository.persist(key, "value");
verify(mockPreparedStatementForPersist, times(0)).executeUpdate();
Expand Down

0 comments on commit 07a8fca

Please sign in to comment.