From 793caea1f278e7b9673e58b2b66643a8f0ffd689 Mon Sep 17 00:00:00 2001 From: linghengqian Date: Sun, 3 Sep 2023 17:25:51 +0800 Subject: [PATCH] Add doc for Row Value Expressions SPI --- .../content/dev-manual/sharding.cn.md | 17 +++++ .../content/dev-manual/sharding.en.md | 17 +++++ docs/document/content/faq/_index.cn.md | 2 +- docs/document/content/faq/_index.en.md | 3 +- .../content/features/sharding/concept.cn.md | 7 ++ .../content/features/sharding/concept.en.md | 9 +++ .../builtin-algorithm/expr.cn.md | 64 ++++++++++++++++++ .../builtin-algorithm/expr.en.md | 66 +++++++++++++++++++ .../builtin-algorithm/sharding.cn.md | 2 +- .../builtin-algorithm/sharding.en.md | 3 +- .../startup/graalvm-native-image.cn.md | 7 +- .../startup/graalvm-native-image.en.md | 12 ++-- .../e2e/engine/type/dml/BaseDMLE2EIT.java | 2 +- 13 files changed, 198 insertions(+), 13 deletions(-) create mode 100644 docs/document/content/user-manual/common-config/builtin-algorithm/expr.cn.md create mode 100644 docs/document/content/user-manual/common-config/builtin-algorithm/expr.en.md diff --git a/docs/document/content/dev-manual/sharding.cn.md b/docs/document/content/dev-manual/sharding.cn.md index dfac191b287e9..e419ccafd2b21 100644 --- a/docs/document/content/dev-manual/sharding.cn.md +++ b/docs/document/content/dev-manual/sharding.cn.md @@ -79,3 +79,20 @@ chapter = true |--------------------------|--------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | DatabaseTimestampService | 从数据库中获取当前时间进行路由 | [`org.apache.shardingsphere.timeservice.type.database.DatabaseTimestampService`](https://github.com/apache/shardingsphere/blob/master/kernel/time-service/type/database/src/main/java/org/apache/shardingsphere/timeservice/type/database/DatabaseTimestampService.java) | | SystemTimestampService | 从应用系统时间中获取当前时间进行路由 | [`org.apache.shardingsphere.timeservice.type.system.SystemTimestampService`](https://github.com/apache/shardingsphere/blob/master/kernel/time-service/type/system/src/main/java/org/apache/shardingsphere/timeservice/type/system/SystemTimestampService.java) | + +## InlineExpressionParser + +### 全限定类名 + +`org.apache.shardingsphere.infra.expr.core.InlineExpressionParser` + +### 定义 + +解析行表达式 + +### 已知实现 + +| *配置标识* | *详细说明* | *全限定类名* | +|----------|-------------------|--------------------------------------------------------------------------------| +| GROOVY | 使用 Groovy 语法的行表达式 | `org.apache.shardingsphere.infra.expr.groovy.GroovyInlineExpressionParser` | +| PURELIST | 使用标准列表的行表达式 | `org.apache.shardingsphere.infra.expr.purelist.PureListInlineExpressionParser` | diff --git a/docs/document/content/dev-manual/sharding.en.md b/docs/document/content/dev-manual/sharding.en.md index aeadbfae50287..0463e27c40cad 100644 --- a/docs/document/content/dev-manual/sharding.en.md +++ b/docs/document/content/dev-manual/sharding.en.md @@ -79,3 +79,20 @@ Obtain the current date for routing definition |--------------------------|--------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | DatabaseTimestampService | Get the current time from the database for routing | [`org.apache.shardingsphere.timeservice.type.database.DatabaseTimestampService`](https://github.com/apache/shardingsphere/blob/master/kernel/time-service/type/database/src/main/java/org/apache/shardingsphere/timeservice/type/database/DatabaseTimestampService.java) | | SystemTimestampService | Get the current time from the application system for routing | [`org.apache.shardingsphere.timeservice.type.system.SystemTimestampService`](https://github.com/apache/shardingsphere/blob/master/kernel/time-service/type/system/src/main/java/org/apache/shardingsphere/timeservice/type/system/SystemTimestampService.java) | + +## InlineExpressionParser + +### Fully-qualified class name + +`org.apache.shardingsphere.infra.expr.core.InlineExpressionParser` + +### Definition + +Row Value Expressions definition + +### Implementation classes + +| *Configuration Type* | *Description* | *Fully-qualified class name* | +|----------------------|---------------------------------------------------|--------------------------------------------------------------------------------| +| GROOVY | Row Value Expressions that uses the Groovy syntax | `org.apache.shardingsphere.infra.expr.groovy.GroovyInlineExpressionParser` | +| PURELIST | Row Value Expressions that uses a standard list | `org.apache.shardingsphere.infra.expr.purelist.PureListInlineExpressionParser` | diff --git a/docs/document/content/faq/_index.cn.md b/docs/document/content/faq/_index.cn.md index d5e3865a03ae7..3797a58860e7f 100644 --- a/docs/document/content/faq/_index.cn.md +++ b/docs/document/content/faq/_index.cn.md @@ -122,7 +122,7 @@ DROP DATABASE sharding_db; 回答： -行表达式标识符可以使用 `${...}` 或 `$->{...}`，但前者与 Spring 本身的属性文件占位符冲突，因此在 Spring 环境中使用行表达式标识符建议使用 `$->{...}`。 +使用 `InlineExpressionParser` SPI 的默认实现的行表达式标识符可以使用 `${...}` 或 `$->{...}`，但前者与 Spring 本身的属性文件占位符冲突，因此在 Spring 环境中使用行表达式标识符建议使用 `$->{...}`。 ### [分片] inline 表达式返回结果为何出现浮点数？ diff --git a/docs/document/content/faq/_index.en.md b/docs/document/content/faq/_index.en.md index f6e7d01b2383b..1bcf0ffbbf32a 100644 --- a/docs/document/content/faq/_index.en.md +++ b/docs/document/content/faq/_index.en.md @@ -130,7 +130,8 @@ Answer: Answer: -`${...}` or `$->{...}` can be used in inline expression identifiers, but the former one clashes with place holders in Spring property files, so `$->{...}` is recommended to be used in Spring as inline expression identifiers. +`${...}` or `$->{...}` can be used in inline expression identifiers using the default implementation of the +`InlineExpressionParser` SPI, but the former one clashes with place holders in Spring property files, so `$->{...}` is recommended to be used in Spring as inline expression identifiers. ### [Sharding] Why does float number appear in the return result of inline expression? diff --git a/docs/document/content/features/sharding/concept.cn.md b/docs/document/content/features/sharding/concept.cn.md index 66cc22c6a806c..194dd2ab7bb84 100644 --- a/docs/document/content/features/sharding/concept.cn.md +++ b/docs/document/content/features/sharding/concept.cn.md @@ -159,6 +159,13 @@ db0.t_order0, db0.t_order1, db1.t_order2, db1.t_order3, db1.t_order4 对于常见的分片算法，使用 Java 代码实现并不有助于配置的统一管理。 通过行表达式书写分片算法，可以有效地将规则配置一同存放，更加易于浏览与存储。 +行表达式作为字符串由两部分组成，分别是字符串开头的对应 SPI 实现的 Type Name 部分和表达式部分。 以 `t_order_${1..3}` 为例，字符 +串`` 部分的子字符串 `GROOVY` 为此行表达式使用的对应 SPI 实现的 Type Name，其被 `<>` 符号包裹来识别。而字符串 `t_order_${1..3}` +为此行表达式的表达式部分。当行表达式不指定 Type Name 时，例如 `t_order_${1..3}`，行表示式默认将使用 `InlineExpressionParser` SPI 的 +`GROOVY` 实现来解析表达式。 + +以下部分介绍 `GROOVY` 实现的语法规则。 + 行表达式的使用非常直观，只需要在配置中使用 `${ expression }` 或 `$->{ expression }` 标识行表达式即可。 目前支持数据节点和分片算法这两个部分的配置。 行表达式的内容使用的是 Groovy 的语法，Groovy 能够支持的所有操作，行表达式均能够支持。 例如： `${begin..end}` 表示范围区间 diff --git a/docs/document/content/features/sharding/concept.en.md b/docs/document/content/features/sharding/concept.en.md index e5da2668f3dce..b72ad2219bb35 100644 --- a/docs/document/content/features/sharding/concept.en.md +++ b/docs/document/content/features/sharding/concept.en.md @@ -159,6 +159,15 @@ Row expressions are designed to address the two main issues of configuration sim For the common sharding algorithm, using Java code implementation does not help to manage the configuration uniformly. But by writing the sharding algorithm through line expressions, the rule configuration can be effectively stored together, which is easier to browse and store. +A Row Value Expressions consists of two parts as a string, the Type Name part of the corresponding SPI implementation at the beginning of the string and the expression part. + +Take `t_order_${1..3}` as sample, the `GROOVY` substring in the part of the `` string is the Type Name used by the corresponding SPI implementation for this Row Value Expressions, which is identified by the `<>` symbol. +And the `t_order_${1..3}` string is the expression part of this Row Value Expressions. When a Row Value Expressions does +not specify a Type Name, such as `t_order_${1..3}`, the Row Value Expressions defaults to parse expressions by `GROOVY` implementation for `InlineExpressionParser` SPI. + +The following sections describe the syntax rules for the `GROOVY` implementation. + + Row expressions are very intuitive, just use `${ expression }` or `$->{ expression }` in the configuration to identify the row expressions. Data nodes and sharding algorithms are currently supported. The content of row expressions uses Groovy syntax, and all operations supported by Groovy are supported by row expressions. For example: `${begin..end}` denotes the range interval diff --git a/docs/document/content/user-manual/common-config/builtin-algorithm/expr.cn.md b/docs/document/content/user-manual/common-config/builtin-algorithm/expr.cn.md new file mode 100644 index 0000000000000..c55718e6b5711 --- /dev/null +++ b/docs/document/content/user-manual/common-config/builtin-algorithm/expr.cn.md @@ -0,0 +1,64 @@ ++++ +title = "行表达式" +weight = 7 ++++ + +## 使用 Groovy 语法的行表达式 + +类型：GROOVY + +可配置属性： + +无 + +## 使用标准列表的行表达式 + +类型：PURELIST + +可配置属性： + +无 + +## 操作步骤 + +使用需要使用 `行表达式` 的属性时， 如在 `数据分片` 功能中， 在 `actualDataNodes` 属性下指明特定的 SPI 实现的 Type Name 即可。 + +若 `行表达式` 不指明 SPI 的 Type Name，默认将使用 `GROOVY` 的 SPI 实现。 + +## 配置示例 + +```yaml +rules: +- !SHARDING + tables: + t_order: + actualDataNodes: ds_0.t_order_0, ds_0.t_order_1, ds_1.t_order_0, ds_1.t_order_1 + tableStrategy: + standard: + shardingColumn: order_id + shardingAlgorithmName: t_order_inline + keyGenerateStrategy: + column: order_id + keyGeneratorName: snowflake + defaultDatabaseStrategy: + standard: + shardingColumn: user_id + shardingAlgorithmName: database_inline + shardingAlgorithms: + database_inline: + type: INLINE + props: + algorithm-expression: ds_${user_id % 2} + t_order_inline: + type: INLINE + props: + algorithm-expression: t_order_${order_id % 2} + keyGenerators: + snowflake: + type: SNOWFLAKE +``` + +## 相关参考 + +- [核心概念](/docs/document/content/features/sharding/concept.cn.md) +- [数据分片](/docs/document/content/dev-manual/sharding.cn.md) diff --git a/docs/document/content/user-manual/common-config/builtin-algorithm/expr.en.md b/docs/document/content/user-manual/common-config/builtin-algorithm/expr.en.md new file mode 100644 index 0000000000000..216ce754d3943 --- /dev/null +++ b/docs/document/content/user-manual/common-config/builtin-algorithm/expr.en.md @@ -0,0 +1,66 @@ ++++ +title = "Row Value Expressions" +weight = 7 ++++ + +## Row Value Expressions that uses the Groovy syntax + +Type: GROOVY + +Attributes: + +None + +## Row Value Expressions that uses a standard list + +Type: PURELIST + +Attributes: + +None + +## Procedure + +When using attributes that require the use of `Row Value Expressions`, such as in the `data sharding` feature, it is +sufficient to indicate the Type Name of the specific SPI implementation under the `actualDataNodes` attribute. + +If the `Row Value Expressions` does not indicate the Type Name of the SPI, the SPI implementation of `GROOVY` will be +used by default. + +## Sample + +```yaml +rules: +- !SHARDING + tables: + t_order: + actualDataNodes: ds_0.t_order_0, ds_0.t_order_1, ds_1.t_order_0, ds_1.t_order_1 + tableStrategy: + standard: + shardingColumn: order_id + shardingAlgorithmName: t_order_inline + keyGenerateStrategy: + column: order_id + keyGeneratorName: snowflake + defaultDatabaseStrategy: + standard: + shardingColumn: user_id + shardingAlgorithmName: database_inline + shardingAlgorithms: + database_inline: + type: INLINE + props: + algorithm-expression: ds_${user_id % 2} + t_order_inline: + type: INLINE + props: + algorithm-expression: t_order_${order_id % 2} + keyGenerators: + snowflake: + type: SNOWFLAKE +``` + +## Related References + +- [Core Concept](/docs/document/content/features/sharding/concept.en.md) +- [Data Sharding](/docs/document/content/dev-manual/sharding.en.md) diff --git a/docs/document/content/user-manual/common-config/builtin-algorithm/sharding.cn.md b/docs/document/content/user-manual/common-config/builtin-algorithm/sharding.cn.md index ba4a751c8ad71..0cd29976d13fe 100644 --- a/docs/document/content/user-manual/common-config/builtin-algorithm/sharding.cn.md +++ b/docs/document/content/user-manual/common-config/builtin-algorithm/sharding.cn.md @@ -73,7 +73,7 @@ Apache ShardingSphere 内置的标准分片算法实现类包括： #### 行表达式分片算法 -使用 Groovy 的表达式，提供对 SQL 语句中的 `=` 和 `IN` 的分片操作支持，只支持单分片键。 +使用 `InlineExpressionParser` SPI 的默认实现的 Groovy 的表达式，提供对 SQL 语句中的 `=` 和 `IN` 的分片操作支持，只支持单分片键。 对于简单的分片算法，可以通过简单的配置使用，从而避免繁琐的 Java 代码开发，如: `t_user_$->{u_id % 8}` 表示 `t_user` 表根据 `u_id` 模 8，而分成 8 张表，表名称为 `t_user_0` 到 `t_user_7`。 详情请参见[行表达式](/cn/dev-manual/sharding/#implementation-classes)。 diff --git a/docs/document/content/user-manual/common-config/builtin-algorithm/sharding.en.md b/docs/document/content/user-manual/common-config/builtin-algorithm/sharding.en.md index 7ba7d66d7933b..4c1e7bae24544 100644 --- a/docs/document/content/user-manual/common-config/builtin-algorithm/sharding.en.md +++ b/docs/document/content/user-manual/common-config/builtin-algorithm/sharding.en.md @@ -75,7 +75,8 @@ Apache ShardingSphere built-in standard sharding algorithm are: #### Inline Sharding Algorithm -With Groovy expressions, `InlineShardingStrategy` provides single-key support for the sharding operation of `=` and `IN` in SQL. +With Groovy expressions that uses the default implementation of the `InlineExpressionParser` SPI, +`InlineShardingStrategy` provides single-key support for the sharding operation of `=` and `IN` in SQL. Simple sharding algorithms can be used through a simple configuration to avoid laborious Java code developments. For example, `t_user_$->{u_id % 8}` means table t_user is divided into 8 tables according to u_id, with table names from `t_user_0` to `t_user_7`. Please refer to [Inline Expression](/en/dev-manual/sharding/#implementation-classes) for more details. diff --git a/docs/document/content/user-manual/shardingsphere-proxy/startup/graalvm-native-image.cn.md b/docs/document/content/user-manual/shardingsphere-proxy/startup/graalvm-native-image.cn.md index 973e550be6911..501f9f99d00eb 100644 --- a/docs/document/content/user-manual/shardingsphere-proxy/startup/graalvm-native-image.cn.md +++ b/docs/document/content/user-manual/shardingsphere-proxy/startup/graalvm-native-image.cn.md @@ -37,9 +37,10 @@ services: - `org.apache.shardingsphere.sharding.algorithm.sharding.inline.ComplexInlineShardingAlgorithm` - `org.apache.shardingsphere.sharding.algorithm.sharding.hint.HintInlineShardingAlgorithm` -- 当前阶段，GraalVM Native Image 形态的 ShardingSphere Proxy 不支持使用带 Groovy - 语法的 `行表达式`， 这首先导致 `数据分片` 功能的`actualDataNodes`属性只能使用纯列表来配置， 例如 `ds_0.t_order_0, ds_0.t_order_1` - 或 `ds_0.t_user_0, ds_15.t_user_1023`。此问题在 https://github.com/oracle/graal/issues/5522 追踪。 +- 当前阶段，GraalVM Native Image 形态的 ShardingSphere Proxy 不支持使用 `InlineExpressionParser` SPI 的默认实现的 `行表达式`， + 这首先导致 `数据分片` 功能的`actualDataNodes` 属性只能使用其他 `InlineExpressionParser` SPI 的实现来配置， 例如使用 + `InlineExpressionParser` SPI 实现为 `PURELIST` 的 `行表达式`, 即 `ds_0.t_order_0, ds_0.t_order_1` + 或 `ds_0.t_user_0, ds_15.t_user_1023`。 - 本节假定处于 Linux（amd64，aarch64）， MacOS（amd64）或 Windows（amd64）环境。 如果你位于 MacOS（aarch64/M1） 环境，你需要关注尚未关闭的 https://github.com/oracle/graal/issues/2666 。 diff --git a/docs/document/content/user-manual/shardingsphere-proxy/startup/graalvm-native-image.en.md b/docs/document/content/user-manual/shardingsphere-proxy/startup/graalvm-native-image.en.md index 993d9dcd48263..fe0d38acfa86c 100644 --- a/docs/document/content/user-manual/shardingsphere-proxy/startup/graalvm-native-image.en.md +++ b/docs/document/content/user-manual/shardingsphere-proxy/startup/graalvm-native-image.en.md @@ -42,16 +42,18 @@ services: - `org.apache.shardingsphere.sharding.algorithm.sharding.inline.ComplexInlineShardingAlgorithm` - `org.apache.shardingsphere.sharding.algorithm.sharding.hint.HintInlineShardingAlgorithm` -- At this stage, ShardingSphere Proxy in the form of GraalVM Native Image does not support the use - of `Row Value Expressions` with Groovy syntax, which first results in the `actualDataNodes` property of the `Sharding` - feature being only configurable using a pure list, such as `ds_0.t_order_0, ds_0.t_order_1` - or `ds_0.t_user_0, ds_15.t_user_1023`. This issue is tracked in https://github.com/oracle/graal/issues/5522 . +- At this stage, ShardingSphere Proxy in the form of GraalVM Native Image does not support `row expressions` using the + default implementation of the `InlineExpressionParser` SPI. + This first results in the `actualDataNodes` property of the `data sharding` feature that can only be configured using + other implementations of the `InlineExpressionParser` SPI, for example using `PURELIST` implemented + `InlineExpressionParser` SPI for `row expression`, i.e. `ds_0.t_order_0, ds_0.t_order_1` + or `ds_0.t_user_0, ds_15.t_user_1023`. - This section assumes a Linux (amd64, aarch64), MacOS (amd64) or Windows (amd64) environment. If you are on MacOS (aarch64/M1) environment, you need to follow https://github.com/oracle/graal/issues/2666 which is not closed yet. -- 'org.apache.shardingsphere:shardingsphere-cluster-mode-repository-etcd' is affected by +- `org.apache.shardingsphere:shardingsphere-cluster-mode-repository-etcd` is affected by https://github.com/micronaut-projects/micronaut-gcp/issues/532 and cannot be used. ## Premise diff --git a/test/e2e/sql/src/test/java/org/apache/shardingsphere/test/e2e/engine/type/dml/BaseDMLE2EIT.java b/test/e2e/sql/src/test/java/org/apache/shardingsphere/test/e2e/engine/type/dml/BaseDMLE2EIT.java index 022e253ea237c..1d6cb531ef12e 100644 --- a/test/e2e/sql/src/test/java/org/apache/shardingsphere/test/e2e/engine/type/dml/BaseDMLE2EIT.java +++ b/test/e2e/sql/src/test/java/org/apache/shardingsphere/test/e2e/engine/type/dml/BaseDMLE2EIT.java @@ -91,7 +91,7 @@ protected final void assertDataSet(final AssertionTestParameter testParam, final assertDataSet(testParam, containerComposer, each); } } - + private void assertDataSet(final AssertionTestParameter testParam, final SingleE2EContainerComposer containerComposer, final DataSetMetaData expectedDataSetMetaData) throws SQLException { for (String each : InlineExpressionParserFactory.newInstance(expectedDataSetMetaData.getDataNodes()).splitAndEvaluate()) { DataNode dataNode = new DataNode(each);