Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[GLUTEN-6920][CORE] Redesign and move trait GlutenPlan to gluten-core #8036

Open
wants to merge 13 commits into
base: main
Choose a base branch
from

Conversation

zhztheplayer
Copy link
Member

@zhztheplayer zhztheplayer commented Nov 25, 2024

Part of #6920

This work factors out GlutenPlan to separate the APIs to 2 places:

  1. For APIs related to native validation, e.g., doValidate, move to new API ValidatablePlan and TransformSupport
  2. Other convention-related APIs, move to gluten-core, and still be in GlutenPlan

After the change, GlutenPlan will become a thin and base interface for query plans of all Gluten backends. APIs in GlutenPlan will be used by query planner to decide whether pre-transition or post-transition is needed for this plan node. Backends could still define their own validation logic if needed, by extending the trait by themselves.

GlutenPlan will hide supportsColumnar and supportsRowBased (for Spark version > 3.2) up and expose Gluten APIs rowType and batchType to sub-classes instead. The later two are more flexible since Gluten's query planner could distinguish between different batch types and add essential transitions among them and vanilla rows.

@github-actions github-actions bot added CORE works for Gluten Core VELOX CLICKHOUSE labels Nov 25, 2024
Copy link

#6920

Copy link

Run Gluten Clickhouse CI on x86

@zhztheplayer zhztheplayer changed the title [GLUTEN-6920][CORE] Move trait GlutenPlan to gluten-core [GLUTEN-6920][CORE] Move trait GlutenPlan to gluten-core Nov 25, 2024
Copy link

Run Gluten Clickhouse CI on x86

2 similar comments
Copy link

Run Gluten Clickhouse CI on x86

Copy link

Run Gluten Clickhouse CI on x86

Copy link

Run Gluten Clickhouse CI on x86

@github-actions github-actions bot added the INFRA label Nov 26, 2024
Copy link

Run Gluten Clickhouse CI on x86

8 similar comments
Copy link

Run Gluten Clickhouse CI on x86

Copy link

Run Gluten Clickhouse CI on x86

Copy link

Run Gluten Clickhouse CI on x86

Copy link

Run Gluten Clickhouse CI on x86

Copy link

Run Gluten Clickhouse CI on x86

Copy link

Run Gluten Clickhouse CI on x86

Copy link

Run Gluten Clickhouse CI on x86

Copy link

Run Gluten Clickhouse CI on x86

@github-actions github-actions bot removed the INFRA label Nov 26, 2024
Copy link

Run Gluten Clickhouse CI on x86

Copy link

Run Gluten Clickhouse CI on x86

Copy link

Run Gluten Clickhouse CI on x86

Copy link

Run Gluten Clickhouse CI on x86

1 similar comment
@zhztheplayer
Copy link
Member Author

Run Gluten Clickhouse CI on x86

Copy link

Run Gluten Clickhouse CI on x86

Copy link

Run Gluten Clickhouse CI on x86

Copy link

Run Gluten Clickhouse CI on x86

Copy link

Run Gluten Clickhouse CI on x86

@zhztheplayer zhztheplayer marked this pull request as ready for review November 27, 2024 02:12
@zhztheplayer zhztheplayer changed the title [GLUTEN-6920][CORE] Move trait GlutenPlan to gluten-core [GLUTEN-6920][CORE] Redesign and move trait GlutenPlan to gluten-core Nov 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CLICKHOUSE CORE works for Gluten Core VELOX
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant