Implement indexes (currently only unique indexes are supported) #65

KKould · 2023-09-23T18:27:35Z

What is changed and how it works?

support unique field option declaration
support unique index
- ddl
  - create table
  - drop table
- dml
  - insert
  - delete
  - update
- dql
  - filter
added RBO rules:
- SimplifyFilter:
  - perform constant folding on the conditional expression in the Filter operator
- PushPredicateIntoScan:
  - filter whether the field matches the index and push down to Scan
fixed RBO rule:
- PushProjectThroughChild:
  - fixed the problem of missing fields when pushing down
- PushLimitThroughJoin
  - fixed the problem that when the on condition in Join generates multiple same number of connection rows, the limit is exceeded
rename
- Transaction -> Iter
- Table -> Transaction
TableCodec
- IndexMeta
- Index

Code changes

Has Rust code change
Has CI related scripts change

Check List

Tests

Unit test
Integration test
Manual test (add detailed scripts or steps below)
No code

Side effects

Performance regression: Consumes more CPU
Performance regression: Consumes more Memory
Breaking backward compatibility

Note for reviewer

…ate a unique index for it

…esponding unique index, insert the index and check whether it already exists

used to extract the constant binary expression information corresponding to Column in the condition of the where clause

Fusion of marginal values with values within a range

constant folding -> expression extraction -> aggregation (and) -> rearrangement (or)

…ates one

- RBO Rule: `PushProjectThroughChild`: fixed the problem of missing fields when pushing down - RBO Rule: `PushLimitThroughJoin`: fixed the problem that when the on condition in Join generates multiple same number of connection rows, the limit limit is exceeded. -

…ue key

KipData#68

KKould added 21 commits September 24, 2023 01:57

feat: implement DataValue::to_index_key

76f0356

feat: implement Index and encoding in TableCodec

b2c8c49

feat: implementing when create table, detect the unique field and cre…

3250f08

…ate a unique index for it

feat: when inserting data, it will check whether the field has a corr…

0677ccb

…esponding unique index, insert the index and check whether it already exists

feat: Processing of unique indexes when adding update and delete

c760431

feat: Processing of unique indexes when truncate

9c7ef27

style: rename Table -> Transaction, Transaction -> Iter

628f7b1

feat: added ScalarExpression::convert_binary

40e669e

used to extract the constant binary expression information corresponding to Column in the condition of the where clause

style: code optimization

90b8752

feat: Implement RBO rule -> SimplifyFilter

f6005fa

test: add test case for simplification.rs

72f4a37

feat: add RBO RuleSimplifyFilter

73b8652

perf: ConstantBinary::scope_aggregation

e0e71ac

Fusion of marginal values with values within a range

feat: implement ConstantBinary::rearrange

088aeda

constant folding -> expression extraction -> aggregation (and) -> rearrangement (or)

fix: ConstantBinary::scope_aggregation selection of Eq/NotEq and Scope

1c6da7c

fix: ConstantBinary::scope_aggregation the eq condition only aggreg…

93665d3

…ates one

feat: add RBO RulePushPredicateIntoScan

8a6a4a5

feat: implement IndexScan

35fa96c

feat: implement offset and limit for IndexScan

d16e4a3

fix: many bugs

602ef4c

- RBO Rule: `PushProjectThroughChild`: fixed the problem of missing fields when pushing down - RBO Rule: `PushLimitThroughJoin`: fixed the problem that when the on condition in Join generates multiple same number of connection rows, the limit limit is exceeded. -

fix: resolve merge conflicts

c26cb66

KKould requested a review from loloxwg September 23, 2023 18:29

KKould self-assigned this Sep 23, 2023

KKould added bug Something isn't working enhancement New feature or request invalid This doesn't seem right labels Sep 23, 2023

KKould added 3 commits September 24, 2023 02:40

style: code format

654e75e

fix: check or in c1 > c2 or c1 > 1

ded2c95

docs: supplementary index related

69bb3b4

KKould linked an issue Sep 24, 2023 that may be closed by this pull request

select * from t1 where a > 0 order by a, b error on test test_run_sql #63

Closed

KKould added 6 commits September 25, 2023 15:17

perf: ScalarExpression::check_or optimize implementation

4b899ec

Merge remote-tracking branch 'upstream/main'

8a711c2

feat: implemented for Decimal type DataValue::to_index_key

f7a205d

perf: Optimized DataValue conversion to bitwise sequence

1f48edc

perf: optimized DataValue::Utf8 convert to encoding of primary/uniq…

cdf3694

…ue key

refactor: reconstruct the Key encoding of each structure of TableCodec

026f34b

KipData#68

This was linked to issues Sep 27, 2023

Keyword splicing abbreviation in TableCodec #68

Closed

DataValue::to_index_key and DataValue::to_primary_key abbreviation #67

Closed

KKould merged commit 6691037 into KipData:main Sep 30, 2023
1 of 2 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement indexes (currently only unique indexes are supported) #65

Implement indexes (currently only unique indexes are supported) #65

KKould commented Sep 23, 2023 •

edited

Loading

Implement indexes (currently only unique indexes are supported) #65

Implement indexes (currently only unique indexes are supported) #65

Conversation

KKould commented Sep 23, 2023 • edited Loading

What is changed and how it works?

Code changes

Check List

Note for reviewer

KKould commented Sep 23, 2023 •

edited

Loading