feat(query): Virtual column support alias name

[b41sh]

I hereby agree to the terms of the CLA available at: https://docs.databend.com/dev/policies/cla/

Summary

virtual columns allow set alias names, user can use alias name to query variant inner column

for example:

root@0.0.0.0:48000/default> create table t1 (a int null, v json null);
0 row written in 0.084 sec. Processed 0 row, 0 B (0 row/s, 0 B/s)

root@0.0.0.0:48000/default> insert into t1 values(1, parse_json('{"a":[1,2,3],"b":{"c":10}}'));
┌─────────────────────────┐
│ number of rows inserted │
│          UInt64         │
├─────────────────────────┤
│                       1 │
└─────────────────────────┘
1 row written in 0.076 sec. Processed 1 row, 81 B (13.16 rows/s, 1.04 KiB/s)

root@0.0.0.0:48000/default> create virtual column (v['a'][0] as v1, v['b'] as v2) for t1;
0 row written in 0.036 sec. Processed 0 row, 0 B (0 row/s, 0 B/s)

root@0.0.0.0:48000/default> refresh virtual column for t1;
0 row read in 0.032 sec. Processed 1 row, 76 B (31.25 rows/s, 2.32 KiB/s)

root@0.0.0.0:48000/default> select a, v1, v2 from t1;
┌─────────────────────────────────────────────────────────┐
│        a        │         v1        │         v2        │
│ Nullable(Int32) │ Nullable(Variant) │ Nullable(Variant) │
├─────────────────┼───────────────────┼───────────────────┤
│               1 │ 1                 │ {"c":10}          │
└─────────────────────────────────────────────────────────┘
1 row read in 0.042 sec. Processed 1 row, 64 B (23.81 rows/s, 1.49 KiB/s)

root@0.0.0.0:48000/default> explain select a, v1, v2 from t1;
-[ EXPLAIN ]-----------------------------------
TableScan
├── table: default.default.t1
├── output columns: [a (#0), v['a'][0] (#2), v['b'] (#3)]
├── read rows: 1
├── read size: < 1 KiB
├── partitions total: 1
├── partitions scanned: 1
├── pruning stats: [segments: <range pruning: 1 to 1>, blocks: <range pruning: 1 to 1>]
├── push downs: [filters: [], limit: NONE]
├── virtual columns: [v['a'][0], v['b']]
└── estimated rows: 1.00

11 rows explain in 0.033 sec. Processed 0 rows, 0 B (0 row/s, 0 B/s)

• fixes: #[Link the issue here]

Tests

• Unit Test
• Logic Test
• Benchmark Test
• No Test - Explain why

Type of change

• Bug Fix (non-breaking change which fixes an issue)
• New Feature (non-breaking change which adds functionality)
• Breaking Change (fix or feature that could cause existing functionality not to work as expected)
• Documentation Update
• Refactoring
• Performance Improvement
• Other (please describe):

---

This change is 

[drmingdrmer]

Reviewed 43 of 43 files at r1, all commit messages.
Reviewable status: all files reviewed, 4 unresolved discussions (waiting on @b41sh and @sundy-li)

---

src/meta/protos/proto/virtual_column.proto line 50 at r1 (raw file):

  // if true, virtual columns are auto generated,
  // otherwise virtual columns are created by user.
  optional bool auto_generated = 7;

This is explicitly set to be an Option. What does it mean if it is None? I see that None is replaced with the default value false thus optional seems to be unnecessary?

---

src/meta/app/src/schema/virtual_column.rs line 52 at r1 (raw file):

}

#[derive(serde::Serialize, serde::Deserialize, Clone, Debug, Eq, PartialEq)]

Why do VirtualColumnMeta and VirtualField need serde?

---

src/meta/app/src/schema/virtual_column.rs line 59 at r1 (raw file):

    pub created_on: DateTime<Utc>,
    pub updated_on: Option<DateTime<Utc>>,
    pub auto_generated: bool,

This field needs to be explained about its purpose and usage.

---

src/meta/proto-conv/src/virtual_column_from_to_protobuf_impl.rs line 62 at r1 (raw file):

            };
            let alias_name = if let Some(alias_name) = p.alias_names.get(i) {
                if !alias_name.is_empty() {

I do believe any string including empty string "" could be a valid name. 🤔

[drmingdrmer]

Cleaner and simpler! Good job!

Reviewed 7 of 7 files at r2, all commit messages.
Reviewable status: complete! all files reviewed, all discussions resolved (waiting on @b41sh)

---

src/meta/protos/proto/virtual_column.proto line 47 at r2 (raw file):

  // virtual column alias names,
  // key is the index of `virtual_columns` field.
  map<uint64, string> alias_names = 6;

Nice shot! Using a map significantly simplified the data conversion logic.

---

src/meta/app/src/schema/virtual_column.rs line 75 at r2 (raw file):

    pub updated_on: Option<DateTime<Utc>>,
    // Whether the virtual columns are auto-generated,
    // true for auto-generated, false for user-defined.

I think it's meant to be:

Suggestion:

    // Whether the virtual column alias names are auto-generated,
    // true for auto-generated, false for user-defined.

[b41sh]

Cleaner and simpler! Good job!

Reviewed 7 of 7 files at r2, all commit messages.
Reviewable status: complete! all files reviewed, all discussions resolved (waiting on @b41sh)

src/meta/protos/proto/virtual_column.proto line 47 at r2 (raw file):

  // virtual column alias names,
  // key is the index of `virtual_columns` field.
  map<uint64, string> alias_names = 6;

Nice shot! Using a map significantly simplified the data conversion logic.

src/meta/app/src/schema/virtual_column.rs line 75 at r2 (raw file):

    pub updated_on: Option<DateTime<Utc>>,
    // Whether the virtual columns are auto-generated,
    // true for auto-generated, false for user-defined.

I think it's meant to be:

Suggestion:

    // Whether the virtual column alias names are auto-generated,
    // true for auto-generated, false for user-defined.

Thanks for the review. @drmingdrmer auto-generated means whether the virtual columns are auto-generated or not, not the alias names. this field is not used yet, it's a feature I'm planning to do, just this PR by the way to change the meta first.

[drmingdrmer]

Thanks for the review. @drmingdrmer auto-generated means whether the virtual columns are auto-generated or not, not the alias names. this field is not used yet, it's a feature I'm planning to do, just this PR by the way to change the meta first.

Then everything is good to me :)