disable conflict check for those row_id generated by rowIdGenerator

[st1page]

We have find the conflict check can be the bottleneck in #14635, even if we have optimized it with bloom filter and operator cach.

For the AppendOnly table without primary key, we do not need conflict check because all the row_id is generated by rowIdGenerator.

dev=> explain create table t(v int) append only;
                                                       QUERY PLAN                                                        
-------------------------------------------------------------------------------------------------------------------------
 StreamMaterialize { columns: [v, _row_id(hidden)], stream_key: [_row_id], pk_columns: [_row_id], pk_conflict: NoCheck }
 └─StreamRowIdGen { row_id_index: 1 }
   └─StreamUnion { all: true }
     └─StreamExchange [no_shuffle] { dist: SomeShard }
       └─StreamDml { columns: [v, _row_id] }
         └─StreamSource
(6 rows)

when the table is non-append only, we still need to check for for the delete and update DML statment. But the the op rows are most from insert or the table's upstream connector like kafka. So if we find a way to determine those rows whose row_id is generated by rowIdGenerator, we can save the conflict check.

dev=> explain create table t(v int);
                                                        QUERY PLAN                                                         
---------------------------------------------------------------------------------------------------------------------------
 StreamMaterialize { columns: [v, _row_id(hidden)], stream_key: [_row_id], pk_columns: [_row_id], pk_conflict: Overwrite }
 └─StreamRowIdGen { row_id_index: 1 }
   └─StreamUnion { all: true }
     └─StreamExchange { dist: HashShard(_row_id) }
       └─StreamDml { columns: [v, _row_id] }
         └─StreamSource
(6 rows)

[chenzl25]

Is it possible that _row_id is deleted from DML?

[st1page]

Is it possible that _row_id is deleted from DML?

Yes, So we need some way to let MaterializeExecutor if the row needs to be checked (comes from Update/delete DML)

[lmatz]

This only applies to the case where there is no downstream MV or index, right?

With MV or index, we have to do the checks for inserts anyways.

[st1page]

This only applies to the case where there is no downstream MV or index, right?

With MV or index, we have to do the checks for inserts anyways.

It can optimized for all tables without primary key, whether there are MVs downstream or not

[github-actions]

This issue has been open for 60 days with no activity.

If you think it is still relevant today, and needs to be done in the near future, you can comment to update the status, or just manually remove the no-issue-activity label.

You can also confidently close this issue as not planned to keep our backlog clean.
Don't worry if you think the issue is still valuable to continue in the future.
It's searchable and can be reopened when it's time. 😄