test: add duckdb cte tests & fix some minor bugs

[xxchan]

I hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.

What's changed and what's your intention?

#5987

fix #12511
fix #12526
not fixed yet: #12513

Checklist

• I have written necessary rustdoc comments
• I have added necessary unit tests and integration tests
• I have added fuzzing tests or opened an issue to track them. (Optional, recommended for new SQL features Sqlsmith: Sql feature generation #7934).
• My PR contains breaking changes. (If it deprecates some features, please create a tracking issue to remove them in the future).
• All checks passed in ./risedev check (or alias, ./risedev c)
• My PR changes performance-critical code. (Please run macro/micro-benchmarks and show the results.)

• My PR contains critical fixes that are necessary to be merged into the latest release. (Please check out the details)

Documentation

• My PR needs documentation updates. (Please use the Release note section below to summarize the impact on users)

Release note

If this PR includes changes that directly affect users or other significant modifications relevant to the community, kindly draft a release note to provide a concise summary of these changes. Please prioritize highlighting the impact these changes will have on users.

[xxchan]

So that risedev slt doesn't need to add -p 4566 -d dev any more. (If flags are present, they should have higher precedence)

[BugenZhao]

What about leveraging the risedev_env way?

https://github.com/risingwavelabs/risingwave/blob/2ab9b70b4d3c649b5bf51c216074b487db9e3016/src/risedevtool/src/risedev_env.rs

[xxchan]

Why?

[BugenZhao]

Hard-coded 4566 is not always the listen port. 🤔

[xxchan]

2022-03-17T08:04:40.367655Z  WARN node{id=7 name="frontend-1"}:task{id=606}:process_query_msg_one_stmt{session_id=0 stmt=WITH with_0 AS (SELECT hop_3.c11 AS col_0 FROM (lineitem AS t_1 LEFT JOIN supplier AS t_2 ON t_1.l_partkey = t_2.s_suppkey) FULL JOIN hop(alltypes2, alltypes2.c11, INTERVAL '78', INTERVAL '78') AS hop_3 ON t_2.s_nationkey = hop_3.c3 AND true WHERE (true) GROUP BY t_1.l_tax, hop_3.c8, t_1.l_returnflag, t_2.s_phone, hop_3.c2, hop_3.c4, hop_3.c10, t_1.l_shipinstruct, t_2.s_address, t_1.l_orderkey, hop_3.c3, hop_3.c5, t_1.l_partkey, t_1.l_linenumber, hop_3.c6, t_1.l_receiptdate, hop_3.c11) SELECT TIME '12:56:56' AS col_0 FROM with_0 WHERE EXISTS (WITH with_4 AS (SELECT t_7.col_0 AS col_0, 'VETiYBlOYM' AS col_1 FROM m8 AS t_7 GROUP BY t_7.col_0 HAVING true LIMIT 21) SELECT 'lhji9mKwl6' AS col_0, ((INTERVAL '66') * (1)) AS col_1 FROM with_4 LIMIT 38) LIMIT 24}: risingwave_frontend::scheduler::plan_fragmenter: The stage has single distribution, but contains a scan of table `lineitem` with 6 partitions. A single random worker will be assigned
thread '<unnamed>' panicked at src/batch/src/executor/join/local_lookup_join.rs:375:9:
assertion failed: !vnode_mapping.is_empty()

🤔️

[BugenZhao]

Rest LGTM

[BugenZhao]

What about leveraging the risedev_env way?

https://github.com/risingwavelabs/risingwave/blob/2ab9b70b4d3c649b5bf51c216074b487db9e3016/src/risedevtool/src/risedev_env.rs

[codecov]

Codecov Report

Merging #12527 (cc868d5) into main (f38554e) will increase coverage by 0.01%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main   #12527      +/-   ##
==========================================
+ Coverage   69.43%   69.45%   +0.01%     
==========================================
  Files        1438     1438              
  Lines      238698   238700       +2     
==========================================
+ Hits       165740   165787      +47     
+ Misses      72958    72913      -45     

Flag	Coverage Δ
rust	69.45% <100.00%> (+0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files	Coverage Δ
src/frontend/src/binder/relation/mod.rs	71.74% <100.00%> (+0.18%)	⬆️
...rc/frontend/src/optimizer/plan_node/batch_limit.rs	67.34% <100.00%> (ø)
src/risedevtool/src/risedev_env.rs	0.00% <ø> (ø)

... and 5 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

[chenzl25]

LGTM

[chenzl25]

Could you please elaborate more about this case about lookup join panic?

[xxchan]

Basically, I used self.clone() in two_phase_limit instead of self.clone_with_input, so its input is the one before to_distributed is called: LocalLookupJoin + BatchScan (Single dist).

What's interesting is that without where exists (select 1) the result is correct.

dev=> explain (verbose) select 1 from (select * from t1 left join t3 on t1.v1 = t3.v1 order by t1.v1, t1.v2) where exists (select 1) limit 1;
                                    QUERY PLAN
-----------------------------------------------------------------------------------
 BatchExchange { order: [], dist: Single }
 └─BatchProject { exprs: [1:Int32] }
   └─BatchLimit { limit: 1, offset: 0 }
     └─BatchNestedLoopJoin { type: LeftSemi, predicate: true, output: all }
       ├─BatchLookupJoin { type: LeftOuter, predicate: t1.v1 = t3.v1, output: [] }
       │ └─BatchScan { table: t1, columns: [t1.v1], distribution: Single }
       └─BatchValues { rows: [[]] }
(7 rows)

dev=>
dev=> explain (verbose) select 1 from (select * from t1 left join t3 on t1.v1 = t3.v1 order by t1.v1, t1.v2) limit 1;
                                    QUERY PLAN
-----------------------------------------------------------------------------------
 BatchProject { exprs: [1:Int32] }
 └─BatchLimit { limit: 1, offset: 0 }
   └─BatchExchange { order: [], dist: Single }
     └─BatchLimit { limit: 1, offset: 0 }
       └─BatchLookupJoin { type: LeftOuter, predicate: t1.v1 = t3.v1, output: [] }
         └─BatchExchange { order: [], dist: UpstreamHashShard(t1.v1) }
           └─BatchScan { table: t1, columns: [t1.v1], distribution: SomeShard }
(7 rows)