chore(sink): exclude java iceberg and deltalake sink from connector node #14277
Conversation
wenym1
force-pushed
the
yiming/remove-java-iceberg-deltalake
branch
from
122641a to
904bffa
Compare
wenym1
force-pushed
the
yiming/remove-java-iceberg-deltalake
branch
from
b0f3eee to
5397607
Compare
| connector = 'deltalake_rust', | ||
| connector = 'deltalake', |
There was a problem hiding this comment.
Is it a user facing change?
| // iceberg with multiple parallelism will fail easily with concurrent commit | ||
| // on metadata | ||
| // TODO: reset iceberg sink to have multiple parallelism |
There was a problem hiding this comment.
What's the parallelism requirement for iceberg rust version
There was a problem hiding this comment.
The comment was for previous iceberg sink. Now there is no singleton requirement on both iceberg sink and delta lake sink.
wenym1
requested review from
liurenjie1024,
hzxa21,
chenzl25,
ZENOTME and
xxhZs
Just save for reference given that there is a simple way to exclude it from packaging. We can remove the code when it needs extra effort to maintain in the future. |
How about also remove related tests? |
I hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.
What's changed and what's your intention?
Exclude the module of java
iceberganddeltalakesink from connector node.The previous
deltalake_rustwill take over the current java version of delta lake and deltalake sink created withconnector = 'deltalake'will use the rust implementation now.The package size reduced from 407,850,813 to 119,393,608. Fix #13113.
Checklist
./risedev check(or alias,./risedev c)Documentation
Release note
connector = 'iceberg_java'is deprecated, and users can only create sink with rust version of iceberg.