Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(sink): fix sink in to Cassandra failed when using column name containing upper case letter #17493

Merged
merged 14 commits into from
Aug 20, 2024

Conversation

xxhZs
Copy link
Contributor

@xxhZs xxhZs commented Jun 28, 2024

I hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.

What's changed and what's your intention?

#17490
#17855

Checklist

  • I have written necessary rustdoc comments
  • I have added necessary unit tests and integration tests
  • I have added test labels as necessary. See details.
  • I have added fuzzing tests or opened an issue to track them. (Optional, recommended for new SQL features Sqlsmith: Sql feature generation #7934).
  • My PR contains breaking changes. (If it deprecates some features, please create a tracking issue to remove them in the future).
  • All checks passed in ./risedev check (or alias, ./risedev c)
  • My PR changes performance-critical code. (Please run macro/micro-benchmarks and show the results.)
  • My PR contains critical fixes that are necessary to be merged into the latest release. (Please check out the details)

Documentation

  • My PR needs documentation updates. (Please use the Release note section below to summarize the impact on users)

Release note

If this PR includes changes that directly affect users or other significant modifications relevant to the community, kindly draft a release note to provide a concise summary of these changes. Please prioritize highlighting the impact these changes will have on users.

@xxhZs xxhZs requested a review from yuhao-su June 28, 2024 06:12
@github-actions github-actions bot added the type/fix Bug fix label Jun 28, 2024
@yuhao-su
Copy link
Contributor

Can you help add some test to cover this case?

fix
@xxhZs xxhZs force-pushed the xxh/fix-cassandra-column_name branch from e4143c8 to a00f692 Compare June 28, 2024 06:58
@yuhao-su yuhao-su requested review from fuyufjh and xiangjinwu June 28, 2024 15:43
@xxhZs xxhZs force-pushed the xxh/fix-cassandra-column_name branch from 7693be2 to acf7086 Compare July 1, 2024 02:39
fix ci

fi x
@xxhZs xxhZs force-pushed the xxh/fix-cassandra-column_name branch from acf7086 to ef1cf1a Compare July 1, 2024 04:27
Copy link
Contributor

@xiangjinwu xiangjinwu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we have a schema check phase during sink creation stage? Such column mismatch issues could have been caught before any insert/update/delete entries actually come.

@xxhZs
Copy link
Contributor Author

xxhZs commented Jul 2, 2024

Do we have a schema check phase during sink creation stage? Such column mismatch issues could have been caught before any insert/update/delete entries actually come.

We have checked, and the reason for this bug is that the Rw and Cassandra names are completely consistent, but our CQL statement concatenation is incorrect

@xxhZs xxhZs requested a review from wenym1 July 9, 2024 04:34
@yuhao-su yuhao-su requested a review from xiangjinwu July 22, 2024 17:21
@graphite-app graphite-app bot requested a review from xxchan July 26, 2024 02:51
@xxhZs xxhZs force-pushed the xxh/fix-cassandra-column_name branch from 53393b8 to 3e570d8 Compare July 29, 2024 07:43
fix ci

fix
@xxhZs xxhZs force-pushed the xxh/fix-cassandra-column_name branch from 3e570d8 to b5cfeb7 Compare July 29, 2024 08:24
@xxhZs xxhZs force-pushed the xxh/fix-cassandra-column_name branch from ee5f887 to 887a6c5 Compare July 30, 2024 06:31
fix

fix
@xxhZs xxhZs force-pushed the xxh/fix-cassandra-column_name branch from 887a6c5 to ef71d85 Compare July 30, 2024 07:05
@lmatz lmatz added the need-cherry-pick-release-1.10 Open a cherry-pick PR to branch release-1.10 after the current PR is merged label Aug 20, 2024
Comment on lines 65 to 73
if cat ./query_result2.csv | awk -F "," '{
exit !($1 == 1 && $2 == 1 && $3 == "1\r"); }'; then
echo "Cassandra sink check passed"
else
echo "The output is not as expected."
echo "output:"
cat ./query_result2.csv
exit 1
fi
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: I think we can also put the validation logic in cassandra_sink.slt using system command

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

okk

@xxhZs xxhZs enabled auto-merge August 20, 2024 05:36
@xxhZs xxhZs added this pull request to the merge queue Aug 20, 2024
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Aug 20, 2024
@xxhZs xxhZs enabled auto-merge August 20, 2024 06:25
@xxhZs xxhZs added this pull request to the merge queue Aug 20, 2024
Merged via the queue into main with commit aba3232 Aug 20, 2024
30 of 31 checks passed
@xxhZs xxhZs deleted the xxh/fix-cassandra-column_name branch August 20, 2024 06:50
github-actions bot pushed a commit that referenced this pull request Aug 20, 2024
github-merge-queue bot pushed a commit that referenced this pull request Aug 20, 2024
lmatz pushed a commit that referenced this pull request Sep 24, 2024
@hzxa21 hzxa21 mentioned this pull request Oct 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ci/run-e2e-cassandra-sink-tests need-cherry-pick-release-1.10 Open a cherry-pick PR to branch release-1.10 after the current PR is merged need-cherry-pick-release-2.0 type/fix Bug fix
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants