Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bug: insert null as pk causing jdbc sink panic #12933

Closed
tabVersion opened this issue Oct 18, 2023 · 4 comments
Closed

bug: insert null as pk causing jdbc sink panic #12933

tabVersion opened this issue Oct 18, 2023 · 4 comments
Assignees
Labels
type/bug Something isn't working
Milestone

Comments

@tabVersion
Copy link
Contributor

Describe the bug

image

the above sql will result in a channel close error, which users can not understand

the root casue found in CN, the pk is null and make sink exec panic

	
{"timestamp":"2023-10-18T03:03:34.181682374Z","level":"ERROR","fields":{"message":"failed to collect barrier: Actor 29 exit unexpectedly: Executor error: Sink error: Remote sink error: gRPC error (Internal error): Error when exec INSERT INTO public.room_user_visit_1min(report_ts, uv, room_id) VALUES (NULL, 1, '1364688900') ON CONFLICT (report_ts, room_id) DO UPDATE SET report_ts=EXCLUDED.report_ts, uv=EXCLUDED.uv, room_id=EXCLUDED.room_id
RETURNING *, message ERROR: null value in column \"report_ts\" of relation \"room_user_visit_1min\" violates not-null constraint
  Detail: Failing row contains (null, 1364688900, 1).
  backtrace of `StreamExecutorError`:
   0: <risingwave_stream::executor::error::Inner as core::convert::From<risingwave_stream::executor::error::ErrorKind>>::from
             at ./risingwave/src/stream/src/executor/error.rs:40:10
   1: <T as core::convert::Into<U>>::into
             at ./rustc/f0411ffcebcd7f75ac02ed45feb53ffd07b75398/library/core/src/convert/mod.rs:717:9
   2: <risingwave_stream::executor::error::StreamExecutorError as core::convert::From<risingwave_stream::executor::error::ErrorKind>>::from
             at ./risingwave/src/stream/src/executor/error.rs:143:34
   3: <T as core::convert::Into<U>>::into
             at ./rustc/f0411ffcebcd7f75ac02ed45feb53ffd07b75398/library/core/src/convert/mod.rs:717:9
   4: <risingwave_stream::executor::error::StreamExecutorError as core::convert::From<risingwave_connector::sink::SinkError>>::from
             at ./risingwave/src/stream/src/executor/error.rs:209:33
   5: <core::result::Result<T,F> as core::ops::try_trait::FromResidual<core::result::Result<core::convert::Infallible,E>>>::from_residual
             at ./rustc/f0411ffcebcd7f75ac02ed45feb53ffd07b75398/library/core/src/result.rs:1961:27
   6: risingwave_stream::executor::sink::SinkExecutor<F>::execute_consume_log::{{closure}}
             at ./risingwave/src/stream/src/executor/sink.rs:279:25

Error message/log

No response

To Reproduce

No response

Expected behavior

No response

How did you deploy RisingWave?

No response

The version of RisingWave

No response

Additional context

No response

@tabVersion tabVersion added the type/bug Something isn't working label Oct 18, 2023
@github-actions github-actions bot added this to the release-1.4 milestone Oct 18, 2023
@tabVersion tabVersion changed the title bug: validate not null for pk when creating jdbc sink bug: insert null as pk causing jdbc sink panic Oct 18, 2023
@StrikeW
Copy link
Contributor

StrikeW commented Oct 18, 2023

Kernel version 1.2.0.
Sink error causes the Actor exit, then trigger the recovery procedure which will reset the compute node. This is by design w/o sink decouple.

@tabVersion
Copy link
Contributor Author

tabVersion commented Oct 18, 2023

had a discussion with @st1page, the expected behavior is to discard the record with NULL pk, and report a warning. This rule applies to all connector x format with the required primary_key.

The only problem left is kinesis x append-only, kinesis requires a message key but it does not mean anything in append-only, what do you think @xiangjinwu

@xiangjinwu
Copy link
Contributor

You meant as a natural extension kinesis x append-only would also discard records with nulls in pk, but it is also okay (maybe preferred) not to discard them?

As long as it writes to a message queue, having nulls in some pk columns (including all nulls) is not as severe as inserts into a DB. The actual meaning is partition key #9768.

@tabVersion
Copy link
Contributor Author

After some discussion, I think it is a expected behavior. We've proposed log store to handle such case. And we don't need to add extra behavior than let it stuck on.

@tabVersion tabVersion closed this as not planned Won't fix, can't repro, duplicate, stale Oct 19, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants