Discussion: report user errors and abort mv creation directly during backfilling?

[TennyZhuang]

Currently, we'll ignore and just warn all parsing and expression errors of streaming jobs (mview). It may be friendly for online jobs, but confusing for the mview creator.

It'll be better if the mview creation will be aborted when meeting errors during backfilling.

[chenzl25]

We have background ddl right now, for background ddl it seems we can't report user errors easily. Furthermore, we will have sink into table later. It also relies on backfilling, but it looks like it should not abort because it will cause a partial update. However, this idea is good for debugging, maybe we can apply it the the frontend mv creation and provide a system variable to let users decide which mode to use.

[kwannoel]

+1 for system / session variable support

[xxchan]

Can this be solved by #13108?

It seems this provided the reporting mechanism.

The main problem for this is that "parsing and expression errors" are ignored/tolerated, so they are not considered "errors", and will also not be propagated.

We might need to add a mechanism to change the behavior of error handling, like ignore when streaming, but report when backfilling. One challenge may be that the code for tolerating errors are scattered in many places.

[BugenZhao]

We might need to add a mechanism to change the behavior of error handling, like ignore when streaming, but report when backfilling. One challenge may be that the code for tolerating errors are scattered in many places.

The idea makes sense to me, but it makes error-handling become a bit too difficult to explain to users. Prefer KISS now and keeping the behavior as slightly ignoring the parsing errors.

Is source error also the case? Somehow similar to #15587 (comment).

[xxchan]

We might need to add a mechanism to change the behavior of error handling, like ignore when streaming, but report when backfilling. One challenge may be that the code for tolerating errors are scattered in many places.

The idea makes sense to me, but it makes error-handling become a bit too difficult to explain to users. Prefer KISS now and keeping the behavior as slightly ignoring the parsing errors.

Per offline discussion: reporting error on backfilling is still useful.

• Especially for source parsing error, now if users need to know whether parsing is correct, they have to check the logs, which is troublesome. The "strict mode" mechanism can offer better UX. https://github.com/orgs/risingwavelabs/discussions/16686#discussioncomment-9380613
• On the other hand, there is no harm in reporting an error: the MV is still not visible, so it's ok to fails it. If the user really doesn't care about the error, the message can hint to bypass using non-blocking DDL.

Any updates?

I think not started yet. There are a few design problems to investigate.