choose to detect inconsistency in materialized view executor

[lmatz]

We intend to detect inconsistencies as upstream as possible.

sometimes, the inconsistency error may not originate from the table that produces such an error. The error may occur in the upstream MVs.

When it happens, we can turn this check on and let users recreate all the MVs once to see if we can catch the bug.

[BugenZhao]

Do you mean enabling the "dry-run" of conflict handling for materialized views?

risingwave/proto/catalog.proto

Lines 321 to 326 in 295295d

	enum HandleConflictBehavior {
	HANDLE_CONFLICT_BEHAVIOR_UNSPECIFIED = 0;
	HANDLE_CONFLICT_BEHAVIOR_OVERWRITE = 1;
	HANDLE_CONFLICT_BEHAVIOR_IGNORE = 2;
	HANDLE_CONFLICT_BEHAVIOR_NO_CHECK = 3;
	}

This sounds like a good point as it reuses the table state of Materialize to check the consistency without introducing new states. Some downsides:

• The coverage of cases is limited. Since there are typically Join or Agg operations in every streaming job, we must also detect (and tolerate) any inconsistencies there first to make it possible for Materialize to find them.
• Additional computation and I/O have to be paid on checking. It would be beneficial if we could configure it on the fly, as long as the computation is deterministic and reproducible.

[github-actions]

This issue has been open for 60 days with no activity. Could you please update the status? Feel free to continue discussion or close as not planned.