fix: Correctly handle the Parallelism inference under the DefaultParallelism configuration.

[shanicky]

I hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.

What's changed and what's your intention?

This PR correctly handles the compatibility between default parallelism and table parallelism.

	DefaultParallelism::Full	DefaultParallelism::Default(n)
Adaptive	Adaptive, all	Fixed, n
Fixed(m)	Fixed(m), m	Fixed, m

And during restarts, it automatically scales the logic to address the impacts that may arise from old versions. For instance, when the default parallelism is set to n, older versions may still be set to Adaptive mode. This PR, upon startup, will determine if the actual parallelism is n and will update it to Fixed(n).

Checklist

• I have written necessary rustdoc comments
• I have added necessary unit tests and integration tests
• I have added test labels as necessary. See details.
• I have added fuzzing tests or opened an issue to track them. (Optional, recommended for new SQL features Sqlsmith: Sql feature generation #7934).
• My PR contains breaking changes. (If it deprecates some features, please create a tracking issue to remove them in the future).
• All checks passed in ./risedev check (or alias, ./risedev c)
• My PR changes performance-critical code. (Please run macro/micro-benchmarks and show the results.)

• My PR contains critical fixes that are necessary to be merged into the latest release. (Please check out the details)

Documentation

• My PR needs documentation updates. (Please use the Release note section below to summarize the impact on users)

Release note

If this PR includes changes that directly affect users or other significant modifications relevant to the community, kindly draft a release note to provide a concise summary of these changes. Please prioritize highlighting the impact these changes will have on users.

[yezizp2012]

LGTM, @neverchanje PTAL.

[neverchanje]

DefaultParallelism::Full	DefaultParallelism::Default(n)
Adaptive	Fixed(n) for creating MV, while remaining Adaptive for existing MV
Fixed(m)	Fixed(m)

If a users explicitly alter a MV to adaptive, with default_parallelism preconfigured as n in the meanwhile, the parallelism should be changed to adaptive.

[neverchanje]

Is this function only called when MV is being created? alter mv and create mv are two different handling paths. Only create mv should evaluate default_parallelism, alter mv should not.

The logic is pretty simple from my understanding.

// Let's divide the handling into two phases:
fn create_mv() {
  // handle `default_parallelism` first.
  let mut parallelism = match config.default_parallelism {
    Some(n) => Fixed(n)
    None => Adaptive
  };
  // Evaluate the sesssion variable  "streaming_parallelism".
  if let Some(p) = session_variable.streaming_parallelism {
    parallelism = p;
  }
}

fn alter_mv(p: TableParallelism) {
  // `p` will be the target parallelism, no matter what value `default_parallelism` is.
}

[shanicky]

This modification is made to handle offline scaling during startup recovery, addressing the situation in version 1.7 where new creations using default parallelism with an adaptive policy might exist. This aims to prevent a mass scale-out occurrence immediately after going live.

[neverchanje]

new creations using default parallelism

It sounds good.

I am not familar with the code so I cannot tell if it's correct. Maybe you can manually test it, and make sure:

1. alter mv will apply regardless of default_parallelism .
2. Restarting the cluster will not change the parallelism of existing materialized views back to default_parallelism.