Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: Meta notifies vec[(target, operation)] to Frontend/Compactor/Compute/Hummock #13683

Closed
Rossil2012 opened this issue Nov 28, 2023 · 2 comments

Comments

@Rossil2012
Copy link
Contributor

For now, meta send notification like this:

message SubscribeResponse {
  enum Operation {
    UNSPECIFIED = 0;
    ADD = 1;
    DELETE = 2;
    UPDATE = 3;
    SNAPSHOT = 4;
  }
  common.Status status = 1;
  Operation operation = 2;
  uint64 version = 3;
  oneof info {
    catalog.Database database = 4;
    catalog.Schema schema = 5;
    catalog.Function function = 6;
    user.UserInfo user = 11;
    // for streaming
    FragmentParallelUnitMapping parallel_unit_mapping = 12;
    common.WorkerNode node = 13;
    hummock.HummockSnapshot hummock_snapshot = 14;
    hummock.HummockVersionDeltas hummock_version_deltas = 15;
    MetaSnapshot snapshot = 16;
    backup_service.MetaBackupManifestId meta_backup_manifest_id = 17;
    SystemParams system_params = 19;
    hummock.WriteLimits hummock_write_limits = 20;
    RelationGroup relation_group = 21;
    catalog.Connection connection = 22;
    FragmentParallelUnitMappings serving_parallel_unit_mappings = 23;
    hummock.HummockVersionStats hummock_stats = 24;
  }
}

which indicates that one notification can only invoke one modification on one target. However, many sql statements need to modify a batch of objects in one query. For instance, ALTER DATABASE db OWNER to new_owner needs to modify db's owner and grant CONNECT privilege to new_owner at the same time.
A suggested Proto would be like:

message SubscribeResponse {
  message Info {
    enum Operation {
      UNSPECIFIED = 0;
      ADD = 1;
      DELETE = 2;
      UPDATE = 3;
      SNAPSHOT = 4;
    }
    Operation operation = 1;

    oneof target {
      catalog.Database database = 4;
      catalog.Schema schema = 5;
      catalog.Function function = 6;
      user.UserInfo user = 11;
      // for streaming
      FragmentParallelUnitMapping parallel_unit_mapping = 12;
      common.WorkerNode node = 13;
      hummock.HummockSnapshot hummock_snapshot = 14;
      hummock.HummockVersionDeltas hummock_version_deltas = 15;
      MetaSnapshot snapshot = 16;
      backup_service.MetaBackupManifestId meta_backup_manifest_id = 17;
      SystemParams system_params = 19;
      hummock.WriteLimits hummock_write_limits = 20;
      RelationGroup relation_group = 21;
      catalog.Connection connection = 22;
      FragmentParallelUnitMappings serving_parallel_unit_mappings = 23;
      hummock.HummockVersionStats hummock_stats = 24;
    }
  }
  common.Status status = 1;
  Operation operation = 2;
  uint64 version = 3;
  repeated Info infos = 4;
}
@github-actions github-actions bot added this to the release-1.5 milestone Nov 28, 2023
@Rossil2012
Copy link
Contributor Author

cc @yezizp2012

@fuyufjh fuyufjh modified the milestones: release-1.5, release-1.6 Dec 4, 2023
@fuyufjh fuyufjh modified the milestones: release-1.6, release-1.7 Jan 9, 2024
@yezizp2012 yezizp2012 modified the milestones: release-1.7, release-1.8 Mar 6, 2024
@yezizp2012 yezizp2012 removed this from the release-1.8 milestone Apr 8, 2024
Copy link
Contributor

This issue has been open for 60 days with no activity. Could you please update the status? Feel free to continue discussion or close as not planned.

@yezizp2012 yezizp2012 closed this as not planned Won't fix, can't repro, duplicate, stale Jul 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants