fix(meta): use per table mce and safe epoch in metadata backup #17227
Conversation
# Conflicts: # proto/backup_service.proto # src/storage/backup/src/lib.rs
|
FYI, I have to revert the adoption of the macro introduced in #17225 when restoring a backup, because insertion requires a specific order to satisfy foreign key constraints, which is different with the one in That means, now to include a new model in metadata backup, 2 steps are required:
|
| @@ -58,6 +60,8 @@ pub struct MetaSnapshotMetadata { | |||
| #[serde(default)] | |||
| pub format_version: u32, | |||
| pub remarks: Option<String>, | |||
| #[serde(with = "table_id_key_map")] | |||
There was a problem hiding this comment.
If we set key type as string or u32, we may not need to impl the serde by ourselves?
There was a problem hiding this comment.
serde_json requires map key to be String. So either using String as key or impl the serde.
The downside of using String as key is additional type conversion when visiting the map.
| #[async_trait::async_trait] | ||
| impl Writer<MetadataV2> for WriterModelV2ToMetaStoreV2 { | ||
| async fn write(&self, target_snapshot: MetaSnapshot<MetadataV2>) -> BackupResult<()> { | ||
| let metadata = target_snapshot.metadata; | ||
| let db = &self.meta_store.conn; | ||
| write_model_v2_to_meta_store_v2(&metadata, db).await?; | ||
| insert_models(metadata.seaql_migrations.clone(), db).await?; |
There was a problem hiding this comment.
Any reason to manually expand the macro here?
I hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.
What's changed and what's your intention?
After #17161, batch query over metadata backup won't work correctly, because it
try_get_hummock_version(find a queryable backup) using global safe_epoch and MCE, whilevalidate_safe_epochusing per table safe_epoch and MCE.This PR fixes it by storing and using per table safe_epoch and MCE.
Checklist
./risedev check(or alias,./risedev c)Documentation
Release note
If this PR includes changes that directly affect users or other significant modifications relevant to the community, kindly draft a release note to provide a concise summary of these changes. Please prioritize highlighting the impact these changes will have on users.