Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bug(source): schema.registry.name.strategy is not followed for encode protobuf #18319

Open
xiangjinwu opened this issue Aug 29, 2024 · 2 comments
Assignees

Comments

@xiangjinwu
Copy link
Contributor

#11384

async fn extract_protobuf_table_schema(
schema: &ProtobufSchema,
with_properties: &WithOptionsSecResolved,
format_encode_options: &mut BTreeMap<String, String>,
) -> Result<Vec<ColumnCatalog>> {
let info = StreamSourceInfo {
proto_message_name: schema.message_name.0.clone(),
row_schema_location: schema.row_schema_location.0.clone(),
use_schema_registry: schema.use_schema_registry,
format: FormatType::Plain.into(),
row_encode: EncodeType::Protobuf.into(),
format_encode_options: format_encode_options.clone(),
..Default::default()
};

Just acknowledging this fact. It will be fixed after format_encode_options parsing logic unification.

@xiangjinwu xiangjinwu self-assigned this Aug 29, 2024
@github-actions github-actions bot added this to the release-2.1 milestone Aug 29, 2024
@xiangjinwu
Copy link
Contributor Author

Not verified end-to-end:

diff --git a/src/frontend/src/handler/create_source.rs b/src/frontend/src/handler/create_source.rs
index 3f83c25fb3..bfede244da 100644
--- a/src/frontend/src/handler/create_source.rs
+++ b/src/frontend/src/handler/create_source.rs
@@ -193,20 +193,11 @@ async fn extract_debezium_avro_table_pk_columns(
 
 /// Map a protobuf schema to a relational schema.
 async fn extract_protobuf_table_schema(
-    schema: &ProtobufSchema,
+    info: &StreamSourceInfo,
     with_properties: &WithOptionsSecResolved,
     format_encode_options: &mut BTreeMap<String, String>,
 ) -> Result<Vec<ColumnCatalog>> {
-    let info = StreamSourceInfo {
-        proto_message_name: schema.message_name.0.clone(),
-        row_schema_location: schema.row_schema_location.0.clone(),
-        use_schema_registry: schema.use_schema_registry,
-        format: FormatType::Plain.into(),
-        row_encode: EncodeType::Protobuf.into(),
-        format_encode_options: format_encode_options.clone(),
-        ..Default::default()
-    };
-    let parser_config = SpecificParserConfig::new(&info, with_properties)?;
+    let parser_config = SpecificParserConfig::new(info, with_properties)?;
     try_consume_string_from_options(format_encode_options, SCHEMA_REGISTRY_USERNAME);
     try_consume_string_from_options(format_encode_options, SCHEMA_REGISTRY_PASSWORD);
     consume_aws_config_from_options(format_encode_options);
@@ -392,7 +383,7 @@ pub(crate) async fn bind_columns_from_source(
 
             Some(
                 extract_protobuf_table_schema(
-                    &protobuf_schema,
+                    &stream_source_info,
                     &options_with_secret,
                     &mut format_encode_options_to_consume,
                 )

@xiangjinwu
Copy link
Contributor Author

xiangjinwu commented Oct 17, 2024

schema.registry.name.strategy is being phased out.
https://github.com/risingwavelabs/risingwave-docs/pull/2660

Moving to a later milestone to either fix it or remove it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant