refactor(sink): (early preview) support both json and proto encoding #12125
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.
What's changed and what's your intention?
related: #11995
preview 1 (933072a)
RowEncoder
SinkFormatter
MessageSink
TODOs:
RowEncoder
generalize ofserde_json::Value::null
on upsert op::deletenull
). It is for kafka log compaction.SinkFormatter
support multiple output on debezium_json op::deleteimpl SerToString for String
clones unnecessarilyEmptyEncoder
as keyMessageSink
for nats and redispreview 0 (c5f87ad)
As an example, only touched {kinesis,kafka} x append-only.
RecordEncoder
forJsonEncoder
andProtoEncoder(MessageDescriptor)
ToBytes
forMap<String, serde_json::Value>
andDynamicMessage
pk_to_json
andrecord_to_json
has been superseded bygen_append_only_message_stream
has been replaced by a sync Iterator-likeGenAppendOnlyMsg
.Checklist
./risedev check
(or alias,./risedev c
)Documentation
Release note
If this PR includes changes that directly affect users or other significant modifications relevant to the community, kindly draft a release note to provide a concise summary of these changes. Please prioritize highlighting the impact these changes will have on users.