Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(source): format upsert encode avro shall no longer fetch key schema from registry #17670

Closed
xiangjinwu opened this issue Jul 12, 2024 · 0 comments · Fixed by #17759
Closed
Assignees
Milestone

Comments

@xiangjinwu
Copy link
Contributor

xiangjinwu commented Jul 12, 2024

After the introduction of include key as clause, we agree that kafka key will always be ingested as raw bytea and further decoding into text / json will be handled with SQL expressions.

  • convert_from('\x613141'::bytea, 'utf8') -> 'a1A'::varchar
  • convert_from('\x7b2279656172223a323032347d'::bytea, 'utf8')::jsonb -> '{"year": 2024}'::jsonb
  • convert_from('\x313233'::bytea, 'utf8')::int8 -> 123::int8
  • int8recv('\x000000000000007b'::bytea) -> 123::int8

The old logic of decoding key according to schema registry should be completely cleaned up. For now it still tries to read schema registry and reports an error when it is not found. But the schema is never used for decoding.

Note: format debezium encode avro is still reading key schema from registry.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
1 participant