Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bug(postgres-cdc): after recreate enum, debezium fail to update. #16880

Open
KeXiangWang opened this issue May 21, 2024 · 0 comments
Open

bug(postgres-cdc): after recreate enum, debezium fail to update. #16880

KeXiangWang opened this issue May 21, 2024 · 0 comments
Labels
type/bug Something isn't working

Comments

@KeXiangWang
Copy link
Contributor

KeXiangWang commented May 21, 2024

Describe the bug

In Pg, when delete and recreate a enum. The oid of enum is changed. However, debezium fails to capture this change. The root cause seems to be this https://issues.redhat.com/browse/DBZ-2045.

Error message/log

No response

To Reproduce

SQLs to create enum

CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');

SELECT oid
FROM pg_type
WHERE typname = 'mood';
--- for example, if we get 16390 from the above query.

Create a table based on the enum:

CREATE TABLE IF NOT EXISTS postgres_enum(
     c_boolean boolean,
     c_enum mood,
     PRIMARY KEY (c_boolean)
);
INSERT INTO postgres_single_types VALUES( False, 'happy' );

In rw, create a source and a table capturing the postgres_enum table.
Then drop everything, including the tables in RW and PG, and the enum.
Redo the steps again. In compute node's log, you will find sth like:

2024-05-21T14:17:31.586834-04:00  WARN risingwave_connector_node: Unknown OID 16390 requested thread="debezium-postgresconnector-RW_CDC_5-change-event-source-coordinator" class="io.debezium.connector.postgresql.TypeRegistry"
2024-05-21T14:17:31.58767-04:00  WARN risingwave_connector_node: Unknown OID -1 requested thread="debezium-postgresconnector-RW_CDC_5-change-event-source-coordinator" class="io.debezium.connector.postgresql.TypeRegistry"
2024-05-21T14:17:31.587756-04:00  WARN risingwave_connector_node: Unexpected JDBC type '-2147483648' for column 'c_enum' that will be ignored thread="debezium-postgresconnector-RW_CDC_5-change-event-source-coordinator" class="io.debezium.relational.TableSchemaBuilder"
2024-05-21T14:17:31.58823-04:00  WARN risingwave_connector_node: Unknown OID -1 requested thread="debezium-postgresconnector-RW_CDC_5-change-event-source-coordinator" class="io.debezium.connector.postgresql.TypeRegistry"
2024-05-21T14:17:31.588272-04:00  WARN risingwave_connector_node: No converter found for column public.postgres_single_types.c_enum of type unknown. The column will not be part of change events for that table. thread="debezium-postgresconnector-RW_CDC_5-change-event-source-coordinator" class="io.debezium.relational.TableSchemaBuilder"

The DBz is still querying with the old oid of the enum.
When you scan the newly create rw table, the c_enum will be NULL

Expected behavior

No response

How did you deploy RisingWave?

In any way.

The version of RisingWave

No response

Additional context

The best solution for now is restarting the RW cluster.

@KeXiangWang KeXiangWang added the type/bug Something isn't working label May 21, 2024
@github-actions github-actions bot added this to the release-1.10 milestone May 21, 2024
@fuyufjh fuyufjh removed this from the release-1.10 milestone Jul 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants