Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Extend the SQL connector API to support adapting to primary key changes #2729

Closed
edgarrmondragon opened this issue Oct 21, 2024 · 0 comments · Fixed by #2730
Closed
Assignees
Labels
kind/Feature New feature or request SQL Support for SQL taps and targets valuestream/SDK
Milestone

Comments

@edgarrmondragon
Copy link
Collaborator

Feature scope

Targets (data type handling, batching, SQL object generation, tests, etc.)

Description

Right now the flow IIUC goes like this:

  1. SQLConnector.prepare_table is called
    1. If the table doesn't exist or load_method is overwrite a new table is created.
    2. Else, SQLConnector.prepare_column is called for each of the table's columns.
      1. If the column doesn't exist, it's created.
      2. Else, an attempt to update the column type is made, if necessary.

So, there's a probably missing SQLConnector.prepare_primary_key API, to be called after columns are updated that updates the table primary keys if necessary.

Related:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/Feature New feature or request SQL Support for SQL taps and targets valuestream/SDK
Projects
None yet
1 participant