Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Regression] Fix psycopg2 version install #113

Merged
merged 42 commits into from
Jun 18, 2024

Conversation

mikealfare
Copy link
Contributor

resolves #96

Problem

We broke folks who were installing dbt-postgres on Linux and expecting psycopg2-binary instead of psycopg2. This is an issue because psycopg2 expects some OS level dependencies whereas psycopg2-binary does not. In particular, this broke a lot of CI workflows which traditionally run on some flavor of Linux.

Solution

  • default to psycopg2-binary everywhere
  • add a hatch post-build hook that looks for the environment variable DBT_PSYCOPG2_NAME and swaps out psycopg2-binary for psycopg2 and hard pins to the same version
    • this aligns with the previously existing approach
  • update relevant tests

Checklist

  • I have read the contributing guide and understand what's expected of me
  • I have run this code in development and it appears to resolve the stated issue
  • This PR includes tests, or tests are not required/relevant for this PR
  • This PR has no interface changes (e.g. macros, cli, logs, json artifacts, config files, adapter interface, etc) or this PR has already received feedback and approval from Product or DX

@mikealfare mikealfare self-assigned this Jun 6, 2024
@cla-bot cla-bot bot added the cla:yes label Jun 6, 2024
Copy link

github-actions bot commented Jun 6, 2024

Thank you for your pull request! We could not find a changelog entry for this change. For details on how to document a change, see the dbt-postgres contributing guide.

hatch_hooks.py Outdated Show resolved Hide resolved
@mikealfare mikealfare marked this pull request as ready for review June 7, 2024 19:01
@VersusFacit
Copy link
Contributor

Just a naive question -- does doing this lose us anything? I assume no but worth asking as we're moving to the binary.

Copy link
Contributor

@VersusFacit VersusFacit left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking for commentary and possibly some more programmatic accessing of package data.

@mikealfare
Copy link
Contributor Author

Just a naive question -- does doing this lose us anything? I assume no but worth asking as we're moving to the binary.

The only thing this really impacts is if you're looking for better performance out of psycopg2, which is really a prod thing. psycopg2-binary is fine for development and CI testing. So the issue is that folks will need to update their prod installations with the three line workaround in the README.md until we can automate that for folks.

@mikealfare mikealfare marked this pull request as draft June 14, 2024 16:08
@mikealfare mikealfare marked this pull request as ready for review June 14, 2024 17:12

PSYCOPG2_NAME=$((pip show psycopg2 || pip show psycopg2-binary) | grep Name | cut -d " " -f 2)
if [[ "$PSYCOPG2_NAME" != "$PSYCOPG2_EXPECTED_NAME" ]]; then
echo -e 'Expected: "$PSYCOPG2_EXPECTED_NAME" but found: "$PSYCOPG2_NAME"'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: can we pipe this to stderr >&2

Copy link
Contributor

@VersusFacit VersusFacit left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One nit, otherwise looking good

@colin-rogers-dbt colin-rogers-dbt merged commit 7c19b33 into main Jun 18, 2024
21 checks passed
@colin-rogers-dbt colin-rogers-dbt deleted the fix-psycopg2-version-install branch June 18, 2024 21:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Regression] Forced psycopg2 dependency on Linux causes sudden build failures
4 participants