Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update dbt-metricflow dependencies to use dbt-* 1.8 packages #1244

Merged
merged 8 commits into from
Jun 5, 2024
Merged

Update dbt-metricflow dependencies to use dbt-* 1.8 packages #1244

merged 8 commits into from
Jun 5, 2024

Conversation

plypaul
Copy link
Contributor

@plypaul plypaul commented Jun 4, 2024 •
edited
Loading

Resolves #1243

Description

This PR makes the following project configuration changes:

  • Use requirements.txt-style files to share dependency definitions between metricflow and dbt-metricflow.
  • Remove the editable installs that are run when setting up the hatch environment used for development. It seems to work with out them now.
  • Update dbt-metricflow dependencies to use dbt-* 1.8 packages

@cla-bot cla-bot bot added the cla:yes label Jun 4, 2024
@plypaul plypaul force-pushed the p--py312--10 branch 2 times, most recently from 4de8598 to e55b4a3 Compare June 4, 2024 17:44
@plypaul plypaul changed the title Remove editable installs and update dbt-metricflow dependencies to use dbt-* 1.8 packages Update dbt-metricflow dependencies to use dbt-* 1.8 packages Jun 4, 2024
@plypaul plypaul added the Run Tests With Other SQL Engines Runs the test suite against the SQL engines in our target environment label Jun 4, 2024
@plypaul plypaul temporarily deployed to DW_INTEGRATION_TESTS June 4, 2024 17:49 — with GitHub Actions Inactive
@plypaul plypaul temporarily deployed to DW_INTEGRATION_TESTS June 4, 2024 17:49 — with GitHub Actions Inactive
@plypaul plypaul temporarily deployed to DW_INTEGRATION_TESTS June 4, 2024 17:49 — with GitHub Actions Inactive
@plypaul plypaul temporarily deployed to DW_INTEGRATION_TESTS June 4, 2024 17:49 — with GitHub Actions Inactive
@github-actions github-actions bot removed the Run Tests With Other SQL Engines Runs the test suite against the SQL engines in our target environment label Jun 4, 2024
@plypaul plypaul changed the base branch from p--py312--09 to main June 4, 2024 18:26
@plypaul plypaul added the Run Tests With Other SQL Engines Runs the test suite against the SQL engines in our target environment label Jun 4, 2024
@plypaul plypaul temporarily deployed to DW_INTEGRATION_TESTS June 4, 2024 18:41 — with GitHub Actions Inactive
@plypaul plypaul had a problem deploying to DW_INTEGRATION_TESTS June 4, 2024 18:41 — with GitHub Actions Failure
@plypaul plypaul temporarily deployed to DW_INTEGRATION_TESTS June 4, 2024 18:41 — with GitHub Actions Inactive
@plypaul plypaul temporarily deployed to DW_INTEGRATION_TESTS June 4, 2024 18:41 — with GitHub Actions Inactive
@plypaul plypaul marked this pull request as ready for review June 4, 2024 18:42
@plypaul plypaul temporarily deployed to DW_INTEGRATION_TESTS June 4, 2024 22:21 — with GitHub Actions Inactive
@plypaul plypaul temporarily deployed to DW_INTEGRATION_TESTS June 4, 2024 22:21 — with GitHub Actions Inactive
@plypaul plypaul temporarily deployed to DW_INTEGRATION_TESTS June 4, 2024 22:21 — with GitHub Actions Inactive
@plypaul plypaul temporarily deployed to DW_INTEGRATION_TESTS June 4, 2024 22:21 — with GitHub Actions Inactive
@github-actions github-actions bot removed the Run Tests With Other SQL Engines Runs the test suite against the SQL engines in our target environment label Jun 4, 2024
tlento
tlento reviewed Jun 5, 2024
View reviewed changes
Copy link
Contributor

@tlento tlento left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unless I'm missing something I think you've removed the optional install for dbt-trino. I'm assuming that's a mistake?

dbt-metricflow/pyproject.toml
snowflake = [
"dbt-snowflake>=1.7.0, <1.8.0"
]
trino = [
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wait, what happened to trino?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch - added.

dbt-metricflow/extra-hatch-configuration/requirements-dbt-bigquery.txt Outdated
@@ -0,0 +1 @@
dbt-bigquery>=1.7.0, <1.8.0
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this really the only way to do this? Having to update all these files is a drag.

We should be able to automate it if we need to do it often enough, so that's nice. Or maybe we can remove the dependency range and let the dbt-core version dictate what's acceptable via the pip resolver.

dbt-metricflow/pyproject.toml Outdated
]
redshift = [
"dbt-redshift>=1.7.0, <1.8.0"
dbt-bigquery = [
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: I really like alphabetical order for this kind of thing.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated.

pyproject.toml
@@ -110,132 +104,123 @@ packages = ["metricflow", "metricflow-semantics/metricflow_semantics"]

[tool.hatch.envs.dev-env]
description = "Environment for development. Includes a DuckDB-backed client."

pre-install-commands = [
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment block above this is is now out of date. Maybe update to explain what's going on here - like why do we need to install metricflow-semantics as a dev env dependency when it's already listed as part of the build?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do we need to install metricflow-semantics as a dev env dependency when it's already listed as part of the build?

Forgot to make the update in this commit - it was fixed in: #1246

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added the fix for this branch.

pyproject.toml Outdated Show resolved Hide resolved
pyproject.toml Outdated Show resolved Hide resolved
@plypaul plypaul force-pushed the p--py312--10 branch 2 times, most recently from ceb6079 to 01dc251 Compare June 5, 2024 01:10
tlento
tlento approved these changes Jun 5, 2024
View reviewed changes
Copy link
Contributor

@tlento tlento left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here's hoping the dev workflows remain reasonable!🤞

@plypaul plypaul merged commit dde021e into main Jun 5, 2024
15 checks passed
@plypaul plypaul deleted the p--py312--10 branch June 5, 2024 22:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@tlento tlento tlento approved these changes

Assignees
No one assigned
Labels
cla:yes
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[SL-2323] [Feature] Update dbt-metricflow dependencies to use dbt-* 1.8 packages
2 participants
@plypaul @tlento