Skip to content

Commit

Permalink
Make duckdb the standard in the dev-env environment
Browse files Browse the repository at this point in the history
MetricFlow's testing configuration currently tags sql engine types
to hatch execution environment. The standard one, which we use for
`make test`, is meant to be duckdb, but a stale environment variable
can cause inscrutable failures on any test requiring an engine
connection.

This change updates the test environment configuration to make duckdb
the standard for any runs in the hatch dev-env, including any
invocation such as `make test`. It also removes the default fallback
to duckdb, which was causing inconsistent behaviors in certain
environment configurations, and replaces it with a consistent error.

Developers with heavily customized environment configurations might
have a bad time with this, but they were already having a bad time
with other things, and at least this gives them an actionable
error message.
  • Loading branch information
tlento committed Aug 15, 2023
1 parent 772eb32 commit 76d4aeb
Show file tree
Hide file tree
Showing 3 changed files with 14 additions and 12 deletions.
6 changes: 6 additions & 0 deletions .changes/unreleased/Under the Hood-20230814-225729.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
kind: Under the Hood
body: Make duckdb the standard for all dev-env environment runs, including `make test`
time: 2023-08-14T22:57:29.775028-07:00
custom:
Author: tlento
Issue: "723"
19 changes: 7 additions & 12 deletions metricflow/test/fixtures/setup_fixtures.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

import datetime
import logging
import os
import warnings
from dataclasses import dataclass

Expand Down Expand Up @@ -68,24 +69,18 @@ def pytest_addoption(parser: _pytest.config.argparsing.Parser) -> None:
)


class MetricFlowTestEnvironmentVariables:
"""Environment variables to setup the testing environment."""

MF_SQL_ENGINE_URL = EnvironmentVariable("MF_SQL_ENGINE_URL")
MF_SQL_ENGINE_PASSWORD = EnvironmentVariable("MF_SQL_ENGINE_PASSWORD")


@pytest.fixture(scope="session")
def mf_test_session_state( # noqa: D
request: FixtureRequest,
disable_sql_alchemy_deprecation_warning: None,
source_table_snapshot_repository: SqlTableSnapshotRepository,
) -> MetricFlowTestSessionState:
engine_url = MetricFlowTestEnvironmentVariables.MF_SQL_ENGINE_URL.get_optional()
if engine_url is None:
logger.info(f"{MetricFlowTestEnvironmentVariables.MF_SQL_ENGINE_URL.name} has not been set, so using DuckDb")
engine_url = "duckdb://"
engine_password = MetricFlowTestEnvironmentVariables.MF_SQL_ENGINE_PASSWORD.get_optional() or ""
engine_url = os.environ.get("MF_SQL_ENGINE_URL")
assert engine_url is not None, (
"MF_SQL_ENGINE_URL environment variable has not been set! Are you running in a properly configured "
"environment? Check out our CONTRIBUTING.md for pointers to our environment configurations."
)
engine_password = os.environ.get("MF_SQL_ENGINE_PASSWORD", "")

current_time = datetime.datetime.now().strftime("%Y_%m_%d")
random_suffix = random_id()
Expand Down
1 change: 1 addition & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -118,6 +118,7 @@ features = [

[tool.hatch.envs.dev-env.env-vars]
MF_TEST_ADAPTER_TYPE="duckdb"
MF_SQL_ENGINE_URL="duckdb://"

[tool.hatch.envs.postgres-env.env-vars]
MF_SQL_ENGINE_URL="postgresql://metricflow@localhost:5432/metricflow"
Expand Down

0 comments on commit 76d4aeb

Please sign in to comment.