Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ensure wheel distribution includes pydantic shim #243

Merged

Conversation

QMalcolm
Copy link
Collaborator

@QMalcolm QMalcolm commented Jan 4, 2024

Resolves #241

Description

To ensure the pydantic shim was included in the wheel distribution we had to explicitly set what packages we wanted to be included for the wheel in the pyproject.toml.

Manual Testing

Basic testing

  1. ran hatch build in my local dbt_semantic_interfaces repository
  2. uninstalled DSI in my dbt-core main venv pip uninstall dbt-semantic-interfaces
  3. installed my locally built wheel to my dbt-core main venv pip install /Users/quigleymalcolm/Developer/dbt-labs/dbt-semantic-interfaces/dist/dbt_semantic_interfaces-0.5.0a2-py3-none-any.whl
  4. checked that both dbt-semantic-interfaces and dsi_pydantic_shim.py were present in dbt-core/venv/lib/python3.8/site-packages/

Functionally testing

(This requires at least step 1 from the basic testing)

  1. updated the core/setup.py to use my local dbt-semantic interfaces wheel
    a. update the dependency from "dbt-semantic-interfaces~=0.5.0a2" to "dbt-semantic-interfaces" in the install_requires param
    b. added the following param/value dependency_links=["/Users/quigleymalcolm/Developer/dbt-labs/dbt-semantic-interfaces/dist/dbt_semantic_interfaces-0.5.0a2-py3-none-any.whl"] to the setup config
  2. cleared my tox cache rm -rf .tox in dbt-core
  3. ran dbt-core tests make test
  4. double checked that both dbt-semantic-interfaces and dsi_pydantic_shim.py were in the tox cache .tox/py/lib/python3.8/site-packages/

Checklist

…chance

The pydantic shim we provide alongside the dbt-semantic-interfaces package is
installed outside of the dbt-semantif-interfaces by pip. That is when the
wheel is installed the venv lib looks something like

```
venv/
  ...
  lib/
    ...other packages
    dbt-semantic-interfaces
    ...other packages
    pydantic_shim.py
    ...
```

The problem with this is that if another package is installed which also provides
a pydantic shim, we risk a name collision. Put another way if `package_foo` and
`dbt-semantic-interfaces` both provide a `pydantic_shim.py` alongside their package,
then pip will likely be unhappy. To avoid this we've renamed our shim to
`dsi_pydantic_shim.py` to denote our shim as being specific to dbt-semantic-interfaces.
…building

Previously we weren't specifying what was to be included in the wheel build,
thus the wheel distribution was going through the default paths, which were
as documented [here](https://hatch.pypa.io/1.9/plugins/builder/wheel/#default-file-selection)

* <NAME>/__init__.py
* src/<NAME>/__init__.py
* <NAME>.py
* <NAMESPACE>/<NAME>/__init__.py

This meant that our pydantic shim was being excluded. This commit alters
the pyproject.toml to now begin specifying the packages to be included
in the wheel distribution, and notably adds our pydantic shim to it.
@QMalcolm QMalcolm added the Backport 0.4.latest Fix should be backported to 0.4.latest label Jan 4, 2024
@cla-bot cla-bot bot added the cla:yes label Jan 4, 2024
@QMalcolm QMalcolm requested review from MichelleArk and tlento January 4, 2024 20:14
@QMalcolm QMalcolm changed the title Qmalcolm 241 ensure wheel distribution includes pydantic shim Ensure wheel distribution includes pydantic shim Jan 4, 2024
@QMalcolm QMalcolm merged commit 85c6c3a into main Jan 5, 2024
65 checks passed
@QMalcolm QMalcolm deleted the qmalcolm--241-ensure-wheel-distribution-includes-pydantic-shim branch January 5, 2024 01:17
QMalcolm added a commit that referenced this pull request Jan 8, 2024
[BACKPORT] #243 to 0.4.latest (wheel distribution fix)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Backport 0.4.latest Fix should be backported to 0.4.latest cla:yes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[SL-1489] [Bug] The pydantic shim isn't being included in the wheel distributions of the package
2 participants