Exclusive extra markers do not work as expected when the package is a dependency #10120
Comments
kikones34
added
kind/bug
|
I am not at all sure of the status of negative extras, they strike me as dubious at best. Is there any standard support for this or is it a poetry-invented attempt to workaround torch unpleasantness? Anyway, even if you can get the resolution to work as you intend this still will not work. The You will have to be explicit in your top level project. |
I looked around a bit, and it seems some people have used them outside of Poetry (e.g. in a setup.py), although it doesn't seem very widespread. Either way, I believe it truly is the only way to properly deal with CPU / GPU packaging spaghetti. Or at least, I have not been able to figure out any alternate solution, other than:
Ah, that explains why I was having version resolution issues when including specific torch cpu / cuda versions. In any case, I only used that example because I figured it is a "manual example" and should work, since it's straight from the docs. The need to specify different sources is not really relevant to the main problem, however. Below is an example toml for "dependency" which causes the same issue when trying to install it in "dependent". pyproject.toml[project]
name = "dependency"
version = "1.0.0"
requires-python = ">=3.10"
dependencies = ["onnxruntime; extra != 'cuda'"]
[project.optional-dependencies]
cuda = ["onnxruntime-gpu"]
[tool.poetry]
requires-poetry = ">=2.0"Again, this will work as the docs suggest when installing the dependencies of If this is not a bug and is expected behavior, then I believe the docs are misleading, as they dedicate a section to this exact use case and suggest that it is well-supported (as long as |
|
There's an active proposal PEP771 for default extras which is likely relevant here, and an ongoing discussion here. That looks like the intended standards approach for this sort of situation, So far as I can see no-one on the thread is saying "hold on, negative extras are already a thing, this is all unnecessary". If you believe that's the case then you may want to chime in. The proposal looks to be progressing towards general agreement, I have no insight into whether or when poetry might implement it. If the PEP is agreed then perhaps you'd like to contribute. |
|
Thanks for the heads up, I wasn't aware of that PEP. The more I look into it, the more convinced I am that the In any case, right now it is presumably supported by Poetry, so either we make it work all the way, or we add a disclaimer regarding the limitations of the feature. I could try to contribute myself if I can find the time. @reesehyde @radoering I'm sorry to bother you, but do you have any input regarding this? Was it expected that exclusive extras would propagate correctly down dependency chains? TL;DR context: the feature implemented in #9553 doesn't seem to be working when a package specifying exclusive extras is a dependency of another package. The dependencies marked by "negative" extras ( |
Description
When a package uses extra markers for its dependencies, it works as expected. However, when this package is a dependency of another package, they do not.
Consider the example in the documentation about exclusive extra markers, and modify it slightly to be valid as a package:
`dependency` package `pyproject.toml`
(Note: I also removed the explicit torch versions because it was giving me resolution issues).
When executing
poetry syncandpoetry sync --extras cuda, Poetry behaves as expected (installs either the CPU or CUDA version of pytorch).Now, make another package that depends on this package, and "pass through" the
cudaextra.`dependent` package `pyproject.toml`
When executing
poetry sync, thedependencypackage gets installed, but no version of torch gets installed (unexpected). When executingpoetry sync --extras cuda, the CUDA version of torch gets installed (expected).Workarounds
I've been unable to find any workaround after trying for a while.
Poetry Installation Method
pip
Operating System
Ubuntu 22.04.5 LTS
Poetry Version
Poetry (version 2.0.1)
Poetry Configuration
Python Sysconfig
sysconfig.log
Example pyproject.toml
See description.
Poetry Runtime Logs
poetry -vvv sync
The text was updated successfully, but these errors were encountered: