Single workspace member with abi3 feature will force this for all members

[adamreichold]

Bug Description

The rust-numpy repository is a Cargo workspace containing the main crate and three example crates which are Python extensions, one of which enables usage of the stable Python ABI via PyO3's abi3 features. It appears this alone is enough to force auto-detection of abi3 usage for the other examples as well which will silently fail until PyO3 0.16.3 and fails a consistency check since PyO3 0.16.4: Maturin detects abi3, sets PYO3_NO_PYTHON but the crate itself does indeed not have any of the abi3 features enabled.

Is it possible that the abi3 detection logic is per-workspace instead of per-package?

Your Python version (python -V)

3.8.13

Your pip version (pip -V)

22.0.4

What bindings you're using

pyo3

Does cargo build work?

• Yes, it works

If on windows, have you checked that you aren't accidentally using unix path (those with the forward slash /)?

• Yes

Steps to Reproduce

https://github.com/PyO3/rust-numpy/runs/6029114285?check_suite_focus=true

[adamreichold]

(Conversely, if I remove the usage of the abi3 feature from the one example where it is supposed to be used, all other examples revert back to a normal PyO3-based build.)

[messense]

Opened #877

[adamreichold]

Reading Maturin's source code and considering that Cargo unifies features across the whole of a workspace, I came to the conclusion that the way I use features in the rust-numpy repository is wrong and that I should break out the examples into separate workspace to test their features independently.

Hence, I opened PyO3/rust-numpy#322 and think we should be closing this instead.

[adamreichold]

(I am sorry for writing this only now, but it took me a while to understand my mistake and figure out that what I did should not be working in the first place.)

[messense]

https://doc.rust-lang.org/cargo/reference/resolver.html#features

When building multiple packages in a workspace (such as with --workspace or multiple -p flags), the features of the dependencies of all of those packages are unified. If you have a circumstance where you want to avoid that unification for different workspace members, you will need to build them via separate cargo invocations.

My read of this is that when you do cargo build without --workspace and multiple -p flags, features are not unified?

[adamreichold]

Agreed, but then instead of #877, the way the Cargo meta-data is collected should be changed so that the above applies?

[adamreichold]

But I am not sure if Cargo actually exposes this information, i.e. if I run

cargo metadata --manifest-path examples/linalg/Cargo.toml | python3 -m json.tool > deps.json

in the rust-numpy repsitory, I find

{
    "id": "pyo3 0.16.4 (registry+https://github.com/rust-lang/crates.io-index)",
    "dependencies": [
        "cfg-if 1.0.0 (registry+https://github.com/rust-lang/crates.io-index)",
        "indoc 1.0.4 (registry+https://github.com/rust-lang/crates.io-index)",
        "inventory 0.2.2 (registry+https://github.com/rust-lang/crates.io-index)",
        "libc 0.2.123 (registry+https://github.com/rust-lang/crates.io-index)",
        "parking_lot 0.12.0 (registry+https://github.com/rust-lang/crates.io-index)",
        "pyo3-build-config 0.16.4 (registry+https://github.com/rust-lang/crates.io-index)",
        "pyo3-ffi 0.16.4 (registry+https://github.com/rust-lang/crates.io-index)",
        "pyo3-macros 0.16.4 (registry+https://github.com/rust-lang/crates.io-index)",
        "unindent 0.1.8 (registry+https://github.com/rust-lang/crates.io-index)"
    ],
    "deps": [
        {
            "name": "cfg_if",
            "pkg": "cfg-if 1.0.0 (registry+https://github.com/rust-lang/crates.io-index)",
            "dep_kinds": [
                {
                    "kind": null,
                    "target": null
                }
            ]
        },
        {
            "name": "indoc",
            "pkg": "indoc 1.0.4 (registry+https://github.com/rust-lang/crates.io-index)",
            "dep_kinds": [
                {
                    "kind": null,
                    "target": null
                }
            ]
        },
        {
            "name": "inventory",
            "pkg": "inventory 0.2.2 (registry+https://github.com/rust-lang/crates.io-index)",
            "dep_kinds": [
                {
                    "kind": null,
                    "target": null
                }
            ]
        },
        {
            "name": "libc",
            "pkg": "libc 0.2.123 (registry+https://github.com/rust-lang/crates.io-index)",
            "dep_kinds": [
                {
                    "kind": null,
                    "target": null
                }
            ]
        },
        {
            "name": "parking_lot",
            "pkg": "parking_lot 0.12.0 (registry+https://github.com/rust-lang/crates.io-index)",
            "dep_kinds": [
                {
                    "kind": null,
                    "target": null
                }
            ]
        },
        {
            "name": "pyo3_build_config",
            "pkg": "pyo3-build-config 0.16.4 (registry+https://github.com/rust-lang/crates.io-index)",
            "dep_kinds": [
                {
                    "kind": "build",
                    "target": null
                }
            ]
        },
        {
            "name": "pyo3_ffi",
            "pkg": "pyo3-ffi 0.16.4 (registry+https://github.com/rust-lang/crates.io-index)",
            "dep_kinds": [
                {
                    "kind": null,
                    "target": null
                }
            ]
        },
        {
            "name": "pyo3_macros",
            "pkg": "pyo3-macros 0.16.4 (registry+https://github.com/rust-lang/crates.io-index)",
            "dep_kinds": [
                {
                    "kind": null,
                    "target": null
                }
            ]
        },
        {
            "name": "unindent",
            "pkg": "unindent 0.1.8 (registry+https://github.com/rust-lang/crates.io-index)",
            "dep_kinds": [
                {
                    "kind": null,
                    "target": null
                }
            ]
        }
    ],
    "features": [
        "abi3",
        "abi3-py310",
        "abi3-py37",
        "abi3-py38",
        "abi3-py39",
        "auto-initialize",
        "default",
        "extension-module",
        "indoc",
        "inventory",
        "macros",
        "multiple-pymethods",
        "pyo3-macros",
        "pyproto",
        "unindent"
    ]
},

even though rust-linalg does enable neither multiple-pymethods nor abi3.

[adamreichold]

But I am not sure if Cargo actually exposes this information,

So essentially, this seems to be a mismatch between what cargo metadata does and what cargo build does.

[messense]

So essentially, this seems to be a mismatch between what cargo metadata does and what cargo build does.

Yes, I think we should find or open an issue in rust-lang/cargo.

[messense]

Found it: rust-lang/cargo#7754

[adamreichold]

I think I will break out the examples from the workspace for now, since it will probably take a while to resolve this upstream. (I need to do this any way for our MSRV tests and the loss of usability during development can be handled via the xtask script.)